diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000000..0fa80d1f86 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,29 @@ +--- +name: Bug report +about: Our code behaves incorrectly? +title: '' +labels: bug +assignees: '' + +--- + + + +**Describe the bug** + +What happened? What should have happened instead? + +**Provide a Reproducer** + +* If possible, please provide a small self-contained project (or even just a single file) where the issue reproduces. +* If you can't pinpoint the issue, please provide at least *some* project where this reproduces, for example, your production one. If you are not ready to show the project publicly, we are open to discussing the details privately. +* If you really can't provide any code, please do still open an issue. This may prompt other people to chime in with their reproducers. diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000000..145cd391f2 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,5 @@ +blank_issues_enabled: true +contact_links: + - name: Kotlinlang Slack + url: https://surveys.jetbrains.com/s3/kotlin-slack-sign-up + about: Please ask and answer usage-related questions here. diff --git a/.github/ISSUE_TEMPLATE/design_considerations.md b/.github/ISSUE_TEMPLATE/design_considerations.md new file mode 100644 index 0000000000..cca209d81b --- /dev/null +++ b/.github/ISSUE_TEMPLATE/design_considerations.md @@ -0,0 +1,45 @@ +--- +name: Design considerations +about: We didn't think things through? +title: '' +labels: design +assignees: '' + +--- + + + +**What do we have now?** + +Preferably with specific code examples. + +**What should be instead?** + +Preferably with specific code examples. + +**Why?** + +The upsides of your proposal. +* Who would benefit from this and how? + - Would it be possible to cover new use cases? + - Would some code become clearer? + - Would the library become conceptually simpler? + - etc. + +**Why not?** + +The downsides of your proposal that you already see. +* Is this a breaking change? +* Are there use cases that are better solved by what we have now? +* Does some code become less clear after this change? +* etc. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000000..4fb7a0b7ff --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,33 @@ +--- +name: Feature request +about: We're missing something? +title: '' +labels: enhancement +assignees: '' + +--- + + + +**Use case** + +Explain what *specifically* you are trying to do and why. +- Example: "I have a `SharedFlow` that represents readings from an external device. The readings arrive in a set interval of 100 milliseconds. However, I also need to be able to calibrate the state of the external device by setting the readings from inside the program. When I set the state, the `SharedFlow` must immediately emit the value that was set and ignore any values coming from the device in the following 10 milliseconds since they are considered outdated, as the device is only guaranteed to recalibrate to the updated value after that period." +- Non-example: "I have a `SharedFlow` that has several sources of its values, and these sources need to have priorities attached to them so that one source always takes precedence over the other in a close race." +- Non-example: "RxJava has feature X, so the coroutines library should also." + +**The Shape of the API** + +What could the desired API look like? What would some sample code using the new feature look like? If you don't have a clear idea, pseudocode or just explaining the API shape is also perfectly fine. + +**Prior Art** + +(Optional) Maybe you have seen something like the feature you need, but in other libraries, or there is something very similar but not quite sufficient in `kotlinx.coroutines`? Maybe there's already a way to do it, but it's too cumbersome and unclear? diff --git a/.github/ISSUE_TEMPLATE/guide.md b/.github/ISSUE_TEMPLATE/guide.md new file mode 100644 index 0000000000..f78d5878a9 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/guide.md @@ -0,0 +1,20 @@ +--- +name: Issue with the Coroutines guide +about: Problems on https://kotlinlang.org/docs/coroutines-guide.html +title: '' +labels: guide +assignees: '' + +--- + +**Which page?** + +Drop a link to a specific page, unless you're reporting something that's missing entirely. + +**What can be improved?** + +Describe the problematic part. Is it misleading, unclear, or just something that didn’t work for you? + +**Is this explained better somewhere else?** + +Show us the explanation that made it click for you. diff --git a/.github/ISSUE_TEMPLATE/rc_feedback.md b/.github/ISSUE_TEMPLATE/rc_feedback.md new file mode 100644 index 0000000000..02f06dda9e --- /dev/null +++ b/.github/ISSUE_TEMPLATE/rc_feedback.md @@ -0,0 +1,28 @@ +--- +name: Release Candidate release feedback +about: Something used to work, but broke in an RC release? +title: 'RC:' +labels: bug +assignees: '' + +--- + + + +**What broke?** + +Even a vague description suffices. If you want to search for a reproducer or +narrow the issue down, this would help a lot, but the most important thing is +letting us know that there is a problem at all as soon as possible. Otherwise, +we'll just publish a stable release with this problem while you're looking for a +reproducer! + +**Did I check that setting the version to the latest stable release fixes the problem?** + +Yes. diff --git a/.gitignore b/.gitignore index 35cf719595..8dc03d4e1f 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,17 @@ -/.idea +**/.idea/* +!/.idea/icon.png +!/.idea/vcs.xml +!/.idea/copyright +!/.idea/codeStyleSettings.xml +!/.idea/codeStyles +!/.idea/dictionaries *.iml -target \ No newline at end of file +.gradle +.gradletasknamecache +build +out +target +local.properties +benchmarks.jar +/kotlin-js-store +/.kotlin diff --git a/.idea/codeStyleSettings.xml b/.idea/codeStyleSettings.xml new file mode 100644 index 0000000000..2287edb44c --- /dev/null +++ b/.idea/codeStyleSettings.xml @@ -0,0 +1,8 @@ + + + + + + \ No newline at end of file diff --git a/.idea/codeStyles/Project.xml b/.idea/codeStyles/Project.xml new file mode 100644 index 0000000000..461a31ed76 --- /dev/null +++ b/.idea/codeStyles/Project.xml @@ -0,0 +1,15 @@ + + + + + + + + diff --git a/.idea/codeStyles/codeStyleConfig.xml b/.idea/codeStyles/codeStyleConfig.xml new file mode 100644 index 0000000000..79ee123c2b --- /dev/null +++ b/.idea/codeStyles/codeStyleConfig.xml @@ -0,0 +1,5 @@ + + + + \ No newline at end of file diff --git a/.idea/copyright/profiles_settings.xml b/.idea/copyright/profiles_settings.xml new file mode 100644 index 0000000000..5e22a9977e --- /dev/null +++ b/.idea/copyright/profiles_settings.xml @@ -0,0 +1,3 @@ + + + \ No newline at end of file diff --git a/.idea/dictionaries/shared.xml b/.idea/dictionaries/shared.xml new file mode 100644 index 0000000000..45cbedcf1f --- /dev/null +++ b/.idea/dictionaries/shared.xml @@ -0,0 +1,14 @@ + + + + Alistarh + Elizarov + Koval + kotlinx + lincheck + linearizability + linearizable + redirector + + + \ No newline at end of file diff --git a/.idea/icon.png b/.idea/icon.png new file mode 100644 index 0000000000..d957c64af5 Binary files /dev/null and b/.idea/icon.png differ diff --git a/.idea/vcs.xml b/.idea/vcs.xml new file mode 100644 index 0000000000..fd137b62c9 --- /dev/null +++ b/.idea/vcs.xml @@ -0,0 +1,22 @@ + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/CHANGES.md b/CHANGES.md index b15c306520..c289a5d8b0 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,171 +1,252 @@ -# Change log for kotlinx.coroutines - -## Version 0.15 - -* Switched to Kotlin version 1.1.2 (can still be used with 1.1.0). -* `CoroutineStart` enum is introduced for `launch`/`async`/`actor` builders: - * The usage of `luanch(context, start = false)` is deprecated and is replaced with - `launch(context, CoroutineStart.LAZY)` - * `CoroutineStart.UNDISPATCHED` is introduced to start coroutine execution immediately in the invoker thread, - so that `async(context, CoroutineStart.UNDISPATCHED)` is similar to the behavior of C# `async`. - * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) mentions the use of it to optimize - the start of coroutines from UI threads. -* Introduced `BroadcastChannel` interface in `kotlinx-coroutines-core` module: - * It extends `SendChannel` interface and provides `open` function to create subscriptions. - * Subscriptions are represented with `SubscriptionReceiveChannel` interface. - * The corresponding `SubscriptionReceiveChannel` interfaces are removed from [reactive](reactive) implementation - modules. They use an interface defined in `kotlinx-coroutines-core` module. - * `ConflatedBroadcastChannel` implementation is provided for state-observation-like use-cases, where a coroutine or a - regular code (in UI, for example) updates the state that subscriber coroutines shall react to. - * `ArrayBroadcastChannel` implementation is provided for event-bus-like use-cases, where a sequence of events shall - be received by multiple subscribers without any omissions. - * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) includes - "Rx Subject vs BroadcastChannel" section. -* Pull requests from Konrad Kamiński are merged into reactive stream implementations: - * Support for Project Reactor `Mono` and `Flux`. - See [`kotlinx-coroutines-reactor`](reactive/kotlinx-coroutines-reactor) module. - * Implemented Rx1 `Completable.awaitCompleted`. - * Added support for Rx2 `Maybe`. -* Better timeout support: - * Introduced `withTimeoutOrNull` function. - * Implemented `onTimeout` clause for `select` expressions. - * Fixed spurious concurrency inside `withTimeout` blocks on their cancellation. - * Changed behavior of `withTimeout` when `CancellationException` is suppressed inside the block. - Invocation of `withTimeout` now always returns the result of execution of its inner block. -* The `channel` property in `ActorScope` is promoted to a wider `Channel` type, so that an actor - can have an easy access to its own inbox send channel. -* Renamed `Mutex.withMutex` to `Mutex.withLock`, old name is deprecated. - -## Version 0.14 - -* Switched to Kotlin version 1.1.1 (can still be used with 1.1.0). -* Introduced `consumeEach` helper function for channels and reactive streams, Rx 1.x, and Rx 2.x. - * It ensures that streams are unsubscribed from on any exception. - * Iteration with `for` loop on reactive streams is **deprecated**. - * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) is updated virtually - all over the place to reflect these important changes. -* Implemented `awaitFirstOrDefault` extension for reactive streams, Rx 1.x, and Rx 2.x. -* Added `Mutex.withMutex` helper function. -* `kotlinx-coroutines-android` module has `provided` dependency on of Android APIs to - eliminate warnings when using it in android project. - - -## Version 0.13 - -* New `kotlinx-coroutinex-android` module with Android `UI` context implementation. -* Introduced `whileSelect` convenience function. -* Implemented `ConflatedChannel`. -* Renamed various `toXXX` conversion functions to `asXXX` (old names are deprecated). -* `run` is optimized with fast-path case and no longer has `CoroutineScope` in its block. -* Fixed dispatching logic of `withTimeout` (removed extra dispatch). -* `EventLoop` that is used by `runBlocking` now implements Delay, giving more predictable test behavior. -* Various refactorings related to resource management and timeouts: - * `Job.Registration` is renamed to `DisposableHandle`. - * `EmptyRegistration` is renamed to `NonDisposableHandle`. - * `Job.unregisterOnCompletion` is renamed to `Job.disposeOnCompletion`. - * `Delay.invokeOnTimeout` is introduced. - * `withTimeout` now uses `Delay.invokeOnTimeout` when available. -* A number of improvement for reactive streams and Rx: - * Introduced `rxFlowable` builder for Rx 2.x. - * `Scheduler.asCoroutineDispatcher` extension for Rx 2.x. - * Fixed bug with sometimes missing `onComplete` in `publish`, `rxObservable`, and `rxFlowable` builders. - * Channels that are open for reactive streams are now `Closeable`. - * Fixed `CompletableSource.await` and added test for it. - * Removed `rx.Completable.await` due to name conflict. -* New documentation: - * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) - * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) -* Code is published to JCenter repository. - -## Version 0.12 - -* Switched to Kotlin version 1.1.0 release. -* Reworked and updated utilities for - [Reactive Streams](kotlinx-coroutines-reactive), - [Rx 1.x](kotlinx-coroutines-rx1), and - [Rx 2.x](kotlinx-coroutines-rx2) with library-specific - coroutine builders, suspending functions, converters and iteration support. -* `LinkedListChannel` with unlimited buffer (`offer` always succeeds). -* `onLock` select clause and an optional `owner` parameter in all `Mutex` functions. -* `selectUnbiased` function. -* `actor` coroutine builder. -* Couple more examples for "Shared mutable state and concurrency" section and - "Channels are fair" section with ping-pong table example - in [coroutines guide](coroutines-guide.md). - -## Version 0.11-rc - -* `select` expression with onJoin/onAwait/onSend/onReceive clauses. -* `Mutex` is moved to `kotlinx.coroutines.experimental.sync` package. -* `ClosedSendChannelException` is a subclass of `CancellationException` now. -* New sections on "Shared mutable state and concurrency" and "Select expression" - in [coroutines guide](coroutines-guide.md). - -## Version 0.10-rc - -* Switched to Kotlin version 1.1.0-rc-91. -* `Mutex` synchronization primitive is introduced. -* `buildChannel` is renamed to `produce`, old name is deprecated. -* `Job.onCompletion` is renamed to `Job.invokeOnCompletion`, old name is deprecated. -* `delay` implementation in Swing, JavaFx, and scheduled executors is fixed to avoid an extra dispatch. -* `CancellableContinuation.resumeUndispatched` is introduced to make this efficient implementation possible. -* Remove unnecessary creation of `CancellationException` to improve performance, plus other performance improvements. -* Suppress deprecated and internal APIs from docs. -* Better docs at top level with categorized summary of classes and functions. - -## Version 0.8-beta - -* `defer` coroutine builder is renamed to `async`. -* `lazyDefer` is deprecated, `async` has an optional `start` parameter instead. -* `LazyDeferred` interface is deprecated, lazy start functionality is integrated into `Job` interface. -* `launch` has an optional `start` parameter for lazily started coroutines. -* `Job.start` and `Job.isCompleted` are introduced. -* `Deferred.isCompletedExceptionally` and `Deferred.isCancelled` are introduced. -* `Job.getInactiveCancellationException` is renamed to `getCompletionException`. -* `Job.join` is now a member function. -* Internal `JobSupport` state machine is enhanced to support _new_ (not-started-yet) state. - So, lazy coroutines do not need a separate state variable to track their started/not-started (new/active) status. -* Exception transparency in `Job.cancel` (original cause is rethrown). -* Clarified possible states for `Job`/`CancellableContinuation`/`Deferred` in docs. -* Example on async-style functions and links to API reference site from [coroutines guide](coroutines-guide.md). - -## Version 0.7-beta - -* Buffered and unbuffered channels are introduced: `Channel`, `SendChannel`, and `ReceiveChannel` interfaces, - `RendezvousChannel` and `ArrayChannel` implementations, `Channel()` factory function and `buildChannel{}` - coroutines builder. -* `Here` context is renamed to `Unconfined` (the old name is deprecated). -* A [guide on coroutines](coroutines-guide.md) is expanded: sections on contexts and channels. - -## Version 0.6-beta - -* Switched to Kotlin version 1.1.0-beta-37. -* A [guide on coroutines](coroutines-guide.md) is expanded. - -## Version 0.5-beta - -* Switched to Kotlin version 1.1.0-beta-22 (republished version). -* Removed `currentCoroutineContext` and related thread-locals without replacement. - Explicitly pass coroutine context around if needed. -* `lazyDefer(context) {...}` coroutine builder and `LazyDeferred` interface are introduced. -* The default behaviour of all coroutine dispatchers is changed to always schedule execution of new coroutine - for later in this thread or thread pool. Correspondingly, `CoroutineDispatcher.isDispatchNeeded` function - has a default implementation that returns `true`. -* `NonCancellable` context is introduced. -* Performance optimizations for cancellable continuations (fewer objects created). -* A [guide on coroutines](coroutines-guide.md) is added. - -## Version 0.4-beta - -* Switched to Kotlin version 1.1.0-beta-18 (republished version). -* `CoroutineDispatcher` methods now have `context` parameter. -* Introduced `CancellableContinuation.isCancelled` -* Introduced `EventLoop` dispatcher and made it a default for `runBlocking { ... }` -* Introduced `CoroutineScope` interface with `isActive` and `context` properties; - standard coroutine builders include it as receiver for convenience. -* Introduced `Executor.toCoroutineDispatcher()` extension. -* Delay scheduler thread is not daemon anymore, but times out automatically. -* Debugging facilities in `newCoroutineContext` can be explicitly disabled with `-Dkotlinx.coroutines.debug=off`. -* xxx-test files are renamed to xxx-example for clarity. -* Fixed NPE in Job implementation when starting coroutine with already cancelled parent job. -* Support cancellation in `kotlinx-coroutines-nio` module +# Change log for kotlinx.coroutines + +## Version 1.10.2 + +* Fixed the `kotlinx-coroutines-debug` JAR file including the `module-info.class` file twice, resulting in failures in various tooling (#4314). Thanks, @RyuNen344! +* Fixed `Flow.stateIn` hanging when the scope is cancelled in advance or the flow is empty (#4322). Thanks, @francescotescari! +* Improved handling of dispatcher failures in `.limitedParallelism` (#4330) and during flow collection (#4272). +* Fixed `runBlocking` failing to run its coroutine to completion in some cases if its JVM thread got interrupted (#4399). +* Small tweaks, fixes, and documentation improvements. + +## Version 1.10.1 + +* Fixed binary incompatibility introduced for non-JVM targets in #4261 (#4309). + +## Version 1.10.0 + +* Kotlin was updated to 2.1.0 (#4284). +* Introduced `Flow.any`, `Flow.all`, and `Flow.none` (#4212). Thanks, @CLOVIS-AI! +* Reorganized `kotlinx-coroutines-debug` and `kotlinx-coroutines-core` code to avoid a split package between the two artifacts (#4247). Note that directly referencing `kotlinx.coroutines.debug.AgentPremain` must now be replaced with `kotlinx.coroutines.debug.internal.AgentPremain`. Thanks, @sellmair! +* No longer shade byte-buddy in `kotlinx-coroutines-debug`, reducing the artifact size and simplifying the build configuration of client code. Thanks, @sellmair! +* Fixed `NullPointerException` when using Java-deserialized `kotlinx-coroutines-core` exceptions (#4291). Thanks, @AlexRiedler! +* Properly report exceptions thrown by `CoroutineDispatcher.dispatch` instead of raising internal errors (#4091). Thanks, @zuevmaxim! +* Fixed a bug that delayed scheduling of a `Dispatchers.Default` or `Dispatchers.IO` task after a `yield()` in rare scenarios (#4248). +* Fixed a bug that prevented the `main()` coroutine on Wasm/WASI from executing after a `delay()` call in some scenarios (#4239). +* Fixed scheduling of `runBlocking` tasks on Kotlin/Native that arrive after the `runBlocking` block was exited (#4245). +* Fixed some terminal `Flow` operators sometimes resuming without taking cancellation into account (#4254). Thanks, @jxdabc! +* Fixed a bug on the JVM that caused coroutine-bound `ThreadLocal` values not to get cleaned when using non-`CoroutineDispatcher` continuation interceptors (#4296). +* Small tweaks, fixes, and documentation improvements. + +## Version 1.9.0 + +### Features + +* Wasm/WASI target support (#4064). Thanks, @igoriakovlev! +* `limitedParallelism` now optionally accepts the name of the dispatcher view for easier debugging (#4023). +* No longer initialize `Dispatchers.IO` on the JVM when other standard dispatchers are accessed (#4166). Thanks, @metalhead8816! +* Introduced the `Flow.chunked(size: Int): Flow>` operator that groups emitted values into groups of the given size (#1290). +* Closeable dispatchers are instances of `AutoCloseable` now (#4123). + +### Fixes + +* Calling `hasNext` on a `Channel`'s iterator is idempotent (#4065). Thanks, @gitpaxultek! +* `CoroutineScope()` created without an explicit dispatcher uses `Dispatchers.Default` on Native (#4074). Thanks, @whyoleg! +* Fixed a bug that prevented non-Android `Dispatchers.Main` from initializing when the Firebase dependency is used (#3914). +* Ensured a more intuitive ordering of tasks in `runBlocking` (#4134). +* Forbid casting a `Mutex` to `Semaphore` (#4176). +* Worked around a stack overflow that may occur when calling `asDeferred` on a `Future` many times (#4156). + +### Deprecations and promotions + +* Advanced the deprecation levels for `BroadcastChannel`-based API (#4197). +* Advanced the deprecation levels for the old `kotlinx-coroutines-test` API (#4198). +* Deprecated `Job.cancelFutureOnCompletion` (#4173). +* Promoted `CoroutineDispatcher.limitedParallelism` to stable (#3864). +* Promoted `CoroutineStart.ATOMIC` from `ExperimentalCoroutinesApi` to `DelicateCoroutinesApi` (#4169). +* Promoted `CancellableContinuation.resume` with an `onCancellation` lambda to stable, providing extra arguments to the lambda (#4088). +* Marked the classes and interfaces that are not supposed to be inherited from with the new `InternalForInheritanceCoroutinesApi` opt-in (#3770). +* Marked the classes and interfaces inheriting from which is not stable with the new `ExperimentalForInheritanceCoroutinesApi` opt-in (#3770). + +### Other + +* Kotlin was updated to 2.0 (#4137). +* Reworked the documentation for `CoroutineStart` and `Channel`-based API (#4147, #4148, #4167). Thanks, @globsterg! +* Simplified the internal implementation of `Job` (#4053). +* Small tweaks, fixes, and documentation improvements. + +## Version 1.9.0-RC.2 + +* Advanced the deprecation levels for `BroadcastChannel`-based API (#4197). +* Advanced the deprecation levels for the old `kotlinx-coroutines-test` API (#4198). +* Promoted `CoroutineStart.ATOMIC` from `ExperimentalCoroutinesApi` to `DelicateCoroutinesApi` (#4169). +* Reworked the documentation for `CoroutineStart` and `Channel`-based API (#4147, #4148, #4167). Thanks, @globsterg! +* Forbid casting a `Mutex` to `Semaphore` (#4176). +* Deprecated `Job.cancelFutureOnCompletion` (#4173). +* Worked around a stack overflow that may occur when calling `asDeferred` on a `Future` many times (#4156). +* Fixed a bug that disallowed setting a custom `probeCoroutineResumed` when starting coroutines with `UNDISPATCHED` (#4162). +* No longer initialize `Dispatchers.IO` on the JVM when other standard dispatchers are accessed (#4166). Thanks, @metalhead8816! +* Small tweaks, fixes, and documentation improvements. + +## Version 1.9.0-RC + +* Kotlin was updated to 2.0 (#4137). +* Introduced the `Flow.chunked(size: Int): Flow>` operator that groups emitted values into groups of the given size (#1290). +* Closeable dispatchers are instances of `AutoCloseable` now (#4123). +* `limitedParallelism` now optionally accepts the name of the dispatcher view for easier debugging (#4023). +* Marked the classes and interfaces that are not supposed to be inherited from with the new `InternalForInheritanceCoroutinesApi` opt-in (#3770). +* Marked the classes and interfaces inheriting from which is not stable with the new `ExperimentalForInheritanceCoroutinesApi` opt-in (#3770). +* Wasm/WASI target support (#4064). Thanks, @igoriakovlev! +* Promoted `CoroutineDispatcher.limitedParallelism` to stable (#3864). +* Promoted `CancellableContinuation.resume` with an `onCancellation` lambda to stable, providing extra arguments to the lambda (#4088). +* Ensured a more intuitive ordering of tasks in `runBlocking` (#4134). +* Simplified the internal implementation of `Job` (#4053). +* Fixed a bug that prevented non-Android `Dispatchers.Main` from initializing when the Firebase dependency is used (#3914). +* Calling `hasNext` on a `Channel`'s iterator is idempotent (#4065). Thanks, @gitpaxultek! +* `CoroutineScope()` created without an explicit dispatcher uses `Dispatchers.Default` on Native (#4074). Thanks, @whyoleg! +* Small tweaks and documentation fixes. + +## Version 1.8.1 + +* Remove the `@ExperimentalTime` annotation from usages of `TimeSource` (#4046). Thanks, @hfhbd! +* Introduce a workaround for an Android bug that caused an occasional `NullPointerException` when setting the `StateFlow` value on old Android devices (#3820). +* No longer use `kotlin.random.Random` as part of `Dispatchers.Default` and `Dispatchers.IO` initialization (#4051). +* `Flow.timeout` throws the exception with which the channel was closed (#4071). +* Small tweaks and documentation fixes. + +### Changelog relative to version 1.8.1-Beta + +* `Flow.timeout` throws the exception with which the channel was closed (#4071). +* Small documentation fixes. + +## Version 1.8.1-Beta + +* Remove the `@ExperimentalTime` annotation from usages of `TimeSource` (#4046). Thanks, @hfhbd! +* Attempt a workaround for an Android bug that caused an occasional `NullPointerException` when setting the `StateFlow` value on old Android devices (#3820). +* No longer use `kotlin.random.Random` as part of `Dispatchers.Default` and `Dispatchers.IO` initialization (#4051). +* Small tweaks. + +## Version 1.8.0 + +* Implement the library for the Web Assembly (Wasm) for JavaScript (#3713). Thanks @igoriakovlev! +* Major Kotlin version update: was 1.8.20, became 1.9.21. +* On Android, ensure that `Dispatchers.Main != Dispatchers.Main.immediate` (#3545, #3963). +* Fixed a bug that caused `Flow` operators that limit cancel the upstream flow to forget that they were already finished if there is another such operator upstream (#4035, #4038) +* `kotlinx-coroutines-debug` is published with the correct Java 9 module info (#3944). +* `kotlinx-coroutines-debug` no longer requires manually setting `DebugProbes.enableCoroutineCreationStackTraces` to `false`, it's the default (#3783). +* `kotlinx-coroutines-test`: set the default timeout of `runTest` to 60 seconds, added the ability to configure it on the JVM with the `kotlinx.coroutines.test.default_timeout=10s` (#3800). +* `kotlinx-coroutines-test`: fixed a bug that could lead to not all uncaught exceptions being reported after some tests failed (#3800). +* `delay(Duration)` rounds nanoseconds up to whole milliseconds and not down (#3920). Thanks @kevincianfarini! +* `Dispatchers.Default` and the default thread for background work are guaranteed to use the same context classloader as the object containing it them (#3832). +* It is guaranteed that by the time `SharedFlow.collect` suspends for the first time, it's registered as a subscriber for that `SharedFlow` (#3885). Before, it was also true, but not documented. +* Atomicfu version is updated to 0.23.1, and Kotlin/Native atomic transformations are enabled, reducing the footprint of coroutine-heavy code (#3954). +* Added a workaround for miscompilation of `withLock` on JS (#3881). Thanks @CLOVIS-AI! +* Small tweaks and documentation fixes. + +### Changelog relative to version 1.8.0-RC2 + +* `kotlinx-coroutines-debug` no longer requires manually setting `DebugProbes.enableCoroutineCreationStackTraces` to `false`, it's the default (#3783). +* Fixed a bug that caused `Flow` operators that limit cancel the upstream flow to forget that they were already finished if there is another such operator upstream (#4035, #4038) +* Small documentation fixes. + +## Version 1.8.0-RC2 + +* Fixed a bug introduced in 1.8.0-RC where `Mutex.onLock` would not unlock if a non-local return was performed (#3985). +* Fixed a bug introduced in 1.8.0-RC where depending on kotlinx-coroutines in Native code failed with a compilation error `Could not find "org.jetbrains.kotlinx:atomicfu-cinterop-interop"` (#3968). +* Small documentation fixes. + +## Version 1.8.0-RC + +* Implement the library for the Web Assembly (Wasm) for JavaScript (#3713). Thanks @igoriakovlev! +* On Android, ensure that `Dispatchers.Main != Dispatchers.Main.immediate` (#3545, #3963). +* `kotlinx-coroutines-debug` is published with the correct Java 9 module info (#3944). +* Major Kotlin version update: was 1.8.20, became 1.9.21. +* `kotlinx-coroutines-test`: set the default timeout of `runTest` to 60 seconds, added the ability to configure it on the JVM with the `kotlinx.coroutines.test.default_timeout=10s` (#3800). +* `kotlinx-coroutines-test`: fixed a bug that could lead to not all uncaught exceptions being reported after some tests failed (#3800). +* `delay(Duration)` rounds nanoseconds up to whole milliseconds and not down (#3920). Thanks @kevincianfarini! +* `Dispatchers.Default` and the default thread for background work are guaranteed to use the same context classloader as the object containing it them (#3832). +* It is guaranteed that by the time `SharedFlow.collect` suspends for the first time, it's registered as a subscriber for that `SharedFlow` (#3885). Before, it was also true, but not documented. +* Atomicfu version is updated to 0.23.1, and Kotlin/Native atomic transformations are enabled, reducing the footprint of coroutine-heavy code (#3954). +* Added a workaround for miscompilation of `withLock` on JS (#3881). Thanks @CLOVIS-AI! +* Small tweaks and documentation fixes. + +## Version 1.7.3 + +* Disabled the publication of the multiplatform library metadata for the old (1.6 and earlier) KMP Gradle plugin (#3809). +* Fixed a bug introduced in 1.7.2 that disabled the coroutine debugger in IDEA (#3822). + +## Version 1.7.2 + +### Bug fixes and improvements + +* Coroutines debugger no longer keeps track of coroutines with empty coroutine context (#3782). +* `CopyableThreadContextElement` now properly copies an element when crossing the coroutine boundary in `flowOn` (#3787). Thanks @wanyingd1996! +* Coroutine timeouts no longer prevent K/N `newSingleThreadContext` from closing (#3768). +* A non-linearizability in `Mutex` during `tryLock`/`unlock` sequence with owners is fixed (#3745). +* Atomicfu version is updated to 0.21.0. + +## Version 1.7.1 + +### Bug fixes and improvements + +* Special characters in coroutine names in JSON dumps are supported (#3747) +* The binary compatibility of the experimental overload of `runTest` is restored (#3673) +* Channels that don't use `onUndeliveredElement` now allocate less memory (#3646) + +## Version 1.7.0 + +### Core API significant improvements + +* New `Channel` implementation with significant performance improvements across the API (#3621). +* New `select` operator implementation: faster, more lightweight, and more robust (#3020). +* `Mutex` and `Semaphore` now share the same underlying data structure (#3020). +* `Dispatchers.IO` is added to K/N (#3205) + * `newFixedThreadPool` and `Dispatchers.Default` implementations on K/N were wholly rewritten to support graceful growth under load (#3595). +* `kotlinx-coroutines-test` rework: + - Add the `timeout` parameter to `runTest` for the whole-test timeout, 10 seconds by default (#3270). This replaces the configuration of quiescence timeouts, which is now deprecated (#3603). + - The `withTimeout` exception messages indicate if the timeout used the virtual time (#3588). + - `TestCoroutineScheduler`, `runTest`, and `TestScope` API are promoted to stable (#3622). + - `runTest` now also fails if there were uncaught exceptions in coroutines not inherited from the test coroutine (#1205). + +### Breaking changes + +* Old K/N memory model is no longer supported (#3375). +* New generic upper bounds were added to reactive integration API where the language since 1.8.0 dictates (#3393). +* `kotlinx-coroutines-core` and `kotlinx-coroutines-jdk8` artifacts were merged into a single artifact (#3268). +* Artificial stackframes in stacktrace recovery no longer contain the `\b` symbol and are now navigable in IDE and supplied with proper documentation (#2291). +* `CoroutineContext.isActive` returns `true` for contexts without any job in them (#3300). + +### Bug fixes and improvements + +* Kotlin version is updated to 1.8.20 +* Atomicfu version is updated to 0.20.2. +* `JavaFx` version is updated to 17.0.2 in `kotlinx-coroutines-javafx` (#3671).. +* JPMS is supported (#2237). Thanks @lion7! +* `BroadcastChannel` and all the corresponding API are deprecated (#2680). +* Added all supported K/N targets (#3601, #812, #855). +* K/N `Dispatchers.Default` is backed by the number of threads equal to the number of available cores (#3366). +* Fixed an issue where some coroutines' internal exceptions were not properly serializable (#3328). +* Introduced `Job.parent` API (#3201). +* Fixed a bug when `TestScheduler` leaked cancelled jobs (#3398). +* `TestScope.timeSource` now provides comparable time marks (#3617). Thanks @hfhbd! +* Fixed an issue when cancelled `withTimeout` handles were preserved in JS runtime (#3440). +* Ensure `awaitFrame` only awaits a single frame when used from the main looper (#3432). Thanks @pablobaxter! +* Obsolete `Class-Path` attribute was removed from `kotlinx-coroutines-debug.jar` manifest (#3361). +* Fixed a bug when `updateThreadContext` operated on the parent context (#3411). +* Added new `Flow.filterIsInstance` extension (#3240). +* `Dispatchers.Default` thread name prefixes are now configurable with system property (#3231). +* Added `Flow.timeout` operator as `@FlowPreview` (#2624). Thanks @pablobaxter! +* Improved the performance of the `future` builder in case of exceptions (#3475). Thanks @He-Pin! +* `Mono.awaitSingleOrNull` now waits for the `onComplete` signal (#3487). +* `Channel.isClosedForSend` and `Channel.isClosedForReceive` are promoted from experimental to delicate (#3448). +* Fixed a data race in native `EventLoop` (#3547). +* `Dispatchers.IO.limitedParallelism(valueLargerThanIOSize)` no longer creates an additional wrapper (#3442). Thanks @dovchinnikov! +* Various `@FlowPreview` and `@ExperimentalCoroutinesApi` are promoted to experimental and stable respectively (#3542, #3097, #3548). +* Performance improvements in `Dispatchers.Default` and `Dispatchers.IO` (#3416, #3418). +* Fixed a bug when internal `suspendCancellableCoroutineReusable` might have hanged (#3613). +* Introduced internal API to process events in the current system dispatcher (#3439). +* Global `CoroutineExceptionHandler` is no longer invoked in case of unprocessed `future` failure (#3452). +* Performance improvements and reduced thread-local pressure for the `withContext` operator (#3592). +* Improved performance of `DebugProbes` (#3527). +* Fixed a bug when the coroutine debugger might have detected the state of a coroutine incorrectly (#3193). +* `CoroutineDispatcher.asExecutor()` runs tasks without dispatching if the dispatcher is unconfined (#3683). Thanks @odedniv! +* `SharedFlow.toMutableList` and `SharedFlow.toSet` lints are introduced (#3706). +* `Channel.invokeOnClose` is promoted to stable API (#3358). +* Improved lock contention in `Dispatchers.Default` and `Dispatchers.IO` during the startup phase (#3652). +* Fixed a bug that led to threads oversubscription in `Dispatchers.Default` (#3642). +* Fixed a bug that allowed `limitedParallelism` to perform dispatches even after the underlying dispatcher was closed (#3672). +* Fixed a bug that prevented stacktrace recovery when the exception's constructor from `cause` was selected (#3714). +* Improved sanitizing of stracktrace-recovered traces (#3714). +* Introduced an internal flag to disable uncaught exceptions reporting in tests as a temporary migration mechanism (#3736). +* Various documentation improvements and fixes. + +Changelog for previous versions may be found in [CHANGES_UP_TO_1.7.md](CHANGES_UP_TO_1.7.md) diff --git a/CHANGES_UP_TO_1.7.md b/CHANGES_UP_TO_1.7.md new file mode 100644 index 0000000000..c59e3b306e --- /dev/null +++ b/CHANGES_UP_TO_1.7.md @@ -0,0 +1,1606 @@ +# Change log for kotlinx.coroutines + +## Version 1.7.0 + +### Core API significant improvements + +* New `Channel` implementation with significant performance improvements across the API (#3621). +* New `select` operator implementation: faster, more lightweight, and more robust (#3020). +* `Mutex` and `Semaphore` now share the same underlying data structure (#3020). +* `Dispatchers.IO` is added to K/N (#3205) + * `newFixedThreadPool` and `Dispatchers.Default` implementations on K/N were wholly rewritten to support graceful growth under load (#3595). +* `kotlinx-coroutines-test` rework: + - Add the `timeout` parameter to `runTest` for the whole-test timeout, 10 seconds by default (#3270). This replaces the configuration of quiescence timeouts, which is now deprecated (#3603). + - The `withTimeout` exception messages indicate if the timeout used the virtual time (#3588). + - `TestCoroutineScheduler`, `runTest`, and `TestScope` API are promoted to stable (#3622). + - `runTest` now also fails if there were uncaught exceptions in coroutines not inherited from the test coroutine (#1205). + +### Breaking changes + +* Old K/N memory model is no longer supported (#3375). +* New generic upper bounds were added to reactive integration API where the language since 1.8.0 dictates (#3393). +* `kotlinx-coroutines-core` and `kotlinx-coroutines-jdk8` artifacts were merged into a single artifact (#3268). +* Artificial stackframes in stacktrace recovery no longer contain the `\b` symbol and are now navigable in IDE and supplied with proper documentation (#2291). +* `CoroutineContext.isActive` returns `true` for contexts without any job in them (#3300). + +### Bug fixes and improvements + +* Kotlin version is updated to 1.8.20 +* Atomicfu version is updated to 0.20.2. +* `JavaFx` version is updated to 17.0.2 in `kotlinx-coroutines-javafx` (#3671).. +* JPMS is supported (#2237). Thanks @lion7! +* `BroadcastChannel` and all the corresponding API are deprecated (#2680). +* Added all supported K/N targets (#3601, #812, #855). +* K/N `Dispatchers.Default` is backed by the number of threads equal to the number of available cores (#3366). +* Fixed an issue where some coroutines' internal exceptions were not properly serializable (#3328). +* Introduced `Job.parent` API (#3201). +* Fixed a bug when `TestScheduler` leaked cancelled jobs (#3398). +* `TestScope.timeSource` now provides comparable time marks (#3617). Thanks @hfhbd! +* Fixed an issue when cancelled `withTimeout` handles were preserved in JS runtime (#3440). +* Ensure `awaitFrame` only awaits a single frame when used from the main looper (#3432). Thanks @pablobaxter! +* Obsolete `Class-Path` attribute was removed from `kotlinx-coroutines-debug.jar` manifest (#3361). +* Fixed a bug when `updateThreadContext` operated on the parent context (#3411). +* Added new `Flow.filterIsInstance` extension (#3240). +* `Dispatchers.Default` thread name prefixes are now configurable with system property (#3231). +* Added `Flow.timeout` operator as `@FlowPreview` (#2624). Thanks @pablobaxter! +* Improved the performance of the `future` builder in case of exceptions (#3475). Thanks @He-Pin! +* `Mono.awaitSingleOrNull` now waits for the `onComplete` signal (#3487). +* `Channel.isClosedForSend` and `Channel.isClosedForReceive` are promoted from experimental to delicate (#3448). +* Fixed a data race in native `EventLoop` (#3547). +* `Dispatchers.IO.limitedParallelism(valueLargerThanIOSize)` no longer creates an additional wrapper (#3442). Thanks @dovchinnikov! +* Various `@FlowPreview` and `@ExperimentalCoroutinesApi` are promoted to experimental and stable respectively (#3542, #3097, #3548). +* Performance improvements in `Dispatchers.Default` and `Dispatchers.IO` (#3416, #3418). +* Fixed a bug when internal `suspendCancellableCoroutineReusable` might have hanged (#3613). +* Introduced internal API to process events in the current system dispatcher (#3439). +* Global `CoroutineExceptionHandler` is no longer invoked in case of unprocessed `future` failure (#3452). +* Performance improvements and reduced thread-local pressure for the `withContext` operator (#3592). +* Improved performance of `DebugProbes` (#3527). +* Fixed a bug when the coroutine debugger might have detected the state of a coroutine incorrectly (#3193). +* `CoroutineDispatcher.asExecutor()` runs tasks without dispatching if the dispatcher is unconfined (#3683). Thanks @odedniv! +* `SharedFlow.toMutableList` and `SharedFlow.toSet` lints are introduced (#3706). +* `Channel.invokeOnClose` is promoted to stable API (#3358). +* Improved lock contention in `Dispatchers.Default` and `Dispatchers.IO` during the startup phase (#3652). +* Fixed a bug that led to threads oversubscription in `Dispatchers.Default` (#3642). +* Fixed a bug that allowed `limitedParallelism` to perform dispatches even after the underlying dispatcher was closed (#3672). +* Fixed a bug that prevented stacktrace recovery when the exception's constructor from `cause` was selected (#3714). +* Improved sanitizing of stracktrace-recovered traces (#3714). +* Introduced an internal flag to disable uncaught exceptions reporting in tests as a temporary migration mechanism (#3736). +* Various documentation improvements and fixes. + +### Changelog relative to version 1.7.0-RC + +* Fixed a bug that prevented stacktrace recovery when the exception's constructor from `cause` was selected (#3714). +* Improved sanitizing of stracktrace-recovered traces (#3714). +* Introduced an internal flag to disable uncaught exceptions reporting in tests as a temporary migration mechanism (#3736). + +## Version 1.7.0-RC + +* Kotlin version is updated to 1.8.20. +* Atomicfu version is updated to 0.20.2. +* `JavaFx` version is updated to 17.0.2 in `kotlinx-coroutines-javafx` (#3671). +* `previous-compilation-data.bin` file is removed from JAR resources (#3668). +* `CoroutineDispatcher.asExecutor()` runs tasks without dispatching if the dispatcher is unconfined (#3683). Thanks @odedniv! +* `SharedFlow.toMutableList` lint overload is undeprecated (#3706). +* `Channel.invokeOnClose` is promoted to stable API (#3358). +* Improved lock contention in `Dispatchers.Default` and `Dispatchers.IO` during the startup phase (#3652). +* Fixed a bug that led to threads oversubscription in `Dispatchers.Default` (#3642). +* Fixed a bug that allowed `limitedParallelism` to perform dispatches even after the underlying dispatcher was closed (#3672). +* Restored binary compatibility of previously experimental `TestScope.runTest(Long)` (#3673). + +## Version 1.7.0-Beta + +### Core API significant improvements + +* New `Channel` implementation with significant performance improvements across the API (#3621). +* New `select` operator implementation: faster, more lightweight, and more robust (#3020). +* `Mutex` and `Semaphore` now share the same underlying data structure (#3020). +* `Dispatchers.IO` is added to K/N (#3205) + * `newFixedThreadPool` and `Dispatchers.Default` implementations on K/N were wholly rewritten to support graceful growth under load (#3595). +* `kotlinx-coroutines-test` rework: + - Add the `timeout` parameter to `runTest` for the whole-test timeout, 10 seconds by default (#3270). This replaces the configuration of quiescence timeouts, which is now deprecated (#3603). + - The `withTimeout` exception messages indicate if the timeout used the virtual time (#3588). + - `TestCoroutineScheduler`, `runTest`, and `TestScope` API are promoted to stable (#3622). + - `runTest` now also fails if there were uncaught exceptions in coroutines not inherited from the test coroutine (#1205). + +### Breaking changes + +* Old K/N memory model is no longer supported (#3375). +* New generic upper bounds were added to reactive integration API where the language since 1.8.0 dictates (#3393). +* `kotlinx-coroutines-core` and `kotlinx-coroutines-jdk8` artifacts were merged into a single artifact (#3268). +* Artificial stackframes in stacktrace recovery no longer contain the `\b` symbol and are now navigable in IDE and supplied with proper documentation (#2291). +* `CoroutineContext.isActive` returns `true` for contexts without any job in them (#3300). + +### Bug fixes and improvements + +* Kotlin version is updated to 1.8.10. +* JPMS is supported (#2237). Thanks @lion7! +* `BroadcastChannel` and all the corresponding API are deprecated (#2680). +* Added all supported K/N targets (#3601, #812, #855). +* K/N `Dispatchers.Default` is backed by the number of threads equal to the number of available cores (#3366). +* Fixed an issue where some coroutines' internal exceptions were not properly serializable (#3328). +* Introduced `Job.parent` API (#3201). +* Fixed a bug when `TestScheduler` leaked cancelled jobs (#3398). +* `TestScope.timeSource` now provides comparable time marks (#3617). Thanks @hfhbd! +* Fixed an issue when cancelled `withTimeout` handles were preserved in JS runtime (#3440). +* Ensure `awaitFrame` only awaits a single frame when used from the main looper (#3432). Thanks @pablobaxter! +* Obsolete `Class-Path` attribute was removed from `kotlinx-coroutines-debug.jar` manifest (#3361). +* Fixed a bug when `updateThreadContext` operated on the parent context (#3411). +* Added new `Flow.filterIsInstance` extension (#3240). +* `Dispatchers.Default` thread name prefixes are now configurable with system property (#3231). +* Added `Flow.timeout` operator as `@FlowPreview` (#2624). Thanks @pablobaxter! +* Improved the performance of the `future` builder in case of exceptions (#3475). Thanks @He-Pin! +* `Mono.awaitSingleOrNull` now waits for the `onComplete` signal (#3487). +* `Channel.isClosedForSend` and `Channel.isClosedForReceive` are promoted from experimental to delicate (#3448). +* Fixed a data race in native `EventLoop` (#3547). +* `Dispatchers.IO.limitedParallelism(valueLargerThanIOSize)` no longer creates an additional wrapper (#3442). Thanks @dovchinnikov! +* Various `@FlowPreview` and `@ExperimentalCoroutinesApi` are promoted to experimental and stable respectively (#3542, #3097, #3548). +* Performance improvements in `Dispatchers.Default` and `Dispatchers.IO` (#3416, #3418). +* Fixed a bug when internal `suspendCancellableCoroutineReusable` might have hanged (#3613). +* Introduced internal API to process events in the current system dispatcher (#3439). +* Global `CoroutineExceptionHandler` is no longer invoked in case of unprocessed `future` failure (#3452). +* Performance improvements and reduced thread-local pressure for the `withContext` operator (#3592). +* Improved performance of `DebugProbes` (#3527). +* Fixed a bug when the coroutine debugger might have detected the state of a coroutine incorrectly (#3193). +* Various documentation improvements and fixes. + +## Version 1.6.4 + +* Added `TestScope.backgroundScope` for launching coroutines that perform work in the background and need to be cancelled at the end of the test (#3287). +* Fixed the POM of `kotlinx-coroutines-debug` having an incorrect reference to `kotlinx-coroutines-bom`, which cause the builds of Maven projects using the debug module to break (#3334). +* Fixed the `Publisher.await` functions in `kotlinx-coroutines-reactive` not ensuring that the `Subscriber` methods are invoked serially (#3360). Thank you, @EgorKulbachka! +* Fixed a memory leak in `withTimeout` on K/N with the new memory model (#3351). +* Added the guarantee that all `Throwable` implementations in the core library are serializable (#3328). +* Moved the documentation to (#3342). +* Various documentation improvements. + +## Version 1.6.3 + +* Updated atomicfu version to 0.17.3 (#3321), fixing the projects using this library with JS IR failing to build (#3305). + +## Version 1.6.2 + +* Fixed a bug with `ThreadLocalElement` not being correctly updated when the most outer `suspend` function was called directly without `kotlinx.coroutines` (#2930). +* Fixed multiple data races: one that might have been affecting `runBlocking` event loop, and a benign data race in `Mutex` (#3250, #3251). +* Obsolete `TestCoroutineContext` is removed, which fixes the `kotlinx-coroutines-test` JPMS package being split between `kotlinx-coroutines-core` and `kotlinx-coroutines-test` (#3218). +* Updated the ProGuard rules to further shrink the size of the resulting DEX file with coroutines (#3111, #3263). Thanks, @agrieve! +* Atomicfu is updated to `0.17.2`, which includes a more efficient and robust JS IR transformer (#3255). +* Kotlin is updated to `1.6.21`, Gradle version is updated to `7.4.2` (#3281). Thanks, @wojtek-kalicinski! +* Various documentation improvements. + +## Version 1.6.1 + +* Rollback of time-related functions dispatching on `Dispatchers.Main`. + This behavior was introduced in 1.6.0 and then found inconvenient and erroneous (#3106, #3113). +* Reworked the newly-introduced `CopyableThreadContextElement` to solve issues uncovered after the initial release (#3227). +* Fixed a bug with `ThreadLocalElement` not being properly updated in racy scenarios (#2930). +* Reverted eager loading of default `CoroutineExceptionHandler` that triggered ANR on some devices (#3180). +* New API to convert a `CoroutineDispatcher` to a Rx scheduler (#968, #548). Thanks @recheej! +* Fixed a memory leak with the very last element emitted from `flow` builder being retained in memory (#3197). +* Fixed a bug with `limitedParallelism` on K/N with new memory model throwing `ClassCastException` (#3223). +* `CoroutineContext` is added to the exception printed to the default `CoroutineExceptionHandler` to improve debuggability (#3153). +* Static memory consumption of `Dispatchers.Default` was significantly reduced (#3137). +* Updated slf4j version in `kotlinx-coroutines-slf4j` from 1.7.25 to 1.7.32. + +## Version 1.6.0 + +Note that this is a full changelog relative to the 1.5.2 version. Changelog relative to 1.6.0-RC3 can be found at the end. + +### kotlinx-coroutines-test rework + +* `kotlinx-coroutines-test` became a multiplatform library usable from K/JVM, K/JS, and K/N. +* Its API was completely reworked to address long-standing issues with consistency, structured concurrency and correctness (#1203, #1609, #2379, #1749, #1204, #1390, #1222, #1395, #1881, #1910, #1772, #1626, #1742, #2082, #2102, #2405, #2462 + ). +* The old API is deprecated for removal, but the new API is based on the similar concepts ([README](kotlinx-coroutines-test/README.md)), and the migration path is designed to be graceful: [migration guide](kotlinx-coroutines-test/MIGRATION.md). + +### Dispatchers + +* Introduced `CoroutineDispatcher.limitedParallelism` that allows obtaining a view of the original dispatcher with limited parallelism (#2919). +* `Dispatchers.IO.limitedParallelism` usages ignore the bound on the parallelism level of `Dispatchers.IO` itself to avoid starvation (#2943). +* Introduced new `Dispatchers.shutdown` method for containerized environments (#2558). +* `newSingleThreadContext` and `newFixedThreadPoolContext` are promoted to delicate API (#2919). + +### Breaking changes + +* When racing with cancellation, the `future` builder no longer reports unhandled exceptions into the global `CoroutineExceptionHandler`. Thanks @vadimsemenov! (#2774, #2791). +* `Mutex.onLock` is deprecated for removal (#2794). +* `Dispatchers.Main` is now used as the default source of time for `delay` and `withTimeout` when present(#2972). + * To opt-out from this behaviour, `kotlinx.coroutines.main.delay` system property can be set to `false`. +* Java target of coroutines build is now 8 instead of 6 (#1589). +* **Source-breaking change**: extension `collect` no longer resolves when used with a non-in-place argument of a functional type. This is a candidate for a fix, uncovered after 1.6.0, see #3107 for the additional details. + +### Bug fixes and improvements + +* Kotlin is updated to 1.6.0. +* Kotlin/Native [new memory model](https://blog.jetbrains.com/kotlin/2021/08/try-the-new-kotlin-native-memory-manager-development-preview/) is now supported in regular builds of coroutines conditionally depending on whether `kotlin.native.binary.memoryModel` is enabled (#2914). +* Introduced `CopyableThreadContextElement` for mutable context elements shared among multiple coroutines. Thanks @yorickhenning! (#2893). +* `transformWhile`, `awaitClose`, `ProducerScope`, `merge`, `runningFold`, `runingReduce`, and `scan` are promoted to stable API (#2971). +* `SharedFlow.subscriptionCount` no longer conflates incoming updates and gives all subscribers a chance to observe a short-lived subscription (#2488, #2863, #2871). +* `Flow` exception transparency mechanism is improved to be more exception-friendly (#3017, #2860). +* Cancellation from `flat*` operators that leverage multiple coroutines is no longer propagated upstream (#2964). +* `SharedFlow.collect` now returns `Nothing` (#2789, #2502). +* `DisposableHandle` is now `fun interface`, and corresponding inline extension is removed (#2790). +* `FlowCollector` is now `fun interface`, and corresponding inline extension is removed (#3047). +* Deprecation level of all previously deprecated signatures is raised (#3024). +* The version file is shipped with each JAR as a resource (#2941). +* Unhandled exceptions on K/N are passed to the standard library function `processUnhandledException` (#2981). +* A direct executor is used for `Task` callbacks in `kotlinx-coroutines-play-services` (#2990). +* Metadata of coroutines artifacts leverages Gradle platform to have all versions of dependencies aligned (#2865). +* Default `CoroutineExceptionHandler` is loaded eagerly and does not invoke `ServiceLoader` on its exception-handling path (#2552). +* Fixed the R8 rules for `ServiceLoader` optimization (#2880). +* Fixed BlockHound integration false-positives (#2894, #2866, #2937). +* Fixed the exception handler being invoked several times on Android, thanks to @1zaman (#3056). +* `SendChannel.trySendBlocking` is now available on Kotlin/Native (#3064). +* The exception recovery mechanism now uses `ClassValue` when available (#2997). +* JNA is updated to 5.9.0 to support Apple M1 (#3001). +* Obsolete method on internal `Delay` interface is deprecated (#2979). +* Support of deprecated `CommonPool` is removed. +* `@ExperimentalTime` is no longer needed for methods that use `Duration` (#3041). +* JDK 1.6 is no longer required for building the project (#3043). +* New version of Dokka is used, fixing the memory leak when building the coroutines and providing brand new reference visuals (https://kotlinlang.org/api/kotlinx.coroutines/) (#3051, #3054). + +### Changelog relative to version 1.6.0-RC3 + +* Restored MPP binary compatibility on K/JS and K/N (#3104). +* Fixed Dispatchers.Main not being fully initialized on Android and Swing (#3101). + +## Version 1.6.0-RC3 + +* Fixed the error in 1.6.0-RC2 because of which `Flow.collect` couldn't be called due to the `@InternalCoroutinesApi` annotation (#3082) +* Fixed some R8 warnings introduced in 1.6.0-RC (#3090) +* `TestCoroutineScheduler` now provides a `TimeSource` with its virtual time via the `timeSource` property. Thanks @hfhbd! (#3087) + +## Version 1.6.0-RC2 + +* `@ExperimentalTime` is no longer needed for methods that use `Duration` (#3041). +* `FlowCollector` is now `fun interface`, and corresponding inline extension is removed (#3047). +* Fixed the exception handler being invoked several times on Android, thanks to @1zaman (#3056). +* The deprecated `TestCoroutineScope` is no longer sealed, to simplify migration from it (#3072). +* `runTest` gives more informative errors when it times out waiting for external completion (#3071). +* `SendChannel.trySendBlocking` is now available on Kotlin/Native (#3064). +* Fixed the bug due to which `Dispatchers.Main` was not used for `delay` and `withTimeout` (#3046). +* JDK 1.6 is no longer required for building the project (#3043). +* New version of Dokka is used, fixing the memory leak when building the coroutines and providing brand new reference visuals (https://kotlinlang.org/api/kotlinx.coroutines/) (#3051, #3054). + +## Version 1.6.0-RC + +### kotlinx-coroutines-test rework + +* `kotlinx-coroutines-test` became a multiplatform library usable from K/JVM, K/JS, and K/N. +* Its API was completely reworked to address long-standing issues with consistency, structured concurrency and correctness (#1203, #1609, #2379, #1749, #1204, #1390, #1222, #1395, #1881, #1910, #1772, #1626, #1742, #2082, #2102, #2405, #2462 + ). +* The old API is deprecated for removal, but the new API is based on the similar concepts ([README](kotlinx-coroutines-test/README.md)), and the migration path is designed to be graceful: [migration guide](kotlinx-coroutines-test/MIGRATION.md) + +### Dispatchers + +* Introduced `CoroutineDispatcher.limitedParallelism` that allows obtaining a view of the original dispatcher with limited parallelism (#2919). +* `Dispatchers.IO.limitedParallelism` usages ignore the bound on the parallelism level of `Dispatchers.IO` itself to avoid starvation (#2943). +* Introduced new `Dispatchers.shutdown` method for containerized environments (#2558). +* `newSingleThreadContext` and `newFixedThreadPoolContext` are promoted to delicate API (#2919). + +### Breaking changes + +* When racing with cancellation, the `future` builder no longer reports unhandled exceptions into the global `CoroutineExceptionHandler`. Thanks @vadimsemenov! (#2774, #2791). +* `Mutex.onLock` is deprecated for removal (#2794). +* `Dispatchers.Main` is now used as the default source of time for `delay` and `withTimeout` when present(#2972). + * To opt-out from this behaviour, `kotlinx.coroutines.main.delay` system property can be set to `false`. +* Java target of coroutines build is now 8 instead of 6 (#1589). + +### Bug fixes and improvements + +* Kotlin is updated to 1.6.0. +* Kotlin/Native [new memory model](https://blog.jetbrains.com/kotlin/2021/08/try-the-new-kotlin-native-memory-manager-development-preview/) is now supported in regular builds of coroutines conditionally depending on whether `kotlin.native.binary.memoryModel` is enabled (#2914). +* Introduced `CopyableThreadContextElement` for mutable context elements shared among multiple coroutines. Thanks @yorickhenning! (#2893). +* `transformWhile`, `awaitClose`, `ProducerScope`, `merge`, `runningFold`, `runingReduce`, and `scan` are promoted to stable API (#2971). +* `SharedFlow.subscriptionCount` no longer conflates incoming updates and gives all subscribers a chance to observe a short-lived subscription (#2488, #2863, #2871). +* `Flow` exception transparency mechanism is improved to be more exception-friendly (#3017, #2860). +* Cancellation from `flat*` operators that leverage multiple coroutines is no longer propagated upstream (#2964). +* `SharedFlow.collect` now returns `Nothing` (#2789, #2502). +* `DisposableHandle` is now `fun interface`, and corresponding inline extension is removed (#2790). +* Deprecation level of all previously deprecated signatures is raised (#3024). +* The version file is shipped with each JAR as a resource (#2941). +* Unhandled exceptions on K/N are passed to the standard library function `processUnhandledException` (#2981). +* A direct executor is used for `Task` callbacks in `kotlinx-coroutines-play-services` (#2990). +* Metadata of coroutines artifacts leverages Gradle platform to have all versions of dependencies aligned (#2865). +* Default `CoroutineExceptionHandler` is loaded eagerly and does not invoke `ServiceLoader` on its exception-handling path (#2552). +* Fixed the R8 rules for `ServiceLoader` optimization (#2880). +* Fixed BlockHound integration false-positives (#2894, #2866, #2937). +* The exception recovery mechanism now uses `ClassValue` when available (#2997). +* JNA is updated to 5.9.0 to support Apple M1 (#3001). +* Obsolete method on internal `Delay` interface is deprecated (#2979). +* Support of deprecated `CommonPool` is removed. + +## Version 1.5.2 + +* Kotlin is updated to 1.5.30. +* New native targets for Apple Silicon are introduced. +* Fixed a bug when `onUndeliveredElement` was incorrectly called on a properly received elements on JS (#2826). +* Fixed `Dispatchers.Default` on React Native, it now fully relies on `setTimeout` instead of stub `process.nextTick`. Thanks to @Legion2 (#2843). +* Optimizations of `Mutex` implementation (#2581). +* `Mutex` implementation is made completely lock-free as stated (#2590). +* Various documentation and guides improvements. Thanks to @MasoodFallahpoor and @Pihanya. + +## Version 1.5.1 + +* Atomic `update`, `getAndUpdate`, and `updateAndGet` operations of `MutableStateFlow` (#2720). +* `Executor.asCoroutineDispatcher` implementation improvements (#2601): + * If the target executor is `ScheduledExecutorService`, then its `schedule` API is used for time-related coroutine operations. + * `RemoveOnCancelPolicy` is now part of the public contract. +* Introduced overloads for `Task.asDeferred` and `Task.await` that accept `CancellationTokenSource` for bidirectional cancellation (#2527). +* Reactive streams are updated to `1.0.3` (#2740). +* `CopyableThrowable` is allowed to modify the exception message during stacktrace recovery (#1931). +* `CoroutineDispatcher.releaseInterceptedContinuation` is now a `final` method (#2785). +* Closing a Handler underlying `Handler.asCoroutineDispatcher` now causes the dispatched coroutines to be canceled on `Dispatchers.IO (#2778)`. +* Kotlin is updated to 1.5.20. +* Fixed a spurious `ClassCastException` in `releaseInterceptedContinuation` and `IllegalStateException` from `tryReleaseClaimedContinuation` (#2736, #2768). +* Fixed inconsistent exception message during stacktrace recovery for non-suspending channel iterators (#2749). +* Fixed linear stack usage for `CompletableFuture.asDeferred` when the target future has a long chain of listeners (#2730). +* Any exceptions from `CoroutineDispatcher.isDispatchNeeded` are now considered as fatal and are propagated to the caller (#2733). +* Internal `DebugProbesKt` (used in the debugger implementation) are moved from `debug` to `core` module. + +## Version 1.5.0 + +Note that this is a full changelog relative to 1.4.3 version. Changelog relative to 1.5.0-RC can be found in the end. + +### Channels API + +* Major channels API rework (#330, #974). Existing `offer`, `poll`, and `sendBlocking` methods are deprecated, internal `receiveCatching` and `onReceiveCatching` removed, `receiveOrNull` and `onReceiveOrNull` are completely deprecated. Previously deprecated `SendChannel.isFull` declaration is removed. Channel operators deprecated with `ERROR` are now `HIDDEN`. +* New methods `receiveCatching`, `onReceiveCatching` `trySend`, `tryReceive`, and `trySendBlocking` along with the new result type `ChannelResult` are introduced. They provide better type safety, are less error-prone, and have a consistent future-proof naming scheme. The full rationale behind this change can be found [here](https://github.com/Kotlin/kotlinx.coroutines/issues/974#issuecomment-806569582). +* `BroadcastChannel` and `ConflatedBroadcastChannel` are marked as `ObsoleteCoroutinesApi` in the favor or `SharedFlow` and `StateFlow`. The migration scheme can be found in their documentation. These classes will be deprecated in the next major release. +* `callbackFlow` and `channelFlow` are promoted to stable API. + +### Reactive integrations + +* All existing API in modules `kotlinx-coroutines-rx2`, `kotlinx-coroutines-rx3`, `kotlinx-coroutines-reactive`, `kotlinx-coroutines-reactor`, and `kotlinx-coroutines-jdk9` were revisited and promoted to stable (#2545). +* `publish` is no longer allowed to emit `null` values (#2646). +* Misleading `awaitSingleOr*` functions on `Publisher` type are deprecated (#2591). +* `MaybeSource.await` is deprecated in the favor of `awaitSingle`, additional lint functions for `Mono` are added in order to prevent ambiguous `Publisher` usages (#2628, #1587). +* `ContextView` support in `kotlinx-coroutines-reactor` (#2575). +* All reactive builders no longer ignore inner cancellation exceptions preventing their completion (#2262, #2646). +* `MaybeSource.collect` and `Maybe.collect` properly finish when they are completed without a value (#2617). +* All exceptions are now consistently handled according to reactive specification, whether they are considered 'fatal' or not by reactive frameworks (#2646). + +### Other improvements + +* Kotlin version is upgraded to 1.5.0 and JVM target is updated to 1.8. +* `Flow.last` and `Flow.lastOrNull` operators (#2246). +* `Flow.runningFold` operator (#2641). +* `CoroutinesTimeout` rule for JUnit5 (#2197). +* Internals of `Job` and `AbstractCoroutine` was reworked, resulting in smaller code size, less memory footprint, and better performance (#2513, #2512). +* `CancellationException` from Kotlin standard library is used for cancellation on Kotlin/JS and Kotlin/Native (#2638). +* Introduced new `DelicateCoroutinesApi` annotation that warns users about potential target API pitfalls and suggests studying API's documentation first. The only delicate API right now is `GlobalScope` (#2637). +* Fixed bug introduced in `1.4.3` when `kotlinx-coroutines-core.jar` triggered IDEA debugger failure (#2619). +* Fixed memory leak of `ChildHandlerNode` with reusable continuations (#2564). +* Various documentation improvements (#2555, #2589, #2592, #2583, #2437, #2616, #2633, #2560). + +### Changelog relative to version 1.5.0-RC + +* Fail-fast during `emitAll` called from cancelled `onCompletion` operator (#2700). +* Flows returned by `stateIn`/`shareIn` keep strong reference to sharing job (#2557). +* Rename internal `TimeSource` to `AbstractTimeSource` due to import issues (#2691). +* Reverted the change that triggered IDEA coroutines debugger crash (#2695, reverted #2291). +* `watchosX64` target support for Kotlin/Native (#2524). +* Various documentation fixes and improvements. + +## Version 1.5.0-RC + +### Channels API + +* Major channels API rework (#330, #974). Existing `offer`, `poll`, and `sendBlocking` methods are deprecated, internal `receiveCatching` and `onReceiveCatching` removed, `receiveOrNull` and `onReceiveOrNull` are completely deprecated. Previously deprecated `SendChannel.isFull` declaration is removed. Channel operators deprecated with `ERROR` are now `HIDDEN`. +* New methods `receiveCatching`, `onReceiveCatching` `trySend`, `tryReceive`, and `trySendBlocking` along with the new result type `ChannelResult` are introduced. They provide better type safety, are less error-prone, and have a consistent future-proof naming scheme. The full rationale behind this change can be found [here](https://github.com/Kotlin/kotlinx.coroutines/issues/974#issuecomment-806569582). +* `BroadcastChannel` and `ConflatedBroadcastChannel` are marked as `ObsoleteCoroutinesApi` in the favor or `SharedFlow` and `StateFlow`. The migration scheme can be found in their documentation. These classes will be deprecated in the next major release. +* `callbackFlow` and `channelFlow` are promoted to stable API. + +### Reactive integrations + +* All existing API in modules `kotlinx-coroutines-rx2`, `kotlinx-coroutines-rx3`, `kotlinx-coroutines-reactive`, `kotlinx-coroutines-reactor`, and `kotlinx-coroutines-jdk9` were revisited and promoted to stable (#2545). +* `publish` is no longer allowed to emit `null` values (#2646). +* Misleading `awaitSingleOr*` functions on `Publisher` type are deprecated (#2591). +* `MaybeSource.await` is deprecated in the favor of `awaitSingle`, additional lint functions for `Mono` are added in order to prevent ambiguous `Publisher` usages (#2628, #1587). +* `ContextView` support in `kotlinx-coroutines-reactor` (#2575). +* All reactive builders no longer ignore inner cancellation exceptions preventing their completion (#2262, #2646). +* `MaybeSource.collect` and `Maybe.collect` properly finish when they are completed without a value (#2617). +* All exceptions are now consistently handled according to reactive specification, whether they are considered 'fatal' or not by reactive frameworks (#2646). + +### Other improvements + +* `Flow.last` and `Flow.lastOrNull` operators (#2246). +* `Flow.runningFold` operator (#2641). +* `CoroutinesTimeout` rule for JUnit5 (#2197). +* Internals of `Job` and `AbstractCoroutine` was reworked, resulting in smaller code size, less memory footprint, and better performance (#2513, #2512). +* `CancellationException` from Kotlin standard library is used for cancellation on Kotlin/JS and Kotlin/Native (#2638). +* Introduced new `DelicateCoroutineApi` annotation that warns users about potential target API pitfalls and suggests studying API's documentation first. The only delicate API right now is `GlobalScope` (#2637). +* Fixed bug introduced in `1.4.3` when `kotlinx-coroutines-core.jar` triggered IDEA debugger failure (#2619). +* Fixed memory leak of `ChildHandlerNode` with reusable continuations (#2564). +* Various documentation improvements (#2555, #2589, #2592, #2583, #2437, #2616, #2633, #2560). + +## Version 1.4.3 + +### General changes + +* Thread context is properly preserved and restored for coroutines without `ThreadContextElement` (#985) +* `ThreadContextElement`s are now restored in the opposite order from update (#2195) +* Improved performance of combine with 4 parameters, thanks to @alexvanyo (#2419) +* Debug agent sanitizer leaves at least one frame with source location (#1437) +* Update Reactor version in `kotlinx-coroutines-reactor` to `3.4.1`, thanks to @sokomishalov (#2432) +* `callInPlace` contract added to `ReceiveChannel.consume` (#941) +* `CoroutineStart.UNDISPATCHED` promoted to stable API (#1393) +* Kotlin updated to 1.4.30 +* `kotlinx.coroutines` are now released directly to MavenCentral +* Reduced the size of `DispatchedCoroutine` by a field +* Internal class `TimeSource` renamed to `SchedulerTimeSource` to prevent wildcard import issues (#2537) + +### Bug fixes + +* Fixed the problem that prevented implementation via delegation for `Job` interface (#2423) +* Fixed incorrect ProGuard rules that allowed shrinking volatile felds (#1564) +* Fixed `await`/`asDeferred` for `MinimalStage` implementations in jdk8 module (#2456) +* Fixed bug when `onUndeliveredElement` wasn't called for unlimited channels (#2435) +* Fixed a bug when `ListenableFuture.isCancelled` returned from `asListenableFuture` could have thrown an exception, thanks to @vadimsemenov (#2421) +* Coroutine in `callbackFlow` and `produce` is properly cancelled when the channel was closed separately (#2506) + +## Version 1.4.2 + +* Fixed `StackOverflowError` in `Job.toString` when `Job` is observed in its intermediate state (#2371). +* Improved liveness and latency of `Dispatchers.Default` and `Dispatchers.IO` in low-loaded mode (#2381). +* Improved performance of consecutive `Channel.cancel` invocations (#2384). +* `SharingStarted` is now `fun` interface (#2397). +* Additional lint settings for `SharedFlow` to catch programmatic errors early (#2376). +* Fixed bug when mutex and semaphore were not released during cancellation (#2390, thanks to @Tilps for reproducing). +* Some corner cases in cancellation propagation between coroutines and listenable futures are repaired (#1442, thanks to @vadimsemenov). +* Fixed unconditional cast to `CoroutineStackFrame` in exception recovery that triggered failures of instrumented code (#2386). +* Platform-specific dependencies are removed from `kotlinx-coroutines-javafx` (#2360). + +## Version 1.4.1 + +This is a patch release with an important fix to the `SharedFlow` implementation. + +* SharedFlow: Fix scenario with concurrent emitters and cancellation of subscriber (#2359, thanks to @vehovsky for the bug report). + +## Version 1.4.0 + +### Improvements + +* `StateFlow`, `SharedFlow` and corresponding operators are promoted to stable API (#2316). +* `Flow.debounce` operator with timeout selector based on each individual element is added (#1216, thanks to @mkano9!). +* `CoroutineContext.job` extension property is introduced (#2159). +* `Flow.combine operator` is reworked: + * Complete fairness is maintained for single-threaded dispatchers. + * Its performance is improved, depending on the use-case, by at least 50% (#2296). + * Quadratic complexity depending on the number of upstream flows is eliminated (#2296). + * `crossinline` and `inline`-heavy internals are removed, fixing sporadic SIGSEGV on Mediatek Android devices (#1683, #1743). +* `Flow.zip` operator performance is improved by 40%. +* Various API has been promoted to stable or its deprecation level has been raised (#2316). + +### Bug fixes + +* Suspendable `stateIn` operator propagates exception to the caller when upstream fails to produce initial value (#2329). +* Fix `SharedFlow` with replay for subscribers working at different speed (#2325). +* Do not fail debug agent installation when security manager does not provide access to system properties (#2311). +* Cancelled lazy coroutines are properly cleaned up from debug agent output (#2294). +* `BlockHound` false-positives are correctly filtered out (#2302, #2190, #2303). +* Potential crash during a race between cancellation and upstream in `Observable.asFlow` is fixed (#2104, #2299, thanks to @LouisCAD and @drinkthestars). + +## Version 1.4.0-M1 + +### Breaking changes + +* The concept of atomic cancellation in channels is removed. All operations in channels + and corresponding `Flow` operators are cancellable in non-atomic way (#1813). +* If `CoroutineDispatcher` throws `RejectedExecutionException`, cancel current `Job` and schedule its execution to `Dispatchers.IO` (#2003). +* `CancellableContinuation.invokeOnCancellation` is invoked if the continuation was cancelled while its resume has been dispatched (#1915). +* `Flow.singleOrNull` operator is aligned with standard library and does not longer throw `IllegalStateException` on multiple values (#2289). + +### New experimental features + +* `SharedFlow` primitive for managing hot sources of events with support of various subscription mechanisms, replay logs and buffering (#2034). +* `Flow.shareIn` and `Flow.stateIn` operators to transform cold instances of flow to hot `SharedFlow` and `StateFlow` respectively (#2047). + +### Other + +* Support leak-free closeable resources transfer via `onUndeliveredElement` in channels (#1936). +* Changed ABI in reactive integrations for Java interoperability (#2182). +* Fixed ProGuard rules for `kotlinx-coroutines-core` (#2046, #2266). +* Lint settings were added to `Flow` to avoid accidental capturing of outer `CoroutineScope` for cancellation check (#2038). + +### External contributions + +* Allow nullable types in `Flow.firstOrNull` and `Flow.singleOrNull` by @ansman (#2229). +* Add `Publisher.awaitSingleOrDefault|Null|Else` extensions by @sdeleuze (#1993). +* `awaitCancellation` top-level function by @LouisCAD (#2213). +* Significant part of our Gradle build scripts were migrated to `.kts` by @turansky. + +Thank you for your contributions and participation in the Kotlin community! + +## Version 1.3.9 + +* Support of `CoroutineContext` in `Flow.asPublisher` and similar reactive builders (#2155). +* Kotlin updated to 1.4.0. +* Transition to new HMPP publication scheme for multiplatform usages: + * Artifacts `kotlinx-coroutines-core-common` and `kotlinx-coroutines-core-native` are removed. + * For multiplatform usages, it's enough to [depend directly](README.md#multiplatform) on `kotlinx-coroutines-core` in `commonMain` source-set. + * The same artifact coordinates can be used to depend on platform-specific artifact in platform-specific source-set. + +## Version 1.3.8 + +### New experimental features + +* Added `Flow.transformWhile operator` (#2065). +* Replaced `scanReduce` with `runningReduce` to be consistent with the Kotlin standard library (#2139). + +### Bug fixes and improvements + +* Improve user experience for the upcoming coroutines debugger (#2093, #2118, #2131). +* Debugger no longer retains strong references to the running coroutines (#2129). +* Fixed race in `Flow.asPublisher` (#2109). +* Fixed `ensureActive` to work in the empty context case to fix `IllegalStateException` when using flow from `suspend fun main` (#2044). +* Fixed a problem with `AbortFlowException` in the `Flow.first` operator to avoid erroneous `NoSuchElementException` (#2051). +* Fixed JVM dependency on Android annotations (#2075). +* Removed keep rules mentioning `kotlinx.coroutines.android` from core module (#2061 by @mkj-gram). +* Corrected some docs and examples (#2062, #2071, #2076, #2107, #2098, #2127, #2078, #2135). +* Improved the docs and guide on flow cancellation (#2043). +* Updated Gradle version to `6.3` (it only affects multiplatform artifacts in this release). + +## Version 1.3.7 + +* Fixed problem that triggered Android Lint failure (#2004). +* New `Flow.cancellable()` operator for cooperative cancellation (#2026). +* Emissions from `flow` builder now check cancellation status and are properly cancellable (#2026). +* New `currentCoroutineContext` function to use unambiguously in the contexts with `CoroutineScope` in receiver position (#2026). +* `EXACTLY_ONCE` contract support in coroutine builders. +* Various documentation improvements. + +## Version 1.3.6 + +### Flow + +* `StateFlow`, new primitive for state handling (#1973, #1816, #395). The `StateFlow` is designed to eventually replace `ConflatedBroadcastChannel` for state publication scenarios. Please, try it and share your feedback. Note, that Flow-based primitives to publish events will be added later. For events you should continue to either use `BroadcastChannel(1)`, if you put events into the `StateFlow`, protect them from double-processing with flags. +* `Flow.onEmpty` operator is introduced (#1890). +* Behavioural change in `Flow.onCompletion`, it is aligned with `invokeOnCompletion` now and passes `CancellationException` to its cause parameter (#1693). +* A lot of Flow operators have left its experimental status and are promoted to stable API. + +### Other + +* `runInterruptible` primitive to tie cancellation with thread interruption for blocking calls. Contributed by @jxdabc (#1947). +* Integration module with RxJava3 is introduced. Contributed by @ZacSweers (#1883) +* Integration with [BlockHound](https://github.com/reactor/BlockHound) in `kotlinx-coroutines-debug` module (#1821, #1060). +* Memory leak in ArrayBroadcastChannel is fixed (#1885). +* Behavioural change in `suspendCancellableCoroutine`, cancellation is established before invoking passed block argument (#1671). +* Debug agent internals are moved into `kotlinx-coroutines-core` for better integration with IDEA. It should not affect library users and all the redundant code should be properly eliminated with R8. +* ClassCastException with reusable continuations bug is fixed (#1966). +* More precise scheduler detection for `Executor.asCoroutineDispatcher` (#1992). +* Kotlin updated to 1.3.71. + +## Version 1.3.5 + +* `firstOrNull` operator. Contributed by @bradynpoulsen. +* `java.time` adapters for Flow operators. Contributed by @fvasco. +* `kotlin.time.Duration` support (#1402). Contributed by @fvasco. +* Memory leak with a mix of reusable and non-reusable continuations is fixed (#1855). +* `DebugProbes` are ready for production installation: its performance is increased, the flag to disable creation stacktraces to reduce the footprint is introduced (#1379, #1372). +* Stacktrace recovery workaround for Android 6.0 and earlier bug (#1866). +* New integration module: `kotlinx-coroutines-jdk9` with adapters for `java.util.concurrent.Flow`. +* `BroadcastChannel.close` properly starts lazy coroutine (#1713). +* `kotlinx-coroutines-bom` is published without Gradle metadata. +* Make calls to service loader in reactor integrations optimizable by R8 (#1817). + +## Version 1.3.4 + +### Flow + +* Detect missing `awaitClose` calls in `callbackFlow` to make it less error-prone when used with callbacks (#1762, #1770). This change makes `callbackFlow` **different** from `channelFlow`. +* `ReceiveChannel.asFlow` extension is introduced (#1490). +* Enforce exception transparency invariant in `flow` builder (#1657). +* Proper `Dispatcher` support in `Flow` reactive integrations (#1765). +* Batch `Subscription.request` calls in `Flow` reactive integration (#766). +* `ObservableValue.asFlow` added to JavaFx integration module (#1695). +* `ObservableSource.asFlow` added to RxJava2 integration module (#1768). + +### Other changes + +* `kotlinx-coroutines-core` is optimized for R8, making it much smaller for Android usages (75 KB for `1.3.4` release). +* Performance of `Dispatchers.Default` is improved (#1704, #1706). +* Kotlin is updated to 1.3.70. +* `CoroutineDispatcher` and `ExecutorCoroutineDispatcher` experimental coroutine context keys are introduced (#1805). +* Performance of various `Channel` operations is improved (#1565). + +## Version 1.3.3 + +### Flow +* `Flow.take` performance is significantly improved (#1538). +* `Flow.merge` operator (#1491). +* Reactive Flow adapters are promoted to stable API (#1549). +* Reusable cancellable continuations were introduced that improved the performance of various flow operators and iteration over channels (#1534). +* Fixed interaction of multiple flows with `take` operator (#1610). +* Throw `NoSuchElementException` instead of `UnsupportedOperationException` for empty `Flow` in `reduce` operator (#1659). +* `onCompletion` now rethrows downstream exceptions on emit attempt (#1654). +* Allow non-emitting `withContext` from `flow` builder (#1616). + +### Debugging + +* `DebugProbes.dumpCoroutines` is optimized to be able to print the 6-digit number of coroutines (#1535). +* Properly capture unstarted lazy coroutines in debugger (#1544). +* Capture coroutines launched from within a test constructor with `CoroutinesTimeout` test rule (#1542). +* Stacktraces of `Job`-related coroutine machinery are shortened and prettified (#1574). +* Stacktrace recovery unification that should provide a consistent experience recover of stacktrace (#1597). +* Stacktrace recovery for `withTimeout` is supported (#1625). +* Do not recover exception with a single `String` parameter constructor that is not a `message` (#1631). + +### Other features + +* `Dispatchers.Default` and `Dispatchers.IO` rework: CPU consumption is significantly lower, predictable idle threads termination (#840, #1046, #1286). +* Avoid `ServiceLoader` for loading `Dispatchers.Main` (#1572, #1557, #878, #1606). +* Consistently handle undeliverable exceptions in RxJava and Reactor integrations (#252, #1614). +* `yield` support in immediate dispatchers (#1474). +* `CompletableDeferred.completeWith(result: Result)` is introduced. +* Added support for tvOS and watchOS-based Native targets (#1596). + +### Bug fixes and improvements + +* Kotlin version is updated to 1.3.61. +* `CoroutineDispatcher.isDispatchNeeded` is promoted to stable API (#1014). +* Livelock and stackoverflows in mutual `select` expressions are fixed (#1411, #504). +* Properly handle `null` values in `ListenableFuture` integration (#1510). +* Making ReceiveChannel.cancel linearizability-friendly. +* Linearizability of Channel.close in a complex contended cases (#1419). +* ArrayChannel.isBufferEmpty atomicity is fixed (#1526). +* Various documentation improvements. +* Reduced bytecode size of `kotlinx-coroutines-core`, reduced size of minified `dex` when using basic functionality of `kotlinx-coroutines`. + +## Version 1.3.2 + +This is a maintenance release that does not include any new features or bug fixes. + +* Reactive integrations for `Flow` are promoted to stable API. +* Obsolete reactive API is deprecated. +* Deprecation level for API deprecated in 1.3.0 is increased. +* Various documentation improvements. + +## Version 1.3.1 + +This is a minor update with various fixes: +* Flow: Fix recursion in combineTransform (#1466). +* Fixed race in the Semaphore (#1477). +* Repaired some of ListenableFuture.kt's cancellation corner cases (#1441). +* Consistently unwrap exception in slow path of CompletionStage.asDeferred (#1479). +* Various fixes in documentation (#1496, #1476, #1470, #1468). +* Various cleanups and additions in tests. + +Note: Kotlin/Native artifacts are now published with Gradle metadata format version 1.0, so you will need +Gradle version 5.3 or later to use this version of kotlinx.coroutines in your Kotlin/Native project. + +## Version 1.3.0 + +### Flow + +This version is the first stable release with [`Flow`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html) API. + +All `Flow` API not marked with `@FlowPreview` or `@ExperimentalCoroutinesApi` annotations are stable and here to stay. +Flow declarations marked with `@ExperimentalCoroutinesApi` have [the same guarantees](/docs/topics/compatibility.md#experimental-api) as regular experimental API. +Please note that API marked with `@FlowPreview` have [weak guarantees](/docs/topics/compatibility.md#flow-preview-api) on source, binary and semantic compatibility. + +### Changelog + +* A new [guide section](/docs/topics/flow.md) about Flow. +* `CoroutineDispatcher.asExecutor` extension (#1450). +* Fixed bug when `select` statement could report the same exception twice (#1433). +* Fixed context preservation in `flatMapMerge` in a case when collected values were immediately emitted to another flow (#1440). +* Reactive Flow integrations enclosing files are renamed for better interoperability with Java. +* Default buffer size in all Flow operators is increased to 64. +* Kotlin updated to 1.3.50. + +## Version 1.3.0-RC2 + +### Flow improvements +* Operators for UI programming are reworked for the sake of consistency, naming scheme for operator overloads is introduced: + * `combineLatest` is deprecated in the favor of `combine`. + * `combineTransform` operator for non-trivial transformations (#1224). + * Top-level `combine` and `combineTransform` overloads for multiple flows (#1262). + * `switchMap` is deprecated. `flatMapLatest`, `mapLatest` and `transformLatest` are introduced instead (#1335). + * `collectLatest` terminal operator (#1269). + +* Improved cancellation support in `flattenMerge` (#1392). +* `channelFlow` cancellation does not leak to the parent (#1334). +* Fixed flow invariant enforcement for `suspend fun main` (#1421). +* `delayEach` and `delayFlow` are deprecated (#1429). + +### General changes +* Integration with Reactor context + * Propagation of the coroutine context of `await` calls into Mono/Flux builder. + * Publisher.asFlow propagates coroutine context from `collect` call to the Publisher. + * New `Flow.asFlux ` builder. + +* ServiceLoader-code is adjusted to avoid I/O on the Main thread on newer (3.6.0+) Android toolchain. +* Stacktrace recovery support for minified builds on Android (#1416). +* Guava version in `kotlinx-coroutines-guava` updated to `28.0`. +* `setTimeout`-based JS dispatcher for platforms where `process` is unavailable (#1404). +* Native, JS and common modules are added to `kotlinx-coroutines-bom`. +* Fixed bug with ignored `acquiredPermits` in `Semaphore` (#1423). + +## Version 1.3.0-RC + +### Flow + +* Core `Flow` API is promoted to stable +* New basic `Flow` operators: `withIndex`, `collectIndexed`, `distinctUntilChanged` overload +* New core `Flow` operators: `onStart` and `onCompletion` +* `ReceiveChannel.consumeAsFlow` and `emitAll` (#1340) + +### General changes + +* Kotlin updated to 1.3.41 +* Added `kotlinx-coroutines-bom` with Maven Bill of Materials (#1110) +* Reactive integrations are seriously improved + * All builders now are top-level functions instead of extensions on `CoroutineScope` and prohibit `Job` instance in their context to simplify lifecycle management + * Fatal exceptions are handled consistently (#1297) + * Integration with Reactor Context added (#284) +* Stacktrace recovery for `suspend fun main` (#1328) +* `CoroutineScope.cancel` extension with message (#1338) +* Protection against non-monotonic clocks in `delay` (#1312) +* `Duration.ZERO` is handled properly in JDK 8 extensions (#1349) +* Library code is adjusted to be more minification-friendly + +## Version 1.3.0-M2 + + * Kotlin updated to 1.3.40. + * `Flow` exception transparency concept. + * New declarative `Flow` operators: `onCompletion`, `catch`, `retryWhen`, `launchIn`. `onError*` operators are deprecated in favour of `catch`. (#1263) + * `Publisher.asFlow` is integrated with `buffer` operator. + * `Publisher.openSubscription` default request size is `1` instead of `0` (#1267). + +## Version 1.3.0-M1 + +Flow: + * Core `Flow` interfaces and operators are graduated from preview status to experimental. + * Context preservation invariant rework (#1210). + * `channelFlow` and `callbackFlow` replacements for `flowViaChannel` for concurrent flows or callback-based APIs. + * `flow` prohibits emissions from non-scoped coroutines by default and recommends to use `channelFlow` instead to avoid most of the concurrency-related bugs. + * Flow cannot be implemented directly + * `AbstractFlow` is introduced for extension (e.g. for managing state) and ensures all context preservation invariants. + * Buffer size is decoupled from all operators that imply channel usage (#1233) + * `buffer` operator can be used to adjust buffer size of any buffer-dependent operator (e.g. `channelFlow`, `flowOn` and `flatMapMerge`). + * `conflate` operator is introduced. + * Flow performance is significantly improved. + * New operators: `scan`, `scanReduce`, `first`, `emitAll`. + * `flowWith` and `flowViaChannel` are deprecated. + * `retry` ignores cancellation exceptions from upstream when the flow was externally cancelled (#1122). + * `combineLatest` overloads for multiple flows (#1193). + * Fixed numerical overflow in `drop` operator. + +Channels: + * `consumeEach` is promoted to experimental API (#1080). + * Conflated channels always deliver the latest value after closing (#332, #1235). + * Non-suspending `ChannelIterator.next` to improve iteration performance (#1162). + * Channel exception types are consistent with `produce` and are no longer swallowed as cancellation exceptions in case of programmatic errors (#957, #1128). + * All operators on channels (that were prone to coroutine leaks) are deprecated in the favor of `Flow`. + +General changes: + * Kotlin updated to 1.3.31 + * `Semaphore` implementation (#1088) + * Loading of `Dispatchers.Main` is tweaked so the latest version of R8 can completely remove I/O when loading it (#1231). + * Performace of all JS dispatchers is significantly improved (#820). + * `withContext` checks cancellation status on exit to make reasoning about sequential concurrent code easier (#1177). + * Consistent exception handling mechanism for complex hierarchies (#689). + * Convenient overload for `CoroutinesTimeout.seconds` (#1184). + * Fix cancellation bug in onJoin (#1130). + * Prevent internal names clash that caused errors for ProGuard (#1159). + * POSIX's `nanosleep` as `delay` in `runBlocking ` in K/N (#1225). + +## Version 1.2.2 + +* Kotlin updated to 1.3.40. + +## Version 1.2.1 + +Major: + * Infrastructure for testing coroutine-specific code in `kotlinx-coroutines-test`: `runBlockingTest`, `TestCoroutineScope` and `TestCoroutineDispatcher`, contributed by Sean McQuillan (@objcode). Obsolete `TestCoroutineContext` from `kotlinx-coroutines-core` is deprecated. + * `Job.asCompletableFuture` extension in jdk8 module (#1113). + +Flow improvements: + * `flowViaChannel` rework: block parameter is no longer suspending, but provides `CoroutineScope` receiver and allows conflated channel (#1081, #1112). + * New operators: `switchMap`, `sample`, `debounce` (#1107). + * `consumerEach` is deprecated on `Publisher`, `ObservableSource` and `MaybeSource`, `collect` extension is introduced instead (#1080). + +Other: + * Race in Job.join and concurrent cancellation is fixed (#1123). + * Stacktrace recovery machinery improved: cycle detection works through recovered exceptions, cancellation exceptions are recovered on cancellation fast-path. + * Atomicfu-related bug fixes: publish transformed artifacts, do not propagate transitive atomicfu dependency (#1064, #1116). + * Publication to NPM fixed (#1118). + * Misplaced resources are removed from the final jar (#1131). + +## Version 1.2.0 + + * Kotlin updated to 1.3.30. + * New API: `CancellableContinuation.resume` with `onCancelling` lambda (#1044) to consistently handle closeable resources. + * Play services task version updated to 16.0.1. + * `ReceiveChannel.isEmpty` is no longer deprecated + +A lot of `Flow` improvements: + * Purity property is renamed to context preservation and became more restrictive. + * `zip` and `combineLatest` operators. + * Integration with RxJava2 + * `flatMap`, `merge` and `concatenate` are replaced with `flattenConcat`, `flattenMerge`, `flatMapConcat` and `flatMapMerge`. + * Various documentation improvements and minor bug fixes. + +Note that `Flow` **is not** leaving its [preview status](/docs/topics/compatibility.md#flow-preview-api). + +## Version 1.2.0-alpha-2 + +This release contains major [feature preview](/docs/topics/compatibility.md#flow-preview-api): cold streams aka `Flow` (#254). + +Performance: +* Performance of `Dispatcher.Main` initialization is significantly improved (#878). + +## Version 1.2.0-alpha + +* Major debug agent improvements. Real stacktraces are merged with coroutine stacktraces for running coroutines, merging heuristic is improved, API is cleaned up and is on its road to stabilization (#997). +* `CoroutineTimeout` rule or JUnit4 is introduced to simplify coroutines debugging (#938). +* Stacktrace recovery improvements. Exceptions with custom properties are no longer copied, `CopyableThrowable` interface is introduced, machinery is [documented](https://github.com/Kotlin/kotlinx.coroutines/blob/develop/docs/debugging.md) (#921, #950). +* `Dispatchers.Unconfined`, `MainCoroutineDispatcher.immediate`, `MainScope` and `CoroutineScope.cancel` are promoted to stable API (#972). +* `CompletableJob` is introduced (#971). +* Structured concurrency is integrated into futures and listenable futures (#1008). +* `ensurePresent` and `isPresent` extensions for `ThreadLocal` (#1028). +* `ensureActive` extensions for `CoroutineContext`, `CoroutineScope` and `Job` (#963). +* `SendChannel.isFull` and `ReceiveChannel.isEmpty` are deprecated (#1053). +* `withContext` checks cancellation on entering (#962). +* Operator `invoke` on `CoroutineDispatcher` (#428). +* Java 8 extensions for `delay` and `withTimeout` now properly handle too large values (#428). +* A global exception handler for fatal exceptions in coroutines is introduced (#808, #773). +* Major improvements in cancellation machinery and exceptions delivery consistency. Cancel with custom exception is completely removed. +* Kotlin version is updated to 1.3.21. +* Do not use private API on newer Androids to handle exceptions (#822). + +Bug fixes: +* Proper `select` support in debug agent (#931). +* Proper `supervisorScope` support in debug agent (#915). +* Throwing `initCause` does no longer trigger an internal error (#933). +* Lazy actors are started when calling `close` in order to cleanup their resources (#939). +* Minor bugs in reactive integrations are fixed (#1008). +* Experimental scheduler shutdown sequence is fixed (#990). + +## Version 1.1.1 + +* Maintenance release, no changes in the codebase +* Kotlin is updated to 1.3.20 +* Gradle is updated to 4.10 +* Native module is published with Gradle metadata v0.4 + +## Version 1.1.0 + +* Kotlin version updated to 1.3.11. +* Resumes to `CancellableContinuation` in the final state produce `IllegalStateException` (#901). This change does not affect #830, races between resume and cancellation do not lead to an exceptional situation. +* `runBlocking` is integrated with `Dispatchers.Unconfined` by sharing an internal event loop. This change does not affect the semantics of the previously correct code but allows to mix multiple `runBlocking` and unconfined tasks (#860). + +## Version 1.1.0-alpha + +### Major improvements in coroutines testing and debugging +* New module: [`kotlinx-coroutines-debug`](https://github.com/Kotlin/kotlinx.coroutines/blob/master/core/kotlinx-coroutines-debug/README.md). Debug agent that improves coroutines stacktraces, allows to print all active coroutines and its hierarchies and can be installed as Java agent. +* New module: [`kotlinx-coroutines-test`](https://github.com/Kotlin/kotlinx.coroutines/blob/master/core/kotlinx-coroutines-test/README.md). Allows setting arbitrary `Dispatchers.Main` implementation for tests (#810). +* Stacktrace recovery mechanism. Exceptions from coroutines are recovered from current coroutine stacktraces to simplify exception diagnostic. Enabled in debug mode, controlled by `kotlinx.coroutines.debug` system property (#493). + +### Other improvements +* `MainScope` factory and `CoroutineScope.cancel` extension (#829). One line `CoroutineScope` integration! +* `CancellableContinuation` race between `resumeWithException` and `cancel` is addressed, exceptions during cancellation are no longer reported to exception handler (#830, #892). +* `Dispatchers.Default` now consumes much less CPU on JVM (#840). +* Better diagnostic and fast failure if an uninitialized dispatcher is used (#880). +* Conflated channel becomes linearizable. +* Fixed inconsistent coroutines state when the result of the coroutine had type `DisposableHandle` (#835). +* Fixed `JavaFx` initialization bug (#816). +* `TimeoutCancellationException` is thrown by `withTimeout` instead of `CancellationException` if negative timeout is supplied (#870). +* Kotlin/Native single-threaded workers support: coroutines can be safely used in multiple independent K/N workers. +* jsdom support in `Dispatchers.Default` on JS. +* rxFlowable generic parameter is now restricted with Any. +* Guava 27 support in `kotlinx-coroutines-guava`. +* Coroutines are now built with progressive mode. +* Various fixes in the documentation. + +## Version 1.0.1 + +* Align `publisher` implementation with Reactive TCK. +* Reimplement `future` coroutine builders on top of `AbstractCoroutine` (#751). +* Performance optimizations in `Dispatchers.Default` and `Dispatchers.IO`. +* Use only public API during `JavaFx` instantiation, fixes warnings on Java 9 and build on Java 11 (#463). +* Updated contract of `CancellableContinuation.resumeWithException` (documentation fix, see #712). +* Check cancellation on fast-path of all in-place coroutine builders (`withContext`, `coroutineScope`, `supervisorScope`, `withTimeout` and `withTimeoutOrNull`). +* Add optional prefix to thread names of `ExperimentalCoroutineDispatcher` (#661). +* Fixed bug when `ExperimentalCoroutineDispatcher` could end up in inconsistent state if `Thread` constructor throws an exception (#748). + +## Version 1.0.0 + +* All Kotlin dependencies updated to 1.3 release version. +* Fixed potential memory leak in `HandlerDispatcher.scheduleResumeAfterDelay`, thanks @cbeyls. +* `yield` support for `Unconfined` and immediate dispatchers (#737). +* Various documentation improvements. + +## Version 1.0.0-RC1 + +* Coroutines API is updated to Kotlin 1.3. +* Deprecated API is removed or marked as `internal`. +* Experimental and internal coroutine API is marked with corresponding `kotlin.experimental.Experimental` annotation. If you are using `@ExperimentalCoroutinesApi` or `@InternalCoroutinesApi` you should explicitly opt-in, otherwise compilation warning (or error) will be produced. +* `Unconfined` dispatcher (and all dispatchers which support immediate invocation) forms event-loop on top of current thread, thus preventing all `StackOverflowError`s. `Unconfined` dispatcher is now much safer for the general use and may leave its experimental status soon (#704). +* Significantly improved performance of suspending hot loops in `kotlinx.coroutines` (#537). +* Proguard rules are embedded into coroutines JAR to assist jettifier (#657) +* Fixed bug in shutdown sequence of `runBlocking` (#692). +* `ReceiveChannel.receiveOrNull` is marked as obsolete and deprecated. +* `Job.cancel(cause)` and `ReceiveChannel.cancel(cause)` are deprecated, `cancel()` returns `Unit` (#713). + +## Version 0.30.2 + +* `Dispatchers.Main` is instantiated lazily (see #658 and #665). +* Blocking coroutine dispatcher views are now shutdown properly (#678). +* Prevent leaking Kotlin 1.3 from atomicfu dependency (#659). +* Thread-pool based dispatcher factories are marked as obsolete (#261). +* Fixed exception loss on `withContext` cancellation (#675). + +## Version 0.30.1 + +Maintenance release: +* Added `Dispatchers.Main` to common dispatchers, which can be used from Android, Swing and JavaFx projects if a corresponding integration library is added to dependencies. +* With `Dispatchers.Main` improvement tooling bug in Android Studio #626 is mitigated, so Android users now can safely start the migration to the latest `kotlinx.coroutines` version. +* Fixed bug with thread unsafety of shutdown sequence in `EventLoop`. +* Experimental coroutine dispatcher now has `close` contract similar to Java `Executor`, so it can be safely instantiated and closed multiple times (affects only unit tests). +* Atomicfu version is updated with fixes in JS transformer (see #609) + +## Version 0.30.0 + +* **[Major]** Further improvements in exception handling — no failure exception is lost. + * `async` and async-like builders cancel parent on failure (it affects `CompletableDeferred`, and all reactive integration builders). + * This makes parallel decomposition exception-safe and reliable without having to rember about `awaitAll` (see #552). + * `Job()` wih parent now also cancels parent on failure consistently with other scopes. + * All coroutine builders and `Job` implementations propagate failure to the parent unless it is a `CancellationException`. + * Note, "scoping" builders don't "cancel the parent" verbatim, but rethrow the corresponding exception to the caller for handling. + * `SupervisorJob()` and `supervisorScope { ... }` are introduced, allowing for a flexible implementation of custom exception-handling policies, see a [new section in the guide on supervision](docs/topics/exception-handling.md#supervision). + * Got rid of `awaitAll` in documentation and rewrote `currentScope` section (see #624). +* **[Major]** Coroutine scheduler is used for `Dispatchers.Default` by default instead of deprecated `CommonPool`. + * "`DefaultDispatcher`" is used as a public name of the default impl (you'll see it thread names and in the guide). + * `-Dkotlinx.coroutines.scheduler=off` can be used to switch back to `CommonPool` for a time being (until deprecated CommonPool is removed). +* Make `CoroutineStart.ATOMIC` experimental as it covers important use-case with resource cleanup in finally block (see #627). +* Restored binary compatibility of `Executor.asCoroutineDispatcher` (see #629). +* Fixed OOM in thread-pool dispatchers (see #571). +* Check for cancellation when starting coroutine with `Dispatchers.Unconfined` (see #621). +* A bunch of various performance optimizations and docs fixes, including contributions from @AlexanderPrendota, @PaulWoitaschek. + +## Version 0.27.0 + +* **[Major]** Public API revision. All public API was reviewed and marked as preparation to `1.0` release: + 1. `@Deprecated` API. All API marked as deprecated will be removed in 1.0 release without replacement. + 2. `@ExperimentalCoroutinesApi` API. This API is experimental and may change in the future, but migration mechanisms will be provided. Signature, binary compatibility and semantics can be changed. + 3. `@InternalCoroutinesApi`. This API is intended to be used **only** from within `kotlinx.coroutines`. It can and will be changed, broken + and removed in the future releases without any warnings and migration aids. If you find yourself using this API, it is better to report + your use-case to Github issues, so decent, stable and well-tested alternative can be provided. + 4. `@ObsoleteCoroutinesApi`. This API has serious known flaws and will be replaced with a better alternative in the nearest releases. + 5. Regular public API. This API is proven to be stable and is not going to be changed. If at some point it will be discovered that such API + has unfixable design flaws, it will be gradually deprecated with proper replacement and migration aid, but won't be removed for at least a year. +* **[Major]** Job state machine is reworked. It includes various performance improvements, fixes in +data-races which could appear in a rare circumstances and consolidation of cancellation and exception handling. +Visible consequences of include more robust exception handling for large coroutines hierarchies and for different kinds of `CancellationException`, transparent parallel decomposition and consistent view of coroutines hierarchy in terms of its state (see #220 and #585). +* NIO, Quasar and Rx1 integration modules are removed with no replacement (see #595, #601, #603). +* `withContext` is now aligned with structured concurrency and awaits for all launched tasks, its performance is significantly improved (see #553 and #617). +* Added integration module with Play Services Task API. Thanks @SUPERCILEX and @lucasvalenteds for the contribution! +* Integration with Rx2 now respects nullability in type constraints (see #347). Thanks @Dmitry-Borodin for the contribution! +* `CompletableFuture.await` and `ListenableFuture.await` now propagate cancellation to the future (see #611). +* Cancellation of `runBlocking` machinery is improved (see #589). +* Coroutine guide is restructured and split to multiple files for the sake of simplicity. +* `CoroutineScope` factory methods add `Job` if it is missing from the context to enforce structured concurrency (see #610). +* `Handler.asCoroutineDispatcher` has a `name` parameter for better debugging (see #615). +* Fixed bug when `CoroutineSchedule` was closed from one of its threads (see #612). +* Exceptions from `CoroutineExceptionHandler` are reported by default exception handler (see #562). +* `CoroutineName` is now available from common modules (see #570). +* Update to Kotlin 1.2.70. + +## Version 0.26.1 + +* Android `Main` dispatcher is `async` by default which may significantly improve UI performance. Contributed by @JakeWharton (see #427). +* Fixed bug when lazily-started coroutine with registered cancellation handler was concurrently started and cancelled. +* Improved termination sequence in IO dispatcher. +* Fixed bug with `CoroutineScope.plus` operator (see #559). +* Various fixes in the documentation. Thanks to @SUPERCILEX, @yorlov, @dualscyther and @soudmaijer! + +## Version 0.26.0 + +* Major rework of `kotlinx.coroutines` concurrency model (see #410 for a full explanation of the rationale behind this change): + * All coroutine builders are now extensions on `CoroutineScope` and inherit its `coroutineContext`. Standalone builders are deprecated. + * As a consequence, all nested coroutines launched via builders now automatically establish parent-child relationship and inherit `CoroutineDispatcher`. + * All coroutine builders use `Dispatchers.Default` by default if `CoroutineInterceptor` is not present in their context. + * [CoroutineScope](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html) became the first-class citizen in `kolinx.coroutines`. + * `withContext` `block` argument has `CoroutineScope` as a receiver. + * [GlobalScope](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-global-scope/index.html) is introduced to simplify migration to new API and to launch global-level coroutines. + * `currentScope` and `coroutineScope` builders are introduced to extract and provide `CoroutineScope`. + * Factory methods to create `CoroutineScope` from `CoroutineContext` are introduced. + * `CoroutineScope.isActive` became an extension property. + * New sections about structured concurrency in core guide: ["Structured concurrency"](docs/topics/coroutines-guide.md#structured-concurrency), ["Scope builder"](docs/topics/coroutines-guide.md#scope-builder) and ["Structured concurrency with async"](docs/topics/coroutines-guide.md#structured-concurrency-with-async). + * New section in UI guide with Android example: ["Structured concurrency, lifecycle and coroutine parent-child hierarchy"](ui/coroutines-guide-ui.md#structured-concurrency,-lifecycle-and-coroutine-parent-child-hierarchy). + * Deprecated reactive API is removed. +* Dispatchers are renamed and grouped in the Dispatchers object (see #41 and #533): + * Dispatcher names are consistent. + * Old dispatchers including `CommonPool` are deprecated. +* Fixed bug with JS error in rare cases in `invokeOnCompletion(onCancelling = true)`. +* Fixed loading of Android exception handler when `Thread.contextClassLoader` is mocked (see #530). +* Fixed bug when `IO` dispatcher silently hung (see #524 and #525) . + +## Version 0.25.3 + +* Distribution no longer uses multi-version jar which is not supported on Android (see #510). +* JS version of the library does not depend on AtomicFu anymore: +  All the atomic boxes in JS are fully erased. +* Note that versions 0.25.1-2 are skipped for technical reasons (they were not fully released). + +## Version 0.25.0 + +* Major rework on exception-handling and cancellation in coroutines (see #333, #452 and #451): + * New ["Exception Handling" section in the guide](docs/topics/coroutines-guide.md#exception-handling) explains exceptions in coroutines. + * Semantics of `Job.cancel` resulting `Boolean` value changed — `true` means exception was handled by the job, caller shall handle otherwise. + * Exceptions are properly propagated from children to parents. + * Installed `CoroutineExceptionHandler` for a family of coroutines receives one aggregated exception in case of failure. + * Change `handleCoroutineException` contract, so custom exception handlers can't break coroutines machinery. + * Unwrap `JobCancellationException` properly to provide exception transparency over whole call chain. +* Introduced support for thread-local elements in coroutines context (see #119): + * `ThreadContextElement` API for custom thread-context sensitive context elements. + * `ThreadLocal.asContextElement()` extension function to convert an arbitrary thread-local into coroutine context element. + * New ["Thread-local data" subsection in the guide](docs/topics/coroutines-guide.md#thread-local-data) with examples. + * SLF4J Mapped Diagnostic Context (MDC) integration is provided via `MDCContext` element defined in [`kotlinx-coroutines-slf4j`](integration/kotlinx-coroutines-slf4j/README.md) integration module. +* Introduced IO dispatcher to offload blocking I/O-intensive tasks (see #79). +* Introduced `ExecutorCoroutineDispatcher` instead of `CloseableCoroutineDispatcher` (see #385). +* Built with Kotlin 1.2.61 and Kotlin/Native 0.8.2. +* JAR files for `kotlinx-coroutines` are now [JEP 238](https://openjdk.java.net/jeps/238) multi-release JAR files. + * On JDK9+ `VarHandle` is used for atomic operations instead of `Atomic*FieldUpdater` for better performance. + * See [AtomicFu](https://github.com/Kotlin/kotlinx.atomicfu/blob/master/README.md) project for details. +* Reversed addition of `BlockingChecker` extension point to control where `runBlocking` can be used (see #227). + * `runBlocking` can be used anywhere without limitations (again), but it would still cause problems if improperly used on UI thread. +* Corrected return-type of `EventLoop` pseudo-constructor (see #477, PR by @Groostav). +* Fixed `as*Future()` integration functions to catch all `Throwable` exceptions (see #469). +* Fixed `runBlocking` cancellation (see #501). +* Fixed races and timing bugs in `withTimeoutOrNull` (see #498). +* Execute `EventLoop.invokeOnTimeout` in `DefaultDispatcher` to allow busy-wait loops inside `runBlocking` (see #479). +* Removed `kotlinx-coroutines-io` module from the project, it has moved to [kotlinx-io](https://github.com/kotlin/kotlinx-io/). +* Provide experimental API to create limited view of experimental dispatcher (see #475). +* Various minor fixes by @LouisCAD, @Dmitry-Borodin. + +## Version 0.24.0 + +* Fully multiplatform release with Kotlin/Native support (see #246): + * Only single-threaded operation inside `runBlocking` event loop is supported at this moment. + * See details on setting up build environment [here](native/README.md). +* Improved channels: + * Introduced `SendChannel.invokeOnClose` (see #341). + * Make `close`, `cancel`, `isClosedForSend`, `isClosedForReceive` and `offer` linearizable with other operations (see #359). + * Fixed bug when send operation can be stuck in channel forever. + * Fixed broadcast channels on JS (see #412). +* Provides `BlockingChecker` mechanism which checks current context (see #227). + * Attempts to use `runBlocking` from any supported UI thread (Android, JavaFx, Swing) will result in exception. +* Android: + * Worked around Android bugs with zero-size ForkJoinPool initialization (see #432, #288). + * Introduced `UI.immediate` extension as performance-optimization to immediately execute tasks which are invoked from the UI thread (see #381). + * Use it only when absolutely needed. It breaks asynchrony of coroutines and may lead to surprising and unexpected results. +* Fixed materialization of a `cause` exception for `Job` onCancelling handlers (see #436). +* Fixed JavaFx `UI` on Java 9 (see #443). +* Fixed and documented the order between cancellation handlers and continuation resume (see #415). +* Fixed resumption of cancelled continuation (see #450). +* Includes multiple fixes to documentation contributed by @paolop, @SahilLone, @rocketraman, @bdavisx, @mtopolnik, @Groostav. +* Experimental coroutines scheduler preview (JVM only): + * Written from scratch and optimized for communicating coroutines. + * Performs significantly better than ForkJoinPool on coroutine benchmarks and for connected applications with [ktor](https://ktor.io). + * Supports automatic creating of new threads for blocking operations running on the same thread pool (with an eye on solving #79), but there is no stable public API for it just yet. + * For preview, run JVM with `-Dkotlinx.coroutines.scheduler` option. In this case `DefaultDispatcher` is set to new experimental scheduler instead of FJP-based `CommonPool`. + * Submit your feedback to issue #261. + +## Version 0.23.4 + +* Recompiled with Kotlin 1.2.51 to solve broken metadata problem (see [KT-24944](https://youtrack.jetbrains.com/issue/KT-24944)). + +## Version 0.23.3 + +* Kotlin 1.2.50. +* JS: Moved to atomicfu version 0.10.3 that properly matches NPM & Kotlin/JS module names (see #396). +* Improve source-code compatibility with previous (0.22.x) version of `openChannel().use { ... }` pattern by providing deprecated extension function `use` on `ReceiveChannel`. + +## Version 0.23.2 + +* IO: fix joining and continuous writing byte array interference. + +## Version 0.23.1 + +* JS: Fix dependencies in NPM: add "kotlinx-atomicfu" dependency (see #370). +* Introduce `broadcast` coroutine builder (see #280): + * Support `BroadcastChannel.cancel` method to drop the buffer. + * Introduce `ReceiveChannel.broadcast()` extension. +* Fixed a bunch of doc typos (PRs by @paolop). +* Corrected previous version's release notes (PR by @ansman). + +## Version 0.23.0 + +* Kotlin 1.2.41 +* **Coroutines core module is made mostly cross-platform for JVM and JS**: + * Migrate channels and related operators to common, so channels can be used from JS (see #201). + * Most of the code is shared between JVM and JS versions using cross-platform version of [AtomicFU](https://github.com/Kotlin/kotlinx.atomicfu) library. + * The recent version of Kotlin allows default parameters in common code (see #348). + * The project is built using Gradle 4.6. +* **Breaking change**: `CancellableContinuation` is not a `Job` anymore (see #219): + * It does not affect casual users of `suspendCancellableCoroutine`, since all the typically used functions are still there. + * `CancellableContinuation.invokeOnCompletion` is deprecated now and its semantics had subtly changed: + * `invokeOnCancellation` is a replacement for `invokeOnCompletion` to install a handler. + * The handler is **not** invoked on `resume` which corresponds to the typical usage pattern. + * There is no need to check for `cont.isCancelled` in a typical handler code anymore (since handler is invoked only when continuation is cancelled). + * Multiple cancellation handlers cannot be installed. + * Cancellation handlers cannot be removed (disposed of) anymore. + * This change is designed to allow better performance of suspending cancellable functions: + * Now `CancellableContinuation` implementation has simpler state machine and is implemented more efficiently. + * Exception handling in `AbstractContinuation` (that implements `CancellableContinuation`) is now consistent: + * Always prefer exception thrown from coroutine as exceptional reason, add cancellation cause as suppressed exception. +* **Big change**: Deprecate `CoroutineScope.coroutineContext`: + * It is replaced with top-level `coroutineContext` function from Kotlin standard library. +* Improve `ReceiveChannel` operators implementations to guarantee closing of the source channels under all circumstances (see #279): + * `onCompletion` parameter added to `produce` and all other coroutine builders. + * Introduce `ReceiveChannel.consumes(): CompletionHandler` extension function. +* Replace `SubscriptionReceiveChannel` with `ReceiveChannel` (see #283, PR by @deva666). + * `ReceiveChannel.use` extension is introduced to preserve source compatibility, but is deprecated. + * `consume` or `consumeEach` extensions should be used for channels. + * When writing operators, `produce(onCompletion=consumes()) { ... }` pattern shall be used (see #279 above). +* JS: Kotlin is declared as peer dependency (see #339, #340, PR by @ansman). +* Invoke exception handler for actor on cancellation even when channel was successfully closed, so exceptions thrown by actor are always reported (see #368). +* Introduce `awaitAll` and `joinAll` for `Deferred` and `Job` lists correspondingly (see #171). +* Unwrap `CompletionException` exception in `CompletionStage.await` slow-path to provide consistent results (see #375). +* Add extension to `ExecutorService` to return `CloseableCoroutineDispatcher` (see #278, PR by @deva666). +* Fail with proper message during build if JDK_16 is not set (see #291, PR by @venkatperi). +* Allow negative timeouts in `delay`, `withTimeout` and `onTimeout` (see #310). +* Fix a few bugs (leaks on cancellation) in `delay`: + * Invoke `clearTimeout` on cancellation in JSDispatcher. + * Remove delayed task on cancellation from internal data structure on JVM. +* Introduce `ticker` function to create "ticker channels" (see #327): + * It provides analogue of RX `Observable.timer` for coroutine channels. + * It is currently supported on JVM only. +* Add a test-helper class `TestCoroutineContext` (see #297, PR by @streetsofboston). + * It is currently supported on JVM only. + * Ticker channels (#327) are not yet compatible with it. +* Implement a better way to set `CoroutineContext.DEBUG` value (see #316, PR by @dmytrodanylyk): + * Made `CoroutineContext.DEBUG_PROPERTY_NAME` constant public. + * Introduce public constants with `"on"`, `"off"`, `"auto"` values. +* Introduce system property to control `CommonPool` parallelism (see #343): + * `CommonPool.DEFAULT_PARALLELISM_PROPERTY_NAME` constant is introduced with a value of "kotlinx.coroutines.default.parallelism". +* Include package-list files into documentation site (see #290). +* Fix various typos in docs (PRs by @paolop and @ArtsiomCh). + +## Version 0.22.5 + +* JS: Fixed main file reference in [NPM package](https://www.npmjs.com/package/kotlinx-coroutines-core) +* Added context argument to `Channel.filterNot` (PR by @jcornaz). +* Implemented debug `toString` for channels (see #185). + +## Version 0.22.4 + +* JS: Publish to NPM (see #229). +* JS: Use node-style dispatcher on ReactNative (see #236). +* [jdk8 integration](integration/kotlinx-coroutines-jdk8/README.md) improvements: + * Added conversion from `CompletionStage` to `Deferred` (see #262, PR by @jcornaz). + * Use fast path in `CompletionStage.await` and make it cancellable. + +## Version 0.22.3 + +* Fixed `produce` builder to close the channel on completion instead of cancelling it, which lead to lost elements with buffered channels (see #256). +* Don't use `ForkJoinPool` if there is a `SecurityManager` present to work around JNLP problems (see #216, PR by @NikolayMetchev). +* JS: Check for undefined `window.addEventListener` when choosing default coroutine dispatcher (see #230, PR by @ScottPierce). +* Update 3rd party dependencies: + * [kotlinx-coroutines-rx1](reactive/kotlinx-coroutines-rx1) to RxJava version `1.3.6`. + * [kotlinx-coroutines-rx2](reactive/kotlinx-coroutines-rx2) to RxJava version `2.1.9`. + * [kotlinx-coroutines-guava](integration/kotlinx-coroutines-guava) to Guava version `24.0-jre`. + +## Version 0.22.2 + +* Android: Use @Keep annotation on AndroidExceptionPreHandler to fix the problem on Android with minification enabled (see #214). +* Reactive: Added `awaitFirstOrDefault` and `awaitFirstOrNull` extensions (see #224, PR by @konrad-kaminski). +* Core: Fixed `withTimeout` and `withTimeoutOrNull` that should not use equals on result (see #212, PR by @konrad-kaminski). +* Core: Fixed hanged receive from a closed subscription of BroadcastChannel (see #226). +* IO: fixed error propagation (see https://github.com/ktorio/ktor/issues/301). +* Include common sources into sources jar file to work around KT-20971. +* Fixed bugs in documentation due to MPP. + +## Version 0.22.1 + +* Migrated to Kotlin 1.2.21. +* Improved `actor` builder documentation (see #210) and fixed bugs in rendered documentation due to multiplatform. +* Fixed `runBlocking` to properly support specified dispatchers (see #209). +* Fixed data race in `Job` implementation (it was hanging at `LockFreeLinkedList.helpDelete` on certain stress tests). +* `AbstractCoroutine.onCancellation` is invoked before cancellation handler that is set via `invokeOnCompletion`. +* Ensure that `launch` handles uncaught exception before another coroutine that uses `join` on it resumes (see #208). + +## Version 0.22 + +* Migrated to Kotlin 1.2.20. +* Introduced stable public API for `AbstractCoroutine`: + * Implements `Job`, `Continuation`, and `CoroutineScope`. + * Has overridable `onStart`, `onCancellation`, `onCompleted` and `onCompletedExceptionally` functions. + * Reactive integration modules are now implemented using public API only. + * Notifies onXXX before all the installed handlers, so `launch` handles uncaught exceptions before "joining" coroutines wakeup (see #208). + +## Version 0.21.2 + +* Fixed `openSubscription` extension for reactive `Publisher`/`Observable`/`Flowable` when used with `select { ... }` and added an optional `request` parameter to specify how many elements are requested from publisher in advance on subscription (see #197). +* Simplified implementation of `Channel.flatMap` using `toChannel` function to work around Android 5.0 APK install SIGSEGV (see #205). + +## Version 0.21.1 + +* Improved performance of coroutine dispatching (`DispatchTask` instance is no longer allocated). +* Fixed `Job.cancel` and `CompletableDeferred.complete` to support cancelling/completing states and properly wait for their children to complete on join/await (see #199). +* Fixed a bug in binary heap implementation (used internally by `delay`) which could have resulted in wrong delay time in rare circumstances. +* Coroutines library for [Kotlin/JS](js/README.md): + * `Promise.asDeferred` immediately installs handlers to avoid "Unhandled promise rejection" warning. + * Use `window.postMessage` instead of `setTimeout` for coroutines inside the browser to avoid timeout throttling (see #194). + * Use custom queue in `Window.awaitAnimationFrame` to align all animations and reduce overhead. + * Introduced `Window.asCoroutineDispatcher()` extension function. + +## Version 0.21 + +* Migrated to Kotlin 1.2.10. +* Coroutines library for [Kotlin/JS](js/README.md) and [multiplatform projects](https://kotlinlang.org/docs/reference/multiplatform.html) (see #33): + * `launch` and `async` coroutine builders. + * `Job` and `Deferred` light-weight future with cancellation support. + * `delay` and `yield` top-level suspending functions. + * `await` extension for JS `Promise` and `asPromise`/`asDeferred` conversions. + * `promise` coroutine builder. + * `Job()` and `CompletableDeferred()` factories. + * Full support for parent-child coroutine hierarchies. + * `Window.awaitAnimationFrame` extension function. + * [Sample frontend Kotlin/JS application](js/example-frontend-js/README.md) with coroutine-driven animations. +* `run` is deprecated and renamed to `withContext` (see #134). +* `runBlocking` and `EventLoop` implementations optimized (see #190). + +## Version 0.20 + +* Migrated to Kotlin 1.2.0. +* Channels: + * Sequence-like `filter`, `map`, etc extensions on `ReceiveChannel` are introduced (see #88 by @fvasco and #69 by @konrad-kaminski). + * Introduced `ReceiveChannel.cancel` method. + * All operators on `ReceiveChannel` fully consume the original channel (`cancel` it when they are done) using a helper `consume` extension. + * Deprecated `ActorJob` and `ProducerJob`; `actor` now returns `SendChannel` and `produce` returns `ReceiveChannel` (see #127). + * `SendChannel.sendBlocking` extension method (see #157 by @@fvasco). +* Parent-child relations between coroutines: + * Introduced an optional `parent` job parameter for all coroutine builders so that code with an explict parent `Job` is more natural. + * Added `parent` parameter to `CompletableDeferred` constructor. + * Introduced `Job.children` property. + * `Job.cancelChildren` is now an extension (member is deprecated and hidden). + * `Job.joinChildren` extension is introduced. + * Deprecated `Job.attachChild` as a error-prone API. + * Fixed StackOverflow when waiting for a lot of completed children that did not remove their handlers from the parent. +* Use `java.util.ServiceLoader` to find default instances of `CoroutineExceptionHandler`. +* Android UI integration: + * Use `Thread.getUncaughtExceptionPreHandler` to make sure that exceptions are logged before crash (see #148). + * Introduce `UI.awaitFrame` for animation; added sample coroutine-based animation application for Android [here](ui/kotlinx-coroutines-android/animation-app). + * Fixed `delay(Long.MAX_VALUE)` (see #161) +* Added missing `DefaultDispatcher` on some reactive operators (see #174 by @fvasco) +* Fixed `actor` and `produce` so that a cancellation of a Job cancels the underlying channel (closes and removes all the pending messages). +* Fixed sporadic failure of `example-context-06` (see #160) +* Fixed hang of `Job.start` on lazy coroutine with attached `invokeOnCompletion` handler. +* A more gradual introduction to `runBlocking` and coroutines in the [guide](docs/topics/coroutines-guide.md) (see #166). + +## Version 0.19.3 + +* Fixed `send`/`openSubscription` race in `ArrayBroadcastChannel`. + This race lead to stalled (hanged) `send`/`receive` invocations. +* Project build has been migrated to Gradle. + +## Version 0.19.2 + +* Fixed `ArrayBroadcastChannel` receive of stale elements on `openSubscription`. + Only elements that are sent after invocation of `openSubscription` are received now. +* Added a default value for `context` parameter to `rxFlowable` (see #146 by @PhilGlass). +* Exception propagation logic from cancelled coroutines is adjusted (see #152): + * When cancelled coroutine crashes due to some other exception, this other exception becomes the cancellation reason + of the coroutine, while the original cancellation reason is suppressed. + * `UnexpectedCoroutineException` is no longer used to report those cases as is removed. + * This fixes a race between crash of CPU-consuming coroutine and cancellation which resulted in an unhandled exception + and lead to crashes on Android. +* `run` uses cancelling state & propagates exceptions when cancelled (see #147): + * When coroutine that was switched into a different dispatcher using `run` is cancelled, the run invocation does not + complete immediately, but waits until the body completes. + * If the body completes with exception, then this exception is propagated. +* No `Job` in `newSingleThreadContext` and `newFixedThreadPoolContext` anymore (see #149, #151): + * This resolves the common issue of using `run(ctx)` where ctx comes from either `newSingleThreadContext` or + `newFixedThreadPoolContext` invocation. They both used to return a combination of dispatcher + job, + and this job was overriding the parent job, thus preventing propagation of cancellation. Not anymore. + * `ThreadPoolDispatcher` class is now public and is the result type for both functions. + It has the `close` method to release the thread pool. + +## Version 0.19.1 + +* Failed parent Job cancels all children jobs, then waits for them them. + This makes parent-child hierarchies easier to get working right without + having to use `try/catch` or other exception handlers. +* Fixed a race in `ArrayBroadcastChannel` between `send` and `openChannel` invocations + (see #138). +* Fixed quite a rare race in `runBlocking` that resulted in `AssertionError`. + Unfortunately, cannot write a reliable stress-test to reproduce it. +* Updated Reactor support to leverage Bismuth release train + (contributed by @sdeleuze, see PR #141) + +## Version 0.19 + +* This release is published to Maven Central. +* `DefaultDispatcher` is introduced (see #136): + * `launch`, `async`, `produce`, `actor` and other integration-specific coroutine builders now use + `DefaultDispatcher` as the default value for their `context` parameter. + * When a context is explicitly specified, `newCoroutineContext` function checks if there is any + interceptor/dispatcher defined in the context and uses `DefaultDispatcher` if there is none. + * `DefaultDispatcher` is currently defined to be equal to `CommonPool`. + * Examples in the [guide](docs/topics/coroutines-guide.md) now start with `launch { ... }` code and explanation on the nature + and the need for coroutine context starts in "Coroutine context and dispatchers" section. +* Parent coroutines now wait for their children (see #125): + * Job _completing_ state is introduced in documentation as a state in which parent coroutine waits for its children. + * `Job.attachChild` and `Job.cancelChildren` are introduced. + * `Job.join` now always checks cancellation status of invoker coroutine for predictable behavior when joining + failed child coroutine. + * `Job.cancelAndJoin` extension is introduced. + * `CoroutineContext.cancel` and `CoroutineContext.cancelChildren` extensions are introduced for convenience. + * `withTimeout`/`withTimeoutOrNull` blocks become proper coroutines that have `CoroutineScope` and wait for children, too. + * Diagnostics in cancellation and unexpected exception messages are improved, + coroutine name is included in debug mode. + * Fixed cancellable suspending functions to throw `CancellationException` (as was documented before) even when + the coroutine is cancelled with another application-specific exception. + * `JobCancellationException` is introduced as a specific subclass of `CancellationException` which is + used for coroutines that are cancelled without cause and to wrap application-specific exceptions. + * `Job.getCompletionException` is renamed to `Job.getCancellationException` and return a wrapper exception if needed. + * Introduced `Deferred.getCompletionExceptionOrNull` to get not-wrapped exception result of `async` task. + * Updated docs for `Job` & `Deferred` to explain parent/child relations. +* `select` expression is modularized: + * `SelectClause(0,1,2)` interfaces are introduced, so that synchronization + constructs can define their select clauses without having to modify + the source of the `SelectBuilder` in `kotlinx-corounes-core` module. + * `Job.onJoin`, `Deferred.onAwait`, `Mutex.onLock`, `SendChannel.onSend`, `ReceiveChannel.onReceive`, etc + that were functions before are now properties returning the corresponding select clauses. Old functions + are left in bytecode for backwards compatibility on use-site, but any outside code that was implementing those + interfaces by itself must be updated. + * This opens road to moving channels into a separate module in future updates. +* Renamed `TimeoutException` to `TimeoutCancellationException` (old name is deprecated). +* Fixed various minor problems: + * JavaFx toolkit is now initialized by `JavaFx` context (see #108). + * Fixed lost ACC_STATIC on methods (see #116). + * Fixed link to source code from documentation (see #129). + * Fixed `delay` in arbitrary contexts (see #133). +* `kotlinx-coroutines-io` module is introduced. It is a work-in-progress on `ByteReadChannel` and `ByteWriteChannel` + interfaces, their implementations, and related classes to enable convenient coroutine integration with various + asynchronous I/O libraries and sockets. It is currently _unstable_ and **will change** in the next release. + +## Version 0.18 + +* Kotlin 1.1.4 is required to use this version, which enables: + * `withLock` and `consumeEach` functions are now inline suspend functions. + * `JobSupport` class implementation is optimized (one fewer field). +* `TimeoutException` is public (see #89). +* Improvements to `Mutex` (courtesy of @fvasco): + * Introduced `holdsLock` (see #92). + * Improved documentation on `Mutex` fairness (see #90). +* Fixed NPE when `ArrayBroadcastChannel` is closed concurrently with receive (see #97). +* Fixed bug in internal class LockFreeLinkedList that resulted in ISE under stress in extremely rare circumstances. +* Integrations: + * [quasar](integration/kotlinx-coroutines-quasar): Introduced integration with suspendable JVM functions + that are instrumented with [Parallel Universe Quasar](https://docs.paralleluniverse.co/quasar/) + (thanks to the help of @pron). + * [reactor](reactive/kotlinx-coroutines-reactor): Replaced deprecated `setCancellation` with `onDipose` and + updated to Aluminium-SR3 release (courtesy of @yxf07, see #96) + * [jdk8](integration/kotlinx-coroutines-jdk8): Added adapters for `java.time` classes (courtesy of @fvasco, see #93) + +## Version 0.17 + +* `CompletableDeferred` is introduced as a set-once event-like communication primitive (see #70). + * [Coroutines guide](docs/topics/coroutines-guide.md) uses it in a section on actors. + * `CompletableDeferred` is an interface with private impl (courtesy of @fvasco, see #86). + * It extends `Deferred` interface with `complete` and `completeExceptionally` functions. +* `Job.join` and `Deferred.await` wait until a cancelled coroutine stops execution (see #64). + * `Job` and `Deferred` have a new _cancelling_ state which they enter on invocation of `cancel`. + * `Job.invokeOnCompletion` has an additional overload with `onCancelling: Boolean` parameter to + install handlers that are fired as soon as coroutine enters _cancelling_ state as opposed + to waiting until it _completes_. + * Internal `select` implementation is refactored to decouple it from `JobSupport` internal class + and to optimize its state-machine. + * Internal `AbstractCoroutine` class is refactored so that it is extended only by true coroutines, + all of which support the new _cancelling_ state. +* `CoroutineScope.context` is renamed to `coroutineContext` to avoid conflicts with other usages of `context` + in applications (like Android context, see #75). +* `BroadcastChannel.open` is renamed to `openSubscription` (see #54). +* Fixed `StackOverflowError` in a convoy of `Mutex.unlock` invokers with `Unconfined` dispatcher (see #80). +* Fixed `SecurityException` when trying to use coroutines library with installed `SecurityManager`. +* Fixed a bug in `withTimeoutOrNull` in case with nested timeouts when coroutine was cancelled before it was + ever suspended. +* Fixed a minor problem with `awaitFirst` on reactive streams that would have resulted in spurious stack-traces printed + on the console when used with publishers/observables that continue to invoke `onNext` despite being cancelled/disposed + (which they are technically allowed to do by specification). +* All factory functions for various interfaces are implemented as top-level functions + (affects `Job`, `Channel`, `BroadcastChannel`, `Mutex`, `EventLoop`, and `CoroutineExceptionHandler`). + Previous approach of using `operator invoke` on their companion objects is deprecated. +* Nicer-to-use debug `toString` implementations for coroutine dispatcher tasks and continuations. +* A default dispatcher for `delay` is rewritten and now shares code with `EventLoopImpl` that is used by + `runBlocking`. It internally supports non-default `TimeSource` so that delay-using tests can be written + with "virtual time" by replacing their time source for the duration of tests (this feature is not available + outside of the library). + +## Version 0.16 + +* Coroutines that are scheduled for execution are cancellable by default now + * `suspendAtomicCancellableCoroutine` function is introduced for funs like +  `send`/`receive`/`receiveOrNull` that require atomic cancellation +  (they cannot be cancelled after decision was made) + * Coroutines started with default mode using +  `async`/`launch`/`actor` builders can be cancelled before their execution starts + * `CoroutineStart.ATOMIC` is introduced as a start mode to specify that +  coroutine cannot be cancelled before its execution starts + * `run` function is also cancellable in the same way and accepts an optional + `CoroutineStart` parameter to change this default. +* `BroadcastChannel` factory function is introduced +* `CoroutineExceptionHandler` factory function is introduced by @konrad-kaminski +* [`integration`](integration) directory is introduced for all 3rd party integration projects + * It has [contribution guidelines](integration/README.md#contributing) and contributions from community are welcome + * Support for Guava `ListenableFuture` in the new [`kotlinx-coroutines-guava`](integration/kotlinx-coroutines-guava) module + * Rx1 Scheduler support by @konrad-kaminski +* Fixed a number of `Channel` and `BroadcastChannel` implementation bugs related to concurrent + send/close/close of channels that lead to hanging send, offer or close operations (see #66). + Thanks to @chrisly42 and @cy6erGn0m for finding them. +* Fixed `withTimeoutOrNull` which was returning `null` on timeout of inner or outer `withTimeout` blocks (see #67). + Thanks to @gregschlom for finding the problem. +* Fixed a bug where `Job` fails to dispose a handler when it is the only handler by @uchuhimo + +## Version 0.15 + +* Switched to Kotlin version 1.1.2 (can still be used with 1.1.0). +* `CoroutineStart` enum is introduced for `launch`/`async`/`actor` builders: + * The usage of `luanch(context, start = false)` is deprecated and is replaced with + `launch(context, CoroutineStart.LAZY)` + * `CoroutineStart.UNDISPATCHED` is introduced to start coroutine execution immediately in the invoker thread, + so that `async(context, CoroutineStart.UNDISPATCHED)` is similar to the behavior of C# `async`. + * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) mentions the use of it to optimize + the start of coroutines from UI threads. +* Introduced `BroadcastChannel` interface in `kotlinx-coroutines-core` module: + * It extends `SendChannel` interface and provides `open` function to create subscriptions. + * Subscriptions are represented with `SubscriptionReceiveChannel` interface. + * The corresponding `SubscriptionReceiveChannel` interfaces are removed from [reactive](reactive) implementation + modules. They use an interface defined in `kotlinx-coroutines-core` module. + * `ConflatedBroadcastChannel` implementation is provided for state-observation-like use-cases, where a coroutine or a + regular code (in UI, for example) updates the state that subscriber coroutines shall react to. + * `ArrayBroadcastChannel` implementation is provided for event-bus-like use-cases, where a sequence of events shall + be received by multiple subscribers without any omissions. + * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) includes + "Rx Subject vs BroadcastChannel" section. +* Pull requests from Konrad Kamiński are merged into reactive stream implementations: + * Support for Project Reactor `Mono` and `Flux`. + See [`kotlinx-coroutines-reactor`](reactive/kotlinx-coroutines-reactor) module. + * Implemented Rx1 `Completable.awaitCompleted`. + * Added support for Rx2 `Maybe`. +* Better timeout support: + * Introduced `withTimeoutOrNull` function. + * Implemented `onTimeout` clause for `select` expressions. + * Fixed spurious concurrency inside `withTimeout` blocks on their cancellation. + * Changed behavior of `withTimeout` when `CancellationException` is suppressed inside the block. + Invocation of `withTimeout` now always returns the result of execution of its inner block. +* The `channel` property in `ActorScope` is promoted to a wider `Channel` type, so that an actor + can have an easy access to its own inbox send channel. +* Renamed `Mutex.withMutex` to `Mutex.withLock`, old name is deprecated. + +## Version 0.14 + +* Switched to Kotlin version 1.1.1 (can still be used with 1.1.0). +* Introduced `consumeEach` helper function for channels and reactive streams, Rx 1.x, and Rx 2.x. + * It ensures that streams are unsubscribed from on any exception. + * Iteration with `for` loop on reactive streams is **deprecated**. + * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) is updated virtually + all over the place to reflect these important changes. +* Implemented `awaitFirstOrDefault` extension for reactive streams, Rx 1.x, and Rx 2.x. +* Added `Mutex.withMutex` helper function. +* `kotlinx-coroutines-android` module has `provided` dependency on of Android APIs to + eliminate warnings when using it in android project. + +## Version 0.13 + +* New `kotlinx-coroutinex-android` module with Android `UI` context implementation. +* Introduced `whileSelect` convenience function. +* Implemented `ConflatedChannel`. +* Renamed various `toXXX` conversion functions to `asXXX` (old names are deprecated). +* `run` is optimized with fast-path case and no longer has `CoroutineScope` in its block. +* Fixed dispatching logic of `withTimeout` (removed extra dispatch). +* `EventLoop` that is used by `runBlocking` now implements Delay, giving more predictable test behavior. +* Various refactorings related to resource management and timeouts: + * `Job.Registration` is renamed to `DisposableHandle`. + * `EmptyRegistration` is renamed to `NonDisposableHandle`. + * `Job.unregisterOnCompletion` is renamed to `Job.disposeOnCompletion`. + * `Delay.invokeOnTimeout` is introduced. + * `withTimeout` now uses `Delay.invokeOnTimeout` when available. +* A number of improvement for reactive streams and Rx: + * Introduced `rxFlowable` builder for Rx 2.x. + * `Scheduler.asCoroutineDispatcher` extension for Rx 2.x. + * Fixed bug with sometimes missing `onComplete` in `publish`, `rxObservable`, and `rxFlowable` builders. + * Channels that are open for reactive streams are now `Closeable`. + * Fixed `CompletableSource.await` and added test for it. + * Removed `rx.Completable.await` due to name conflict. +* New documentation: + * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) + * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) +* Code is published to JCenter repository. + +## Version 0.12 + +* Switched to Kotlin version 1.1.0 release. +* Reworked and updated utilities for + [Reactive Streams](kotlinx-coroutines-reactive), + [Rx 1.x](kotlinx-coroutines-rx1), and + [Rx 2.x](kotlinx-coroutines-rx2) with library-specific + coroutine builders, suspending functions, converters and iteration support. +* `LinkedListChannel` with unlimited buffer (`offer` always succeeds). +* `onLock` select clause and an optional `owner` parameter in all `Mutex` functions. +* `selectUnbiased` function. +* `actor` coroutine builder. +* Couple more examples for "Shared mutable state and concurrency" section and + "Channels are fair" section with ping-pong table example + in [coroutines guide](docs/topics/coroutines-guide.md). + +## Version 0.11-rc + +* `select` expression with onJoin/onAwait/onSend/onReceive clauses. +* `Mutex` is moved to `kotlinx.coroutines.sync` package. +* `ClosedSendChannelException` is a subclass of `CancellationException` now. +* New sections on "Shared mutable state and concurrency" and "Select expression" + in [coroutines guide](docs/topics/coroutines-guide.md). + +## Version 0.10-rc + +* Switched to Kotlin version 1.1.0-rc-91. +* `Mutex` synchronization primitive is introduced. +* `buildChannel` is renamed to `produce`, old name is deprecated. +* `Job.onCompletion` is renamed to `Job.invokeOnCompletion`, old name is deprecated. +* `delay` implementation in Swing, JavaFx, and scheduled executors is fixed to avoid an extra dispatch. +* `CancellableContinuation.resumeUndispatched` is introduced to make this efficient implementation possible. +* Remove unnecessary creation of `CancellationException` to improve performance, plus other performance improvements. +* Suppress deprecated and internal APIs from docs. +* Better docs at top level with categorized summary of classes and functions. + +## Version 0.8-beta + +* `defer` coroutine builder is renamed to `async`. +* `lazyDefer` is deprecated, `async` has an optional `start` parameter instead. +* `LazyDeferred` interface is deprecated, lazy start functionality is integrated into `Job` interface. +* `launch` has an optional `start` parameter for lazily started coroutines. +* `Job.start` and `Job.isCompleted` are introduced. +* `Deferred.isCompletedExceptionally` and `Deferred.isCancelled` are introduced. +* `Job.getInactiveCancellationException` is renamed to `getCompletionException`. +* `Job.join` is now a member function. +* Internal `JobSupport` state machine is enhanced to support _new_ (not-started-yet) state. + So, lazy coroutines do not need a separate state variable to track their started/not-started (new/active) status. +* Exception transparency in `Job.cancel` (original cause is rethrown). +* Clarified possible states for `Job`/`CancellableContinuation`/`Deferred` in docs. +* Example on async-style functions and links to API reference site from [coroutines guide](docs/topics/coroutines-guide.md). + +## Version 0.7-beta + +* Buffered and unbuffered channels are introduced: `Channel`, `SendChannel`, and `ReceiveChannel` interfaces, + `RendezvousChannel` and `ArrayChannel` implementations, `Channel()` factory function and `buildChannel{}` + coroutines builder. +* `Here` context is renamed to `Unconfined` (the old name is deprecated). +* A [guide on coroutines](docs/topics/coroutines-guide.md) is expanded: sections on contexts and channels. + +## Version 0.6-beta + +* Switched to Kotlin version 1.1.0-beta-37. +* A [guide on coroutines](docs/topics/coroutines-guide.md) is expanded. + +## Version 0.5-beta + +* Switched to Kotlin version 1.1.0-beta-22 (republished version). +* Removed `currentCoroutineContext` and related thread-locals without replacement. + Explicitly pass coroutine context around if needed. +* `lazyDefer(context) {...}` coroutine builder and `LazyDeferred` interface are introduced. +* The default behaviour of all coroutine dispatchers is changed to always schedule execution of new coroutine + for later in this thread or thread pool. Correspondingly, `CoroutineDispatcher.isDispatchNeeded` function + has a default implementation that returns `true`. +* `NonCancellable` context is introduced. +* Performance optimizations for cancellable continuations (fewer objects created). +* A [guide on coroutines](docs/topics/coroutines-guide.md) is added. + +## Version 0.4-beta + +* Switched to Kotlin version 1.1.0-beta-18 (republished version). +* `CoroutineDispatcher` methods now have `context` parameter. +* Introduced `CancellableContinuation.isCancelled` +* Introduced `EventLoop` dispatcher and made it a default for `runBlocking { ... }` +* Introduced `CoroutineScope` interface with `isActive` and `context` properties; + standard coroutine builders include it as receiver for convenience. +* Introduced `Executor.toCoroutineDispatcher()` extension. +* Delay scheduler thread is not daemon anymore, but times out automatically. +* Debugging facilities in `newCoroutineContext` can be explicitly disabled with `-Dkotlinx.coroutines.debug=off`. +* xxx-test files are renamed to xxx-example for clarity. +* Fixed NPE in Job implementation when starting coroutine with already cancelled parent job. +* Support cancellation in `kotlinx-coroutines-nio` module diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000000..85ed20dba4 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,4 @@ +## Code of Conduct + +This project and the corresponding community is governed by the [JetBrains Open Source and Community Code of Conduct](https://confluence.jetbrains.com/display/ALL/JetBrains+Open+Source+and+Community+Code+of+Conduct). Please make sure you read it. + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..cc9c3bc998 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,101 @@ +# Contributing Guidelines + +There are two main ways to contribute to the project — submitting issues and submitting +fixes/changes/improvements via pull requests. + +## Submitting issues + +Both bug reports and feature requests are welcome. +Submit issues [here](https://github.com/Kotlin/kotlinx.coroutines/issues). +Questions about usage and general inquiries are better suited for [StackOverflow](https://stackoverflow.com) +or the `#coroutines` channel in [KotlinLang Slack](https://surveys.jetbrains.com/s3/kotlin-slack-sign-up). + +## Submitting PRs + +We love PRs. Submit PRs [here](https://github.com/Kotlin/kotlinx.coroutines/pulls). +However, please keep in mind that maintainers will have to support the resulting code of the project, +so do familiarize yourself with the following guidelines. + +* All development (both new features and bug fixes) is performed in the `develop` branch. + * The `master` branch contains the sources of the most recently released version. + * Base your PRs against the `develop` branch. + * The `develop` branch is pushed to the `master` branch during release. + * Documentation in markdown files can be updated directly in the `master` branch, + unless the documentation is in the source code, and the patch changes line numbers. +* If you fix documentation: + * After fixing/changing code examples in the [`docs`](docs) folder or updating any references in the markdown files + run the [Knit tool](#running-the-knit-tool) and commit the resulting changes as well. + The tests will not pass otherwise. + * If you plan extensive rewrites/additions to the docs, then please [contact the maintainers](#contacting-maintainers) + to coordinate the work in advance. +* If you make any code changes: + * Follow the [Kotlin Coding Conventions](https://kotlinlang.org/docs/reference/coding-conventions.html). + Use 4 spaces for indentation. + Do not add extra newlines in function bodies: if you feel that blocks of code should be logically separated, + then separate them with a comment instead. + * [Build the project](#building) to make sure everything works and passes the tests. +* If you fix a bug: + * Write the test that reproduces the bug. + * Fixes without tests are accepted only in exceptional circumstances if it can be shown that writing the + corresponding test is too hard or otherwise impractical. + * Follow the style of writing tests that is used in this project: + name test functions as `testXxx`. Don't use backticks in test names. +* If you introduce any new public APIs: + * Comment on the existing issue if you want to work on it or create one beforehand. + Ensure that not only the issue describes a problem, but also that the proposed solution has received positive + feedback. Propose a solution if there isn't any. + PRs that add new API without a corresponding issue with positive feedback about the proposed implementation are + unlikely to be approved or reviewed. + * All new APIs must come with documentation and tests. + * All new APIs are initially released with the `@ExperimentalCoroutineApi` annotation and graduate later. + * [Update the public API dumps](#updating-the-public-api-dump) and commit the resulting changes as well. + It will not pass the tests otherwise. + * If you plan large API additions, then please start by submitting an issue with the proposed API design + to gather community feedback. + * [Contact the maintainers](#contacting-maintainers) to coordinate any extensive work in advance. + +## Building + +This library is built with Gradle. + +* Run `./gradlew build` to build. It also runs all the tests. +* Run `./gradlew :check` to test the module you are looking at to speed + things up during development. +* Run `./gradlew :jvmTest` to perform only the fast JVM tests of a multiplatform module. + +You can import this project into IDEA, but you have to delegate build actions +to Gradle (in Preferences -> Build, Execution, Deployment -> Build Tools -> Gradle -> Build and run). + +### Environment requirements + +* JDK >= 11 referred to by the `JAVA_HOME` environment variable. +* JDK 1.8 referred to by the `JDK_18` environment variable. Only used by nightly stress-tests. + It is OK to have `JDK_18` point to a non-1.8 JDK (e.g. `JAVA_HOME`) for external contributions. + +For external contributions you can, for example, add this to your shell startup scripts (e.g. `~/.zshrc`): +```shell +# This assumes JAVA_HOME is set already to a JDK >= 11 version +export JDK_18="$JAVA_HOME" +``` + +### Running the Knit tool + +* Use [Knit](https://github.com/Kotlin/kotlinx-knit/blob/main/README.md) for updates to documentation: + * Run `./gradlew knit` to update the example files, links, tables of content. + * Commit the updated documents and examples together with other changes. + +### Updating the public API dump + +* Use the [Binary Compatibility Validator](https://github.com/Kotlin/binary-compatibility-validator/blob/master/README.md) for updates to public API: + * Run `./gradlew apiDump` to update API index files. + * Commit the updated API indexes together with other changes. + +## Releases + +* The full release procedure checklist is [here](RELEASE.md). + +## Contacting maintainers + +* If something cannot be done, not convenient, or does not work — submit an [issue](#submitting-issues). +* "How to do something" questions — [StackOverflow](https://stackoverflow.com). +* Discussions and general inquiries — use `#coroutines` channel in [KotlinLang Slack](https://kotl.in/slack). diff --git a/LICENSE.txt b/LICENSE.txt new file mode 100644 index 0000000000..9c308d958b --- /dev/null +++ b/LICENSE.txt @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2000-2020 JetBrains s.r.o. and Kotlin Programming Language contributors. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README.md b/README.md index 08fe145774..253c9f8412 100644 --- a/README.md +++ b/README.md @@ -1,96 +1,294 @@ -# kotlinx.coroutines [ ![Download](https://api.bintray.com/packages/kotlin/kotlinx/kotlinx.coroutines/images/download.svg) ](https://bintray.com/kotlin/kotlinx/kotlinx.coroutines/_latestVersion) +# kotlinx.coroutines -Library support for Kotlin coroutines. This is a companion version for Kotlin 1.1 release. +[![Kotlin Stable](https://kotl.in/badges/stable.svg)](https://kotlinlang.org/docs/components-stability.html) +[![JetBrains official project](https://jb.gg/badges/official.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub) +[![GitHub license](https://img.shields.io/badge/license-Apache%20License%202.0-blue.svg?style=flat)](https://www.apache.org/licenses/LICENSE-2.0) +[![Download](https://img.shields.io/maven-central/v/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.10.2)](https://central.sonatype.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.10.2) +[![Kotlin](https://img.shields.io/badge/kotlin-2.0.0-blue.svg?logo=kotlin)](http://kotlinlang.org) +[![KDoc link](https://img.shields.io/badge/API_reference-KDoc-blue)](https://kotlinlang.org/api/kotlinx.coroutines/) +[![Slack channel](https://img.shields.io/badge/chat-slack-green.svg?logo=slack)](https://kotlinlang.slack.com/messages/coroutines/) -## Modules - -Basic modules: +Library support for Kotlin coroutines with [multiplatform](#multiplatform) support. +This is a companion version for the Kotlin `2.0.0` release. -* [kotlinx-coroutines-core](kotlinx-coroutines-core) -- core primitives to work with coroutines. -* [kotlinx-coroutines-jdk8](kotlinx-coroutines-jdk8) -- additional libraries for JDK8 (or Android API level 24). -* [kotlinx-coroutines-nio](kotlinx-coroutines-nio) -- extensions for asynchronous IO on JDK7+. +```kotlin +suspend fun main() = coroutineScope { + launch { + delay(1000) + println("Kotlin Coroutines World!") + } + println("Hello") +} +``` -Modules that provide builders and iteration support for various reactive streams libraries: +> Play with coroutines online [here](https://pl.kotl.in/9zva88r7S) -* [kotlinx-coroutines-reactive](reactive/kotlinx-coroutines-reactive) -- utilities for [Reactive Streams](http://www.reactive-streams.org) -* [kotlinx-coroutines-reactor](reactive/kotlinx-coroutines-reactor) -- utilities for [Reactor](https://projectreactor.io) -* [kotlinx-coroutines-rx1](reactive/kotlinx-coroutines-rx1) -- utilities for [RxJava 1.x](https://github.com/ReactiveX/RxJava/tree/1.x) -* [kotlinx-coroutines-rx2](reactive/kotlinx-coroutines-rx2) -- utilities for [RxJava 2.x](https://github.com/ReactiveX/RxJava) +## Modules -Modules that provide coroutine dispatchers for various single-threaded UI libraries: +* [core](kotlinx-coroutines-core/README.md) — common coroutines across all platforms: + * [launch] and [async] coroutine builders returning [Job] and [Deferred] light-weight futures with cancellation support; + * [Dispatchers] object with [Main][Dispatchers.Main] dispatcher for Android/Swing/JavaFx (which require the corresponding artifacts in runtime) and Darwin (included out of the box), and [Default][Dispatchers.Default] dispatcher for background coroutines; + * [delay] and [yield] top-level suspending functions; + * [Flow] — cold asynchronous stream with [flow][_flow] builder and comprehensive operator set ([filter], [map], etc); + * [Channel], [Mutex], and [Semaphore] communication and synchronization primitives; + * [coroutineScope][_coroutineScope], [supervisorScope][_supervisorScope], [withContext], and [withTimeout] scope builders; + * [MainScope()] for Android and UI applications; + * [SupervisorJob()] and [CoroutineExceptionHandler] for supervision of coroutines hierarchies; + * [select] expression support and more. +* [core/jvm](kotlinx-coroutines-core/jvm/) — additional core features available on Kotlin/JVM: + * [Dispatchers.IO] dispatcher for blocking coroutines; + * [Executor.asCoroutineDispatcher][asCoroutineDispatcher] extension, custom thread pools, and more; + * Integrations with `CompletableFuture` and JVM-specific extensions. +* [core/js](kotlinx-coroutines-core/js/) — additional core features available on Kotlin/JS: + * Integration with `Promise` via [Promise.await] and [promise] builder; + * Integration with `Window` via [Window.asCoroutineDispatcher], etc. +* [test](kotlinx-coroutines-test/README.md) — test utilities for coroutines: + * [Dispatchers.setMain] to override [Dispatchers.Main] in tests; + * [runTest] and [TestScope] to test suspending functions and coroutines. +* [debug](kotlinx-coroutines-debug/README.md) — debug utilities for coroutines: + * [DebugProbes] API to probe, keep track of, print and dump active coroutines; + * [CoroutinesTimeout] test rule to automatically dump coroutines on test timeout. + * Automatic integration with [BlockHound](https://github.com/reactor/BlockHound). +* [reactive](reactive/README.md) — modules that provide builders and iteration support for various reactive streams libraries: + * Reactive Streams ([Publisher.collect], [Publisher.awaitSingle], [kotlinx.coroutines.reactive.publish], etc), + * Flow (JDK 9) (the same interface as for Reactive Streams), + * RxJava 2.x ([rxFlowable], [rxSingle], etc), and + * RxJava 3.x ([rxFlowable], [rxSingle], etc), and + * Project Reactor ([flux], [mono], etc). +* [ui](ui/README.md) — modules that provide the [Main][Dispatchers.Main] dispatcher for various single-threaded UI libraries: + * Android, JavaFX, and Swing. +* [integration](integration/README.md) — modules that provide integration with various asynchronous callback- and future-based libraries: + * Guava [ListenableFuture.await], and Google Play Services [Task.await]; + * SLF4J MDC integration via [MDCContext]. -* [kotlinx-coroutines-android](ui/kotlinx-coroutines-android) -- `UI` context for Android applications. -* [kotlinx-coroutines-javafx](ui/kotlinx-coroutines-javafx) -- `JavaFx` context for JavaFX UI applications. -* [kotlinx-coroutines-swing](ui/kotlinx-coroutines-swing) -- `Swing` context for Swing UI applications. - ## Documentation -* [Guide to kotlinx.coroutines by example](coroutines-guide.md) (**read it first**) -* [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) -* [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) +* Presentations and videos: + * [Kotlin Coroutines in Practice](https://www.youtube.com/watch?v=a3agLJQ6vt8) (Roman Elizarov at KotlinConf 2018, [slides](https://www.slideshare.net/elizarov/kotlin-coroutines-in-practice-kotlinconf-2018)) + * [Deep Dive into Coroutines](https://www.youtube.com/watch?v=YrrUCSi72E8) (Roman Elizarov at KotlinConf 2017, [slides](https://www.slideshare.net/elizarov/deep-dive-into-coroutines-on-jvm-kotlinconf-2017)) + * [History of Structured Concurrency in Coroutines](https://www.youtube.com/watch?v=Mj5P47F6nJg) (Roman Elizarov at Hydra 2019, [slides](https://speakerdeck.com/elizarov/structured-concurrency)) +* Guides and manuals: + * [Guide to kotlinx.coroutines by example](https://kotlinlang.org/docs/coroutines-guide.html) (**read it first**) + * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) + * [Debugging capabilities in kotlinx.coroutines](docs/topics/debugging.md) +* [Compatibility policy and experimental annotations](docs/topics/compatibility.md) * [Change log for kotlinx.coroutines](CHANGES.md) -* [Coroutines design document (KEEP)](https://github.com/Kotlin/kotlin-coroutines/blob/master/kotlin-coroutines-informal.md) -* [Full kotlinx.coroutines API reference](http://kotlin.github.io/kotlinx.coroutines) +* [Coroutines design document (KEEP)](https://github.com/Kotlin/KEEP/blob/master/proposals/coroutines.md) +* [Full kotlinx.coroutines API reference](https://kotlinlang.org/api/kotlinx.coroutines/) ## Using in your projects -> Note that these libraries are experimental and are subject to change. - -The libraries are published to [kotlinx](https://bintray.com/kotlin/kotlinx/kotlinx.coroutines) bintray repository -and also linked to [JCenter](https://bintray.com/bintray/jcenter?filterByPkgName=kotlinx.coroutines). - -These libraries require kotlin compiler version `1.1.x` and -require kotlin runtime of the same version as a dependency. - ### Maven -Add Bintray JCenter repository to `` section: - -```xml - - central - http://jcenter.bintray.com - -``` - Add dependencies (you can also add other modules that you need): ```xml org.jetbrains.kotlinx kotlinx-coroutines-core - 0.15 + 1.10.2 ``` -And make sure that you use the right Kotlin version: +And make sure that you use the latest Kotlin version: ```xml - 1.1.2 + 2.0.0 ``` ### Gradle -Add Bintray JCenter repository: +Add dependencies (you can also add other modules that you need): + +```kotlin +dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.10.2") +} +``` + +And make sure that you use the latest Kotlin version: -```groovy +```kotlin +plugins { + // For build.gradle.kts (Kotlin DSL) + kotlin("jvm") version "2.0.0" + + // For build.gradle (Groovy DSL) + id "org.jetbrains.kotlin.jvm" version "2.0.0" +} +``` + +Make sure that you have `mavenCentral()` in the list of repositories: + +```kotlin repositories { - jcenter() + mavenCentral() } ``` -Add dependencies (you can also add other modules that you need): +### Android -```groovy -compile 'org.jetbrains.kotlinx:kotlinx-coroutines-core:0.15' +Add [`kotlinx-coroutines-android`](ui/kotlinx-coroutines-android) +module as a dependency when using `kotlinx.coroutines` on Android: + +```kotlin +implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.10.2") ``` -And make sure that you use the right Kotlin version: +This gives you access to the Android [Dispatchers.Main] +coroutine dispatcher and also makes sure that in case of a crashed coroutine with an unhandled exception that +this exception is logged before crashing the Android application, similarly to the way uncaught exceptions in +threads are handled by the Android runtime. + +#### R8 and ProGuard + +R8 and ProGuard rules are bundled into the [`kotlinx-coroutines-android`](ui/kotlinx-coroutines-android) module. +For more details see ["Optimization" section for Android](ui/kotlinx-coroutines-android/README.md#optimization). + +#### Avoiding including the debug infrastructure in the resulting APK -```groovy -buildscript { - ext.kotlin_version = '1.1.2' +The `kotlinx-coroutines-core` artifact contains a resource file that is not required for the coroutines to operate +normally and is only used by the debugger. To exclude it at no loss of functionality, add the following snippet to the +`android` block in your Gradle file for the application subproject: + +```kotlin +packagingOptions { + resources.excludes += "DebugProbesKt.bin" +} +``` + +### Multiplatform + +Core modules of `kotlinx.coroutines` are also available for +[Kotlin/JS](https://kotlinlang.org/docs/reference/js-overview.html) and [Kotlin/Native](https://kotlinlang.org/docs/reference/native-overview.html). + +In common code that should get compiled for different platforms, you can add a dependency to `kotlinx-coroutines-core` right to the `commonMain` source set: + +```kotlin +commonMain { + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.10.2") + } } ``` + +Platform-specific dependencies are recommended to be used only for non-multiplatform projects that are compiled only for target platform. + +#### JS + +Kotlin/JS version of `kotlinx.coroutines` is published as +[`kotlinx-coroutines-core-js`](https://central.sonatype.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-js/1.10.2) +(follow the link to get the dependency declaration snippet). + +#### Native + +Kotlin/Native version of `kotlinx.coroutines` is published as +[`kotlinx-coroutines-core-$platform`](https://central.sonatype.com/search?q=kotlinx-coroutines-core&namespace=org.jetbrains.kotlinx) where `$platform` is +the target Kotlin/Native platform. +Targets are provided in accordance with [official K/N target support](https://kotlinlang.org/docs/native-target-support.html). +## Building and Contributing + +See [Contributing Guidelines](CONTRIBUTING.md). + + + + +[launch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html +[async]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[Deferred]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/index.html +[Dispatchers]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/index.html +[Dispatchers.Main]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-main.html +[Dispatchers.Default]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-default.html +[delay]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html +[yield]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/yield.html +[_coroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/coroutine-scope.html +[_supervisorScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/supervisor-scope.html +[withContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-context.html +[withTimeout]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-timeout.html +[MainScope()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-main-scope.html +[SupervisorJob()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-supervisor-job.html +[CoroutineExceptionHandler]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-exception-handler/index.html +[Dispatchers.IO]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-i-o.html +[asCoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/as-coroutine-dispatcher.html +[Promise.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/await.html +[promise]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/[js]promise.html +[Window.asCoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/as-coroutine-dispatcher.html + + + +[Flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html +[_flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flow.html +[filter]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/filter.html +[map]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/map.html + + + +[Channel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-channel/index.html + + + +[select]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.selects/select.html + + + +[Mutex]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/index.html +[Semaphore]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-semaphore/index.html + + + + +[Dispatchers.setMain]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/set-main.html +[runTest]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/run-test.html +[TestScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-test-scope/index.html + + + + +[DebugProbes]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug/-debug-probes/index.html + + + +[CoroutinesTimeout]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug.junit4/-coroutines-timeout/index.html + + + + +[MDCContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-slf4j/kotlinx.coroutines.slf4j/-m-d-c-context/index.html + + + + + + +[ListenableFuture.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-guava/kotlinx.coroutines.guava/await.html + + + + +[Task.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-play-services/kotlinx.coroutines.tasks/await.html + + + + +[Publisher.collect]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/collect.html +[Publisher.awaitSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/await-single.html +[kotlinx.coroutines.reactive.publish]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/publish.html + + + + +[rxFlowable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/rx-flowable.html +[rxSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/rx-single.html + + + + + + +[flux]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.reactor/flux.html +[mono]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.reactor/mono.html + + diff --git a/RELEASE.md b/RELEASE.md new file mode 100644 index 0000000000..4a793bff6d --- /dev/null +++ b/RELEASE.md @@ -0,0 +1,81 @@ +# kotlinx.coroutines release checklist + +To release a new `` of `kotlinx-coroutines`: + +1. Checkout the `develop` branch:
+ `git checkout develop` + +2. Retrieve the most recent `develop`:
+ `git pull` + +3. Make sure the `master` branch is fully merged into `develop`: + `git merge origin/master` + +4. Search & replace `` with `` across the project files. Should replace in: + * Docs + * [`README.md`](README.md) (native, core, test, debug, modules) + * [`kotlinx-coroutines-debug/README.md`](kotlinx-coroutines-debug/README.md) + * [`kotlinx-coroutines-test/README.md`](kotlinx-coroutines-test/README.md) + * [`coroutines-guide-ui.md`](ui/coroutines-guide-ui.md) + * Properties + * [`gradle.properties`](gradle.properties) + * [`integration-testing/gradle.properties`](integration-testing/gradle.properties) + * Make sure to **exclude** `CHANGES.md` from replacements. + + As an alternative approach, you can use `./bump-version.sh new_version` + +5. Write release notes in [`CHANGES.md`](CHANGES.md): + * Use the old releases for style guidance. + * Write each change on a single line (don't wrap with CR). + * Look through the commit messages since the previous release. + +6. Create the branch for this release: + `git checkout -b version-` + +7. Commit the updated files to the new version branch:
+ `git commit -a -m "Version "` + +8. Push the new version to GitHub:
+ `git push -u origin version-` + +9. Create a Pull-Request on GitHub from the `version-` branch into `master`: + * Review it. + * Make sure it builds on CI. + * Get approval for it. + +0. On [TeamCity integration server](https://teamcity.jetbrains.com/project.html?projectId=KotlinTools_KotlinxCoroutines): + * Wait until "Build" configuration for committed `version-` branch passes tests. + * Run "Deploy (Configure, RUN THIS ONE)" configuration with the corresponding new version: + - Use the `version-` branch + - Set the `DeployVersion` build parameter to `` + * Wait until all four "Deploy" configurations finish. + +1. In [Nexus](https://oss.sonatype.org/#stagingRepositories) admin interface: + * Close the repository and wait for it to verify. + * Release the repository. + +2. Merge the new version branch into `master`:
+ `git checkout master`
+ `git merge version-`
+ `git push` + +3. In [GitHub](https://github.com/kotlin/kotlinx.coroutines) interface: + * Create a release named ``, creating the `` tag. + * Cut & paste lines from [`CHANGES.md`](CHANGES.md) into description. + +4. Announce the new release in [Slack](https://kotlinlang.slack.com) + +5. Switch onto the `develop` branch:
+ `git checkout develop` + +6. Fetch the latest `master`:
+ `git fetch` + +7. Merge the release from `master`:
+ `git merge origin/master` + +8. Push the updates to GitHub:
+ `git push` + +9. Propose the website documentation update:
+ * Set new value for [`KOTLINX_COROUTINES_RELEASE_TAG`](https://github.com/JetBrains/kotlin-web-site/blob/master/.teamcity/BuildParams.kt), creating a Pull Request in the website's repository. diff --git a/benchmarks/build.gradle.kts b/benchmarks/build.gradle.kts new file mode 100644 index 0000000000..2b124f6208 --- /dev/null +++ b/benchmarks/build.gradle.kts @@ -0,0 +1,59 @@ +@file:Suppress("UnstableApiUsage") + +import org.jetbrains.kotlin.gradle.tasks.* +import org.jetbrains.kotlin.gradle.dsl.JvmTarget + +plugins { + id("me.champeau.jmh") +} + +repositories { + maven("https://repo.typesafe.com/typesafe/releases/") +} + +java { + sourceCompatibility = JavaVersion.VERSION_1_8 + targetCompatibility = JavaVersion.VERSION_1_8 +} + +tasks.named("compileJmhKotlin") { + compilerOptions { + jvmTarget = JvmTarget.JVM_1_8 + freeCompilerArgs.add("-Xjvm-default=all") + } +} + +val jmhJarTask = tasks.named("jmhJar") { + archiveBaseName = "benchmarks" + archiveClassifier = null + archiveVersion = null + archiveVersion.convention(null as String?) + destinationDirectory = rootDir +} + +tasks { + // For some reason the DuplicatesStrategy from jmh is not enough + // and errors with duplicates appear unless I force it to WARN only: + withType { + duplicatesStrategy = DuplicatesStrategy.WARN + } + + build { + dependsOn(jmhJarTask) + } +} + +dependencies { + implementation("org.openjdk.jmh:jmh-core:1.35") + implementation("io.projectreactor:reactor-core:${version("reactor")}") + implementation("io.reactivex.rxjava2:rxjava:2.1.9") + implementation("com.github.akarnokd:rxjava2-extensions:0.20.8") + + implementation("com.typesafe.akka:akka-actor_2.12:2.5.0") + implementation(project(":kotlinx-coroutines-core")) + implementation(project(":kotlinx-coroutines-debug")) + implementation(project(":kotlinx-coroutines-reactive")) + + // add jmh dependency on main + "jmhImplementation"(sourceSets.main.get().runtimeClasspath) +} diff --git a/benchmarks/pom.xml b/benchmarks/pom.xml deleted file mode 100644 index 7c2e4f4d7c..0000000000 --- a/benchmarks/pom.xml +++ /dev/null @@ -1,227 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - - - benchmarks - jar - - - 1.18 - 1.8 - default - benchmarks - 2.0.2 - - - - - org.openjdk.jmh - jmh-core - ${jmh.version} - - - com.typesafe.akka - akka-actor - ${akka.version} - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - tests - - - - - - typesafe - Typesafe Repository - http://repo.typesafe.com/typesafe/releases/ - - - - - - - - - kotlin-maven-plugin - org.jetbrains.kotlin - - ${kotlin.version} - - - - process-sources - generate-sources - - compile - - - - ${project.basedir}/src/main/kotlin - - - - - - process-test-sources - test-compile - - test-compile - - - - - - - - - org.codehaus.mojo - exec-maven-plugin - 1.2.1 - - - process-sources - - java - - - true - org.openjdk.jmh.generators.bytecode.JmhBytecodeGenerator - - ${project.basedir}/target/classes/ - ${project.basedir}/target/generated-sources/jmh/ - ${project.basedir}/target/classes/ - ${jmh.generator} - - - - - - - org.openjdk.jmh - jmh-generator-bytecode - ${jmh.version} - - - - - - - - org.codehaus.mojo - build-helper-maven-plugin - 1.8 - - - add-source - process-sources - - add-source - - - - ${project.basedir}/target/generated-sources/jmh - - - - - - - - - - org.apache.maven.plugins - maven-compiler-plugin - 3.1 - - ${javac.target} - ${javac.target} - ${javac.target} - -proc:none - - - - - - - org.apache.maven.plugins - maven-shade-plugin - 2.2 - - - package - - shade - - - ${uberjar.name} - - - org.openjdk.jmh.Main - - - - - - *:* - - META-INF/*.SF - META-INF/*.DSA - META-INF/*.RSA - - - - - - - - - - diff --git a/benchmarks/scripts/generate_plots_flow_flatten_merge.py b/benchmarks/scripts/generate_plots_flow_flatten_merge.py new file mode 100644 index 0000000000..c7f5ebb88b --- /dev/null +++ b/benchmarks/scripts/generate_plots_flow_flatten_merge.py @@ -0,0 +1,75 @@ +# To run this script run the command 'python3 scripts/generate_plots_flow_flatten_merge.py' in the /benchmarks folder + + +import pandas as pd +import sys +import locale +import matplotlib.pyplot as plt +from matplotlib.ticker import FormatStrFormatter + +input_file = "build/reports/jmh/results.csv" +output_file = "out/flow-flatten-merge.svg" +# Please change the value of this variable according to the FlowFlattenMergeBenchmarkKt.ELEMENTS +elements = 100000 +benchmark_name = "benchmarks.flow.FlowFlattenMergeBenchmark.flattenMerge" +csv_columns = ["Benchmark", "Score", "Unit", "Param: concurrency", "Param: flowsNumberStrategy"] +rename_columns = {"Benchmark": "benchmark", "Score" : "score", "Unit" : "unit", + "Param: concurrency" : "concurrency", "Param: flowsNumberStrategy" : "flows"} + +markers = ['.', 'v', '^', '1', '2', '8', 'p', 'P', 'x', 'D', 'd', 's'] +colours = ['red', 'gold', 'sienna', 'olivedrab', 'lightseagreen', 'navy', 'blue', 'm', 'crimson', 'yellow', 'orangered', 'slateblue', 'aqua', 'black', 'silver'] + +def next_colour(): + i = 0 + while True: + yield colours[i % len(colours)] + i += 1 + +def next_marker(): + i = 0 + while True: + yield markers[i % len(markers)] + i += 1 + +def draw(data, plt): + plt.xscale('log', basex=2) + plt.gca().xaxis.set_major_formatter(FormatStrFormatter('%0.f')) + plt.grid(linewidth='0.5', color='lightgray') + if data.unit.unique()[0] != "ops/s": + print("Unexpected time unit: " + data.unit.unique()[0]) + sys.exit(1) + plt.ylabel("elements / ms") + plt.xlabel('concurrency') + plt.xticks(data.concurrency.unique()) + + colour_gen = next_colour() + marker_gen = next_marker() + for flows in data.flows.unique(): + gen_colour = next(colour_gen) + gen_marker = next(marker_gen) + res = data[(data.flows == flows)] +# plt.plot(res.concurrency, res.score*elements/1000, label="flows={}".format(flows), color=gen_colour, marker=gen_marker) + plt.errorbar(x=res.concurrency, y=res.score*elements/1000, yerr=res.score_error*elements/1000, solid_capstyle='projecting', + label="flows={}".format(flows), capsize=4, color=gen_colour, linewidth=2.2) + +langlocale = locale.getdefaultlocale()[0] +locale.setlocale(locale.LC_ALL, langlocale) +dp = locale.localeconv()['decimal_point'] +if dp == ",": + csv_columns.append("Score Error (99,9%)") + rename_columns["Score Error (99,9%)"] = "score_error" +elif dp == ".": + csv_columns.append("Score Error (99.9%)") + rename_columns["Score Error (99.9%)"] = "score_error" +else: + print("Unexpected locale delimeter: " + dp) + sys.exit(1) +data = pd.read_csv(input_file, sep=",", decimal=dp) +data = data[csv_columns].rename(columns=rename_columns) +data = data[(data.benchmark == benchmark_name)] +plt.rcParams.update({'font.size': 15}) +plt.figure(figsize=(12.5, 10)) +draw(data, plt) +plt.legend(loc='upper center', borderpad=0, bbox_to_anchor=(0.5, 1.3), ncol=2, frameon=False, borderaxespad=2, prop={'size': 15}) +plt.tight_layout(pad=12, w_pad=2, h_pad=1) +plt.savefig(output_file, bbox_inches='tight') diff --git a/benchmarks/src/jmh/java/benchmarks/flow/scrabble/RxJava2PlaysScrabble.java b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/RxJava2PlaysScrabble.java new file mode 100644 index 0000000000..c5530f525b --- /dev/null +++ b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/RxJava2PlaysScrabble.java @@ -0,0 +1,159 @@ +package benchmarks.flow.scrabble; + +import benchmarks.flow.scrabble.IterableSpliterator; +import benchmarks.flow.scrabble.ShakespearePlaysScrabble; +import io.reactivex.Flowable; +import io.reactivex.Maybe; +import io.reactivex.Single; +import io.reactivex.functions.Function; +import org.openjdk.jmh.annotations.*; + +import java.util.*; +import java.util.Map.Entry; +import java.util.concurrent.TimeUnit; + +/** + * Shakespeare plays Scrabble with RxJava 2 Flowable. + * @author José + * @author akarnokd + */ +@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +public class RxJava2PlaysScrabble extends ShakespearePlaysScrabble { + + @Benchmark + @Override + public List>> play() throws Exception { + + // Function to compute the score of a given word + Function> scoreOfALetter = letter -> Flowable.just(letterScores[letter - 'a']) ; + + // score of the same letters in a word + Function, Flowable> letterScore = + entry -> + Flowable.just( + letterScores[entry.getKey() - 'a'] * + Integer.min( + (int)entry.getValue().get(), + scrabbleAvailableLetters[entry.getKey() - 'a'] + ) + ) ; + + Function> toIntegerFlowable = + string -> Flowable.fromIterable(IterableSpliterator.of(string.chars().boxed().spliterator())) ; + + // Histogram of the letters in a given word + Function>> histoOfLetters = + word -> toIntegerFlowable.apply(word) + .collect( + () -> new HashMap<>(), + (HashMap map, Integer value) -> + { + LongWrapper newValue = map.get(value) ; + if (newValue == null) { + newValue = () -> 0L ; + } + map.put(value, newValue.incAndSet()) ; + } + + ) ; + + // number of blanks for a given letter + Function, Flowable> blank = + entry -> + Flowable.just( + Long.max( + 0L, + entry.getValue().get() - + scrabbleAvailableLetters[entry.getKey() - 'a'] + ) + ) ; + + // number of blanks for a given word + Function> nBlanks = + word -> histoOfLetters.apply(word) + .flatMapPublisher(map -> Flowable.fromIterable(() -> map.entrySet().iterator())) + .flatMap(blank) + .reduce(Long::sum) ; + + + // can a word be written with 2 blanks? + Function> checkBlanks = + word -> nBlanks.apply(word) + .flatMap(l -> Maybe.just(l <= 2L)) ; + + // score taking blanks into account letterScore1 + Function> score2 = + word -> histoOfLetters.apply(word) + .flatMapPublisher(map -> Flowable.fromIterable(() -> map.entrySet().iterator())) + .flatMap(letterScore) + .reduce(Integer::sum) ; + + // Placing the word on the board + // Building the streams of first and last letters + Function> first3 = + word -> Flowable.fromIterable(IterableSpliterator.of(word.chars().boxed().limit(3).spliterator())) ; + Function> last3 = + word -> Flowable.fromIterable(IterableSpliterator.of(word.chars().boxed().skip(3).spliterator())) ; + + + // Stream to be maxed + Function> toBeMaxed = + word -> Flowable.just(first3.apply(word), last3.apply(word)) + .flatMap(observable -> observable) ; + + // Bonus for double letter + Function> bonusForDoubleLetter = + word -> toBeMaxed.apply(word) + .flatMap(scoreOfALetter) + .reduce(Integer::max) ; + + // score of the word put on the board + Function> score3 = + word -> + Maybe.merge(Arrays.asList( + score2.apply(word), + score2.apply(word), + bonusForDoubleLetter.apply(word), + bonusForDoubleLetter.apply(word), + Maybe.just(word.length() == 7 ? 50 : 0) + ) + ) + .reduce(Integer::sum) ; + + Function>, Single>>> buildHistoOnScore = + score -> Flowable.fromIterable(() -> shakespeareWords.iterator()) + .filter(scrabbleWords::contains) + .filter(word -> checkBlanks.apply(word).blockingGet()) + .collect( + () -> new TreeMap<>(Comparator.reverseOrder()), + (TreeMap> map, String word) -> { + Integer key = score.apply(word).blockingGet() ; + List list = map.get(key) ; + if (list == null) { + list = new ArrayList<>() ; + map.put(key, list) ; + } + list.add(word) ; + } + ) ; + + // best key / value pairs + List>> finalList2 = + buildHistoOnScore.apply(score3) + .flatMapPublisher(map -> Flowable.fromIterable(() -> map.entrySet().iterator())) + .take(3) + .collect( + () -> new ArrayList>>(), + (list, entry) -> { + list.add(entry) ; + } + ) + .blockingGet() ; + return finalList2 ; + } +} diff --git a/benchmarks/src/jmh/java/benchmarks/flow/scrabble/RxJava2PlaysScrabbleOpt.java b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/RxJava2PlaysScrabbleOpt.java new file mode 100644 index 0000000000..bf40759b59 --- /dev/null +++ b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/RxJava2PlaysScrabbleOpt.java @@ -0,0 +1,170 @@ +package benchmarks.flow.scrabble; + +import java.util.*; +import java.util.Map.Entry; +import java.util.concurrent.TimeUnit; + +import hu.akarnokd.rxjava2.math.MathFlowable; +import org.openjdk.jmh.annotations.*; +import benchmarks.flow.scrabble.optimizations.*; +import io.reactivex.*; +import io.reactivex.functions.Function; + +/** + * Shakespeare plays Scrabble with RxJava 2 Flowable optimized. + * @author José + * @author akarnokd + */ +@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +public class RxJava2PlaysScrabbleOpt extends ShakespearePlaysScrabble { + static Flowable chars(String word) { +// return Flowable.range(0, word.length()).map(i -> (int)word.charAt(i)); + return StringFlowable.characters(word); + } + + @Benchmark + @Override + public List>> play() throws Exception { + + // to compute the score of a given word + Function scoreOfALetter = letter -> letterScores[letter - 'a']; + + // score of the same letters in a word + Function, Integer> letterScore = + entry -> + letterScores[entry.getKey() - 'a'] * + Integer.min( + (int)entry.getValue().get(), + scrabbleAvailableLetters[entry.getKey() - 'a'] + ) + ; + + + Function> toIntegerFlowable = + string -> chars(string); + + Map>> histoCache = new HashMap<>(); + // Histogram of the letters in a given word + Function>> histoOfLetters = + word -> { Single> s = histoCache.get(word); + if (s == null) { + s = toIntegerFlowable.apply(word) + .collect( + () -> new HashMap<>(), + (HashMap map, Integer value) -> + { + MutableLong newValue = map.get(value) ; + if (newValue == null) { + newValue = new MutableLong(); + map.put(value, newValue); + } + newValue.incAndSet(); + } + + ); + histoCache.put(word, s); + } + return s; + }; + + // number of blanks for a given letter + Function, Long> blank = + entry -> + Long.max( + 0L, + entry.getValue().get() - + scrabbleAvailableLetters[entry.getKey() - 'a'] + ) + ; + + // number of blanks for a given word + Function> nBlanks = + word -> MathFlowable.sumLong( + histoOfLetters.apply(word).flattenAsFlowable( + map -> map.entrySet() + ) + .map(blank) + ) + ; + + + // can a word be written with 2 blanks? + Function> checkBlanks = + word -> nBlanks.apply(word) + .map(l -> l <= 2L) ; + + // score taking blanks into account letterScore1 + Function> score2 = + word -> MathFlowable.sumInt( + histoOfLetters.apply(word).flattenAsFlowable( + map -> map.entrySet() + ) + .map(letterScore) + ) ; + + // Placing the word on the board + // Building the streams of first and last letters + Function> first3 = + word -> chars(word).take(3) ; + Function> last3 = + word -> chars(word).skip(3) ; + + + // Stream to be maxed + Function> toBeMaxed = + word -> Flowable.concat(first3.apply(word), last3.apply(word)) + ; + + // Bonus for double letter + Function> bonusForDoubleLetter = + word -> MathFlowable.max(toBeMaxed.apply(word) + .map(scoreOfALetter) + ) ; + + // score of the word put on the board + Function> score3 = + word -> + MathFlowable.sumInt(Flowable.concat( + score2.apply(word), + bonusForDoubleLetter.apply(word) + )).map(v -> v * 2 + (word.length() == 7 ? 50 : 0)); + + Function>, Single>>> buildHistoOnScore = + score -> Flowable.fromIterable(shakespeareWords) + .filter(scrabbleWords::contains) + .filter(word -> checkBlanks.apply(word).blockingFirst()) + .collect( + () -> new TreeMap>(Comparator.reverseOrder()), + (TreeMap> map, String word) -> { + Integer key = score.apply(word).blockingFirst() ; + List list = map.get(key) ; + if (list == null) { + list = new ArrayList<>() ; + map.put(key, list) ; + } + list.add(word) ; + } + ) ; + + // best key / value pairs + List>> finalList2 = + buildHistoOnScore.apply(score3).flattenAsFlowable( + map -> map.entrySet() + ) + .take(3) + .collect( + () -> new ArrayList>>(), + (list, entry) -> { + list.add(entry) ; + } + ) + .blockingGet(); + + return finalList2 ; + } +} diff --git a/benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/FlowableCharSequence.java b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/FlowableCharSequence.java new file mode 100644 index 0000000000..63f347f409 --- /dev/null +++ b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/FlowableCharSequence.java @@ -0,0 +1,145 @@ +package benchmarks.flow.scrabble.optimizations; + +import io.reactivex.Flowable; +import io.reactivex.internal.fuseable.QueueFuseable; +import io.reactivex.internal.subscriptions.BasicQueueSubscription; +import io.reactivex.internal.subscriptions.SubscriptionHelper; +import io.reactivex.internal.util.BackpressureHelper; +import org.reactivestreams.Subscriber; + +final class FlowableCharSequence extends Flowable { + + final CharSequence string; + + FlowableCharSequence(CharSequence string) { + this.string = string; + } + + @Override + public void subscribeActual(Subscriber s) { + s.onSubscribe(new CharSequenceSubscription(s, string)); + } + + static final class CharSequenceSubscription + extends BasicQueueSubscription { + + private static final long serialVersionUID = -4593793201463047197L; + + final Subscriber downstream; + + final CharSequence string; + + final int end; + + int index; + + volatile boolean cancelled; + + CharSequenceSubscription(Subscriber downstream, CharSequence string) { + this.downstream = downstream; + this.string = string; + this.end = string.length(); + } + + @Override + public void cancel() { + cancelled = true; + } + + @Override + public void request(long n) { + if (SubscriptionHelper.validate(n)) { + if (BackpressureHelper.add(this, n) == 0) { + if (n == Long.MAX_VALUE) { + fastPath(); + } else { + slowPath(n); + } + } + } + } + + void fastPath() { + int e = end; + CharSequence s = string; + Subscriber a = downstream; + + for (int i = index; i != e; i++) { + if (cancelled) { + return; + } + + a.onNext((int)s.charAt(i)); + } + + if (!cancelled) { + a.onComplete(); + } + } + + void slowPath(long r) { + long e = 0L; + int i = index; + int f = end; + CharSequence s = string; + Subscriber a = downstream; + + for (;;) { + + while (e != r && i != f) { + if (cancelled) { + return; + } + + a.onNext((int)s.charAt(i)); + + i++; + e++; + } + + if (i == f) { + if (!cancelled) { + a.onComplete(); + } + return; + } + + r = get(); + if (e == r) { + index = i; + r = addAndGet(-e); + if (r == 0L) { + break; + } + e = 0L; + } + } + } + + @Override + public int requestFusion(int requestedMode) { + return requestedMode & QueueFuseable.SYNC; + } + + @Override + public Integer poll() { + int i = index; + if (i != end) { + index = i + 1; + return (int)string.charAt(i); + } + return null; + } + + @Override + public boolean isEmpty() { + return index == end; + } + + @Override + public void clear() { + index = end; + } + } + +} diff --git a/benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/FlowableSplit.java b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/FlowableSplit.java new file mode 100644 index 0000000000..020285cb32 --- /dev/null +++ b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/FlowableSplit.java @@ -0,0 +1,323 @@ +package benchmarks.flow.scrabble.optimizations; + +import io.reactivex.Flowable; +import io.reactivex.FlowableTransformer; +import io.reactivex.exceptions.Exceptions; +import io.reactivex.internal.fuseable.ConditionalSubscriber; +import io.reactivex.internal.fuseable.SimplePlainQueue; +import io.reactivex.internal.queue.SpscArrayQueue; +import io.reactivex.internal.subscriptions.SubscriptionHelper; +import io.reactivex.internal.util.BackpressureHelper; +import io.reactivex.plugins.RxJavaPlugins; +import org.reactivestreams.Publisher; +import org.reactivestreams.Subscriber; +import org.reactivestreams.Subscription; + +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicLong; +import java.util.regex.Pattern; + +final class FlowableSplit extends Flowable implements FlowableTransformer { + + final Publisher source; + + final Pattern pattern; + + final int bufferSize; + + FlowableSplit(Publisher source, Pattern pattern, int bufferSize) { + this.source = source; + this.pattern = pattern; + this.bufferSize = bufferSize; + } + + @Override + public Publisher apply(Flowable upstream) { + return new FlowableSplit(upstream, pattern, bufferSize); + } + + @Override + protected void subscribeActual(Subscriber s) { + source.subscribe(new SplitSubscriber(s, pattern, bufferSize)); + } + + static final class SplitSubscriber + extends AtomicInteger + implements ConditionalSubscriber, Subscription { + + static final String[] EMPTY = new String[0]; + + private static final long serialVersionUID = -5022617259701794064L; + + final Subscriber downstream; + + final Pattern pattern; + + final SimplePlainQueue queue; + + final AtomicLong requested; + + final int bufferSize; + + final int limit; + + Subscription upstream; + + volatile boolean cancelled; + + String leftOver; + + String[] current; + + int index; + + int produced; + + volatile boolean done; + Throwable error; + + int empty; + + SplitSubscriber(Subscriber downstream, Pattern pattern, int bufferSize) { + this.downstream = downstream; + this.pattern = pattern; + this.bufferSize = bufferSize; + this.limit = bufferSize - (bufferSize >> 2); + this.queue = new SpscArrayQueue(bufferSize); + this.requested = new AtomicLong(); + } + + @Override + public void request(long n) { + if (SubscriptionHelper.validate(n)) { + BackpressureHelper.add(requested, n); + drain(); + } + } + + @Override + public void cancel() { + cancelled = true; + upstream.cancel(); + + if (getAndIncrement() == 0) { + current = null; + queue.clear(); + } + } + + @Override + public void onSubscribe(Subscription s) { + if (SubscriptionHelper.validate(this.upstream, s)) { + this.upstream = s; + + downstream.onSubscribe(this); + + s.request(bufferSize); + } + } + + @Override + public void onNext(String t) { + if (!tryOnNext(t)) { + upstream.request(1); + } + } + + @Override + public boolean tryOnNext(String t) { + String lo = leftOver; + String[] a; + try { + if (lo == null || lo.isEmpty()) { + a = pattern.split(t, -1); + } else { + a = pattern.split(lo + t, -1); + } + } catch (Throwable ex) { + Exceptions.throwIfFatal(ex); + this.upstream.cancel(); + onError(ex); + return true; + } + + if (a.length == 0) { + leftOver = null; + return false; + } else + if (a.length == 1) { + leftOver = a[0]; + return false; + } + leftOver = a[a.length - 1]; + queue.offer(a); + drain(); + return true; + } + + @Override + public void onError(Throwable t) { + if (done) { + RxJavaPlugins.onError(t); + return; + } + String lo = leftOver; + if (lo != null && !lo.isEmpty()) { + leftOver = null; + queue.offer(new String[] { lo, null }); + } + error = t; + done = true; + drain(); + } + + @Override + public void onComplete() { + if (!done) { + done = true; + String lo = leftOver; + if (lo != null && !lo.isEmpty()) { + leftOver = null; + queue.offer(new String[] { lo, null }); + } + drain(); + } + } + + void drain() { + if (getAndIncrement() != 0) { + return; + } + + SimplePlainQueue q = queue; + + int missed = 1; + int consumed = produced; + String[] array = current; + int idx = index; + int emptyCount = empty; + + Subscriber a = downstream; + + for (;;) { + long r = requested.get(); + long e = 0; + + while (e != r) { + if (cancelled) { + current = null; + q.clear(); + return; + } + + boolean d = done; + + if (array == null) { + array = q.poll(); + if (array != null) { + current = array; + if (++consumed == limit) { + consumed = 0; + upstream.request(limit); + } + } + } + + boolean empty = array == null; + + if (d && empty) { + current = null; + Throwable ex = error; + if (ex != null) { + a.onError(ex); + } else { + a.onComplete(); + } + return; + } + + if (empty) { + break; + } + + if (array.length == idx + 1) { + array = null; + current = null; + idx = 0; + continue; + } + + String v = array[idx]; + + if (v.isEmpty()) { + emptyCount++; + idx++; + } else { + while (emptyCount != 0 && e != r) { + if (cancelled) { + current = null; + q.clear(); + return; + } + a.onNext(""); + e++; + emptyCount--; + } + + if (e != r && emptyCount == 0) { + a.onNext(v); + + e++; + idx++; + } + } + } + + if (e == r) { + if (cancelled) { + current = null; + q.clear(); + return; + } + + boolean d = done; + + if (array == null) { + array = q.poll(); + if (array != null) { + current = array; + if (++consumed == limit) { + consumed = 0; + upstream.request(limit); + } + } + } + + boolean empty = array == null; + + if (d && empty) { + current = null; + Throwable ex = error; + if (ex != null) { + a.onError(ex); + } else { + a.onComplete(); + } + return; + } + } + + if (e != 0L) { + BackpressureHelper.produced(requested, e); + } + + empty = emptyCount; + produced = consumed; + missed = addAndGet(-missed); + if (missed == 0) { + break; + } + } + } + } +} diff --git a/benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/StringFlowable.java b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/StringFlowable.java new file mode 100644 index 0000000000..ee40a5a331 --- /dev/null +++ b/benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/StringFlowable.java @@ -0,0 +1,78 @@ +package benchmarks.flow.scrabble.optimizations; + +import io.reactivex.Flowable; +import io.reactivex.FlowableTransformer; +import io.reactivex.internal.functions.ObjectHelper; +import io.reactivex.plugins.RxJavaPlugins; + +import java.util.regex.Pattern; + +public final class StringFlowable { + /** Utility class. */ + private StringFlowable() { + throw new IllegalStateException("No instances!"); + } + + /** + * Signals each character of the given string CharSequence as Integers. + * @param string the source of characters + * @return the new Flowable instance + */ + public static Flowable characters(CharSequence string) { + ObjectHelper.requireNonNull(string, "string is null"); + return RxJavaPlugins.onAssembly(new FlowableCharSequence(string)); + } + + /** + * Splits the input sequence of strings based on a pattern even across subsequent + * elements if needed. + * @param pattern the Rexexp pattern to split along + * @return the new FlowableTransformer instance + * + * @since 0.13.0 + */ + public static FlowableTransformer split(Pattern pattern) { + return split(pattern, Flowable.bufferSize()); + } + + /** + * Splits the input sequence of strings based on a pattern even across subsequent + * elements if needed. + * @param pattern the Rexexp pattern to split along + * @param bufferSize the number of items to prefetch from the upstream + * @return the new FlowableTransformer instance + * + * @since 0.13.0 + */ + public static FlowableTransformer split(Pattern pattern, int bufferSize) { + ObjectHelper.requireNonNull(pattern, "pattern is null"); + ObjectHelper.verifyPositive(bufferSize, "bufferSize"); + return new FlowableSplit(null, pattern, bufferSize); + } + + /** + * Splits the input sequence of strings based on a pattern even across subsequent + * elements if needed. + * @param pattern the Rexexp pattern to split along + * @return the new FlowableTransformer instance + * + * @since 0.13.0 + */ + public static FlowableTransformer split(String pattern) { + return split(pattern, Flowable.bufferSize()); + } + + /** + * Splits the input sequence of strings based on a pattern even across subsequent + * elements if needed. + * @param pattern the Rexexp pattern to split along + * @param bufferSize the number of items to prefetch from the upstream + * @return the new FlowableTransformer instance + * + * @since 0.13.0 + */ + public static FlowableTransformer split(String pattern, int bufferSize) { + return split(Pattern.compile(pattern), bufferSize); + } + +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkBenchmark.kt new file mode 100644 index 0000000000..51061cce72 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkBenchmark.kt @@ -0,0 +1,67 @@ +package benchmarks + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* +import kotlin.coroutines.* + +@Warmup(iterations = 7, time = 1) +@Measurement(iterations = 5, time = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +@Fork(1) +open class ChannelSinkBenchmark { + private val tl = ThreadLocal.withInitial({ 42 }) + private val tl2 = ThreadLocal.withInitial({ 239 }) + + private val unconfined = Dispatchers.Unconfined + private val unconfinedOneElement = Dispatchers.Unconfined + tl.asContextElement() + private val unconfinedTwoElements = Dispatchers.Unconfined + tl.asContextElement() + tl2.asContextElement() + + @Benchmark + fun channelPipeline(): Int = runBlocking { + run(unconfined) + } + + @Benchmark + fun channelPipelineOneThreadLocal(): Int = runBlocking { + run(unconfinedOneElement) + } + + @Benchmark + fun channelPipelineTwoThreadLocals(): Int = runBlocking { + run(unconfinedTwoElements) + } + + private suspend inline fun run(context: CoroutineContext): Int { + return Channel + .range(1, 10_000, context) + .filter(context) { it % 4 == 0 } + .fold(0) { a, b -> a + b } + } + + private fun Channel.Factory.range(start: Int, count: Int, context: CoroutineContext) = GlobalScope.produce(context) { + for (i in start until (start + count)) + send(i) + } + + // Migrated from deprecated operators, are good only for stressing channels + + private fun ReceiveChannel.filter(context: CoroutineContext = Dispatchers.Unconfined, predicate: suspend (E) -> Boolean): ReceiveChannel = + GlobalScope.produce(context, onCompletion = { cancel() }) { + for (e in this@filter) { + if (predicate(e)) send(e) + } + } + + private suspend inline fun ReceiveChannel.fold(initial: R, operation: (acc: R, E) -> R): R { + var accumulator = initial + consumeEach { + accumulator = operation(accumulator, it) + } + return accumulator + } +} + diff --git a/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkDepthBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkDepthBenchmark.kt new file mode 100644 index 0000000000..18a140f31b --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkDepthBenchmark.kt @@ -0,0 +1,87 @@ +package benchmarks + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* +import kotlin.coroutines.* + +@Warmup(iterations = 7, time = 1) +@Measurement(iterations = 5, time = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +@Fork(2) +open class ChannelSinkDepthBenchmark { + private val tl = ThreadLocal.withInitial({ 42 }) + + private val unconfinedOneElement = Dispatchers.Unconfined + tl.asContextElement() + + @Benchmark + fun depth1(): Int = runBlocking { + run(1, unconfinedOneElement) + } + + @Benchmark + fun depth10(): Int = runBlocking { + run(10, unconfinedOneElement) + } + + @Benchmark + fun depth100(): Int = runBlocking { + run(100, unconfinedOneElement) + } + + @Benchmark + fun depth1000(): Int = runBlocking { + run(1000, unconfinedOneElement) + } + + private suspend inline fun run(callTraceDepth: Int, context: CoroutineContext): Int { + return Channel + .range(1, 10_000, context) + .filter(callTraceDepth, context) { it % 4 == 0 } + .fold(0) { a, b -> a + b } + } + + private fun Channel.Factory.range(start: Int, count: Int, context: CoroutineContext) = + GlobalScope.produce(context) { + for (i in start until (start + count)) + send(i) + } + + // Migrated from deprecated operators, are good only for stressing channels + + private fun ReceiveChannel.filter( + callTraceDepth: Int, + context: CoroutineContext = Dispatchers.Unconfined, + predicate: suspend (Int) -> Boolean + ): ReceiveChannel = + GlobalScope.produce(context, onCompletion = { cancel() }) { + deeplyNestedFilter(this, callTraceDepth, predicate) + } + + private suspend fun ReceiveChannel.deeplyNestedFilter( + sink: ProducerScope, + depth: Int, + predicate: suspend (Int) -> Boolean + ) { + if (depth <= 1) { + for (e in this) { + if (predicate(e)) sink.send(e) + } + } else { + deeplyNestedFilter(sink, depth - 1, predicate) + require(true) // tail-call + } + } + + private suspend inline fun ReceiveChannel.fold(initial: R, operation: (acc: R, E) -> R): R { + var accumulator = initial + consumeEach { + accumulator = operation(accumulator, it) + } + return accumulator + } +} + diff --git a/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkNoAllocationsBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkNoAllocationsBenchmark.kt new file mode 100644 index 0000000000..8dec5f2a19 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/ChannelSinkNoAllocationsBenchmark.kt @@ -0,0 +1,33 @@ +package benchmarks + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* +import kotlin.coroutines.* + +@Warmup(iterations = 3, time = 1) +@Measurement(iterations = 5, time = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +@Fork(1) +open class ChannelSinkNoAllocationsBenchmark { + private val unconfined = Dispatchers.Unconfined + + @Benchmark + fun channelPipeline(): Int = runBlocking { + run(unconfined) + } + + private suspend inline fun run(context: CoroutineContext): Int { + var size = 0 + Channel.range(context).consumeEach { size++ } + return size + } + + private fun Channel.Factory.range(context: CoroutineContext) = GlobalScope.produce(context) { + for (i in 0 until 100_000) + send(Unit) // no allocations + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/ParametrizedDispatcherBase.kt b/benchmarks/src/jmh/kotlin/benchmarks/ParametrizedDispatcherBase.kt new file mode 100644 index 0000000000..da80958516 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/ParametrizedDispatcherBase.kt @@ -0,0 +1,42 @@ +package benchmarks + +import benchmarks.akka.CORES_COUNT +import kotlinx.coroutines.* +import kotlinx.coroutines.scheduling.* +import org.openjdk.jmh.annotations.Param +import org.openjdk.jmh.annotations.Setup +import org.openjdk.jmh.annotations.TearDown +import java.io.Closeable +import java.util.concurrent.* +import kotlin.coroutines.CoroutineContext + +/** + * Base class to use different [CoroutineContext] in benchmarks via [Param] in inheritors. + * Currently allowed values are "fjp" for [CommonPool] and ftp_n for [ThreadPoolDispatcher] with n threads. + */ +abstract class ParametrizedDispatcherBase : CoroutineScope { + + abstract var dispatcher: String + override lateinit var coroutineContext: CoroutineContext + private var closeable: Closeable? = null + + @Setup + open fun setup() { + coroutineContext = when { + dispatcher == "fjp" -> ForkJoinPool.commonPool().asCoroutineDispatcher() + dispatcher == "scheduler" -> { + Dispatchers.Default + } + dispatcher.startsWith("ftp") -> { + newFixedThreadPoolContext(dispatcher.substring(4).toInt(), dispatcher).also { closeable = it } + } + else -> error("Unexpected dispatcher: $dispatcher") + } + } + + @TearDown + fun tearDown() { + closeable?.close() + } + +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/SequentialSemaphoreBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/SequentialSemaphoreBenchmark.kt new file mode 100644 index 0000000000..ed1a4ed89a --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/SequentialSemaphoreBenchmark.kt @@ -0,0 +1,39 @@ +package benchmarks + +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.TimeUnit +import kotlin.test.* + +@Warmup(iterations = 5, time = 1) +@Measurement(iterations = 10, time = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +@Fork(1) +open class SequentialSemaphoreAsMutexBenchmark { + val s = Semaphore(1) + + @Benchmark + fun benchmark() : Unit = runBlocking { + val s = Semaphore(permits = 1, acquiredPermits = 1) + var step = 0 + launch(Dispatchers.Unconfined) { + repeat(N) { + assertEquals(it * 2, step) + step++ + s.acquire() + } + } + repeat(N) { + assertEquals(it * 2 + 1, step) + step++ + s.release() + } + } +} + +fun main() = SequentialSemaphoreAsMutexBenchmark().benchmark() + +private val N = 1_000_000 \ No newline at end of file diff --git a/benchmarks/src/jmh/kotlin/benchmarks/akka/PingPongAkkaBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/akka/PingPongAkkaBenchmark.kt new file mode 100644 index 0000000000..30e590b0a6 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/akka/PingPongAkkaBenchmark.kt @@ -0,0 +1,113 @@ +package benchmarks.akka + +import akka.actor.* +import com.typesafe.config.* +import org.openjdk.jmh.annotations.* +import scala.concurrent.* +import scala.concurrent.duration.* +import java.util.concurrent.* + +const val N_MESSAGES = 100_000 + +data class Ball(val count: Int) +class Start +class Stop + +/* + * Benchmark (dispatcher) Mode Cnt Score Error Units + * PingPongAkkaBenchmark.coresCountPingPongs default-dispatcher avgt 10 277.501 ± 38.583 ms/op + * PingPongAkkaBenchmark.coresCountPingPongs single-thread-dispatcher avgt 10 196.192 ± 9.889 ms/op + * + * PingPongAkkaBenchmark.singlePingPong default-dispatcher avgt 10 173.742 ± 41.984 ms/op + * PingPongAkkaBenchmark.singlePingPong single-thread-dispatcher avgt 10 24.181 ± 0.730 ms/op + */ +//@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +//@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +//@Fork(value = 2) +//@BenchmarkMode(Mode.AverageTime) +//@OutputTimeUnit(TimeUnit.MILLISECONDS) +//@State(Scope.Benchmark) +open class PingPongAkkaBenchmark { + + lateinit var system: ActorSystem + + @Param("default-dispatcher", "single-thread-dispatcher") + var dispatcher: String = "akka.actor.default-dispatcher" + + @Setup + fun setup() { + system = ActorSystem.create("PingPong", ConfigFactory.parseString(""" + akka.actor.single-thread-dispatcher { + type = Dispatcher + executor = "thread-pool-executor" + thread-pool-executor { + fixed-pool-size = 1 + } + throughput = 1 + } + """.trimIndent() + )) + } + + @TearDown + fun tearDown() { + Await.ready(system.terminate(), Duration.Inf()) + } + +// @Benchmark + fun singlePingPong() { + runPingPongs(1) + } + +// @Benchmark + fun coresCountPingPongs() { + runPingPongs(Runtime.getRuntime().availableProcessors()) + } + + private fun runPingPongs(pairsCount: Int) { + val latch = CountDownLatch(pairsCount) + repeat(pairsCount) { + val pongRef = system.actorOf(Props.create(PongActorAkka::class.java) + .withDispatcher("akka.actor.$dispatcher")) + val pingRef = system.actorOf(Props.create(PingActorAkka::class.java, pongRef, latch) + .withDispatcher("akka.actor.$dispatcher")) + pingRef.tell(Start(), ActorRef.noSender()) + } + + latch.await() + } + + class PingActorAkka(val pongRef: ActorRef, val stopLatch: CountDownLatch) : UntypedAbstractActor() { + override fun onReceive(msg: Any?) { + when (msg) { + is Start -> { + pongRef.tell(Ball(0), self) + } + is Ball -> { + pongRef.tell(Ball(count = msg.count + 1), self) + } + is Stop -> { + stopLatch.countDown() + context.stop(self) + } + else -> unhandled(msg) + } + } + } + + class PongActorAkka : UntypedAbstractActor() { + override fun onReceive(msg: Any?) { + when (msg) { + is Ball -> { + if (msg.count >= N_MESSAGES) { + sender.tell(Stop(), self) + context.stop(self) + } else { + sender.tell(Ball(msg.count + 1), self) + } + } + else -> unhandled(msg) + } + } + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/akka/StatefulActorAkkaBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/akka/StatefulActorAkkaBenchmark.kt new file mode 100644 index 0000000000..d7be75a56c --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/akka/StatefulActorAkkaBenchmark.kt @@ -0,0 +1,166 @@ +package benchmarks.akka + +import akka.actor.* +import com.typesafe.config.* +import org.openjdk.jmh.annotations.* +import scala.concurrent.* +import scala.concurrent.duration.* +import java.util.concurrent.* + +const val ROUNDS = 10_000 +const val STATE_SIZE = 1024 +val CORES_COUNT = Runtime.getRuntime().availableProcessors() + +/* + * Benchmarks following computation pattern: + * N actors, each has independent state (coefficients), receives numbers and answers with product and + * N requestors, which randomly send requests. N roundtrips over every requestor are measured + * + * Benchmark (dispatcher) Mode Cnt Score Error Units + * StatefulActorAkkaBenchmark.multipleComputationsMultipleRequestors default-dispatcher avgt 14 72.568 ± 10.620 ms/op + * StatefulActorAkkaBenchmark.multipleComputationsMultipleRequestors single-thread-dispatcher avgt 14 70.198 ± 3.594 ms/op + * + * StatefulActorAkkaBenchmark.multipleComputationsSingleRequestor default-dispatcher avgt 14 36.737 ± 3.589 ms/op + * StatefulActorAkkaBenchmark.multipleComputationsSingleRequestor single-thread-dispatcher avgt 14 9.050 ± 0.385 ms/op + * + * StatefulActorAkkaBenchmark.singleComputationMultipleRequestors default-dispatcher avgt 14 446.563 ± 85.577 ms/op + * StatefulActorAkkaBenchmark.singleComputationMultipleRequestors single-thread-dispatcher avgt 14 70.250 ± 3.104 ms/op + * + * StatefulActorAkkaBenchmark.singleComputationSingleRequestor default-dispatcher avgt 14 39.964 ± 2.343 ms/op + * StatefulActorAkkaBenchmark.singleComputationSingleRequestor single-thread-dispatcher avgt 14 10.214 ± 2.152 ms/op + */ +//@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +//@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +//@Fork(value = 2) +//@BenchmarkMode(Mode.AverageTime) +//@OutputTimeUnit(TimeUnit.MILLISECONDS) +//@State(Scope.Benchmark) +open class StatefulActorAkkaBenchmark { + + lateinit var system: ActorSystem + + @Param("default-dispatcher", "single-thread-dispatcher") + var dispatcher: String = "akka.actor.default-dispatcher" + + @Setup + fun setup() { + // TODO extract it to common AkkaBase if new benchmark will appear + system = ActorSystem.create("StatefulActors", ConfigFactory.parseString(""" + akka.actor.single-thread-dispatcher { + type = Dispatcher + executor = "thread-pool-executor" + thread-pool-executor { + fixed-pool-size = 1 + } + throughput = 1 + } + """.trimIndent() + )) + } + + @TearDown + fun tearDown() { + Await.ready(system.terminate(), Duration.Inf()) + } + +// @Benchmark + fun singleComputationSingleRequestor() { + run(1, 1) + } + +// @Benchmark + fun singleComputationMultipleRequestors() { + run(1, CORES_COUNT) + } + +// @Benchmark + fun multipleComputationsSingleRequestor() { + run(CORES_COUNT, 1) + } + +// @Benchmark + fun multipleComputationsMultipleRequestors() { + run(CORES_COUNT, CORES_COUNT) + } + + private fun run(computationActors: Int, requestorActors: Int) { + val stopLatch = CountDownLatch(requestorActors) + /* + * For complex setups Akka creates actors slowly, + * so first start message may become dead letter (and freeze benchmark) + */ + val initLatch = CountDownLatch(computationActors + requestorActors) + val computations = createComputationActors(initLatch, computationActors) + val requestors = createRequestorActors(requestorActors, computations, initLatch, stopLatch) + + initLatch.await() + for (requestor in requestors) { + requestor.tell(1L, ActorRef.noSender()) + } + + stopLatch.await() + computations.forEach { it.tell(Stop(), ActorRef.noSender()) } + } + + private fun createRequestorActors(requestorActors: Int, computations: List, initLatch: CountDownLatch, stopLatch: CountDownLatch): List { + return (0 until requestorActors).map { + system.actorOf(Props.create(RequestorActor::class.java, computations, initLatch, stopLatch) + .withDispatcher("akka.actor.$dispatcher")) + } + } + + private fun createComputationActors(initLatch: CountDownLatch, count: Int): List { + return (0 until count).map { + system.actorOf(Props.create( + ComputationActor::class.java, + LongArray(STATE_SIZE) { ThreadLocalRandom.current().nextLong(0, 100) }, initLatch) + .withDispatcher("akka.actor.$dispatcher")) + } + } + + class RequestorActor(val computations: List, val initLatch: CountDownLatch, + val stopLatch: CountDownLatch) : UntypedAbstractActor() { + private var received = 0 + + override fun onReceive(message: Any?) { + when (message) { + is Long -> { + if (++received >= ROUNDS) { + context.stop(self) + stopLatch.countDown() + } else { + computations[ThreadLocalRandom.current().nextInt(0, computations.size)] + .tell(ThreadLocalRandom.current().nextLong(), self) + } + } + else -> unhandled(message) + } + } + + override fun preStart() { + initLatch.countDown() + } + } + + class ComputationActor(val coefficients: LongArray, val initLatch: CountDownLatch) : UntypedAbstractActor() { + override fun onReceive(message: Any?) { + when (message) { + is Long -> { + var result = 0L + for (coefficient in coefficients) { + result += coefficient * message + } + sender.tell(result, self) + } + is Stop -> { + context.stop(self) + } + else -> unhandled(message) + } + } + + override fun preStart() { + initLatch.countDown() + } + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/debug/DebugSequenceOverheadBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/debug/DebugSequenceOverheadBenchmark.kt new file mode 100644 index 0000000000..ec37840514 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/debug/DebugSequenceOverheadBenchmark.kt @@ -0,0 +1,86 @@ +package benchmarks.debug + +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* +import org.openjdk.jmh.annotations.* +import org.openjdk.jmh.annotations.State +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicInteger + +/** + * The benchmark is supposed to show the DebugProbes overhead for a non-concurrent sequence builder. + * The code is actually part of the IDEA codebase, originally reported here: https://github.com/Kotlin/kotlinx.coroutines/issues/3527 + */ +@Warmup(iterations = 5, time = 1) +@Measurement(iterations = 5, time = 1) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class DebugSequenceOverheadBenchmark { + + private fun generateRecursiveSequence( + initialSequence: Sequence, + children: (Node) -> Sequence + ): Sequence { + return sequence { + val initialIterator = initialSequence.iterator() + if (!initialIterator.hasNext()) { + return@sequence + } + val visited = HashSet() + val sequences = ArrayDeque>() + sequences.addLast(initialIterator.asSequence()) + while (sequences.isNotEmpty()) { + val currentSequence = sequences.removeFirst() + for (node in currentSequence) { + if (visited.add(node)) { + yield(node) + sequences.addLast(children(node)) + } + } + } + } + } + + @Param("true", "false") + var withDebugger = false + + @Setup + fun setup() { + DebugProbes.sanitizeStackTraces = false + DebugProbes.enableCreationStackTraces = false + if (withDebugger) { + DebugProbes.install() + } + } + + @TearDown + fun tearDown() { + if (withDebugger) { + DebugProbes.uninstall() + } + } + + // Shows the overhead of sequence builder with debugger enabled + @Benchmark + fun runSequenceSingleThread(): Int = runBlocking { + generateRecursiveSequence((1..100).asSequence()) { + (1..it).asSequence() + }.sum() + } + + // Shows the overhead of sequence builder with debugger enabled and debugger is concurrently stressed out + @Benchmark + fun runSequenceMultipleThreads(): Int = runBlocking { + val result = AtomicInteger(0) + repeat(Runtime.getRuntime().availableProcessors()) { + launch(Dispatchers.Default) { + result.addAndGet(generateRecursiveSequence((1..100).asSequence()) { + (1..it).asSequence() + }.sum()) + } + } + result.get() + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/CombineFlowsBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/CombineFlowsBenchmark.kt new file mode 100644 index 0000000000..207b2453ab --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/CombineFlowsBenchmark.kt @@ -0,0 +1,30 @@ +package benchmarks.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.Throughput) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class CombineFlowsBenchmark { + + @Param("10", "100", "1000") + private var size = 10 + + @Benchmark + fun combine() = runBlocking { + combine((1 until size).map { flowOf(it) }) { a -> a}.collect() + } + + @Benchmark + fun combineTransform() = runBlocking { + val list = (1 until size).map { flowOf(it) }.toList() + combineTransform((1 until size).map { flowOf(it) }) { emit(it) }.collect() + } +} + diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/CombineTwoFlowsBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/CombineTwoFlowsBenchmark.kt new file mode 100644 index 0000000000..dd385353fc --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/CombineTwoFlowsBenchmark.kt @@ -0,0 +1,43 @@ +package benchmarks.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.flow.internal.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.Throughput) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class CombineTwoFlowsBenchmark { + + @Param("100", "100000", "1000000") + private var size = 100000 + + @Benchmark + fun combinePlain() = runBlocking { + val flow = (1 until size.toLong()).asFlow() + flow.combine(flow) { a, b -> a + b }.collect() + } + + @Benchmark + fun combineTransform() = runBlocking { + val flow = (1 until size.toLong()).asFlow() + flow.combineTransform(flow) { a, b -> emit(a + b) }.collect() + } + + @Benchmark + fun combineVararg() = runBlocking { + val flow = (1 until size.toLong()).asFlow() + combine(listOf(flow, flow)) { arr -> arr[0] + arr[1] }.collect() + } + + @Benchmark + fun combineTransformVararg() = runBlocking { + val flow = (1 until size.toLong()).asFlow() + combineTransform(listOf(flow, flow)) { arr -> emit(arr[0] + arr[1]) }.collect() + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/FlatMapMergeBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/FlatMapMergeBenchmark.kt new file mode 100644 index 0000000000..772e1bb11a --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/FlatMapMergeBenchmark.kt @@ -0,0 +1,43 @@ +package benchmarks.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class FlatMapMergeBenchmark { + + // Note: tests only absence of contention on downstream + + @Param("10", "100", "1000") + private var iterations = 100 + + @Benchmark + fun flatMapUnsafe() = runBlocking { + benchmarks.flow.scrabble.flow { + repeat(iterations) { emit(it) } + }.flatMapMerge { value -> + flowOf(value) + }.collect { + if (it == -1) error("") + } + } + + @Benchmark + fun flatMapSafe() = runBlocking { + kotlinx.coroutines.flow.flow { + repeat(iterations) { emit(it) } + }.flatMapMerge { value -> + flowOf(value) + }.collect { + if (it == -1) error("") + } + } + +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/FlowFlattenMergeBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/FlowFlattenMergeBenchmark.kt new file mode 100644 index 0000000000..f89ed9d1fd --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/FlowFlattenMergeBenchmark.kt @@ -0,0 +1,59 @@ +package benchmarks.flow + +import benchmarks.common.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.TimeUnit + +/** + * Benchmark to measure performance of [kotlinx.coroutines.flow.FlowKt.flattenMerge]. + * In addition to that, it can be considered as a macro benchmark for the [kotlinx.coroutines.sync.Semaphore] + */ +@Warmup(iterations = 5, time = 1) +@Measurement(iterations = 5, time = 1) +@BenchmarkMode(Mode.Throughput) +@OutputTimeUnit(TimeUnit.SECONDS) +@State(Scope.Benchmark) +@Fork(1) +open class FlowFlattenMergeBenchmark { + @Param + private var flowsNumberStrategy: FlowsNumberStrategy = FlowsNumberStrategy.`10xConcurrency flows` + + @Param("1", "2", "4", "8") + private var concurrency: Int = 0 + + private lateinit var flow: Flow> + + @Setup + fun setup() { + val n = flowsNumberStrategy.get(concurrency) + val flowElementsToProcess = ELEMENTS / n + + flow = (1..n).asFlow().map { + flow { + repeat(flowElementsToProcess) { + doGeomDistrWork(WORK) + emit(it) + } + } + } + } + + @Benchmark + fun flattenMerge() = runBlocking(Dispatchers.Default) { + flow.flattenMerge(concurrency = concurrency).collect() + } +} + +enum class FlowsNumberStrategy(val get: (concurrency: Int) -> Int) { + `10xConcurrency flows`({ concurrency -> concurrency * 10 }), + `1xConcurrency flows`({ it }), + `100 flows`({ 100 }), + `500 flows`({ 500 }) +} + +// If you change this variable please be sure that you change variable elements in the generate_plots_flow_flatten_merge.py +// python script as well +private const val ELEMENTS = 100_000 +private const val WORK = 100 diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/NumbersBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/NumbersBenchmark.kt new file mode 100644 index 0000000000..697b876cac --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/NumbersBenchmark.kt @@ -0,0 +1,102 @@ +package benchmarks.flow + +import benchmarks.flow.scrabble.flow +import io.reactivex.* +import io.reactivex.functions.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.TimeUnit +import java.util.concurrent.Callable + +@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class NumbersBenchmark { + + companion object { + private const val primes = 100 + private const val natural = 1000L + } + + private fun numbers(limit: Long = Long.MAX_VALUE) = flow { + for (i in 2L..limit) emit(i) + } + + private fun primesFlow(): Flow = flow { + var source = numbers() + while (true) { + val next = source.take(1).single() + emit(next) + source = source.filter { it % next != 0L } + } + } + + private fun rxNumbers() = + Flowable.generate(Callable { 1L }, BiFunction, Long> { state, emitter -> + val newState = state + 1 + emitter.onNext(newState) + newState + }) + + private fun generateRxPrimes(): Flowable = Flowable.generate(Callable { rxNumbers() }, + BiFunction, Emitter, Flowable> { state, emitter -> + // Not the most fair comparison, but here we go + val prime = state.firstElement().blockingGet() + emitter.onNext(prime) + state.filter { it % prime != 0L } + }) + + @Benchmark + fun primes() = runBlocking { + primesFlow().take(primes).count() + } + + @Benchmark + fun primesRx() = generateRxPrimes().take(primes.toLong()).count().blockingGet() + + @Benchmark + fun zip() = runBlocking { + val numbers = numbers(natural) + val first = numbers + .filter { it % 2L != 0L } + .map { it * it } + val second = numbers + .filter { it % 2L == 0L } + .map { it * it } + first.zip(second) { v1, v2 -> v1 + v2 }.filter { it % 3 == 0L }.count() + } + + @Benchmark + fun zipRx() { + val numbers = rxNumbers().take(natural) + val first = numbers + .filter { it % 2L != 0L } + .map { it * it } + val second = numbers + .filter { it % 2L == 0L } + .map { it * it } + first.zipWith(second, { v1, v2 -> v1 + v2 }).filter { it % 3 == 0L }.count() + .blockingGet() + } + + @Benchmark + fun transformations(): Int = runBlocking { + numbers(natural) + .filter { it % 2L != 0L } + .map { it * it } + .filter { (it + 1) % 3 == 0L }.count() + } + + @Benchmark + fun transformationsRx(): Long { + return rxNumbers().take(natural) + .filter { it % 2L != 0L } + .map { it * it } + .filter { (it + 1) % 3 == 0L }.count() + .blockingGet() + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/SafeFlowBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/SafeFlowBenchmark.kt new file mode 100644 index 0000000000..f1b0fafb01 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/SafeFlowBenchmark.kt @@ -0,0 +1,35 @@ +package benchmarks.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* +import benchmarks.flow.scrabble.flow as unsafeFlow +import kotlinx.coroutines.flow.flow as safeFlow + +@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class SafeFlowBenchmark { + + private fun numbersSafe() = safeFlow { + for (i in 2L..1000L) emit(i) + } + + private fun numbersUnsafe() = unsafeFlow { + for (i in 2L..1000L) emit(i) + } + + @Benchmark + fun safeNumbers(): Int = runBlocking { + numbersSafe().count() + } + + @Benchmark + fun unsafeNumbers(): Int = runBlocking { + numbersUnsafe().count() + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/TakeBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/TakeBenchmark.kt new file mode 100644 index 0000000000..b22e5a231a --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/TakeBenchmark.kt @@ -0,0 +1,134 @@ +package benchmarks.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.TimeUnit +import java.util.concurrent.CancellationException +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import benchmarks.flow.scrabble.flow as unsafeFlow + +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class TakeBenchmark { + @Param("1", "10", "100", "1000") + private var size: Int = 0 + + private suspend inline fun Flow.consume() = + filter { it % 2L != 0L } + .map { it * it }.count() + + @Benchmark + fun baseline() = runBlocking { + (0L until size).asFlow().consume() + } + + @Benchmark + fun originalTake() = runBlocking { + (0L..Long.MAX_VALUE).asFlow().originalTake(size).consume() + } + + @Benchmark + fun fastPathTake() = runBlocking { + (0L..Long.MAX_VALUE).asFlow().fastPathTake(size).consume() + } + + @Benchmark + fun mergedStateMachine() = runBlocking { + (0L..Long.MAX_VALUE).asFlow().mergedStateMachineTake(size).consume() + } + + internal class StacklessCancellationException() : CancellationException() { + override fun fillInStackTrace(): Throwable = this + } + + public fun Flow.originalTake(count: Int): Flow { + return unsafeFlow { + var consumed = 0 + try { + collect { value -> + emit(value) + if (++consumed == count) { + throw StacklessCancellationException() + } + } + } catch (e: StacklessCancellationException) { + // Nothing, bail out + } + } + } + + private suspend fun FlowCollector.emitAbort(value: T) { + emit(value) + throw StacklessCancellationException() + } + + public fun Flow.fastPathTake(count: Int): Flow { + return unsafeFlow { + var consumed = 0 + try { + collect { value -> + if (++consumed < count) { + return@collect emit(value) + } else { + return@collect emitAbort(value) + } + } + } catch (e: StacklessCancellationException) { + // Nothing, bail out + } + } + } + + + public fun Flow.mergedStateMachineTake(count: Int): Flow { + return unsafeFlow() { + try { + val takeCollector = FlowTakeCollector(count, this) + collect(takeCollector) + } catch (e: StacklessCancellationException) { + // Nothing, bail out + } + } + } + + + private class FlowTakeCollector( + private val count: Int, + downstream: FlowCollector + ) : FlowCollector, Continuation { + private var consumed = 0 + // Workaround for KT-30991 + private val emitFun = run { + val suspendFun: suspend (T) -> Unit = { downstream.emit(it) } + suspendFun as Function2, Any?> + } + + private var caller: Continuation? = null // lateinit + + override val context: CoroutineContext + get() = caller?.context ?: EmptyCoroutineContext + + override fun resumeWith(result: Result) { + val completion = caller!! + if (++consumed == count) completion.resumeWith(Result.failure(StacklessCancellationException())) + else completion.resumeWith(Result.success(Unit)) + } + + override suspend fun emit(value: T) = suspendCoroutineUninterceptedOrReturn sc@{ + // Invoke it in non-suspending way + caller = it + val result = emitFun.invoke(value, this) + if (result !== COROUTINE_SUSPENDED) { + if (++consumed == count) throw StacklessCancellationException() + else return@sc Unit + } + COROUTINE_SUSPENDED + } + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/FlowPlaysScrabbleBase.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/FlowPlaysScrabbleBase.kt new file mode 100644 index 0000000000..fc2923d9b9 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/FlowPlaysScrabbleBase.kt @@ -0,0 +1,129 @@ +package benchmarks.flow.scrabble + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.openjdk.jmh.annotations.* +import java.lang.Long.max +import java.util.* +import java.util.concurrent.TimeUnit +import kotlin.math.* + +@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class FlowPlaysScrabbleBase : ShakespearePlaysScrabble() { + + @Benchmark + public override fun play(): List>> { + val scoreOfALetter = { letter: Int -> flowOf(letterScores[letter - 'a'.toInt()]) } + + val letterScore = { entry: Map.Entry -> + flowOf( + letterScores[entry.key - 'a'.toInt()] * Integer.min( + entry.value.get().toInt(), + scrabbleAvailableLetters[entry.key - 'a'.toInt()] + ) + ) + } + + val toIntegerStream = { string: String -> + IterableSpliterator.of(string.chars().boxed().spliterator()).asFlow() + } + + val histoOfLetters = { word: String -> + flow { + emit(toIntegerStream(word).fold(HashMap()) { accumulator, value -> + var newValue: LongWrapper? = accumulator[value] + if (newValue == null) { + newValue = LongWrapper.zero() + } + accumulator[value] = newValue.incAndSet() + accumulator + }) + } + } + + val blank = { entry: Map.Entry -> + flowOf(max(0L, entry.value.get() - scrabbleAvailableLetters[entry.key - 'a'.toInt()])) + } + + val nBlanks = { word: String -> + flow { + emit(histoOfLetters(word) + .flatMapConcat { map -> map.entries.iterator().asFlow() } + .flatMapConcat({ blank(it) }) + .reduce { a, b -> a + b }) + } + } + + val checkBlanks = { word: String -> + nBlanks(word).flatMapConcat { l -> flowOf(l <= 2L) } + } + + val score2 = { word: String -> + flow { + emit(histoOfLetters(word) + .flatMapConcat { map -> map.entries.iterator().asFlow() } + .flatMapConcat { letterScore(it) } + .reduce { a, b -> a + b }) + } + } + + val first3 = { word: String -> + IterableSpliterator.of(word.chars().boxed().limit(3).spliterator()).asFlow() + } + + val last3 = { word: String -> + IterableSpliterator.of(word.chars().boxed().skip(3).spliterator()).asFlow() + } + + val toBeMaxed = { word: String -> flowOf(first3(word), last3(word)).flattenConcat() } + + // Bonus for double letter + val bonusForDoubleLetter = { word: String -> + flow { + emit(toBeMaxed(word) + .flatMapConcat { scoreOfALetter(it) } + .reduce { a, b -> max(a, b) }) + } + } + + val score3 = { word: String -> + flow { + emit(flowOf( + score2(word), score2(word), + bonusForDoubleLetter(word), + bonusForDoubleLetter(word), + flowOf(if (word.length == 7) 50 else 0) + ).flattenConcat().reduce { a, b -> a + b }) + } + } + + val buildHistoOnScore: (((String) -> Flow) -> Flow>>) = { score -> + flow { + emit(shakespeareWords.asFlow() + .filter({ scrabbleWords.contains(it) && checkBlanks(it).single() }) + .fold(TreeMap>(Collections.reverseOrder())) { acc, value -> + val key = score(value).single() + var list = acc[key] as MutableList? + if (list == null) { + list = ArrayList() + acc[key] = list + } + list.add(value) + acc + }) + } + } + + return runBlocking { + buildHistoOnScore(score3) + .flatMapConcat { map -> map.entries.iterator().asFlow() } + .take(3) + .toList() + } + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/FlowPlaysScrabbleOpt.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/FlowPlaysScrabbleOpt.kt new file mode 100644 index 0000000000..16190f89ca --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/FlowPlaysScrabbleOpt.kt @@ -0,0 +1,190 @@ +package benchmarks.flow.scrabble + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.flow.Flow +import org.openjdk.jmh.annotations.* +import java.util.* +import java.util.concurrent.* +import kotlin.math.* + +@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class FlowPlaysScrabbleOpt : ShakespearePlaysScrabble() { + + @Benchmark + public override fun play(): List>> { + val histoOfLetters = { word: String -> + flow { + emit(word.asFlow().fold(HashMap()) { accumulator, value -> + var newValue: MutableLong? = accumulator[value] + if (newValue == null) { + newValue = MutableLong() + accumulator[value] = newValue + } + newValue.incAndSet() + accumulator + }) + } + } + + val blank = { entry: Map.Entry -> + max(0L, entry.value.get() - scrabbleAvailableLetters[entry.key - 'a'.toInt()]) + } + + val nBlanks = { word: String -> + flow { + emit(histoOfLetters(word) + .flatMapConcatIterable { it.entries } + .map({ blank(it) }) + .sum() + ) + } + } + + val checkBlanks = { word: String -> + nBlanks(word).map { it <= 2L } + } + + val letterScore = { entry: Map.Entry -> + letterScores[entry.key - 'a'.toInt()] * Integer.min( + entry.value.get().toInt(), + scrabbleAvailableLetters[entry.key - 'a'.toInt()] + ) + } + + val score2 = { word: String -> + flow { + emit(histoOfLetters(word) + .flatMapConcatIterable { it.entries } + .map { letterScore(it) } + .sum()) + } + } + + val first3 = { word: String -> word.asFlow(endIndex = 3) } + val last3 = { word: String -> word.asFlow(startIndex = 3) } + val toBeMaxed = { word: String -> concat(first3(word), last3(word)) } + + val bonusForDoubleLetter = { word: String -> + flow { + emit(toBeMaxed(word) + .map { letterScores[it.toInt() - 'a'.toInt()] } + .max()) + } + } + + val score3 = { word: String -> + flow { + val sum = score2(word).single() + bonusForDoubleLetter(word).single() + emit(sum * 2 + if (word.length == 7) 50 else 0) + } + } + + val buildHistoOnScore: (((String) -> Flow) -> Flow>>) = { score -> + flow { + emit(shakespeareWords.asFlow() + .filter({ scrabbleWords.contains(it) && checkBlanks(it).single() }) + .fold(TreeMap>(Collections.reverseOrder())) { acc, value -> + val key = score(value).single() + var list = acc[key] as MutableList? + if (list == null) { + list = ArrayList() + acc[key] = list + } + list.add(value) + acc + }) + } + } + + return runBlocking { + buildHistoOnScore(score3) + .flatMapConcatIterable { it.entries } + .take(3) + .toList() + } + } +} + +public fun String.asFlow() = flow { + forEach { + emit(it.toInt()) + } +} + +public fun String.asFlow(startIndex: Int = 0, endIndex: Int = length) = + StringByCharFlow(this, startIndex, endIndex.coerceAtMost(this.length)) + +public suspend inline fun Flow.sum(): Int { + val collector = object : FlowCollector { + public var sum = 0 + + override suspend fun emit(value: Int) { + sum += value + } + } + collect(collector) + return collector.sum +} + +public suspend inline fun Flow.max(): Int { + val collector = object : FlowCollector { + public var max = 0 + + override suspend fun emit(value: Int) { + max = max(max, value) + } + } + collect(collector) + return collector.max +} + +@JvmName("longSum") +public suspend inline fun Flow.sum(): Long { + val collector = object : FlowCollector { + public var sum = 0L + + override suspend fun emit(value: Long) { + sum += value + } + } + collect(collector) + return collector.sum +} + +public class StringByCharFlow(private val source: String, private val startIndex: Int, private val endIndex: Int): Flow { + override suspend fun collect(collector: FlowCollector) { + for (i in startIndex until endIndex) collector.emit(source[i]) + } +} + +public fun concat(first: Flow, second: Flow): Flow = flow { + first.collect { value -> + return@collect emit(value) + } + + second.collect { value -> + return@collect emit(value) + } +} + +public fun Flow.flatMapConcatIterable(transformer: (T) -> Iterable): Flow = flow { + collect { value -> + transformer(value).forEach { r -> + emit(r) + } + } +} + +public inline fun flow(@BuilderInference crossinline block: suspend FlowCollector.() -> Unit): Flow { + return object : Flow { + override suspend fun collect(collector: FlowCollector) { + collector.block() + } + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/IterableSpliterator.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/IterableSpliterator.kt new file mode 100644 index 0000000000..cce343cd6f --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/IterableSpliterator.kt @@ -0,0 +1,8 @@ +package benchmarks.flow.scrabble + +import java.util.* + +object IterableSpliterator { + @JvmStatic + public fun of(spliterator: Spliterator): Iterable = Iterable { Spliterators.iterator(spliterator) } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/README.md b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/README.md new file mode 100644 index 0000000000..3ea3dd7b24 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/README.md @@ -0,0 +1,46 @@ +## Reactive scrabble benchmarks + +This package contains reactive scrabble benchmarks. + +Reactive Scrabble benchmarks were originally developed by José Paumard and are [available](https://github.com/JosePaumard/jdk8-stream-rx-comparison-reloaded) under Apache 2.0, +Flow version is adaptation of this work. +All Rx and Reactive benchmarks are based on (or copied from) [David Karnok work](https://github.com/akarnokd/akarnokd-misc). + +### Benchmark classes + +The package (split into two sourcesets, `kotlin` and `java`), contains different benchmarks with different purposes + + * `RxJava2PlaysScrabble` and `RxJava2PlaysScrabbleOpt` are copied as is and used for comparison. The infrastructure (e.g. `FlowableSplit`) + is copied from `akarnokd-misc` in order for the latter benchmark to work. + This is the original benchmark for `RxJava`. + * `ReactorPlaysScrabble` is an original benchmark for `Reactor`, but rewritten into Kotlin. + It is disabled by default and had the only purpose -- verify that Kotlin version performs as the original Java version + (which could have been different due to lambdas translation, implicit boxing, etc.). It is disabled because + it has almost no difference compared to `RxJava` benchmark. + * `FlowPlaysScrabbleBase` is a scrabble benchmark rewritten on top of the `Flow` API without using any optimizations or tricky internals. + * `FlowPlaysScrabbleOpt` is an optimized version of benchmark that follows the same guidelines as `RxJava2PlaysScrabbleOpt`: it still is + lazy, reactive and uses only `Flow` abstraction. + * `SequencePlaysScrabble` is a version of benchmark built on top of `Sequence` without suspensions, used as a lower bound. + * `SaneFlowPlaysScrabble` is a `SequencePlaysScrabble` that produces `Flow`. + This benchmark is not identical (in terms of functions pipelining) to `FlowPlaysScrabbleOpt`, but rather is used as a lower bound of `Flow` performance + on this particular task. + +### Results + +Benchmark results for throughput mode, Java `1.8.0_172` +running on `Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz` +under `Darwin Kernel Version 18.7.0`. +Full command: `java -jar benchmarks.jar -f 2 -jvmArgsPrepend "-XX:+UseParallelGC" '.*Scrabble.*'`. + +``` +Benchmark Mode Cnt Score Error Units + +FlowPlaysScrabbleBase.play avgt 14 62.480 ± 1.018 ms/op +FlowPlaysScrabbleOpt.play avgt 14 13.958 ± 0.278 ms/op + +RxJava2PlaysScrabble.play avgt 14 88.456 ± 0.950 ms/op +RxJava2PlaysScrabbleOpt.play avgt 14 23.653 ± 0.379 ms/op + +SaneFlowPlaysScrabble.play avgt 14 13.608 ± 0.332 ms/op +SequencePlaysScrabble.play avgt 14 9.824 ± 0.190 ms/op +``` diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ReactorPlaysScrabble.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ReactorPlaysScrabble.kt new file mode 100644 index 0000000000..3ab22da8c2 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ReactorPlaysScrabble.kt @@ -0,0 +1,140 @@ +package benchmarks.flow.scrabble + +import reactor.core.publisher.* +import java.lang.Long.* +import java.util.* +import java.util.function.Function + +/*@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark)*/ +open class ReactorPlaysScrabble : ShakespearePlaysScrabble() { + +// @Benchmark + public override fun play(): List>> { + val scoreOfALetter = Function> { letter -> Flux.just(letterScores[letter - 'a'.toInt()]) } + + val letterScore = Function, Flux> { entry -> + Flux.just( + letterScores[entry.key - 'a'.toInt()] * Integer.min( + entry.value.get().toInt(), + scrabbleAvailableLetters[entry.key - 'a'.toInt()] + ) + ) + } + + val toIntegerStream = Function> { string -> + Flux.fromIterable(IterableSpliterator.of(string.chars().boxed().spliterator())) + } + + val histoOfLetters = Function>> { word -> + Flux.from(toIntegerStream.apply(word) + .collect( + { HashMap() }, + { map: HashMap, value: Int -> + var newValue: LongWrapper? = map[value] + if (newValue == null) { + newValue = LongWrapper.zero() + } + map[value] = newValue.incAndSet() + } + + )) + } + + val blank = Function, Flux> { entry -> + Flux.just(max(0L, entry.value.get() - scrabbleAvailableLetters[entry.key - 'a'.toInt()])) + } + + val nBlanks = Function> { word -> + Flux.from(histoOfLetters.apply(word) + .flatMap> { map -> Flux.fromIterable>(Iterable { map.entries.iterator() }) } + .flatMap(blank) + .reduce { a, b -> sum(a, b) }) + } + + val checkBlanks = Function> { word -> + nBlanks.apply(word) + .flatMap { l -> Flux.just(l <= 2L) } + } + + + val score2 = Function> { word -> + Flux.from(histoOfLetters.apply(word) + .flatMap> { map -> Flux.fromIterable>(Iterable { map.entries.iterator() }) } + .flatMap(letterScore) + .reduce { a, b -> Integer.sum(a, b) }) + + } + + val first3 = Function> { word -> Flux.fromIterable( + IterableSpliterator.of( + word.chars().boxed().limit(3).spliterator() + ) + ) } + val last3 = Function> { word -> Flux.fromIterable( + IterableSpliterator.of( + word.chars().boxed().skip(3).spliterator() + ) + ) } + + val toBeMaxed = Function> { word -> + Flux.just(first3.apply(word), last3.apply(word)) + .flatMap { Stream -> Stream } + } + + // Bonus for double letter + val bonusForDoubleLetter = Function> { word -> + Flux.from(toBeMaxed.apply(word) + .flatMap(scoreOfALetter) + .reduce { a, b -> Integer.max(a, b) } + ) + } + + val score3 = Function> { word -> + Flux.from(Flux.just( + score2.apply(word), + score2.apply(word), + bonusForDoubleLetter.apply(word), + bonusForDoubleLetter.apply(word), + Flux.just(if (word.length == 7) 50 else 0) + ) + .flatMap { Stream -> Stream } + .reduce { a, b -> Integer.sum(a, b) }) + } + + val buildHistoOnScore = Function>, Flux>>> { score -> + Flux.from(Flux.fromIterable(Iterable { shakespeareWords.iterator() }) + .filter( { scrabbleWords.contains(it) }) + .filter({ word -> checkBlanks.apply(word).toIterable().iterator().next() }) + .collect( + { TreeMap>(Collections.reverseOrder()) }, + { map: TreeMap>, word: String -> + val key = score.apply(word).toIterable().iterator().next() + var list = map[key] as MutableList? + if (list == null) { + list = ArrayList() + map[key] = list + } + list.add(word) + } + )) + } + + val finalList2 = Flux.from>>>(buildHistoOnScore.apply(score3) + .flatMap>> { map -> Flux.fromIterable>>(Iterable { map.entries.iterator() }) } + .take(3) + .collect>>>( + { ArrayList() }, + { list, entry -> list.add(entry) } + ) + ).toIterable().iterator().next() + + return finalList2 + } + +} + diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/SaneFlowPlaysScrabble.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/SaneFlowPlaysScrabble.kt new file mode 100644 index 0000000000..596a2ad39d --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/SaneFlowPlaysScrabble.kt @@ -0,0 +1,96 @@ +package benchmarks.flow.scrabble + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.openjdk.jmh.annotations.* +import java.lang.Long.* +import java.util.* +import java.util.concurrent.TimeUnit + +@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class SaneFlowPlaysScrabble : ShakespearePlaysScrabble() { + + @Benchmark + public override fun play(): List>> { + val score3: suspend (String) -> Int = { word: String -> + val sum = score2(word) + bonusForDoubleLetter(word) + sum * 2 + if (word.length == 7) 50 else 0 + } + + val buildHistoOnScore: ((suspend (String) -> Int) -> Flow>>) = { score -> + flow { + emit(shakespeareWords.asFlow() + .filter({ scrabbleWords.contains(it) && checkBlanks(it) }) + .fold(TreeMap>(Collections.reverseOrder())) { acc, value -> + val key = score(value) + var list = acc[key] as MutableList? + if (list == null) { + list = ArrayList() + acc[key] = list + } + list.add(value) + acc + }) + } + } + + return runBlocking { + buildHistoOnScore(score3) + .flatMapConcatIterable { it.entries } + .take(3) + .toList() + } + } + + private suspend inline fun score2(word: String): Int { + return buildHistogram(word) + .map { it.letterScore() } + .sum() + } + + private suspend inline fun bonusForDoubleLetter(word: String): Int { + return toBeMaxed(word) + .map { letterScores[it - 'a'.toInt()] } + .max() + } + + private fun Map.Entry.letterScore(): Int = letterScores[key - 'a'.toInt()] * Integer.min( + value.get().toInt(), + scrabbleAvailableLetters[key - 'a'.toInt()]) + + private fun toBeMaxed(word: String) = concat(word.asSequence(), word.asSequence(endIndex = 3)) + + private suspend inline fun checkBlanks(word: String) = numBlanks(word) <= 2L + + private suspend fun numBlanks(word: String): Long { + return buildHistogram(word) + .map { blanks(it) } + .sum() + } + + private fun blanks(entry: Map.Entry): Long = + max(0L, entry.value.get() - scrabbleAvailableLetters[entry.key - 'a'.toInt()]) + + private suspend inline fun buildHistogram(word: String): HashMap { + return word.asSequence().fold(HashMap()) { accumulator, value -> + var newValue: MutableLong? = accumulator[value] + if (newValue == null) { + newValue = MutableLong() + accumulator[value] = newValue + } + newValue.incAndSet() + accumulator + } + } + + private fun String.asSequence(startIndex: Int = 0, endIndex: Int = length) = flow { + for (i in startIndex until endIndex.coerceAtMost(length)) { + emit(get(i).toInt()) + } + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/SequencePlaysScrabble.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/SequencePlaysScrabble.kt new file mode 100644 index 0000000000..eca01dfa4a --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/SequencePlaysScrabble.kt @@ -0,0 +1,99 @@ +package benchmarks.flow.scrabble + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.openjdk.jmh.annotations.* +import java.lang.Long.* +import java.util.* +import java.util.concurrent.TimeUnit + +@Warmup(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 7, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class SequencePlaysScrabble : ShakespearePlaysScrabble() { + + @Benchmark + public override fun play(): List>> { + val score2: (String) -> Int = { word: String -> + buildHistogram(word) + .map { it.letterScore() } + .sum() + } + + val bonusForDoubleLetter: (String) -> Int = { word: String -> + toBeMaxed(word) + .map { letterScores[it - 'a'.toInt()] } + .maxOrNull()!! + } + + val score3: (String) -> Int = { word: String -> + val sum = score2(word) + bonusForDoubleLetter(word) + sum * 2 + if (word.length == 7) 50 else 0 + } + + val buildHistoOnScore: (((String) -> Int) -> Flow>>) = { score -> + flow { + emit(shakespeareWords.asSequence() + .filter({ scrabbleWords.contains(it) && checkBlanks(it) }) + .fold(TreeMap>(Collections.reverseOrder())) { acc, value -> + val key = score(value) + var list = acc[key] as MutableList? + if (list == null) { + list = ArrayList() + acc[key] = list + } + list.add(value) + acc + }) + } + } + + return runBlocking { + buildHistoOnScore(score3) + .flatMapConcatIterable { it.entries } + .take(3) + .toList() + } + } + + private fun Map.Entry.letterScore(): Int = letterScores[key - 'a'.toInt()] * Integer.min( + value.get().toInt(), + scrabbleAvailableLetters[key - 'a'.toInt()]) + + private fun toBeMaxed(word: String) = word.asSequence(startIndex = 3) + word.asSequence(endIndex = 3) + + private fun checkBlanks(word: String) = numBlanks(word) <= 2L + + private fun numBlanks(word: String): Long { + return buildHistogram(word) + .map { blanks(it) } + .sum() + } + + private fun blanks(entry: Map.Entry): Long = + max(0L, entry.value.get() - scrabbleAvailableLetters[entry.key - 'a'.toInt()]) + + private fun buildHistogram(word: String): HashMap { + return word.asSequence().fold(HashMap()) { accumulator, value -> + var newValue: MutableLong? = accumulator[value] + if (newValue == null) { + newValue = MutableLong() + accumulator[value] = newValue + } + newValue.incAndSet() + accumulator + } + } + + private fun String.asSequence(startIndex: Int = 0, endIndex: Int = length) = object : Sequence { + override fun iterator(): Iterator = object : Iterator { + private val _endIndex = endIndex.coerceAtMost(length) + private var currentIndex = startIndex + override fun hasNext(): Boolean = currentIndex < _endIndex + override fun next(): Int = get(currentIndex++).toInt() + } + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ShakespearePlaysScrabble.kt b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ShakespearePlaysScrabble.kt new file mode 100644 index 0000000000..6990c0c5cd --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ShakespearePlaysScrabble.kt @@ -0,0 +1,79 @@ +package benchmarks.flow.scrabble + +import org.openjdk.jmh.annotations.* +import java.io.* +import java.util.stream.* +import java.util.zip.* + +@State(Scope.Benchmark) +abstract class ShakespearePlaysScrabble { + @Throws(Exception::class) + abstract fun play(): List>> + + public class MutableLong { + var value: Long = 0 + fun get(): Long { + return value + } + + fun incAndSet(): MutableLong { + value++ + return this + } + + fun add(other: MutableLong): MutableLong { + value += other.value + return this + } + } + + public interface LongWrapper { + fun get(): Long + + fun incAndSet(): LongWrapper { + return object : LongWrapper { + override fun get(): Long = this@LongWrapper.get() + 1L + } + } + + fun add(other: LongWrapper): LongWrapper { + return object : LongWrapper { + override fun get(): Long = this@LongWrapper.get() + other.get() + } + } + + companion object { + fun zero(): LongWrapper { + return object : LongWrapper { + override fun get(): Long = 0L + } + } + } + } + + @JvmField + public val letterScores: IntArray = intArrayOf(1, 3, 3, 2, 1, 4, 2, 4, 1, 8, 5, 1, 3, 1, 1, 3, 10, 1, 1, 1, 1, 4, 4, 8, 4, 10) + + @JvmField + public val scrabbleAvailableLetters: IntArray = + intArrayOf(9, 2, 2, 1, 12, 2, 3, 2, 9, 1, 1, 4, 2, 6, 8, 2, 1, 6, 4, 6, 4, 2, 2, 1, 2, 1) + + @JvmField + public val scrabbleWords: Set = readResource("ospd.txt.gz") + + @JvmField + public val shakespeareWords: Set = readResource("words.shakespeare.txt.gz") + + private fun readResource(path: String) = + BufferedReader(InputStreamReader(GZIPInputStream(this.javaClass.classLoader.getResourceAsStream(path)))).lines() + .map { it.lowercase() }.collect(Collectors.toSet()) + + init { + val expected = listOf(120 to listOf("jezebel", "quickly"), + 118 to listOf("zephyrs"), 116 to listOf("equinox")) + val actual = play().map { it.key to it.value } + if (expected != actual) { + error("Incorrect benchmark, output: $actual") + } + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/scheduler/DispatchersContextSwitchBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/DispatchersContextSwitchBenchmark.kt new file mode 100644 index 0000000000..fba05101e1 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/DispatchersContextSwitchBenchmark.kt @@ -0,0 +1,69 @@ +package benchmarks.scheduler + +import benchmarks.akka.* +import kotlinx.coroutines.* +import org.openjdk.jmh.annotations.* +import org.openjdk.jmh.annotations.State +import java.lang.Thread.* +import java.util.concurrent.* +import kotlin.concurrent.* +import kotlin.coroutines.* + +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Thread) +open class DispatchersContextSwitchBenchmark { + private val nCoroutines = 10000 + private val delayTimeMs = 1L + private val nRepeatDelay = 10 + + private val fjp = ForkJoinPool.commonPool().asCoroutineDispatcher() + private val ftp = Executors.newFixedThreadPool(CORES_COUNT - 1).asCoroutineDispatcher() + + @TearDown + fun teardown() { + ftp.close() + (ftp.executor as ExecutorService).awaitTermination(1, TimeUnit.SECONDS) + } + + @Benchmark + fun coroutinesIoDispatcher() = runBenchmark(Dispatchers.IO) + + @Benchmark + fun coroutinesDefaultDispatcher() = runBenchmark(Dispatchers.Default) + + @Benchmark + fun coroutinesFjpDispatcher() = runBenchmark(fjp) + + @Benchmark + fun coroutinesFtpDispatcher() = runBenchmark(ftp) + + @Benchmark + fun coroutinesBlockingDispatcher() = runBenchmark(EmptyCoroutineContext) + + @Benchmark + fun threads() { + val threads = List(nCoroutines) { + thread(start = true) { + repeat(nRepeatDelay) { + sleep(delayTimeMs) + } + } + } + threads.forEach { it.join() } + } + + private fun runBenchmark(dispatcher: CoroutineContext) = runBlocking { + repeat(nCoroutines) { + launch(dispatcher) { + repeat(nRepeatDelay) { + delay(delayTimeMs) + } + } + } + } +} + diff --git a/benchmarks/src/jmh/kotlin/benchmarks/scheduler/ForkJoinBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/ForkJoinBenchmark.kt new file mode 100644 index 0000000000..5572858c06 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/ForkJoinBenchmark.kt @@ -0,0 +1,163 @@ +package benchmarks.scheduler + +import benchmarks.* +import kotlinx.coroutines.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +/* + * Comparison of fork-join tasks using specific FJP API and classic [async] jobs. + * FJP job is organized in perfectly balanced binary tree, every leaf node computes + * FPU-heavy sum over its data and intermediate nodes sum results. + * + * Fine-grained batch size (8192 * 1024 tasks, 128 in sequential batch) + * ForkJoinBenchmark.asyncExperimental avgt 10 681.512 ± 32.069 ms/op + * ForkJoinBenchmark.asyncFjp avgt 10 845.386 ± 73.204 ms/op + * ForkJoinBenchmark.fjpRecursiveTask avgt 10 692.120 ± 26.224 ms/op + * ForkJoinBenchmark.fjpTask avgt 10 791.087 ± 66.544 ms/op + * + * Too small tasks (8192 * 1024 tasks, 128 batch, 16 in sequential batch) + * Benchmark Mode Cnt Score Error Units + * ForkJoinBenchmark.asyncExperimental avgt 10 1273.271 ± 190.372 ms/op + * ForkJoinBenchmark.asyncFjp avgt 10 1406.102 ± 216.793 ms/op + * ForkJoinBenchmark.fjpRecursiveTask avgt 10 849.941 ± 141.254 ms/op + * ForkJoinBenchmark.fjpTask avgt 10 831.554 ± 57.276 ms/op + */ +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 2) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class ForkJoinBenchmark : ParametrizedDispatcherBase() { + + companion object { + /* + * Change task size to control global granularity of benchmark + * Change batch size to control affinity/work stealing/scheduling overhead effects + */ + const val TASK_SIZE = 8192 * 1024 + const val BATCH_SIZE = 32 * 8192 + } + + lateinit var coefficients: LongArray + override var dispatcher: String = "scheduler" + + @Setup + override fun setup() { + super.setup() + coefficients = LongArray(TASK_SIZE) { ThreadLocalRandom.current().nextLong(0, 1024 * 1024) } + } + + @Benchmark + fun asyncFjp() = runBlocking { + CoroutineScope(ForkJoinPool.commonPool().asCoroutineDispatcher()).startAsync(coefficients, 0, coefficients.size).await() + } + + @Benchmark + fun asyncExperimental() = runBlocking { + startAsync(coefficients, 0, coefficients.size).await() + } + + @Benchmark + fun fjpRecursiveTask(): Double { + val task = RecursiveAction(coefficients, 0, coefficients.size) + return ForkJoinPool.commonPool().submit(task).join() + } + + @Benchmark + fun fjpTask(): Double { + val task = Task(coefficients, 0, coefficients.size) + return ForkJoinPool.commonPool().submit(task).join() + } + + suspend fun CoroutineScope.startAsync(coefficients: LongArray, start: Int, end: Int): Deferred = async { + if (end - start <= BATCH_SIZE) { + compute(coefficients, start, end) + } else { + val first = startAsync(coefficients, start, start + (end - start) / 2) + val second = startAsync(coefficients, start + (end - start) / 2, end) + first.await() + second.await() + } + } + + class Task(val coefficients: LongArray, val start: Int, val end: Int) : RecursiveTask() { + override fun compute(): Double { + if (end - start <= BATCH_SIZE) { + return compute(coefficients, start, end) + } + + val first = Task(coefficients, start, start + (end - start) / 2).fork() + val second = Task(coefficients, start + (end - start) / 2, end).fork() + + var result = 0.0 + result += first.join() + result += second.join() + return result + } + + private fun compute(coefficients: LongArray, start: Int, end: Int): Double { + var result = 0.0 + for (i in start until end) { + result += Math.sin(Math.pow(coefficients[i].toDouble(), 1.1)) + 1e-8 + } + + return result + } + } + + class RecursiveAction(val coefficients: LongArray, val start: Int, val end: Int, @Volatile var result: Double = 0.0, + parent: RecursiveAction? = null) : CountedCompleter(parent) { + + private var first: ForkJoinTask? = null + private var second: ForkJoinTask? = null + + override fun getRawResult(): Double { + return result + } + + override fun setRawResult(t: Double) { + result = t + } + + override fun compute() { + if (end - start <= BATCH_SIZE) { + rawResult = compute(coefficients, start, end) + } else { + pendingCount = 2 + // One may fork only once here and executing second task here with looping over firstComplete to be even more efficient + first = RecursiveAction( + coefficients, + start, + start + (end - start) / 2, + parent = this + ).fork() + second = RecursiveAction( + coefficients, + start + (end - start) / 2, + end, + parent = this + ).fork() + } + + tryComplete() + } + + override fun onCompletion(caller: CountedCompleter<*>?) { + if (caller !== this) { + rawResult = first!!.rawResult + second!!.rawResult + } + super.onCompletion(caller) + } + } +} + + +private fun compute(coefficients: LongArray, start: Int, end: Int): Double { + var result = 0.0 + for (i in start until end) { + result += Math.sin(Math.pow(coefficients[i].toDouble(), 1.1)) + 1e-8 + } + + return result +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/scheduler/LaunchBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/LaunchBenchmark.kt new file mode 100644 index 0000000000..18c58cdab9 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/LaunchBenchmark.kt @@ -0,0 +1,52 @@ +package benchmarks.scheduler + +import benchmarks.* +import kotlinx.coroutines.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +/* + * Benchmark to measure scheduling overhead in comparison with FJP. + * LaunchBenchmark.massiveLaunch experimental avgt 30 328.662 ± 52.789 us/op + * LaunchBenchmark.massiveLaunch fjp avgt 30 179.762 ± 3.931 us/op + */ +@Warmup(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 2) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class LaunchBenchmark : ParametrizedDispatcherBase() { + + @Param("scheduler", "fjp") + override var dispatcher: String = "fjp" + + private val jobsToLaunch = 100 + private val submitters = 4 + + private val allLaunched = CyclicBarrier(submitters) + private val stopBarrier = CyclicBarrier(submitters + 1) + + @Benchmark + fun massiveLaunch() { + repeat(submitters) { + launch { + // Wait until all cores are occupied + allLaunched.await() + allLaunched.reset() + + (1..jobsToLaunch).map { + launch { + // do nothing + } + }.map { it.join() } + + stopBarrier.await() + } + } + + stopBarrier.await() + stopBarrier.reset() + } + +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/scheduler/StatefulAwaitsBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/StatefulAwaitsBenchmark.kt new file mode 100644 index 0000000000..47d4cb2653 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/StatefulAwaitsBenchmark.kt @@ -0,0 +1,117 @@ +package benchmarks.scheduler + +import benchmarks.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +/* + * Benchmark which launches multiple async jobs each with either own private or global shared state, + * each job iterates over its state multiple times and suspends after every iteration. + * Benchmark is intended to indicate pros and cons of coroutines affinity (assuming threads are rarely migrated) + * and comparison with single thread and ForkJoinPool + * + * Benchmark (dispatcher) (jobsCount) Mode Cnt Score Error Units + * StatefulAsyncBenchmark.dependentStateAsync fjp 1 avgt 10 42.147 ± 11.563 us/op + * StatefulAsyncBenchmark.dependentStateAsync fjp 8 avgt 10 111.053 ± 40.097 us/op + * StatefulAsyncBenchmark.dependentStateAsync fjp 16 avgt 10 239.992 ± 52.839 us/op + * StatefulAsyncBenchmark.dependentStateAsync ftp_1 1 avgt 10 32.851 ± 11.385 us/op + * StatefulAsyncBenchmark.dependentStateAsync ftp_1 8 avgt 10 51.692 ± 0.961 us/op + * StatefulAsyncBenchmark.dependentStateAsync ftp_1 16 avgt 10 101.511 ± 3.060 us/op + * StatefulAsyncBenchmark.dependentStateAsync ftp_8 1 avgt 10 31.549 ± 1.014 us/op + * StatefulAsyncBenchmark.dependentStateAsync ftp_8 8 avgt 10 103.990 ± 1.588 us/op + * StatefulAsyncBenchmark.dependentStateAsync ftp_8 16 avgt 10 156.384 ± 2.914 us/op + * + * StatefulAsyncBenchmark.independentStateAsync fjp 1 avgt 10 32.503 ± 0.721 us/op + * StatefulAsyncBenchmark.independentStateAsync fjp 8 avgt 10 73.000 ± 1.686 us/op + * StatefulAsyncBenchmark.independentStateAsync fjp 16 avgt 10 98.629 ± 7.541 us/op + * StatefulAsyncBenchmark.independentStateAsync ftp_1 1 avgt 10 26.111 ± 0.814 us/op + * StatefulAsyncBenchmark.independentStateAsync ftp_1 8 avgt 10 54.644 ± 1.261 us/op + * StatefulAsyncBenchmark.independentStateAsync ftp_1 16 avgt 10 104.871 ± 1.599 us/op + * StatefulAsyncBenchmark.independentStateAsync ftp_8 1 avgt 10 31.929 ± 0.698 us/op + * StatefulAsyncBenchmark.independentStateAsync ftp_8 8 avgt 10 108.959 ± 1.029 us/op + * StatefulAsyncBenchmark.independentStateAsync ftp_8 16 avgt 10 159.593 ± 5.262 us/op + * + */ +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 2) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +@Suppress("DEPRECATION_ERROR") +open class StatefulAsyncBenchmark : ParametrizedDispatcherBase() { + + private val stateSize = 2048 + private val jobSuspensions = 2 // multiplicative factor for throughput + + // it's useful to have more jobs than cores so run queue always will be non empty + @Param("1", "8", "16") + var jobsCount = 1 + + @Param("fjp", "ftp_1", "dispatcher") + override var dispatcher: String = "fjp" + + @Volatile + private var state: Array? = null + + @Setup + override fun setup() { + super.setup() + state = Array(Runtime.getRuntime().availableProcessors() * 4) { LongArray(stateSize) { ThreadLocalRandom.current().nextLong() } } + } + + @Benchmark + fun independentStateAsync() = runBlocking { + val broadcastChannel = BroadcastChannel(1) + val subscriptionChannel = Channel(jobsCount) + val jobs= (0 until jobsCount).map { launchJob(it, broadcastChannel, subscriptionChannel) }.toList() + + repeat(jobsCount) { + subscriptionChannel.receive() // await all jobs to start + } + + // Fire barrier to start execution + broadcastChannel.send(1) + jobs.forEach { it.await() } + } + + @Benchmark + fun dependentStateAsync() = runBlocking { + val broadcastChannel = BroadcastChannel(1) + val subscriptionChannel = Channel(jobsCount) + val jobs= (0 until jobsCount).map { launchJob(0, broadcastChannel, subscriptionChannel) }.toList() + + repeat(jobsCount) { + subscriptionChannel.receive() // await all jobs to start + } + + // Fire barrier to start execution + broadcastChannel.send(1) + jobs.forEach { it.await() } + } + + private fun launchJob( + stateNum: Int, + channel: BroadcastChannel, + subscriptionChannel: Channel + ): Deferred = + async { + val subscription = channel.openSubscription() + subscriptionChannel.send(1) + subscription.receive() + + var sum = 0L + repeat(jobSuspensions) { + val arr = state!![stateNum] + for (i in 0 until stateSize) { + sum += arr[i] + + } + + yield() + } + sum + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/ConcurrentStatefulActorBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/ConcurrentStatefulActorBenchmark.kt new file mode 100644 index 0000000000..76a674fa33 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/ConcurrentStatefulActorBenchmark.kt @@ -0,0 +1,138 @@ +package benchmarks.scheduler.actors + +import benchmarks.* +import benchmarks.akka.* +import benchmarks.scheduler.actors.StatefulActorBenchmark.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +/* + * Noisy benchmarks useful to measure scheduling fairness and migration of affinity-sensitive tasks. + * + * Benchmark: single actor fans out requests to all (#cores count) computation actors and then ping pongs each in loop. + * Fair benchmark expects that every computation actor will receive exactly N messages, unfair expects N * cores messages received in total. + * + * Benchmark (dispatcher) (stateSize) Mode Cnt Score Error Units + * ConcurrentStatefulActorBenchmark.multipleComputationsFair fjp 1024 avgt 5 215.439 ± 29.685 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsFair ftp_1 1024 avgt 5 85.374 ± 4.477 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsFair ftp_8 1024 avgt 5 418.510 ± 46.906 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsFair experimental 1024 avgt 5 165.250 ± 20.309 ms/op + * + * ConcurrentStatefulActorBenchmark.multipleComputationsFair fjp 8192 avgt 5 220.576 ± 35.596 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsFair ftp_1 8192 avgt 5 298.276 ± 22.256 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsFair ftp_8 8192 avgt 5 426.105 ± 29.870 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsFair experimental 8192 avgt 5 288.546 ± 20.280 ms/op + * + * ConcurrentStatefulActorBenchmark.multipleComputationsFair fjp 262144 avgt 5 4146.057 ± 284.377 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsFair ftp_1 262144 avgt 5 10250.107 ± 1421.253 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsFair ftp_8 262144 avgt 5 6761.283 ± 4091.452 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsFair experimental 262144 avgt 5 6521.436 ± 346.726 ms/op + * + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair fjp 1024 avgt 5 289.875 ± 14.241 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair ftp_1 1024 avgt 5 87.336 ± 5.160 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair ftp_8 1024 avgt 5 430.718 ± 23.497 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair experimental 1024 avgt 5 153.704 ± 13.869 ms/op + * + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair fjp 8192 avgt 5 289.836 ± 9.719 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair ftp_1 8192 avgt 5 299.523 ± 17.357 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair ftp_8 8192 avgt 5 433.959 ± 27.669 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair experimental 8192 avgt 5 283.441 ± 22.740 ms/op + * + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair fjp 262144 avgt 5 7804.066 ± 1386.595 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair ftp_1 262144 avgt 5 11142.530 ± 381.401 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair ftp_8 262144 avgt 5 7739.136 ± 1317.885 ms/op + * ConcurrentStatefulActorBenchmark.multipleComputationsUnfair experimental 262144 avgt 5 7076.911 ± 1971.615 ms/op + * + */ +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class ConcurrentStatefulActorBenchmark : ParametrizedDispatcherBase() { + + @Param("1024", "8192") + var stateSize: Int = -1 + + @Param("fjp", "scheduler") + override var dispatcher: String = "fjp" + + @Benchmark + fun multipleComputationsUnfair() = runBlocking { + val resultChannel: Channel = Channel(1) + val computations = (0 until CORES_COUNT).map { computationActor(stateSize) } + val requestor = requestorActorUnfair(computations, resultChannel) + requestor.send(Letter(Start(), requestor)) + resultChannel.receive() + } + + @Benchmark + fun multipleComputationsFair() = runBlocking { + val resultChannel: Channel = Channel(1) + val computations = (0 until CORES_COUNT).map { computationActor(stateSize) } + val requestor = requestorActorFair(computations, resultChannel) + requestor.send(Letter(Start(), requestor)) + resultChannel.receive() + } + + fun requestorActorUnfair( + computations: List>, + stopChannel: Channel + ) = + actor(capacity = 1024) { + var received = 0 + for (letter in channel) with(letter) { + when (message) { + is Start -> { + computations.shuffled() + .forEach { it.send(Letter(ThreadLocalRandom.current().nextLong(), channel)) } + } + is Long -> { + if (++received >= ROUNDS * 8) { + computations.forEach { it.close() } + stopChannel.send(Unit) + return@actor + } else { + sender.send(Letter(ThreadLocalRandom.current().nextLong(), channel)) + } + } + else -> error("Cannot happen: $letter") + } + } + } + + fun requestorActorFair( + computations: List>, + stopChannel: Channel + ) = + actor(capacity = 1024) { + val received = hashMapOf(*computations.map { it to 0 }.toTypedArray()) + var receivedTotal = 0 + + for (letter in channel) with(letter) { + when (message) { + is Start -> { + computations.shuffled() + .forEach { it.send(Letter(ThreadLocalRandom.current().nextLong(), channel)) } + } + is Long -> { + if (++receivedTotal >= ROUNDS * computations.size) { + computations.forEach { it.close() } + stopChannel.send(Unit) + return@actor + } else { + val receivedFromSender = received[sender]!! + if (receivedFromSender <= ROUNDS) { + received[sender] = receivedFromSender + 1 + sender.send(Letter(ThreadLocalRandom.current().nextLong(), channel)) + } + } + } + else -> error("Cannot happen: $letter") + } + } + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/CycledActorsBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/CycledActorsBenchmark.kt new file mode 100644 index 0000000000..653b769b54 --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/CycledActorsBenchmark.kt @@ -0,0 +1,119 @@ +package benchmarks.scheduler.actors + +import benchmarks.* +import benchmarks.akka.* +import benchmarks.scheduler.actors.PingPongActorBenchmark.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +/* + * Cores count actors chained into single cycle pass message and process it using its private state. + * + * Benchmark (actorStateSize) (dispatcher) Mode Cnt Score Error Units + * CycledActorsBenchmark.cycledActors 1 fjp avgt 14 22.751 ± 1.351 ms/op + * CycledActorsBenchmark.cycledActors 1 ftp_1 avgt 14 4.535 ± 0.076 ms/op + * CycledActorsBenchmark.cycledActors 1 experimental avgt 14 6.728 ± 0.048 ms/op + * + * CycledActorsBenchmark.cycledActors 1024 fjp avgt 14 43.725 ± 14.393 ms/op + * CycledActorsBenchmark.cycledActors 1024 ftp_1 avgt 14 13.827 ± 1.554 ms/op + * CycledActorsBenchmark.cycledActors 1024 experimental avgt 14 23.823 ± 1.643 ms/op + * + * CycledActorsBenchmark.cycledActors 262144 fjp avgt 14 1885.708 ± 532.634 ms/op + * CycledActorsBenchmark.cycledActors 262144 ftp_1 avgt 14 1394.997 ± 101.938 ms/op + * CycledActorsBenchmark.cycledActors 262144 experimental avgt 14 1804.146 ± 57.275 ms/op + */ +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class CycledActorsBenchmark : ParametrizedDispatcherBase() { + + companion object { + val NO_CHANNEL = Channel(0) + } + + @Param("fjp", "ftp_1", "scheduler") + override var dispatcher: String = "fjp" + + @Param("1", "1024") + var actorStateSize = 1 + + @Benchmark + fun cycledActors() = runBlocking { + val stopChannel = Channel(CORES_COUNT) + runCycle(stopChannel) + repeat(CORES_COUNT) { + stopChannel.receive() + } + } + + private suspend fun runCycle(stopChannel: Channel) { + val trailingActor = lastActor(stopChannel) + + var previous = trailingActor + for (i in 1 until CORES_COUNT) { + previous = createActor(previous, stopChannel) + } + + trailingActor.send(Letter(Start(), previous)) + } + + private fun lastActor(stopChannel: Channel) = actor(capacity = 1024) { + var nextChannel: SendChannel? = null + val state = LongArray(actorStateSize) { ThreadLocalRandom.current().nextLong(1024) } + + for (letter in channel) with(letter) { + when (message) { + is Start -> { + nextChannel = sender + sender.send(Letter(Ball(ThreadLocalRandom.current().nextInt(1, 100)), NO_CHANNEL)) + } + is Ball -> { + nextChannel!!.send(Letter(Ball(transmogrify(message.count, state)), NO_CHANNEL)) + } + is Stop -> { + stopChannel.send(Unit) + return@actor + } + else -> error("Can't happen") + } + } + } + + private fun createActor(nextActor: SendChannel, stopChannel: Channel) = actor(capacity = 1024) { + var received = 0 + val state = LongArray(actorStateSize) { ThreadLocalRandom.current().nextLong(1024) } + + for (letter in channel) with(letter) { + when (message) { + is Ball -> { + if (++received > 1_000) { + nextActor.send(Letter(Stop(), NO_CHANNEL)) + stopChannel.send(Unit) + return@actor + } else { + nextActor.send(Letter(Ball(transmogrify(message.count, state)), NO_CHANNEL)) + } + } + is Stop -> { + nextActor.send(Letter(Stop(), NO_CHANNEL)) + stopChannel.send(Unit) + } + else -> error("Can't happen") + } + } + } + + private fun transmogrify(value: Int, coefficients: LongArray): Int { + var result = 0L + for (coefficient in coefficients) { + result += coefficient * value + } + + return result.toInt() + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/PingPongActorBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/PingPongActorBenchmark.kt new file mode 100644 index 0000000000..9c524e845a --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/PingPongActorBenchmark.kt @@ -0,0 +1,99 @@ +package benchmarks.scheduler.actors + +import benchmarks.* +import benchmarks.akka.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +/* + * Benchmark (dispatcher) Mode Cnt Score Error Units + * PingPongActorBenchmark.coresCountPingPongs experimental avgt 10 185.066 ± 21.692 ms/op + * PingPongActorBenchmark.coresCountPingPongs fjp avgt 10 200.581 ± 22.669 ms/op + * PingPongActorBenchmark.coresCountPingPongs ftp_1 avgt 10 494.334 ± 27.450 ms/op + * PingPongActorBenchmark.coresCountPingPongs ftp_2 avgt 10 498.754 ± 27.743 ms/op + * PingPongActorBenchmark.coresCountPingPongs ftp_8 avgt 10 804.498 ± 69.826 ms/op + * + * PingPongActorBenchmark.singlePingPong experimental avgt 10 45.521 ± 3.281 ms/op + * PingPongActorBenchmark.singlePingPong fjp avgt 10 217.005 ± 18.693 ms/op + * PingPongActorBenchmark.singlePingPong ftp_1 avgt 10 57.632 ± 1.835 ms/op + * PingPongActorBenchmark.singlePingPong ftp_2 avgt 10 112.723 ± 5.280 ms/op + * PingPongActorBenchmark.singlePingPong ftp_8 avgt 10 276.958 ± 21.447 ms/op + */ +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class PingPongActorBenchmark : ParametrizedDispatcherBase() { + data class Letter(val message: Any?, val sender: SendChannel) + + @Param("scheduler", "fjp", "ftp_1") + override var dispatcher: String = "fjp" + + @Benchmark + fun singlePingPong() = runBlocking { + runPingPongs(1) + } + + @Benchmark + fun coresCountPingPongs() = runBlocking { + runPingPongs(Runtime.getRuntime().availableProcessors()) + } + + private suspend fun runPingPongs(count: Int) { + val me = Channel() + repeat(count) { + val pong = pongActorCoroutine() + val ping = pingActorCoroutine(pong) + ping.send(Letter(Start(), me)) + } + + repeat(count) { + me.receive() + } + } +} + +fun CoroutineScope.pingActorCoroutine( + pingChannel: SendChannel, + capacity: Int = 1 +) = + actor(capacity = capacity) { + var initiator: SendChannel? = null + for (letter in channel) with(letter) { + when (message) { + is Start -> { + initiator = sender + pingChannel.send(PingPongActorBenchmark.Letter(Ball(0), channel)) + } + is Ball -> { + pingChannel.send(PingPongActorBenchmark.Letter(Ball(message.count + 1), channel)) + } + is Stop -> { + initiator!!.send(PingPongActorBenchmark.Letter(Stop(), channel)) + return@actor + } + else -> error("Cannot happen $message") + } + } + } + +fun CoroutineScope.pongActorCoroutine(capacity: Int = 1) = + actor(capacity = capacity) { + for (letter in channel) with(letter) { + when (message) { + is Ball -> { + if (message.count >= N_MESSAGES) { + sender.send(PingPongActorBenchmark.Letter(Stop(), channel)) + return@actor + } else { + sender.send(PingPongActorBenchmark.Letter(Ball(message.count + 1), channel)) + } + } + else -> error("Cannot happen $message") + } + } + } diff --git a/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/PingPongWithBlockingContext.kt b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/PingPongWithBlockingContext.kt new file mode 100644 index 0000000000..9bebea71dc --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/PingPongWithBlockingContext.kt @@ -0,0 +1,60 @@ +package benchmarks.scheduler.actors + +import benchmarks.akka.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.scheduling.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* +import kotlin.coroutines.* + + +/* + * Benchmark Mode Cnt Score Error Units + * PingPongWithBlockingContext.commonPoolWithContextPingPong avgt 20 972.662 ± 103.448 ms/op + * PingPongWithBlockingContext.limitingDispatcherPingPong avgt 20 136.167 ± 4.971 ms/op + * PingPongWithBlockingContext.withContextPingPong avgt 20 761.669 ± 41.371 ms/op + */ +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class PingPongWithBlockingContext { + + private val experimental = Dispatchers.Default + private val blocking = Dispatchers.IO.limitedParallelism(8) + private val threadPool = newFixedThreadPoolContext(8, "PongCtx") + + @TearDown + fun tearDown() { + threadPool.close() + } + + + @Benchmark + fun limitingDispatcherPingPong() = runBlocking { + runPingPongs(experimental, blocking) + } + + + @Benchmark + fun withContextPingPong() = runBlocking { + runPingPongs(experimental, threadPool) + } + + @Benchmark + fun commonPoolWithContextPingPong() = runBlocking { + runPingPongs(ForkJoinPool.commonPool().asCoroutineDispatcher(), threadPool) + } + + private suspend fun runPingPongs(pingContext: CoroutineContext, pongContext: CoroutineContext) { + val me = Channel() + val pong = CoroutineScope(pongContext).pongActorCoroutine() + val ping = CoroutineScope(pingContext).pingActorCoroutine(pong) + ping.send(PingPongActorBenchmark.Letter(Start(), me)) + + me.receive() + } +} diff --git a/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/StatefulActorBenchmark.kt b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/StatefulActorBenchmark.kt new file mode 100644 index 0000000000..0b03dba57a --- /dev/null +++ b/benchmarks/src/jmh/kotlin/benchmarks/scheduler/actors/StatefulActorBenchmark.kt @@ -0,0 +1,115 @@ +package benchmarks.scheduler.actors + +import benchmarks.* +import benchmarks.akka.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +/* + * kotlinx-based counterpart of [StatefulActorAkkaBenchmark] + * + * Benchmark (dispatcher) Mode Cnt Score Error Units + * StatefulActorBenchmark.multipleComputationsMultipleRequestors fjp avgt 10 81.649 ± 9.671 ms/op + * StatefulActorBenchmark.multipleComputationsMultipleRequestors ftp_1 avgt 10 160.590 ± 50.342 ms/op + * StatefulActorBenchmark.multipleComputationsMultipleRequestors ftp_8 avgt 10 275.798 ± 32.795 ms/op + * + * StatefulActorBenchmark.multipleComputationsSingleRequestor fjp avgt 10 67.206 ± 4.023 ms/op + * StatefulActorBenchmark.multipleComputationsSingleRequestor ftp_1 avgt 10 17.883 ± 1.314 ms/op + * StatefulActorBenchmark.multipleComputationsSingleRequestor ftp_8 avgt 10 77.052 ± 10.132 ms/op + * + * StatefulActorBenchmark.singleComputationMultipleRequestors fjp avgt 10 488.003 ± 53.014 ms/op + * StatefulActorBenchmark.singleComputationMultipleRequestors ftp_1 avgt 10 120.445 ± 24.659 ms/op + * StatefulActorBenchmark.singleComputationMultipleRequestors ftp_8 avgt 10 527.118 ± 51.139 ms/op + * + * StatefulActorBenchmark.singleComputationSingleRequestor fjp avgt 10 95.030 ± 23.850 ms/op + * StatefulActorBenchmark.singleComputationSingleRequestor ftp_1 avgt 10 16.005 ± 0.629 ms/op + * StatefulActorBenchmark.singleComputationSingleRequestor ftp_8 avgt 10 76.435 ± 5.076 ms/op + */ +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class StatefulActorBenchmark : ParametrizedDispatcherBase() { + + data class Letter(val message: Any, val sender: SendChannel) + + @Param("fjp", "ftp_1", "ftp_8", "scheduler") + override var dispatcher: String = "fjp" + + @Benchmark + fun singleComputationSingleRequestor() = runBlocking { + run(1, 1) + } + + @Benchmark + fun singleComputationMultipleRequestors() = runBlocking { + run(1, CORES_COUNT) + } + + @Benchmark + fun multipleComputationsSingleRequestor() = runBlocking { + run(CORES_COUNT, 1) + } + + @Benchmark + fun multipleComputationsMultipleRequestors() = runBlocking { + run(CORES_COUNT, CORES_COUNT) + } + + private suspend fun run(computationActorsCount: Int, requestorActorsCount: Int) { + val resultChannel: Channel = Channel(requestorActorsCount) + val computations = (0 until computationActorsCount).map { computationActor() } + val requestors = (0 until requestorActorsCount).map { requestorActor(computations, resultChannel) } + + for (requestor in requestors) { + requestor.send(Letter(1L, Channel())) + } + + repeat(requestorActorsCount) { + resultChannel.receive() + } + } + + private fun CoroutineScope.requestorActor(computations: List>, stopChannel: Channel) = + actor(capacity = 1024) { + var received = 0 + for (letter in channel) with(letter) { + when (message) { + is Long -> { + if (++received >= ROUNDS) { + stopChannel.send(Unit) + return@actor + } else { + computations[ThreadLocalRandom.current().nextInt(0, computations.size)] + .send(Letter(ThreadLocalRandom.current().nextLong(), channel)) + } + } + else -> error("Cannot happen: $letter") + } + } + } +} + +fun CoroutineScope.computationActor(stateSize: Int = STATE_SIZE) = + actor(capacity = 1024) { + val coefficients = LongArray(stateSize) { ThreadLocalRandom.current().nextLong(0, 100) } + + for (letter in channel) with(letter) { + when (message) { + is Long -> { + var result = 0L + for (coefficient in coefficients) { + result += message * coefficient + } + + sender.send(StatefulActorBenchmark.Letter(result, channel)) + } + is Stop -> return@actor + else -> error("Cannot happen: $letter") + } + } + } diff --git a/benchmarks/src/jmh/resources/ospd.txt.gz b/benchmarks/src/jmh/resources/ospd.txt.gz new file mode 100644 index 0000000000..8a3074e927 Binary files /dev/null and b/benchmarks/src/jmh/resources/ospd.txt.gz differ diff --git a/benchmarks/src/jmh/resources/words.shakespeare.txt.gz b/benchmarks/src/jmh/resources/words.shakespeare.txt.gz new file mode 100644 index 0000000000..456f56cbeb Binary files /dev/null and b/benchmarks/src/jmh/resources/words.shakespeare.txt.gz differ diff --git a/benchmarks/src/main/kotlin/benchmarks/GuideSyncBenchmark.kt b/benchmarks/src/main/kotlin/benchmarks/GuideSyncBenchmark.kt deleted file mode 100644 index 6a19a71166..0000000000 --- a/benchmarks/src/main/kotlin/benchmarks/GuideSyncBenchmark.kt +++ /dev/null @@ -1,107 +0,0 @@ -/* - * Copyright (c) 2014, Oracle America, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * * Neither the name of Oracle nor the names of its contributors may be used - * to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF - * THE POSSIBILITY OF SUCH DAMAGE. - */ - -package benchmarks - -import org.openjdk.jmh.annotations.* -import java.io.OutputStream -import java.io.PrintStream -import java.util.concurrent.TimeUnit - -/* - -Benchmark Mode Cnt Score Error Units -GuideSyncBenchmark.sync01Problem avgt 15 11971.221 ± 1739.891 us/op -GuideSyncBenchmark.sync02Volatile avgt 15 14936.828 ± 142.586 us/op -GuideSyncBenchmark.sync03AtomicInt avgt 15 15505.607 ± 1434.846 us/op -GuideSyncBenchmark.sync04ConfineFine avgt 15 1331453.593 ± 89298.871 us/op -GuideSyncBenchmark.sync05ConfineCoarse avgt 15 2253.270 ± 425.033 us/op -GuideSyncBenchmark.sync06Mutex avgt 15 1075086.511 ± 140589.883 us/op -GuideSyncBenchmark.sync07Actor avgt 15 1075603.512 ± 203901.350 us/op - - */ - -@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) -@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) -@Fork(value = 3) -@BenchmarkMode(Mode.AverageTime) -@OutputTimeUnit(TimeUnit.MICROSECONDS) -@State(Scope.Benchmark) -open class GuideSyncBenchmark { - val oldOut = System.out - - @Setup - fun setup() { - System.setOut(PrintStream(object : OutputStream() { - override fun write(b: Int) {} // empty - })) - } - - @TearDown - fun tearDonw() { - System.setOut(oldOut) - } - - @Benchmark - fun sync01Problem() { - guide.sync.example01.main(emptyArray()) - } - - @Benchmark - fun sync02Volatile() { - guide.sync.example02.main(emptyArray()) - } - - @Benchmark - fun sync03AtomicInt() { - guide.sync.example03.main(emptyArray()) - } - - @Benchmark - fun sync04ConfineFine() { - guide.sync.example04.main(emptyArray()) - } - - @Benchmark - fun sync05ConfineCoarse() { - guide.sync.example05.main(emptyArray()) - } - - @Benchmark - fun sync06Mutex() { - guide.sync.example06.main(emptyArray()) - } - - @Benchmark - fun sync07Actor() { - guide.sync.example07.main(emptyArray()) - } -} diff --git a/benchmarks/src/main/kotlin/benchmarks/PingPongActorBenchmark.kt b/benchmarks/src/main/kotlin/benchmarks/PingPongActorBenchmark.kt deleted file mode 100644 index fab0dbb4d4..0000000000 --- a/benchmarks/src/main/kotlin/benchmarks/PingPongActorBenchmark.kt +++ /dev/null @@ -1,159 +0,0 @@ -package benchmarks - -import akka.actor.* -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.channels.Channel -import kotlinx.coroutines.experimental.channels.SendChannel -import kotlinx.coroutines.experimental.channels.actor -import kotlinx.coroutines.experimental.newFixedThreadPoolContext -import kotlinx.coroutines.experimental.newSingleThreadContext -import kotlinx.coroutines.experimental.runBlocking -import org.openjdk.jmh.annotations.* -import org.openjdk.jmh.annotations.Scope -import java.util.concurrent.TimeUnit -import kotlin.coroutines.experimental.CoroutineContext - -/* - -Benchmark Mode Cnt Score Error Units -PingPongActorBenchmark.pingPongAkka avgt 15 439.419 ± 24.595 ms/op -PingPongActorBenchmark.pingPongCoroutineCommonPool avgt 15 809.122 ± 18.102 ms/op -PingPongActorBenchmark.pingPongCoroutineMain avgt 15 360.072 ± 4.930 ms/op -PingPongActorBenchmark.pingPongCoroutineSingleThread avgt 15 368.429 ± 3.718 ms/op -PingPongActorBenchmark.pingPongCoroutineTwoThreads avgt 15 615.514 ± 5.292 ms/op - -*/ - -private const val N_MESSAGES = 1_000_000 - -@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) -@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) -@Fork(value = 3) -@BenchmarkMode(Mode.AverageTime) -@OutputTimeUnit(TimeUnit.MILLISECONDS) -@State(Scope.Benchmark) -open class PingPongActorBenchmark { - data class Ball(val count: Int) - class Start - class Stop - - class PingActorAkka(val pongRef: ActorRef) : UntypedActor() { - override fun onReceive(msg: Any?) { - when (msg) { - is Start -> { - pongRef.tell(Ball(0), self) - } - is Ball -> { - pongRef.tell(Ball(count = msg.count + 1), self) - } - is Stop -> { - context.system().shutdown() - } - else -> unhandled(msg) - } - } - } - - class PongActorAkka() : UntypedActor() { - override fun onReceive(msg: Any?) { - when (msg) { - is Ball -> { - if (msg.count >= N_MESSAGES) { - sender.tell(Stop()) - context.stop(self) - } else { - sender.tell(Ball(msg.count + 1)) - } - } - else -> unhandled(msg) - } - } - } - - @Benchmark - fun pingPongAkka() { - val system = ActorSystem.create("PingPoing") - val pongRef = system.actorOf(Props(UntypedActorFactory { PongActorAkka() }), "pong") - val pingRef = system.actorOf(Props(UntypedActorFactory { PingActorAkka(pongRef) }), "ping") - pingRef.tell(Start()) - system.awaitTermination() - } - - data class Letter(val msg: Any?, val sender: SendChannel) - - fun pingActorCoroutine(context: CoroutineContext, pingChannel: SendChannel) = actor(context) { - var initiator: SendChannel? = null - for (letter in channel) with(letter) { - when (msg) { - is Start -> { - initiator = sender - pingChannel.send(Letter(Ball(0), channel)) - } - is Ball -> { - pingChannel.send(Letter(Ball(msg.count + 1), channel)) - } - is Stop -> { - initiator!!.send(Letter(Stop(), channel)) - return@actor - } - else -> error("Cannot happen $msg") - } - } - } - - fun pongActorCoroutine(context: CoroutineContext) = actor(context) { - for (letter in channel) with (letter) { - when (msg) { - is Ball -> { - if (msg.count >= N_MESSAGES) { - sender.send(Letter(Stop(), channel)) - return@actor - } else { - sender.send(Letter(Ball(msg.count + 1), channel)) - } - } - else -> error("Cannot happen $msg") - } - } - } - - @Benchmark - fun pingPongCoroutineCommonPool() = runBlocking { - val pong = pongActorCoroutine(CommonPool) - val ping = pingActorCoroutine(CommonPool, pong) - val me = Channel() - ping.send(Letter(Start(), me)) - me.receive() - } - - @Benchmark - fun pingPongCoroutineMain() = runBlocking { - val pong = pongActorCoroutine(context) - val ping = pingActorCoroutine(context, pong) - val me = Channel() - ping.send(Letter(Start(), me)) - me.receive() - } - - val singleThread = newSingleThreadContext("PingPongThread") - - @Benchmark - fun pingPongCoroutineSingleThread() = runBlocking { - val pong = pongActorCoroutine(singleThread) - val ping = pingActorCoroutine(singleThread, pong) - val me = Channel() - ping.send(Letter(Start(), me)) - me.receive() - } - - val twoThreads = newFixedThreadPoolContext(2, "PingPongThreads") - - @Benchmark - fun pingPongCoroutineTwoThreads() = runBlocking { - val pong = pongActorCoroutine(twoThreads) - val ping = pingActorCoroutine(twoThreads, pong) - val me = Channel() - ping.send(Letter(Start(), me)) - me.receive() - } -} diff --git a/benchmarks/src/main/kotlin/benchmarks/common/BenchmarkUtils.kt b/benchmarks/src/main/kotlin/benchmarks/common/BenchmarkUtils.kt new file mode 100644 index 0000000000..21c92c1511 --- /dev/null +++ b/benchmarks/src/main/kotlin/benchmarks/common/BenchmarkUtils.kt @@ -0,0 +1,13 @@ +package benchmarks.common + +import java.util.concurrent.* + +public fun doGeomDistrWork(work: Int) { + // We use geometric distribution here. We also checked on macbook pro 13" (2017) that the resulting work times + // are distributed geometrically, see https://github.com/Kotlin/kotlinx.coroutines/pull/1464#discussion_r355705325 + val p = 1.0 / work + val r = ThreadLocalRandom.current() + while (true) { + if (r.nextDouble() < p) break + } +} diff --git a/build.gradle.kts b/build.gradle.kts new file mode 100644 index 0000000000..7b9248ca49 --- /dev/null +++ b/build.gradle.kts @@ -0,0 +1,171 @@ +import org.jetbrains.kotlin.config.KotlinCompilerVersion +import org.jetbrains.kotlin.gradle.dsl.* +import org.gradle.kotlin.dsl.* + +buildscript { + if (shouldUseLocalMaven(rootProject)) { + repositories { + mavenLocal() + } + } + + repositories { + mavenCentral() + maven(url = "https://plugins.gradle.org/m2/") + addDevRepositoryIfEnabled(this, project) + mavenLocal() + } + + dependencies { + // Please ensure that atomicfu-gradle-plugin is added to the classpath first, do not change the order, for details see #3984. + // The corresponding issue in kotlinx-atomicfu: https://github.com/Kotlin/kotlinx-atomicfu/issues/384 + classpath("org.jetbrains.kotlinx:atomicfu-gradle-plugin:${version("atomicfu")}") + classpath("org.jetbrains.kotlin:kotlin-gradle-plugin:${version("kotlin")}") + classpath("org.jetbrains.dokka:dokka-gradle-plugin:${version("dokka")}") + classpath("org.jetbrains.kotlinx:kotlinx-knit:${version("knit")}") + classpath("org.jetbrains.kotlinx:binary-compatibility-validator:${version("binary_compatibility_validator")}") + classpath("ru.vyarus:gradle-animalsniffer-plugin:${version("animalsniffer")}") // Android API check + classpath("org.jetbrains.kotlin:atomicfu:${version("kotlin")}") + classpath("org.jetbrains.kotlinx:kover-gradle-plugin:${version("kover")}") + } + + with(CacheRedirector) { buildscript.configureBuildScript(rootProject) } +} + +// Configure subprojects with Kotlin sources +apply(plugin = "configure-compilation-conventions") + +allprojects { + val deployVersion = properties["DeployVersion"] + if (deployVersion != null) version = deployVersion + + if (isSnapshotTrainEnabled(rootProject)) { + val skipSnapshotChecks = rootProject.properties["skip_snapshot_checks"] != null + if (!skipSnapshotChecks && version != version("atomicfu")) { + throw IllegalStateException("Current deploy version is $version, but atomicfu version is not overridden (${version("atomicfu")}) for $this") + } + } + + if (shouldUseLocalMaven(rootProject)) { + repositories { + mavenLocal() + } + } + + // This project property is set during nightly stress test + val stressTest = project.properties["stressTest"] + // Copy it to all test tasks + tasks.withType(Test::class).configureEach { + if (stressTest != null) { + systemProperty("stressTest", stressTest) + } + } +} + +plugins { + id("org.jetbrains.kotlinx.binary-compatibility-validator") version "0.16.2" +} + +apply(plugin = "base") +apply(plugin = "kover-conventions") + +apiValidation { + ignoredProjects += unpublished + listOf("kotlinx-coroutines-bom") + if (isSnapshotTrainEnabled(rootProject)) { + ignoredProjects += coreModule + } + ignoredPackages += "kotlinx.coroutines.internal" + @OptIn(kotlinx.validation.ExperimentalBCVApi::class) + klib { + enabled = true + } +} + +// Configure repositories +allprojects { + repositories { + /* + * google should be first in the repository list because some of the play services + * transitive dependencies was removed from jcenter, thus breaking gradle dependency resolution + */ + google() + mavenCentral() + addDevRepositoryIfEnabled(this, project) + } +} + +// needs to be before evaluationDependsOn due to weird Gradle ordering +configure(subprojects) { + fun Project.shouldSniff(): Boolean = + platformOf(project) == "jvm" && project.name !in unpublished && project.name !in sourceless + && project.name !in androidNonCompatibleProjects + // Skip JDK 8 projects or unpublished ones + if (shouldSniff()) { + if (isMultiplatform) { + apply(plugin = "animalsniffer-multiplatform-conventions") + } else { + apply(plugin = "animalsniffer-jvm-conventions") + } + } +} + +configure(subprojects.filter { !sourceless.contains(it.name) }) { + if (isMultiplatform) { + apply(plugin = "kotlin-multiplatform") + apply(plugin = "kotlin-multiplatform-conventions") + } else if (platformOf(this) == "jvm") { + apply(plugin = "kotlin-jvm-conventions") + } else { + val platform = platformOf(this) + throw IllegalStateException("No configuration rules for $platform") + } +} + +configure(subprojects.filter { !sourceless.contains(it.name) && it.name != testUtilsModule }) { + if (isMultiplatform) { + configure { + sourceSets.commonTest.dependencies { implementation(project(":$testUtilsModule")) } + } + } else { + dependencies { add("testImplementation", project(":$testUtilsModule")) } + } +} + +// Add dependency to the core module in all the other subprojects. +configure(subprojects.filter { !sourceless.contains(it.name) && it.name != coreModule }) { + evaluationDependsOn(":$coreModule") + if (isMultiplatform) { + configure { + sourceSets.commonMain.dependencies { api(project(":$coreModule")) } + } + } else { + dependencies { add("api", project(":$coreModule")) } + } +} + +apply(plugin = "bom-conventions") +apply(plugin = "java-modularity-conventions") +apply(plugin = "version-file-conventions") + +rootProject.configureCommunityBuildTweaks() + +apply(plugin = "source-set-conventions") +apply(plugin = "dokka-conventions") +apply(plugin = "knit-conventions") + +/* + * TODO: core and non-core cannot be configured via 'configure(subprojects)' + * because of 'afterEvaluate' issue. This one should be migrated to + * `plugins { id("pub-conventions") }` eventually + */ +configure(subprojects.filter { + !unpublished.contains(it.name) && it.name != coreModule +}) { + apply(plugin = "pub-conventions") +} + +AuxBuildConfiguration.configure(rootProject) +rootProject.registerTopLevelDeployTask() + +// Report Kotlin compiler version when building project +println("Using Kotlin compiler version: ${KotlinCompilerVersion.VERSION}") diff --git a/buildSrc/build.gradle.kts b/buildSrc/build.gradle.kts new file mode 100644 index 0000000000..27b713684c --- /dev/null +++ b/buildSrc/build.gradle.kts @@ -0,0 +1,69 @@ +import java.util.* + +plugins { + `kotlin-dsl` +} + +val cacheRedirectorEnabled = System.getenv("CACHE_REDIRECTOR")?.toBoolean() == true +val buildSnapshotTrain = properties["build_snapshot_train"]?.toString()?.toBoolean() == true +val kotlinDevUrl = project.rootProject.properties["kotlin_repo_url"] as? String + +repositories { + mavenCentral() + if (cacheRedirectorEnabled) { + maven("https://cache-redirector.jetbrains.com/plugins.gradle.org/m2") + } else { + maven("https://plugins.gradle.org/m2") + } + if (!kotlinDevUrl.isNullOrEmpty()) { + maven(kotlinDevUrl) + } + if (buildSnapshotTrain) { + mavenLocal() + } +} + +val gradleProperties = Properties().apply { + file("../gradle.properties").inputStream().use { load(it) } +} + +fun version(target: String): String { + // Intercept reading from properties file + if (target == "kotlin") { + val snapshotVersion = properties["kotlin_snapshot_version"] + if (snapshotVersion != null) return snapshotVersion.toString() + } + val version = "${target}_version" + // Read from CLI first, used in aggregate builds + return properties[version]?.let{"$it"} ?: gradleProperties.getProperty(version) +} + +dependencies { + implementation(kotlin("gradle-plugin", version("kotlin"))) + /* + * Dokka is compiled with language level = 1.4, but depends on Kotlin 1.6.0, while + * our version of Gradle bundles Kotlin 1.4.x and can read metadata only up to 1.5.x, + * thus we're excluding stdlib compiled with 1.6.0 from dependencies. + */ + implementation("org.jetbrains.dokka:dokka-gradle-plugin:${version("dokka")}") { + exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib-jdk8") + exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib-jdk7") + exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib") + } + implementation("org.jetbrains.dokka:dokka-core:${version("dokka")}") { + exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib-jdk8") + exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib-jdk7") + exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib") + } + // Force ASM version, otherwise the one from animalsniffer wins (which is too low for BCV) + implementation("org.ow2.asm:asm:9.6") + implementation("ru.vyarus:gradle-animalsniffer-plugin:${version("animalsniffer")}") // Android API check + implementation("org.jetbrains.kotlinx:kover-gradle-plugin:${version("kover")}") { + exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib-jdk8") + exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib-jdk7") + exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib") + } + implementation("org.jetbrains.kotlinx:kotlinx-benchmark-plugin:${version("benchmarks")}") + implementation("org.jetbrains.kotlinx:kotlinx-knit:${version("knit")}") + implementation("org.jetbrains.kotlinx:atomicfu-gradle-plugin:${version("atomicfu")}") +} diff --git a/buildSrc/settings.gradle.kts b/buildSrc/settings.gradle.kts new file mode 100644 index 0000000000..2ad2ddbea6 --- /dev/null +++ b/buildSrc/settings.gradle.kts @@ -0,0 +1,15 @@ +pluginManagement { + val build_snapshot_train: String? by settings + repositories { + val cacheRedirectorEnabled = System.getenv("CACHE_REDIRECTOR")?.toBoolean() == true + if (cacheRedirectorEnabled) { + println("Redirecting repositories for buildSrc buildscript") + maven("https://cache-redirector.jetbrains.com/plugins.gradle.org/m2") + } else { + maven("https://plugins.gradle.org/m2") + } + if (build_snapshot_train?.toBoolean() == true) { + mavenLocal() + } + } +} diff --git a/buildSrc/src/main/kotlin/AuxBuildConfiguration.kt b/buildSrc/src/main/kotlin/AuxBuildConfiguration.kt new file mode 100644 index 0000000000..be3770e932 --- /dev/null +++ b/buildSrc/src/main/kotlin/AuxBuildConfiguration.kt @@ -0,0 +1,53 @@ +import CacheRedirector.configure +import org.gradle.api.Project +import org.gradle.api.tasks.* +import org.gradle.kotlin.dsl.* + +/** + * Auxiliary build configuration that is grouped in a single place for convenience: + * - Workarounds for Gradle/KGP issues + * - Cache redirector + */ +object AuxBuildConfiguration { + + @JvmStatic + fun configure(rootProject: Project) { + rootProject.allprojects { + CacheRedirector.configure(this) + workaroundForCoroutinesLeakageToClassPath() + } + + CacheRedirector.configureJsPackageManagers(rootProject) + + // Sigh, there is no BuildScanExtension in classpath when there is no --scan + rootProject.extensions.findByName("buildScan")?.withGroovyBuilder { + setProperty("termsOfServiceUrl", "https://gradle.com/terms-of-service") + setProperty("termsOfServiceAgree", "yes") + } + } + + /* + * 'kotlinx-coroutines-core' dependency leaks into test runtime classpath via 'kotlin-compiler-embeddable' + * and conflicts with our own test/runtime incompatibilities (e.g. when class is moved from a main to test), + * so we do substitution here. + * TODO figure out if it's still the problem + */ + private fun Project.workaroundForCoroutinesLeakageToClassPath() { + configurations + .matching { + // Excluding substituted project itself because of circular dependencies, but still do it + // for "*Test*" configurations + name != coreModule || it.name.contains("Test") + } + .configureEach { + resolutionStrategy.dependencySubstitution { + substitute(module("org.jetbrains.kotlinx:$coreModule")) + .using(project(":$coreModule")) + .because( + "Because Kotlin compiler embeddable leaks coroutines into the runtime classpath, " + + "triggering all sort of incompatible class changes errors" + ) + } + } + } +} diff --git a/buildSrc/src/main/kotlin/CacheRedirector.kt b/buildSrc/src/main/kotlin/CacheRedirector.kt new file mode 100644 index 0000000000..85e6eef840 --- /dev/null +++ b/buildSrc/src/main/kotlin/CacheRedirector.kt @@ -0,0 +1,149 @@ +import org.gradle.api.* +import org.gradle.api.artifacts.dsl.* +import org.gradle.api.artifacts.repositories.* +import org.gradle.api.initialization.dsl.* +import org.gradle.kotlin.dsl.* +import org.jetbrains.kotlin.gradle.targets.js.nodejs.* +import org.jetbrains.kotlin.gradle.targets.js.npm.tasks.* +import org.jetbrains.kotlin.gradle.targets.js.yarn.* +import java.net.* + +/** + * Enabled via environment variable, so that it can be reliably accessed from any piece of the build script, + * including buildSrc within TeamCity CI. + */ +private val cacheRedirectorEnabled = System.getenv("CACHE_REDIRECTOR")?.toBoolean() == true + +/** + * The list of repositories supported by cache redirector should be synced with the list at https://cache-redirector.jetbrains.com/redirects_generated.html + * To add a repository to the list create an issue in ADM project (example issue https://youtrack.jetbrains.com/issue/IJI-149) + */ +private val mirroredUrls = listOf( + "https://cdn.azul.com/zulu/bin", + "https://clojars.org/repo", + "https://dl.google.com/android/repository", + "https://dl.google.com/dl/android/maven2", + "https://dl.google.com/dl/android/studio/ide-zips", + "https://dl.google.com/go", + "https://download.jetbrains.com/", + "https://github.com/yarnpkg/yarn/releases/download", + "https://jitpack.io/", + "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/bootstrap", + "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev", + "https://maven.pkg.jetbrains.space/kotlin/p/kotlin/eap", + "https://nodejs.org/dist", + "https://oss.sonatype.org/content/repositories/releases", + "https://oss.sonatype.org/content/repositories/snapshots", + "https://oss.sonatype.org/content/repositories/staging", + "https://packages.confluent.io/maven/", + "https://plugins.gradle.org/m2", + "https://plugins.jetbrains.com/maven", + "https://repo.grails.org/grails/core", + "https://repo.jenkins-ci.org/releases", + "https://repo.maven.apache.org/maven2", + "https://repo.spring.io/milestone", + "https://repo.typesafe.com/typesafe/ivy-releases", + "https://repo1.maven.org/maven2", + "https://services.gradle.org/", + "https://www.exasol.com/artifactory/exasol-releases", + "https://www.jetbrains.com/intellij-repository/nightly", + "https://www.jetbrains.com/intellij-repository/releases", + "https://www.jetbrains.com/intellij-repository/snapshots", + "https://www.myget.org/F/intellij-go-snapshots/maven", + "https://www.myget.org/F/rd-model-snapshots/maven", + "https://www.myget.org/F/rd-snapshots/maven", + "https://www.python.org/ftp", +) + +private val aliases = mapOf( + "https://repo.maven.apache.org/maven2" to "https://repo1.maven.org/maven2" // Maven Central +) + +private fun URI.toCacheRedirectorUri() = URI("https://cache-redirector.jetbrains.com/$host/$path") + +private fun URI.maybeRedirect(): URI? { + val url = toString().trimEnd('/') + val dealiasedUrl = aliases.getOrDefault(url, url) + + return if (mirroredUrls.any { dealiasedUrl.startsWith(it) }) { + URI(dealiasedUrl).toCacheRedirectorUri() + } else { + null + } +} + +private fun URI.isCachedOrLocal() = scheme == "file" || + host == "cache-redirector.jetbrains.com" || + host == "teamcity.jetbrains.com" || + host == "buildserver.labs.intellij.net" + +private fun Project.checkRedirectUrl(url: URI, containerName: String): URI { + val redirected = url.maybeRedirect() + if (redirected == null && !url.isCachedOrLocal()) { + val msg = "Repository $url in $containerName should be cached with cache-redirector" + val details = "Using non cached repository may lead to download failures in CI builds." + + " Check buildSrc/src/main/kotlin/CacheRedirector.kt for details." + logger.warn("WARNING - $msg\n$details") + } + return if (cacheRedirectorEnabled) redirected ?: url else url +} + +private fun Project.checkRedirect(repositories: RepositoryHandler, containerName: String) { + if (cacheRedirectorEnabled) { + logger.info("Redirecting repositories for $containerName") + } + for (repository in repositories) { + when (repository) { + is MavenArtifactRepository -> repository.url = checkRedirectUrl(repository.url, containerName) + is IvyArtifactRepository -> repository.url = checkRedirectUrl(repository.url, containerName) + } + } +} + +private fun Project.configureYarnAndNodeRedirects() { + if (CacheRedirector.isEnabled) { + val yarnRootExtension = extensions.findByType() + yarnRootExtension?.downloadBaseUrl?.let { + yarnRootExtension.downloadBaseUrl = CacheRedirector.maybeRedirect(it) + } + + val nodeJsExtension = rootProject.extensions.findByType() + nodeJsExtension?.downloadBaseUrl?.let { + nodeJsExtension.downloadBaseUrl = CacheRedirector.maybeRedirect(it) + } + } +} + +// Used from Groovy scripts +// TODO get rid of Groovy, come up with a proper convention for rootProject vs arbitrary project argument +object CacheRedirector { + /** + * Substitutes repositories in buildScript { } block. + */ + @JvmStatic + fun ScriptHandler.configureBuildScript(rootProject: Project) { + rootProject.checkRedirect(repositories, "${rootProject.displayName} buildscript") + } + + @JvmStatic + fun configure(project: Project) { + project.checkRedirect(project.repositories, project.displayName) + } + + /** + * Configures JS-specific extensions to use + */ + @JvmStatic + fun configureJsPackageManagers(project: Project) { + project.configureYarnAndNodeRedirects() + } + + @JvmStatic + fun maybeRedirect(url: String): String { + if (!cacheRedirectorEnabled) return url + return URI(url).maybeRedirect()?.toString() ?: url + } + + @JvmStatic + val isEnabled get() = cacheRedirectorEnabled +} diff --git a/buildSrc/src/main/kotlin/CommunityProjectsBuild.kt b/buildSrc/src/main/kotlin/CommunityProjectsBuild.kt new file mode 100644 index 0000000000..0a6b90a5d7 --- /dev/null +++ b/buildSrc/src/main/kotlin/CommunityProjectsBuild.kt @@ -0,0 +1,149 @@ +@file:JvmName("CommunityProjectsBuild") + +import org.gradle.api.* +import org.gradle.api.artifacts.dsl.* +import org.gradle.api.tasks.testing.Test +import org.gradle.kotlin.dsl.* +import java.net.* +import java.util.logging.* +import org.jetbrains.kotlin.gradle.dsl.KotlinVersion + +private val LOGGER: Logger = Logger.getLogger("Kotlin settings logger") + +/** + * Functions in this file are responsible for configuring kotlinx.coroutines build against a custom dev version + * of Kotlin compiler. + * Such configuration is used in a composite community build of Kotlin in order to check whether not-yet-released changes + * are compatible with our libraries (aka "integration testing that substitutes lack of unit testing"). + * + * When `build_snapshot_train` is set to true (and [isSnapshotTrainEnabled] returns `true`), + * - `kotlin_version property` is overridden with `kotlin_snapshot_version` (see [getOverriddenKotlinVersion]), + * - `atomicfu_version` is overwritten by TeamCity environment (AFU is built with snapshot and published to mavenLocal + * as previous step or the snapshot build). + * Additionally, mavenLocal and Sonatype snapshots are added to repository list and stress tests are disabled + * (see [configureCommunityBuildTweaks]). + * + * DO NOT change the name of these properties without adapting the kotlinx.train build chain. +*/ + +/** + * Should be used for running against of non-released Kotlin compiler on a system test level. + * + * @return a Kotlin API version parametrized from command line nor gradle.properties, null otherwise + */ +fun getOverriddenKotlinApiVersion(project: Project): KotlinVersion? { + val apiVersion = project.rootProject.properties["kotlin_api_version"] as? String + return if (apiVersion != null) { + LOGGER.info("""Configured Kotlin API version: '$apiVersion' for project $${project.name}""") + KotlinVersion.fromVersion(apiVersion) + } else { + null + } +} + +/** + * Should be used for running against of non-released Kotlin compiler on a system test level + * + * @return a Kotlin Language version parametrized from command line nor gradle.properties, null otherwise + */ +fun getOverriddenKotlinLanguageVersion(project: Project): KotlinVersion? { + val languageVersion = project.rootProject.properties["kotlin_language_version"] as? String + return if (languageVersion != null) { + LOGGER.info("""Configured Kotlin Language version: '$languageVersion' for project ${project.name}""") + KotlinVersion.fromVersion(languageVersion) + } else { + null + } +} + +/** + * Should be used for running against of non-released Kotlin compiler on a system test level + * Kotlin compiler artifacts are expected to be downloaded from maven central by default. + * In case of compiling with not-published into the MC kotlin compiler artifacts, a kotlin_repo_url gradle parameter should be specified. + * To reproduce a build locally, a kotlin/dev repo should be passed + * + * @return an url for a kotlin compiler repository parametrized from command line nor gradle.properties, empty string otherwise + */ +fun getKotlinDevRepositoryUrl(project: Project): URI? { + val url: String? = project.rootProject.properties["kotlin_repo_url"] as? String + if (url != null) { + LOGGER.info("""Configured Kotlin Compiler repository url: '$url' for project ${project.name}""") + return URI.create(url) + } + return null +} + +/** + * Adds a kotlin-dev space repository with dev versions of Kotlin if Kotlin aggregate build is enabled + */ +fun addDevRepositoryIfEnabled(rh: RepositoryHandler, project: Project) { + val devRepoUrl = getKotlinDevRepositoryUrl(project) ?: return + rh.maven { + url = devRepoUrl + } +} + +/** + * Changes the build config when 'build_snapshot_train' is enabled: + * Disables flaky and Kotlin-specific tests, prints the real version of Kotlin applied (to be sure overridden version of Kotlin is properly picked). + */ +fun Project.configureCommunityBuildTweaks() { + if (!isSnapshotTrainEnabled(this)) return + allprojects { + // Disable stress tests and tests that are flaky on Kotlin version specific + tasks.withType().configureEach { + exclude("**/*LinearizabilityTest*") + exclude("**/*LFTest*") + exclude("**/*StressTest*") + exclude("**/*scheduling*") + exclude("**/*Timeout*") + exclude("**/*definitely/not/kotlinx*") + exclude("**/*PrecompiledDebugProbesTest*") + } + } + + println("Manifest of kotlin-compiler-embeddable.jar for coroutines") + val coreProject = subprojects.single { it.name == coreModule } + configure(listOf(coreProject)) { + configurations.matching { it.name == "kotlinCompilerClasspath" }.configureEach { + val config = resolvedConfiguration.files.single { it.name.contains("kotlin-compiler-embeddable") } + + val manifest = zipTree(config).matching { + include("META-INF/MANIFEST.MF") + }.files.single() + + manifest.readLines().forEach { + println(it) + } + } + } +} + +/** + * Ensures that, if [isSnapshotTrainEnabled] is true, the project is built with a snapshot version of Kotlin compiler. + */ +fun getOverriddenKotlinVersion(project: Project): String? = + if (isSnapshotTrainEnabled(project)) { + val snapshotVersion = project.rootProject.properties["kotlin_snapshot_version"] + ?: error("'kotlin_snapshot_version' should be defined when building with a snapshot compiler") + snapshotVersion.toString() + } else { + null + } + +/** + * Checks if the project is built with a snapshot version of Kotlin compiler. + */ +fun isSnapshotTrainEnabled(project: Project): Boolean { + val buildSnapshotTrain = project.rootProject.properties["build_snapshot_train"] as? String + return !buildSnapshotTrain.isNullOrBlank() +} + +fun shouldUseLocalMaven(project: Project): Boolean { + val hasSnapshotDependency = project.rootProject.properties.any { (key, value) -> + key.endsWith("_version") && value is String && value.endsWith("-SNAPSHOT").also { + if (it) println("NOTE: USING SNAPSHOT VERSION: $key=$value") + } + } + return hasSnapshotDependency || isSnapshotTrainEnabled(project) +} diff --git a/buildSrc/src/main/kotlin/Dokka.kt b/buildSrc/src/main/kotlin/Dokka.kt new file mode 100644 index 0000000000..900375258f --- /dev/null +++ b/buildSrc/src/main/kotlin/Dokka.kt @@ -0,0 +1,22 @@ +import org.gradle.api.* +import org.gradle.kotlin.dsl.* +import org.jetbrains.dokka.gradle.* +import java.io.* +import java.net.* + +/** + * Package-list by external URL for documentation generation. + */ +fun Project.externalDocumentationLink( + url: String, + packageList: File = projectDir.resolve("package.list") +) { + tasks.withType().configureEach { + dokkaSourceSets.configureEach { + externalDocumentationLink { + this.url = URL(url) + packageListUrl = packageList.toPath().toUri().toURL() + } + } + } +} diff --git a/buildSrc/src/main/kotlin/GlobalKotlinCompilerOptions.kt b/buildSrc/src/main/kotlin/GlobalKotlinCompilerOptions.kt new file mode 100644 index 0000000000..5e1667110f --- /dev/null +++ b/buildSrc/src/main/kotlin/GlobalKotlinCompilerOptions.kt @@ -0,0 +1,14 @@ +import org.jetbrains.kotlin.gradle.dsl.KotlinCommonCompilerOptions + +internal fun KotlinCommonCompilerOptions.configureGlobalKotlinArgumentsAndOptIns() { + freeCompilerArgs.addAll("-progressive") + optIn.addAll( + "kotlin.experimental.ExperimentalTypeInference", + // our own opt-ins that we don't want to bother with in our own code: + "kotlinx.coroutines.DelicateCoroutinesApi", + "kotlinx.coroutines.ExperimentalCoroutinesApi", + "kotlinx.coroutines.ObsoleteCoroutinesApi", + "kotlinx.coroutines.InternalCoroutinesApi", + "kotlinx.coroutines.FlowPreview" + ) +} diff --git a/buildSrc/src/main/kotlin/Idea.kt b/buildSrc/src/main/kotlin/Idea.kt new file mode 100644 index 0000000000..615b8aad74 --- /dev/null +++ b/buildSrc/src/main/kotlin/Idea.kt @@ -0,0 +1,5 @@ +object Idea { + @JvmStatic // for Gradle + val active: Boolean + get() = System.getProperty("idea.active") == "true" +} diff --git a/buildSrc/src/main/kotlin/Java9Modularity.kt b/buildSrc/src/main/kotlin/Java9Modularity.kt new file mode 100644 index 0000000000..17cf1fa674 --- /dev/null +++ b/buildSrc/src/main/kotlin/Java9Modularity.kt @@ -0,0 +1,148 @@ +import org.gradle.api.* +import org.gradle.api.attributes.* +import org.gradle.api.file.* +import org.gradle.api.tasks.* +import org.gradle.api.tasks.bundling.* +import org.gradle.api.tasks.compile.* +import org.gradle.jvm.toolchain.* +import org.gradle.kotlin.dsl.* +import org.gradle.work.* +import org.jetbrains.kotlin.gradle.dsl.* + +/** + * This object configures the Java compilation of a JPMS (aka Jigsaw) module descriptor. + * The source file for the module descriptor is expected at /src/module-info.java. + * + * To maintain backwards compatibility with Java 8, the jvm JAR is marked as a multi-release JAR + * with the module-info.class being moved to META-INF/versions/9/module-info.class. + * + * The Java toolchains feature of Gradle is used to detect or provision a JDK 11, + * which is used to compile the module descriptor. + */ +object Java9Modularity { + + /** + * Task that patches `module-info.java` and removes `requires kotlinx.atomicfu` directive. + * + * To have JPMS properly supported, Kotlin compiler **must** be supplied with the correct `module-info.java`. + * The correct module info has to contain `atomicfu` requirement because atomicfu plugin kicks-in **after** + * the compilation process. But `atomicfu` is compile-only dependency that shouldn't be present in the final + * `module-info.java` and that's exactly what this task ensures. + */ + abstract class ProcessModuleInfoFile : DefaultTask() { + @get:InputFile + @get:NormalizeLineEndings + abstract val moduleInfoFile: RegularFileProperty + + @get:OutputFile + abstract val processedModuleInfoFile: RegularFileProperty + + private val projectPath = project.path + + @TaskAction + fun process() { + val sourceFile = moduleInfoFile.get().asFile + if (!sourceFile.exists()) { + throw IllegalStateException("$sourceFile not found in $projectPath") + } + val outputFile = processedModuleInfoFile.get().asFile + sourceFile.useLines { lines -> + outputFile.outputStream().bufferedWriter().use { writer -> + for (line in lines) { + if ("kotlinx.atomicfu" in line) continue + writer.write(line) + writer.newLine() + } + } + } + } + } + + @JvmStatic + fun configure(project: Project) = with(project) { + val javaToolchains = extensions.findByType(JavaToolchainService::class.java) + ?: error("Gradle JavaToolchainService is not available") + val target = when (val kotlin = extensions.getByName("kotlin")) { + is KotlinJvmProjectExtension -> kotlin.target + is KotlinMultiplatformExtension -> kotlin.targets.getByName("jvm") + else -> throw IllegalStateException("Unknown Kotlin project extension in $project") + } + val compilation = target.compilations.getByName("main") + + // Force the use of JARs for compile dependencies, so any JPMS descriptors are picked up. + // For more details, see https://github.com/gradle/gradle/issues/890#issuecomment-623392772 + configurations.getByName(compilation.compileDependencyConfigurationName).attributes { + attribute( + LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, + objects.named(LibraryElements::class, LibraryElements.JAR) + ) + } + + val processModuleInfoFile by tasks.registering(ProcessModuleInfoFile::class) { + moduleInfoFile = file("${target.name.ifEmpty { "." }}/src/module-info.java") + processedModuleInfoFile = project.layout.buildDirectory.file("generated-sources/module-info-processor/module-info.java") + } + + val compileJavaModuleInfo = tasks.register("compileModuleInfoJava", JavaCompile::class.java) { + val moduleName = project.name.replace('-', '.') // this module's name + val compileKotlinTask = + compilation.compileTaskProvider.get() as? org.jetbrains.kotlin.gradle.tasks.KotlinCompile + ?: error("Cannot access Kotlin compile task ${compilation.compileKotlinTaskName}") + val targetDir = compileKotlinTask.destinationDirectory.dir("../java9") + + // Use a Java 11 compiler for the module-info. + javaCompiler = javaToolchains.compilerFor { + languageVersion = JavaLanguageVersion.of(11) + } + + // Always compile kotlin classes before the module descriptor. + dependsOn(compileKotlinTask) + + // Add the module-info source file. + // Note that we use the parent dir and an include filter, + // this is needed for Gradle's module detection to work in + // org.gradle.api.tasks.compile.JavaCompile.createSpec + source(processModuleInfoFile.map { it.processedModuleInfoFile.asFile.get().parentFile }) + val generatedModuleInfoFile = processModuleInfoFile.flatMap { it.processedModuleInfoFile.asFile } + include { it.file == generatedModuleInfoFile.get() } + + // Set the task outputs and destination directory + outputs.dir(targetDir) + destinationDirectory = targetDir + + // Configure JVM compatibility + sourceCompatibility = JavaVersion.VERSION_1_9.toString() + targetCompatibility = JavaVersion.VERSION_1_9.toString() + + // Set the Java release version. + options.release = 9 + + // Ignore warnings about using 'requires transitive' on automatic modules. + // not needed when compiling with recent JDKs, e.g. 17 + options.compilerArgs.add("-Xlint:-requires-transitive-automatic") + + // Patch the compileKotlinJvm output classes into the compilation so exporting packages works correctly. + val destinationDirProperty = compileKotlinTask.destinationDirectory.asFile + options.compilerArgumentProviders.add { + val kotlinCompileDestinationDir = destinationDirProperty.get() + listOf("--patch-module", "$moduleName=$kotlinCompileDestinationDir") + } + + // Use the classpath of the compileKotlinJvm task. + // Also ensure that the module path is used instead of classpath. + classpath = compileKotlinTask.libraries + modularity.inferModulePath = true + } + + tasks.named(target.artifactsTaskName) { + manifest { + attributes("Multi-Release" to true) + } + from(compileJavaModuleInfo) { + // Include **only** file we are interested in as JavaCompile output also contains some tmp files + include("module-info.class") + into("META-INF/versions/9/") + } + } + } +} diff --git a/buildSrc/src/main/kotlin/Platform.kt b/buildSrc/src/main/kotlin/Platform.kt new file mode 100644 index 0000000000..b667a138a8 --- /dev/null +++ b/buildSrc/src/main/kotlin/Platform.kt @@ -0,0 +1,9 @@ +import org.gradle.api.Project + +// Use from Groovy for now +fun platformOf(project: Project): String = + when (project.name.substringAfterLast("-")) { + "js" -> "js" + "common", "native" -> throw IllegalStateException("${project.name} platform is not supported") + else -> "jvm" + } diff --git a/buildSrc/src/main/kotlin/Projects.kt b/buildSrc/src/main/kotlin/Projects.kt new file mode 100644 index 0000000000..48bb938d79 --- /dev/null +++ b/buildSrc/src/main/kotlin/Projects.kt @@ -0,0 +1,47 @@ +@file:JvmName("Projects") + +import org.gradle.api.* +import org.gradle.api.tasks.* + +fun Project.version(target: String): String { + if (target == "kotlin") { + getOverriddenKotlinVersion(this)?.let { return it } + } + return property("${target}_version") as String +} + +val Project.jdkToolchainVersion: Int get() = property("jdk_toolchain_version").toString().toInt() + +/** + * TODO: check if this is still relevant. + * It was introduced in , and the project for which this was + * done is already long finished. + */ +val Project.nativeTargetsAreEnabled: Boolean get() = rootProject.properties["disable_native_targets"] == null + +val Project.sourceSets: SourceSetContainer + get() = extensions.getByName("sourceSets") as SourceSetContainer + +val coreModule = "kotlinx-coroutines-core" +val jdk8ObsoleteModule = "kotlinx-coroutines-jdk8" +val testUtilsModule = "test-utils" + +// Not applicable for Kotlin plugin +val sourceless = setOf("kotlinx.coroutines", "kotlinx-coroutines-bom") + +// Not published +val unpublished = setOf("kotlinx.coroutines", "benchmarks", "android-unit-tests", testUtilsModule) + +val Project.isMultiplatform: Boolean get() = name in setOf(coreModule, "kotlinx-coroutines-test", testUtilsModule) +val Project.isBom: Boolean get() = name == "kotlinx-coroutines-bom" + +// Projects that we do not check for Android API level 14 check due to various limitations +val androidNonCompatibleProjects = setOf( + "kotlinx-coroutines-debug", + "kotlinx-coroutines-swing", + "kotlinx-coroutines-javafx", + "kotlinx-coroutines-jdk8", + "kotlinx-coroutines-jdk9", + "kotlinx-coroutines-reactor", + "kotlinx-coroutines-test" +) diff --git a/buildSrc/src/main/kotlin/Publishing.kt b/buildSrc/src/main/kotlin/Publishing.kt new file mode 100644 index 0000000000..ab82d06c73 --- /dev/null +++ b/buildSrc/src/main/kotlin/Publishing.kt @@ -0,0 +1,212 @@ +@file:Suppress("UnstableApiUsage") + +import groovy.util.Node +import groovy.util.NodeList +import org.gradle.api.Project +import org.gradle.api.XmlProvider +import org.gradle.api.artifacts.dsl.* +import org.gradle.api.publish.PublishingExtension +import org.gradle.api.publish.maven.* +import org.gradle.api.publish.maven.tasks.AbstractPublishToMaven +import org.gradle.api.tasks.* +import org.gradle.api.tasks.bundling.Jar +import org.gradle.kotlin.dsl.* +import org.gradle.plugins.signing.* +import java.net.* + +// Pom configuration + +fun MavenPom.configureMavenCentralMetadata(project: Project) { + name = project.name + description = "Coroutines support libraries for Kotlin" + url = "https://github.com/Kotlin/kotlinx.coroutines" + + licenses { + license { + name = "Apache-2.0" + url = "https://www.apache.org/licenses/LICENSE-2.0.txt" + distribution = "repo" + } + } + + developers { + developer { + id = "JetBrains" + name = "JetBrains Team" + organization = "JetBrains" + organizationUrl = "https://www.jetbrains.com/" + } + } + + scm { + url = "https://github.com/Kotlin/kotlinx.coroutines" + } +} + +/** + * 'libs.space.pub' is a dev option that is set on our CI in order to publish + * dev build into 'https://maven.pkg.jetbrains.space/public/p/kotlinx-coroutines/maven' Maven repository. + * In order to use it, pass the corresponding ENV to the TC 'Deploy' task. + */ +private val spacePublicationEnabled = System.getenv("libs.space.pub")?.equals("true") ?: false + +fun mavenRepositoryUri(): URI { + if (spacePublicationEnabled) { + return URI("https://maven.pkg.jetbrains.space/public/p/kotlinx-coroutines/maven") + } + + val repositoryId: String? = System.getenv("libs.repository.id") + return if (repositoryId == null) { + URI("https://oss.sonatype.org/service/local/staging/deploy/maven2/") + } else { + URI("https://oss.sonatype.org/service/local/staging/deployByRepositoryId/$repositoryId") + } +} + +fun configureMavenPublication(rh: RepositoryHandler, project: Project) { + rh.maven { + url = mavenRepositoryUri() + credentials { + if (spacePublicationEnabled) { + // Configure space credentials + username = project.getSensitiveProperty("libs.space.user") + password = project.getSensitiveProperty("libs.space.password") + } else { + // Configure sonatype credentials + username = project.getSensitiveProperty("libs.sonatype.user") + password = project.getSensitiveProperty("libs.sonatype.password") + } + } + } +} + +fun signPublicationIfKeyPresent(project: Project, publication: MavenPublication) { + val keyId = project.getSensitiveProperty("libs.sign.key.id") + val signingKey = project.getSensitiveProperty("libs.sign.key.private") + val signingKeyPassphrase = project.getSensitiveProperty("libs.sign.passphrase") + if (!signingKey.isNullOrBlank()) { + project.extensions.configure("signing") { + useInMemoryPgpKeys(keyId, signingKey, signingKeyPassphrase) + sign(publication) + } + } +} + +private fun Project.getSensitiveProperty(name: String): String? { + return project.findProperty(name) as? String ?: System.getenv(name) +} + +/** + * This unbelievable piece of engineering^W programming is a workaround for the following issues: + * - https://github.com/gradle/gradle/issues/26132 + * - https://youtrack.jetbrains.com/issue/KT-61313/ + * + * Long story short: + * 1) Single module produces multiple publications + * 2) 'Sign' plugin signs them + * 3) Signature files are re-used, which Gradle detects and whines about an implicit dependency + * + * There are three patterns that we workaround: + * 1) 'Sign' does not depend on 'publish' + * 2) Empty 'javadoc.jar.asc' got reused between publications (kind of a implication of the previous one) + * 3) `klib` signatures are reused where appropriate + * + * It addresses the following failures: + * ``` + * Gradle detected a problem with the following location: 'kotlinx.coroutines/kotlinx-coroutines-core/build/classes/kotlin/macosArm64/main/klib/kotlinx-coroutines-core.klib.asc'. + * Reason: Task ':kotlinx-coroutines-core:linkWorkerTestDebugTestMacosArm64' uses this output of task ':kotlinx-coroutines-core:signMacosArm64Publication' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. + * + * ``` + * and + * ``` + * Gradle detected a problem with the following location: 'kotlinx-coroutines-core/build/libs/kotlinx-coroutines-core-1.7.2-SNAPSHOT-javadoc.jar.asc'. + * Reason: Task ':kotlinx-coroutines-core:publishAndroidNativeArm32PublicationToMavenLocal' uses this output of task ':kotlinx-coroutines-core:signAndroidNativeArm64Publication' without declaring an explicit or implicit dependency. + * ``` + */ +fun Project.establishSignDependencies() { + tasks.withType().configureEach { + val pubName = name.removePrefix("sign").removeSuffix("Publication") + // Gradle#26132 -- establish dependency between sign and link tasks, as well as compile ones + mustRunAfter(tasks.matching { it.name == "linkDebugTest$pubName" }) + mustRunAfter(tasks.matching { it.name == "linkWorkerTestDebugTest$pubName" }) + mustRunAfter(tasks.matching { it.name == "compileTestKotlin$pubName" }) + } + + // Sign plugin issues and publication: + // Establish dependency between 'sign' and 'publish*' tasks + tasks.withType().configureEach { + dependsOn(tasks.withType()) + } +} + +/** + * Re-configure common publication to depend on JVM artifact only in pom.xml. + * It allows us to keep backwards compatibility with pre-multiplatform 'kotlinx-coroutines' publication scheme + * for Maven consumers: + * - Previously, we published 'kotlinx-coroutines-core' as the JVM artifact + * - With a multiplatform enabled as is, 'kotlinx-coroutines-core' is a common artifact not consumable from Maven, + * instead, users should depend on 'kotlinx-coroutines-core-jvm' + * - To keep the compatibility and experience, we do add dependency on 'kotlinx-coroutines-core-jvm' for + * 'kotlinx-coroutines-core' in pom.xml only (e.g. Gradle will keep using the metadata), so Maven users can + * depend on previous coordinates. + * + * Original code comment: + * Publish the platform JAR and POM so that consumers who depend on this module and can't read Gradle module + * metadata can still get the platform artifact and transitive dependencies from the POM. + */ +public fun Project.reconfigureMultiplatformPublication(jvmPublication: MavenPublication) { + val mavenPublications = + extensions.getByType(PublishingExtension::class.java).publications.withType() + val kmpPublication = mavenPublications.getByName("kotlinMultiplatform") + + var jvmPublicationXml: XmlProvider? = null + jvmPublication.pom.withXml { jvmPublicationXml = this } + + kmpPublication.pom.withXml { + val root = asNode() + // Remove the original content and add the content from the platform POM: + root.children().toList().forEach { root.remove(it as Node) } + jvmPublicationXml!!.asNode().children().forEach { root.append(it as Node) } + + // Adjust the self artifact ID, as it should match the root module's coordinates: + ((root["artifactId"] as NodeList).first() as Node).setValue(kmpPublication.artifactId) + + // Set packaging to POM to indicate that there's no artifact: + root.appendNode("packaging", "pom") + + // Remove the original platform dependencies and add a single dependency on the platform module: + val dependencies = (root["dependencies"] as NodeList).first() as Node + dependencies.children().toList().forEach { dependencies.remove(it as Node) } + dependencies.appendNode("dependency").apply { + appendNode("groupId", jvmPublication.groupId) + appendNode("artifactId", jvmPublication.artifactId) + appendNode("version", jvmPublication.version) + appendNode("scope", "compile") + } + } + + // TODO verify if this is still relevant + tasks.matching { it.name == "generatePomFileForKotlinMultiplatformPublication" }.configureEach { + @Suppress("DEPRECATION") + dependsOn(tasks["generatePomFileFor${jvmPublication.name.capitalize()}Publication"]) + } +} + +// Top-level deploy task that publishes all artifacts +public fun Project.registerTopLevelDeployTask() { + assert(this === rootProject) + tasks.register("deploy") { + allprojects { + val publishTasks = tasks.matching { it.name == "publish" } + dependsOn(publishTasks) + } + } +} + +public fun Project.registerEmptyJavadocArtifact(): TaskProvider { + return tasks.register("javadocJar", Jar::class) { + archiveClassifier = "javadoc" + // contents are deliberately left empty + } +} + diff --git a/buildSrc/src/main/kotlin/SourceSets.kt b/buildSrc/src/main/kotlin/SourceSets.kt new file mode 100644 index 0000000000..340fd65cd0 --- /dev/null +++ b/buildSrc/src/main/kotlin/SourceSets.kt @@ -0,0 +1,56 @@ +import org.gradle.api.* +import org.jetbrains.kotlin.gradle.plugin.* +import org.gradle.kotlin.dsl.* + +fun KotlinSourceSet.configureDirectoryPaths() { + if (project.isMultiplatform) { + val srcDir = if (name.endsWith("Main")) "src" else "test" + val platform = name.dropLast(4) + kotlin.srcDir("$platform/$srcDir") + if (name == "jvmMain") { + resources.srcDir("$platform/resources") + } else if (name == "jvmTest") { + resources.srcDir("$platform/test-resources") + } + } else if (platformOf(project) == "jvm") { + when (name) { + "main" -> { + kotlin.srcDir("src") + resources.srcDir("resources") + } + "test" -> { + kotlin.srcDir("test") + resources.srcDir("test-resources") + } + } + } else { + throw IllegalArgumentException("Unclear how to configure source sets for ${project.name}") + } +} + +/** + * Creates shared source sets for a group of source sets. + * + * [reverseDependencies] is a list of prefixes of names of source sets that depend on the new source set. + * [dependencies] is a list of prefixes of names of source sets that the new source set depends on. + * [groupName] is the prefix of the names of the new source sets. + * + * The suffixes of the source sets are "Main" and "Test". + */ +fun NamedDomainObjectContainer.groupSourceSets( + groupName: String, + reverseDependencies: List, + dependencies: List +) { + val sourceSetSuffixes = listOf("Main", "Test") + for (suffix in sourceSetSuffixes) { + register(groupName + suffix) { + for (dep in dependencies) { + dependsOn(get(dep + suffix)) + } + for (revDep in reverseDependencies) { + get(revDep + suffix).dependsOn(this) + } + } + } +} diff --git a/buildSrc/src/main/kotlin/UnpackAar.kt b/buildSrc/src/main/kotlin/UnpackAar.kt new file mode 100644 index 0000000000..88948ac8a8 --- /dev/null +++ b/buildSrc/src/main/kotlin/UnpackAar.kt @@ -0,0 +1,63 @@ +import org.gradle.api.* +import org.gradle.api.artifacts.transform.InputArtifact +import org.gradle.api.artifacts.transform.TransformAction +import org.gradle.api.artifacts.transform.TransformOutputs +import org.gradle.api.artifacts.transform.TransformParameters +import org.gradle.api.attributes.* +import org.gradle.api.file.FileSystemLocation +import org.gradle.api.provider.Provider +import org.gradle.kotlin.dsl.* +import java.io.File +import java.nio.file.Files +import java.util.zip.ZipEntry +import java.util.zip.ZipFile + +// Attributes used by aar dependencies +val artifactType = Attribute.of("artifactType", String::class.java) +val unpackedAar = Attribute.of("unpackedAar", Boolean::class.javaObjectType) + +fun Project.configureAar() = configurations.configureEach { + afterEvaluate { + if (isCanBeResolved && !isCanBeConsumed) { + attributes.attribute(unpackedAar, true) // request all AARs to be unpacked + } + } +} + +fun DependencyHandlerScope.configureAarUnpacking() { + attributesSchema { + attribute(unpackedAar) + } + + artifactTypes { + create("aar") { + attributes.attribute(unpackedAar, false) + } + } + + registerTransform(UnpackAar::class.java) { + from.attribute(unpackedAar, false).attribute(artifactType, "aar") + to.attribute(unpackedAar, true).attribute(artifactType, "jar") + } +} + +@Suppress("UnstableApiUsage") +abstract class UnpackAar : TransformAction { + @get:InputArtifact + abstract val inputArtifact: Provider + + override fun transform(outputs: TransformOutputs) { + ZipFile(inputArtifact.get().asFile).use { zip -> + zip.entries().asSequence() + .filter { !it.isDirectory } + .filter { it.name.endsWith(".jar") } + .forEach { zip.unzip(it, outputs.file(it.name)) } + } + } +} + +private fun ZipFile.unzip(entry: ZipEntry, output: File) { + getInputStream(entry).use { + Files.copy(it, output.toPath()) + } +} diff --git a/buildSrc/src/main/kotlin/VersionFile.kt b/buildSrc/src/main/kotlin/VersionFile.kt new file mode 100644 index 0000000000..46d19840ab --- /dev/null +++ b/buildSrc/src/main/kotlin/VersionFile.kt @@ -0,0 +1,25 @@ +import org.gradle.api.* +import org.gradle.api.tasks.* + +/** + * Adds 'module_name.version' file to the project's JAR META-INF + * for the better toolability. See #2941 + */ +object VersionFile { + fun registerVersionFileTask(project: Project): TaskProvider { + val versionFile = project.layout.buildDirectory.file("${project.name.replace('-', '_')}.version") + val version = project.version.toString() + return project.tasks.register("versionFileTask") { + outputs.file(versionFile) + doLast { + versionFile.get().asFile.writeText(version) + } + } + } + + fun fromVersionFile(target: AbstractCopyTask, versionFileTask: TaskProvider) { + target.from(versionFileTask) { + into("META-INF") + } + } +} diff --git a/buildSrc/src/main/kotlin/animalsniffer-jvm-conventions.gradle.kts b/buildSrc/src/main/kotlin/animalsniffer-jvm-conventions.gradle.kts new file mode 100644 index 0000000000..5ebd5c7248 --- /dev/null +++ b/buildSrc/src/main/kotlin/animalsniffer-jvm-conventions.gradle.kts @@ -0,0 +1,11 @@ +plugins { + id("ru.vyarus.animalsniffer") +} + +project.plugins.withType(JavaPlugin::class.java) { + val signature: Configuration by configurations + dependencies { + signature("net.sf.androidscents.signature:android-api-level-14:4.0_r4@signature") + signature("org.codehaus.mojo.signature:java17:1.0@signature") + } +} diff --git a/buildSrc/src/main/kotlin/animalsniffer-multiplatform-conventions.gradle.kts b/buildSrc/src/main/kotlin/animalsniffer-multiplatform-conventions.gradle.kts new file mode 100644 index 0000000000..08b11d003d --- /dev/null +++ b/buildSrc/src/main/kotlin/animalsniffer-multiplatform-conventions.gradle.kts @@ -0,0 +1,11 @@ +plugins { + id("ru.vyarus.animalsniffer") +} + +project.plugins.withType(KotlinMultiplatformConventionsPlugin::class.java) { + val signature: Configuration by configurations + dependencies { + signature("net.sf.androidscents.signature:android-api-level-14:4.0_r4@signature") + signature("org.codehaus.mojo.signature:java17:1.0@signature") + } +} diff --git a/buildSrc/src/main/kotlin/bom-conventions.gradle.kts b/buildSrc/src/main/kotlin/bom-conventions.gradle.kts new file mode 100644 index 0000000000..14f365c897 --- /dev/null +++ b/buildSrc/src/main/kotlin/bom-conventions.gradle.kts @@ -0,0 +1,16 @@ +import org.gradle.kotlin.dsl.* +import org.jetbrains.kotlin.gradle.dsl.* + + +configure(subprojects.filter { it.name !in unpublished }) { + if (name == "kotlinx-coroutines-bom" || name == "kotlinx.coroutines") return@configure + if (isMultiplatform) { + kotlinExtension.sourceSets.getByName("jvmMain").dependencies { + api(project.dependencies.platform(project(":kotlinx-coroutines-bom"))) + } + } else { + dependencies { + "api"(platform(project(":kotlinx-coroutines-bom"))) + } + } +} diff --git a/buildSrc/src/main/kotlin/configure-compilation-conventions.gradle.kts b/buildSrc/src/main/kotlin/configure-compilation-conventions.gradle.kts new file mode 100644 index 0000000000..26ee664c9c --- /dev/null +++ b/buildSrc/src/main/kotlin/configure-compilation-conventions.gradle.kts @@ -0,0 +1,39 @@ +import org.jetbrains.kotlin.gradle.tasks.* + +configure(subprojects) { + val project = this + if (name in sourceless) return@configure + apply(plugin = "org.jetbrains.kotlinx.atomicfu") + tasks.withType>().configureEach { + val isMainTaskName = name.startsWith("compileKotlin") + compilerOptions { + getOverriddenKotlinLanguageVersion(project)?.let { + languageVersion = it + } + getOverriddenKotlinApiVersion(project)?.let { + apiVersion = it + } + if (isMainTaskName && !unpublished.contains(project.name)) { + allWarningsAsErrors = true + freeCompilerArgs.addAll("-Xexplicit-api=strict", "-Xdont-warn-on-error-suppression") + } + /* Coroutines do not interop with Java and these flags provide a significant + * (i.e. close to double-digit) reduction in both bytecode and optimized dex size */ + if (this@configureEach is KotlinJvmCompile) { + freeCompilerArgs.addAll( + "-Xno-param-assertions", + "-Xno-call-assertions", + "-Xno-receiver-assertions" + ) + } + if (this@configureEach is KotlinNativeCompile) { + optIn.addAll( + "kotlinx.cinterop.ExperimentalForeignApi", + "kotlinx.cinterop.UnsafeNumber", + "kotlin.experimental.ExperimentalNativeApi", + ) + } + } + + } +} diff --git a/buildSrc/src/main/kotlin/dokka-conventions.gradle.kts b/buildSrc/src/main/kotlin/dokka-conventions.gradle.kts new file mode 100644 index 0000000000..966aa98e04 --- /dev/null +++ b/buildSrc/src/main/kotlin/dokka-conventions.gradle.kts @@ -0,0 +1,99 @@ +import org.jetbrains.dokka.gradle.* +import java.net.* + + +plugins { + id("org.jetbrains.dokka") +} + +val knit_version: String by project +private val projetsWithoutDokka = unpublished + "kotlinx-coroutines-bom" + jdk8ObsoleteModule +private val coreModuleDocsUrl = "https://kotlinlang.org/api/kotlinx.coroutines/$coreModule/" +private val coreModuleDocsPackageList = "$projectDir/kotlinx-coroutines-core/build/dokka/htmlPartial/package-list" + +configure(subprojects.filterNot { projetsWithoutDokka.contains(it.name) }) { + apply(plugin = "org.jetbrains.dokka") + configurePathsaver() + condigureDokkaSetup() + configureExternalLinks() +} + +// Setup top-level 'dokkaHtmlMultiModule' with templates +tasks.withType().named("dokkaHtmlMultiModule") { + setupDokkaTemplatesDir(this) +} + +dependencies { + // Add explicit dependency between Dokka and Knit plugin + add("dokkaHtmlMultiModulePlugin", "org.jetbrains.kotlinx:dokka-pathsaver-plugin:$knit_version") +} + +// Dependencies for Knit processing: Knit plugin to work with Dokka +private fun Project.configurePathsaver() { + tasks.withType(DokkaTaskPartial::class).configureEach { + dependencies { + plugins("org.jetbrains.kotlinx:dokka-pathsaver-plugin:$knit_version") + } + } +} + +// Configure Dokka setup +private fun Project.condigureDokkaSetup() { + tasks.withType(DokkaTaskPartial::class).configureEach { + suppressInheritedMembers = true + setupDokkaTemplatesDir(this) + + dokkaSourceSets.configureEach { + jdkVersion = 11 + includes.from("README.md") + noStdlibLink = true + + externalDocumentationLink { + url = URL("https://kotlinlang.org/api/latest/jvm/stdlib/") + packageListUrl = rootProject.projectDir.toPath().resolve("site/stdlib.package.list").toUri().toURL() + } + + // Something suspicious to figure out, probably legacy of earlier days + if (!project.isMultiplatform) { + dependsOn(project.configurations["compileClasspath"]) + } + } + + // Source links + dokkaSourceSets.configureEach { + sourceLink { + localDirectory = rootDir + remoteUrl = URL("https://github.com/kotlin/kotlinx.coroutines/tree/master") + remoteLineSuffix ="#L" + } + } + } +} + +private fun Project.configureExternalLinks() { + tasks.withType() { + dokkaSourceSets.configureEach { + externalDocumentationLink { + url = URL(coreModuleDocsUrl) + packageListUrl = File(coreModuleDocsPackageList).toURI().toURL() + } + } + } +} + +/** + * Setups Dokka templates. While this directory is empty in our repository, + * 'kotlinlang' build pipeline adds templates there when preparing our documentation + * to be published on kotlinlang. + * + * See: + * - Template setup: https://github.com/JetBrains/kotlin-web-site/blob/master/.teamcity/builds/apiReferences/kotlinx/coroutines/KotlinxCoroutinesPrepareDokkaTemplates.kt + * - Templates repository: https://github.com/JetBrains/kotlin-web-site/tree/master/dokka-templates + */ +private fun Project.setupDokkaTemplatesDir(dokkaTask: AbstractDokkaTask) { + dokkaTask.pluginsMapConfiguration = mapOf( + "org.jetbrains.dokka.base.DokkaBase" to """{ "templatesDir" : "${ + project.rootProject.projectDir.toString().replace('\\', '/') + }/dokka-templates" }""" + ) +} diff --git a/buildSrc/src/main/kotlin/java-modularity-conventions.gradle.kts b/buildSrc/src/main/kotlin/java-modularity-conventions.gradle.kts new file mode 100644 index 0000000000..e8df676ea6 --- /dev/null +++ b/buildSrc/src/main/kotlin/java-modularity-conventions.gradle.kts @@ -0,0 +1,13 @@ +// Currently the compilation of the module-info fails for +// kotlinx-coroutines-play-services because it depends on Android JAR's +// which do not have an explicit module-info descriptor. +// Because the JAR's are all named `classes.jar`, +// the automatic module name also becomes `classes`. +// This conflicts since there are multiple JAR's with identical names. +val invalidModules = listOf("kotlinx-coroutines-play-services") + +configure(subprojects.filter { + !unpublished.contains(it.name) && !invalidModules.contains(it.name) && it.extensions.findByName("kotlin") != null +}) { + Java9Modularity.configure(project) +} diff --git a/buildSrc/src/main/kotlin/knit-conventions.gradle.kts b/buildSrc/src/main/kotlin/knit-conventions.gradle.kts new file mode 100644 index 0000000000..e606a514da --- /dev/null +++ b/buildSrc/src/main/kotlin/knit-conventions.gradle.kts @@ -0,0 +1,20 @@ +plugins { + id("kotlinx-knit") +} + +knit { + siteRoot = "https://kotlinlang.org/api/kotlinx.coroutines" + moduleRoots = listOf(".", "integration", "reactive", "ui") + moduleDocs = "build/dokka/htmlPartial" + dokkaMultiModuleRoot = "build/dokka/htmlMultiModule/" +} + +tasks.named("knitPrepare").configure { + val knitTask = this + // In order for knit to operate, it should depend on and collect + // all Dokka outputs from each module + allprojects { + val dokkaTasks = tasks.matching { it.name == "dokkaHtmlMultiModule" } + knitTask.dependsOn(dokkaTasks) + } +} diff --git a/buildSrc/src/main/kotlin/kotlin-jvm-conventions.gradle.kts b/buildSrc/src/main/kotlin/kotlin-jvm-conventions.gradle.kts new file mode 100644 index 0000000000..e467865b8d --- /dev/null +++ b/buildSrc/src/main/kotlin/kotlin-jvm-conventions.gradle.kts @@ -0,0 +1,39 @@ +// Platform-specific configuration to compile JVM modules + +import org.gradle.api.* +import org.jetbrains.kotlin.gradle.dsl.* + +plugins { + kotlin("jvm") +} + +java { + sourceCompatibility = JavaVersion.VERSION_1_8 + targetCompatibility = JavaVersion.VERSION_1_8 +} + +kotlin { + compilerOptions { + jvmTarget = JvmTarget.JVM_1_8 + configureGlobalKotlinArgumentsAndOptIns() + } + jvmToolchain(jdkToolchainVersion) +} + +dependencies { + testImplementation(kotlin("test")) + // Workaround to make addSuppressed work in tests + testImplementation(kotlin("reflect")) + testImplementation(kotlin("stdlib-jdk7")) + testImplementation(kotlin("test-junit")) + testImplementation("junit:junit:${version("junit")}") +} + +tasks.withType { + testLogging { + showStandardStreams = true + events("passed", "failed") + } + val stressTest = project.properties["stressTest"] + if (stressTest != null) systemProperties["stressTest"] = stressTest +} diff --git a/buildSrc/src/main/kotlin/kotlin-multiplatform-conventions.gradle.kts b/buildSrc/src/main/kotlin/kotlin-multiplatform-conventions.gradle.kts new file mode 100644 index 0000000000..f1845cc640 --- /dev/null +++ b/buildSrc/src/main/kotlin/kotlin-multiplatform-conventions.gradle.kts @@ -0,0 +1,156 @@ +import org.gradle.api.* +import org.gradle.api.tasks.testing.logging.* +import org.jetbrains.kotlin.gradle.dsl.* + +plugins { + kotlin("multiplatform") +} + +java { + sourceCompatibility = JavaVersion.VERSION_1_8 + targetCompatibility = JavaVersion.VERSION_1_8 +} + +kotlin { + jvm { + compilations.all { + compileTaskProvider.configure { + compilerOptions { + jvmTarget = JvmTarget.JVM_1_8 + freeCompilerArgs.addAll("-Xjvm-default=disable") + } + } + } + } + jvmToolchain(jdkToolchainVersion) + if (nativeTargetsAreEnabled) { + // According to https://kotlinlang.org/docs/native-target-support.html + // Tier 1 + linuxX64() + macosX64() + macosArm64() + iosSimulatorArm64() + iosX64() + // Tier 2 + linuxArm64() + watchosSimulatorArm64() + watchosX64() + watchosArm32() + watchosArm64() + tvosSimulatorArm64() + tvosX64() + tvosArm64() + iosArm64() + // Tier 3 + androidNativeArm32() + androidNativeArm64() + androidNativeX86() + androidNativeX64() + mingwX64() + watchosDeviceArm64() + } + js { + moduleName = project.name + nodejs() + compilations["main"]?.dependencies { + api("org.jetbrains.kotlinx:atomicfu-js:${version("atomicfu")}") + } + } + @OptIn(org.jetbrains.kotlin.gradle.ExperimentalWasmDsl::class) + wasmJs { + // Module name should be different from the one from JS + // otherwise IC tasks that start clashing different modules with the same module name + moduleName = project.name + "Wasm" + nodejs() + compilations["main"]?.dependencies { + api("org.jetbrains.kotlinx:atomicfu-wasm-js:${version("atomicfu")}") + } + } + @OptIn(org.jetbrains.kotlin.gradle.ExperimentalWasmDsl::class) + wasmWasi { + nodejs() + compilations["main"]?.dependencies { + api("org.jetbrains.kotlinx:atomicfu-wasm-wasi:${version("atomicfu")}") + } + compilations.configureEach { + compileTaskProvider.configure { + compilerOptions { + optIn.add("kotlin.wasm.internal.InternalWasmApi") + } + } + } + } + applyDefaultHierarchyTemplate() + sourceSets { + commonTest { + dependencies { + api("org.jetbrains.kotlin:kotlin-test-common:${version("kotlin")}") + api("org.jetbrains.kotlin:kotlin-test-annotations-common:${version("kotlin")}") + } + } + jvmMain.dependencies { + compileOnly("org.codehaus.mojo:animal-sniffer-annotations:1.20") + // Workaround until https://github.com/JetBrains/kotlin/pull/4999 is picked up + api("org.jetbrains:annotations:23.0.0") + } + jvmTest.dependencies { + api("org.jetbrains.kotlin:kotlin-test:${version("kotlin")}") + // Workaround to make addSuppressed work in tests + api("org.jetbrains.kotlin:kotlin-reflect:${version("kotlin")}") + api("org.jetbrains.kotlin:kotlin-stdlib-jdk7:${version("kotlin")}") + api("org.jetbrains.kotlin:kotlin-test-junit:${version("kotlin")}") + api("junit:junit:${version("junit")}") + } + nativeMain.dependencies { + // workaround for #3968 until this is fixed on atomicfu's side + api("org.jetbrains.kotlinx:atomicfu:0.23.1") + } + jsMain { } + jsTest { + dependencies { + api("org.jetbrains.kotlin:kotlin-test-js:${version("kotlin")}") + } + } + val wasmJsMain by getting { + } + val wasmJsTest by getting { + dependencies { + api("org.jetbrains.kotlin:kotlin-test-wasm-js:${version("kotlin")}") + } + } + val wasmWasiMain by getting { + } + val wasmWasiTest by getting { + dependencies { + api("org.jetbrains.kotlin:kotlin-test-wasm-wasi:${version("kotlin")}") + } + } + groupSourceSets("jsAndWasmJsShared", listOf("js", "wasmJs"), emptyList()) + groupSourceSets("jsAndWasmShared", listOf("jsAndWasmJsShared", "wasmWasi"), listOf("common")) + } + + @OptIn(org.jetbrains.kotlin.gradle.ExperimentalKotlinGradlePluginApi::class) + compilerOptions { + configureGlobalKotlinArgumentsAndOptIns() + freeCompilerArgs.add("-Xexpect-actual-classes") + optIn.add("kotlin.ExperimentalMultiplatform") + } +} + +// Disable intermediate sourceSet compilation because we do not need js-wasm common artifact +tasks.configureEach { + if (name == "compileJsAndWasmSharedMainKotlinMetadata") { + enabled = false + } + if (name == "compileJsAndWasmJsSharedMainKotlinMetadata") { + enabled = false + } +} + +tasks.named("jvmTest", Test::class) { + testLogging { + showStandardStreams = true + events = setOf(TestLogEvent.PASSED, TestLogEvent.FAILED) + } + project.properties["stressTest"]?.let { systemProperty("stressTest", it) } +} diff --git a/buildSrc/src/main/kotlin/kover-conventions.gradle.kts b/buildSrc/src/main/kotlin/kover-conventions.gradle.kts new file mode 100644 index 0000000000..d87e132539 --- /dev/null +++ b/buildSrc/src/main/kotlin/kover-conventions.gradle.kts @@ -0,0 +1,78 @@ +import kotlinx.kover.gradle.plugin.dsl.* + +plugins { + id("org.jetbrains.kotlinx.kover") +} + +val notCovered = sourceless + unpublished + +val expectedCoverage = mutableMapOf( + // These have lower coverage in general, it can be eventually fixed + "kotlinx-coroutines-swing" to 70, // awaitFrame is not tested + "kotlinx-coroutines-javafx" to 35, // JavaFx is not tested on TC because its graphic subsystem cannot be initialized in headless mode + + // Reactor has lower coverage in general due to various fatal error handling features + "kotlinx-coroutines-reactor" to 75 +) + +val conventionProject = project + +subprojects { + val projectName = name + if (projectName in notCovered) return@subprojects + + project.apply(plugin = "org.jetbrains.kotlinx.kover") + conventionProject.dependencies.add("kover", this) + + extensions.configure("kover") { + /* + * Is explicitly enabled on TC in a separate build step. + * Examples: + * ./gradlew :p:check -- doesn't verify coverage + * ./gradlew :p:check -Pkover.enabled=true -- verifies coverage + * ./gradlew :p:koverHtmlReport -Pkover.enabled=true -- generates HTML report + */ + if (properties["kover.enabled"]?.toString()?.toBoolean() != true) { + disable() + } + } + + extensions.configure("kover") { + reports { + total { + html { + htmlDir = conventionProject.layout.buildDirectory.dir("kover/${project.name}/html") + } + + verify { + rule { + /* + * 85 is our baseline that we aim to raise to 90+. + * Missing coverage is typically due to bugs in the agent + * (e.g. signatures deprecated with an error are counted), + * sometimes it's various diagnostic `toString` or `catch` for OOMs/VerificationErrors, + * but some places are definitely worth visiting. + */ + minBound(expectedCoverage[projectName] ?: 85) // COVERED_LINES_PERCENTAGE + } + } + } + } + } +} + +kover { + reports { + total { + verify { + rule { + minBound(85) // COVERED_LINES_PERCENTAGE + } + } + } + } +} + +conventionProject.tasks.register("koverReport") { + dependsOn(conventionProject.tasks.named("koverHtmlReport")) +} diff --git a/buildSrc/src/main/kotlin/pub-conventions.gradle.kts b/buildSrc/src/main/kotlin/pub-conventions.gradle.kts new file mode 100644 index 0000000000..8b7b2b67cb --- /dev/null +++ b/buildSrc/src/main/kotlin/pub-conventions.gradle.kts @@ -0,0 +1,68 @@ +import org.gradle.kotlin.dsl.* + +/* + * For some absolutely cursed reason the name 'publication-conventions' doesn't work in my IDE. + * TODO: recheck after full repair + */ +plugins { + id("maven-publish") + id("signing") +} + +apply(plugin = "maven-publish") +apply(plugin = "signing") + +publishing { + repositories { + configureMavenPublication(this, project) + } + + if (!isMultiplatform && !isBom) { + // Configure java publications for regular non-MPP modules + apply(plugin = "java-library") + + // MPP projects pack their sources automatically, java libraries need to explicitly pack them + project.extensions.getByType(JavaPluginExtension::class.java).withSourcesJar() + + publications { + register("mavenJava", MavenPublication::class) { + from(components["java"]) + } + } + } + + val emptyJavadoc = if (!isBom) registerEmptyJavadocArtifact() else null + publications.withType(MavenPublication::class).all { + pom.configureMavenCentralMetadata(project) + signPublicationIfKeyPresent(project, this) + if (!isBom && name != "kotlinMultiplatform") { + artifact(emptyJavadoc) + } + + val type = name + when (type) { + "kotlinMultiplatform" -> { + // With Kotlin 1.4 & HMPP, the root module should have no suffix in the ID, but for compatibility with + // the consumers who can't read Gradle module metadata, we publish the JVM artifacts in it, too + artifactId = project.name + project.reconfigureMultiplatformPublication(publications.getByName("jvm") as MavenPublication) + } + + "metadata", "jvm", "js", "native" -> { + artifactId = "${project.name}-$type" + } + } + } + + project.establishSignDependencies() +} + + +// Legacy from https://github.com/Kotlin/kotlinx.coroutines/pull/2031 +// Should be fixed with the rest of the hacks around publication +tasks.matching { it.name == "generatePomFileForKotlinMultiplatformPublication" }.configureEach { + dependsOn(tasks.matching { it.name == "generatePomFileForJvmPublication" }) +} + +// Compatibility with old TeamCity configurations that perform :kotlinx-coroutines-core:bintrayUpload +tasks.register("bintrayUpload") { dependsOn(tasks.matching { it.name == "publish" }) } diff --git a/buildSrc/src/main/kotlin/source-set-conventions.gradle.kts b/buildSrc/src/main/kotlin/source-set-conventions.gradle.kts new file mode 100644 index 0000000000..b9b080f9ae --- /dev/null +++ b/buildSrc/src/main/kotlin/source-set-conventions.gradle.kts @@ -0,0 +1,9 @@ +import org.jetbrains.kotlin.gradle.dsl.* + +// Redefine source sets because we are not using 'kotlin/main/fqn' folder convention +// TODO: port benchmarks to the same scheme +configure(subprojects.filter { !sourceless.contains(it.name) && it.name != "benchmarks" }) { + kotlinExtension.sourceSets.forEach { + it.configureDirectoryPaths() + } +} diff --git a/buildSrc/src/main/kotlin/version-file-conventions.gradle.kts b/buildSrc/src/main/kotlin/version-file-conventions.gradle.kts new file mode 100644 index 0000000000..5a807ef5a7 --- /dev/null +++ b/buildSrc/src/main/kotlin/version-file-conventions.gradle.kts @@ -0,0 +1,13 @@ +import org.gradle.api.tasks.bundling.* + +configure(subprojects.filter { !unpublished.contains(it.name) && it.name !in sourceless }) { + val project = this + val jarTaskName = when { + isMultiplatform -> "jvmJar" + else -> "jar" + } + val versionFileTask = VersionFile.registerVersionFileTask(project) + tasks.withType(Jar::class.java).named(jarTaskName) { + VersionFile.fromVersionFile(this, versionFileTask) + } +} diff --git a/bump-version.sh b/bump-version.sh new file mode 100755 index 0000000000..369c88668d --- /dev/null +++ b/bump-version.sh @@ -0,0 +1,96 @@ +#!/bin/sh + +set -efu + +# the list of files that need to have the version updated in them +# +# limitations: +# * no newlines in names +# * no ' char in names +files=" +README.md +kotlinx-coroutines-core/README.md +kotlinx-coroutines-debug/README.md +kotlinx-coroutines-test/README.md +ui/coroutines-guide-ui.md +gradle.properties +integration-testing/gradle.properties +" + +# read gradle.properties to get the old version +set +e +old_version="$(git grep -hoP '(?<=^version=).*(?=-SNAPSHOT$)' gradle.properties)" +set -e +if [ "$?" -ne 0 ] + then + echo "Could not read the old version from gradle.properties." >&2 + if [ "$#" -ne 2 ] + then + echo "Please use this form instead: ./bump-version.sh old_version new_version" + exit 1 + fi +fi + +# check the command-line arguments for mentions of the version +if [ "$#" -eq 2 ] + then + echo "If you want to infer the version automatically, use the form: ./bump-version.sh new_version" >&2 + if [ -n "$old_version" -a "$1" != "$old_version" ] + then + echo "The provided old version ($1) is different from the one in gradle.properties ($old_version)." >&2 + echo "Proceeding anyway with the provided old version." >&2 + fi + old_version=$1 + new_version=$2 + elif [ "$#" -eq 1 ] + then + new_version=$1 + else + echo "Use: ./bump-version.sh new_version" >&2 + exit 1 +fi + + +# Escape dots, e.g. 1.0.0 -> 1\.0\.0 +escaped_old_version="$(printf "%s\n" "$old_version" | sed 's/[.]/\\./g')" + +update_version() { + file=$1 + to_undo=$2 + echo "Updating version from '$old_version' to '$new_version' in $1" >&2 + if [ -n "$(git diff --name-status -- "$file")" ] + then + printf "There are unstaged changes in '$file'. Refusing to proceed.\n" >&2 + [ -z "$to_undo" ] || eval "git checkout$to_undo" + exit 1 + fi + sed -i.bak "s/$escaped_old_version/$new_version/g" "$file" + rm -f "$1.bak" +} + +to_undo=$(printf "%s" "$files" | while read -r file; do + if [ -n "$file" ] + then + update_version "$file" "${to_undo:-}" + to_undo="${to_undo:-} '$file'" + echo -n " '$file'" + fi +done) + +set +e +version_mentions=$( + find . -type f \( -iname '*.properties' -o -iname '*.md' \) \ + -not -iname CHANGES.md -not -iname CHANGES_UP_TO_1.7.md \ + -not -path ./integration/kotlinx-coroutines-jdk8/README.md \ + -exec git grep --fixed-strings --word "$old_version" {} + + ) +set -e +if [ -z "$version_mentions" ] +then + echo "Done. To undo, run this command:" >&2 + printf "git checkout%s\n" "$to_undo" >&2 +else + echo "ERROR: Previous version is present in the project: $version_mentions" + [ -z "$to_undo" ] || eval "git checkout$to_undo" + exit 1 +fi diff --git a/coroutines-guide.md b/coroutines-guide.md index a79084663c..3cc035ae6a 100644 --- a/coroutines-guide.md +++ b/coroutines-guide.md @@ -1,2231 +1,3 @@ - - - - -# Guide to kotlinx.coroutines by example - -This is a short guide on core features of `kotlinx.coroutines` with a series of examples. - -## Introduction and setup - -Kotlin, as a language, provides only minimal low-level APIs in its standard library to enable various other -libraries to utilize coroutines. Unlike many other languages with similar capabilities, `async` and `await` -are not keywords in Kotlin and are not even part of its standard library. - -`kotlinx.coroutines` is one such rich library. It contains a number of high-level -coroutine-enabled primitives that this guide covers, including `async` and `await`. -You need to add a dependency on `kotlinx-coroutines-core` module as explained -[here](README.md#using-in-your-projects) to use primitives from this guide in your projects. - -## Table of contents - - - -* [Coroutine basics](#coroutine-basics) - * [Your first coroutine](#your-first-coroutine) - * [Bridging blocking and non-blocking worlds](#bridging-blocking-and-non-blocking-worlds) - * [Waiting for a job](#waiting-for-a-job) - * [Extract function refactoring](#extract-function-refactoring) - * [Coroutines ARE light-weight](#coroutines-are-light-weight) - * [Coroutines are like daemon threads](#coroutines-are-like-daemon-threads) -* [Cancellation and timeouts](#cancellation-and-timeouts) - * [Cancelling coroutine execution](#cancelling-coroutine-execution) - * [Cancellation is cooperative](#cancellation-is-cooperative) - * [Making computation code cancellable](#making-computation-code-cancellable) - * [Closing resources with finally](#closing-resources-with-finally) - * [Run non-cancellable block](#run-non-cancellable-block) - * [Timeout](#timeout) -* [Composing suspending functions](#composing-suspending-functions) - * [Sequential by default](#sequential-by-default) - * [Concurrent using async](#concurrent-using-async) - * [Lazily started async](#lazily-started-async) - * [Async-style functions](#async-style-functions) -* [Coroutine context and dispatchers](#coroutine-context-and-dispatchers) - * [Dispatchers and threads](#dispatchers-and-threads) - * [Unconfined vs confined dispatcher](#unconfined-vs-confined-dispatcher) - * [Debugging coroutines and threads](#debugging-coroutines-and-threads) - * [Jumping between threads](#jumping-between-threads) - * [Job in the context](#job-in-the-context) - * [Children of a coroutine](#children-of-a-coroutine) - * [Combining contexts](#combining-contexts) - * [Naming coroutines for debugging](#naming-coroutines-for-debugging) - * [Cancellation via explicit job](#cancellation-via-explicit-job) -* [Channels](#channels) - * [Channel basics](#channel-basics) - * [Closing and iteration over channels](#closing-and-iteration-over-channels) - * [Building channel producers](#building-channel-producers) - * [Pipelines](#pipelines) - * [Prime numbers with pipeline](#prime-numbers-with-pipeline) - * [Fan-out](#fan-out) - * [Fan-in](#fan-in) - * [Buffered channels](#buffered-channels) - * [Channels are fair](#channels-are-fair) -* [Shared mutable state and concurrency](#shared-mutable-state-and-concurrency) - * [The problem](#the-problem) - * [Volatiles are of no help](#volatiles-are-of-no-help) - * [Thread-safe data structures](#thread-safe-data-structures) - * [Thread confinement fine-grained](#thread-confinement-fine-grained) - * [Thread confinement coarse-grained](#thread-confinement-coarse-grained) - * [Mutual exclusion](#mutual-exclusion) - * [Actors](#actors) -* [Select expression](#select-expression) - * [Selecting from channels](#selecting-from-channels) - * [Selecting on close](#selecting-on-close) - * [Selecting to send](#selecting-to-send) - * [Selecting deferred values](#selecting-deferred-values) - * [Switch over a channel of deferred values](#switch-over-a-channel-of-deferred-values) -* [Further reading](#further-reading) - - - -## Coroutine basics - -This section covers basic coroutine concepts. - -### Your first coroutine - -Run the following code: - -```kotlin -fun main(args: Array) { - launch(CommonPool) { // create new coroutine in common thread pool - delay(1000L) // non-blocking delay for 1 second (default time unit is ms) - println("World!") // print after delay - } - println("Hello,") // main function continues while coroutine is delayed - Thread.sleep(2000L) // block main thread for 2 seconds to keep JVM alive -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-01.kt) - -Run this code: - -```text -Hello, -World! -``` - - - -Essentially, coroutines are light-weight threads. -They are launched with [launch] _coroutine builder_. -You can achieve the same result replacing -`launch(CommonPool) { ... }` with `thread { ... }` and `delay(...)` with `Thread.sleep(...)`. Try it. - -If you start by replacing `launch(CommonPool)` by `thread`, the compiler produces the following error: - -``` -Error: Kotlin: Suspend functions are only allowed to be called from a coroutine or another suspend function -``` - -That is because [delay] is a special _suspending function_ that does not block a thread, but _suspends_ -coroutine and it can be only used from a coroutine. - -### Bridging blocking and non-blocking worlds - -The first example mixes _non-blocking_ `delay(...)` and _blocking_ `Thread.sleep(...)` in the same -code of `main` function. It is easy to get lost. Let's cleanly separate blocking and non-blocking -worlds by using [runBlocking]: - -```kotlin -fun main(args: Array) = runBlocking { // start main coroutine - launch(CommonPool) { // create new coroutine in common thread pool - delay(1000L) - println("World!") - } - println("Hello,") // main coroutine continues while child is delayed - delay(2000L) // non-blocking delay for 2 seconds to keep JVM alive -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-02.kt) - - - -The result is the same, but this code uses only non-blocking [delay]. - -`runBlocking { ... }` works as an adaptor that is used here to start the top-level main coroutine. -The regular code outside of `runBlocking` _blocks_, until the coroutine inside `runBlocking` is active. - -This is also a way to write unit-tests for suspending functions: - -```kotlin -class MyTest { - @Test - fun testMySuspendingFunction() = runBlocking { - // here we can use suspending functions using any assertion style that we like - } -} -``` - - - -### Waiting for a job - -Delaying for a time while another coroutine is working is not a good approach. Let's explicitly -wait (in a non-blocking way) until the background [Job] that we have launched is complete: - -```kotlin -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { // create new coroutine and keep a reference to its Job - delay(1000L) - println("World!") - } - println("Hello,") - job.join() // wait until child coroutine completes -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-03.kt) - - - -Now the result is still the same, but the code of the main coroutine is not tied to the duration of -the background job in any way. Much better. - -### Extract function refactoring - -Let's extract the block of code inside `launch(CommonPool) { ... }` into a separate function. When you -perform "Extract function" refactoring on this code you get a new function with `suspend` modifier. -That is your first _suspending function_. Suspending functions can be used inside coroutines -just like regular functions, but their additional feature is that they can, in turn, -use other suspending functions, like `delay` in this example, to _suspend_ execution of a coroutine. - -```kotlin -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { doWorld() } - println("Hello,") - job.join() -} - -// this is your first suspending function -suspend fun doWorld() { - delay(1000L) - println("World!") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-04.kt) - - - -### Coroutines ARE light-weight - -Run the following code: - -```kotlin -fun main(args: Array) = runBlocking { - val jobs = List(100_000) { // create a lot of coroutines and list their jobs - launch(CommonPool) { - delay(1000L) - print(".") - } - } - jobs.forEach { it.join() } // wait for all jobs to complete -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-05.kt) - - - -It starts 100K coroutines and, after a second, each coroutine prints a dot. -Now, try that with threads. What would happen? (Most likely your code will produce some sort of out-of-memory error) - -### Coroutines are like daemon threads - -The following code launches a long-running coroutine that prints "I'm sleeping" twice a second and then -returns from the main function after some delay: - -```kotlin -fun main(args: Array) = runBlocking { - launch(CommonPool) { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } - delay(1300L) // just quit after delay -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-06.kt) - -You can run and see that it prints three lines and terminates: - -```text -I'm sleeping 0 ... -I'm sleeping 1 ... -I'm sleeping 2 ... -``` - - - -Active coroutines do not keep the process alive. They are like daemon threads. - -## Cancellation and timeouts - -This section covers coroutine cancellation and timeouts. - -### Cancelling coroutine execution - -In small application the return from "main" method might sound like a good idea to get all coroutines -implicitly terminated. In a larger, long-running application, you need finer-grained control. -The [launch] function returns a [Job] that can be used to cancel running coroutine: - -```kotlin -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to ensure it was cancelled indeed - println("main: Now I can quit.") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-01.kt) - -It produces the following output: - -```text -I'm sleeping 0 ... -I'm sleeping 1 ... -I'm sleeping 2 ... -main: I'm tired of waiting! -main: Now I can quit. -``` - - - -As soon as main invokes `job.cancel`, we don't see any output from the other coroutine because it was cancelled. - -### Cancellation is cooperative - -Coroutine cancellation is _cooperative_. A coroutine code has to cooperate to be cancellable. -All the suspending functions in `kotlinx.coroutines` are _cancellable_. They check for cancellation of -coroutine and throw [CancellationException] when cancelled. However, if a coroutine is working in -a computation and does not check for cancellation, then it cannot be cancelled, like the following -example shows: - -```kotlin -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - var nextPrintTime = 0L - var i = 0 - while (i < 10) { // computation loop - val currentTime = System.currentTimeMillis() - if (currentTime >= nextPrintTime) { - println("I'm sleeping ${i++} ...") - nextPrintTime = currentTime + 500L - } - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to see if it was cancelled.... - println("main: Now I can quit.") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-02.kt) - -Run it to see that it continues to print "I'm sleeping" even after cancellation. - - - -### Making computation code cancellable - -There are two approaches to making computation code cancellable. The first one is to periodically -invoke a suspending function. There is a [yield] function that is a good choice for that purpose. -The other one is to explicitly check the cancellation status. Let us try the later approach. - -Replace `while (true)` in the previous example with `while (isActive)` and rerun it. - -```kotlin -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - var nextPrintTime = 0L - var i = 0 - while (isActive) { // cancellable computation loop - val currentTime = System.currentTimeMillis() - if (currentTime >= nextPrintTime) { - println("I'm sleeping ${i++} ...") - nextPrintTime = currentTime + 500L - } - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to see if it was cancelled.... - println("main: Now I can quit.") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-03.kt) - -As you can see, now this loop can be cancelled. [isActive][CoroutineScope.isActive] is a property that is available inside -the code of coroutines via [CoroutineScope] object. - - - -### Closing resources with finally - -Cancellable suspending functions throw [CancellationException] on cancellation which can be handled in -all the usual way. For example, the `try {...} finally {...}` and Kotlin `use` function execute their -finalization actions normally when coroutine is cancelled: - -```kotlin -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - try { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } finally { - println("I'm running finally") - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to ensure it was cancelled indeed - println("main: Now I can quit.") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-04.kt) - -The example above produces the following output: - -```text -I'm sleeping 0 ... -I'm sleeping 1 ... -I'm sleeping 2 ... -main: I'm tired of waiting! -I'm running finally -main: Now I can quit. -``` - - - -### Run non-cancellable block - -Any attempt to use a suspending function in the `finally` block of the previous example will cause -[CancellationException], because the coroutine running this code is cancelled. Usually, this is not a -problem, since all well-behaving closing operations (closing a file, cancelling a job, or closing any kind of a -communication channel) are usually non-blocking and do not involve any suspending functions. However, in the -rare case when you need to suspend in the cancelled coroutine you can wrap the corresponding code in -`run(NonCancellable) {...}` using [run] function and [NonCancellable] context as the following example shows: - -```kotlin -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - try { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } finally { - run(NonCancellable) { - println("I'm running finally") - delay(1000L) - println("And I've just delayed for 1 sec because I'm non-cancellable") - } - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to ensure it was cancelled indeed - println("main: Now I can quit.") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-05.kt) - - - -### Timeout - -The most obvious reason to cancel coroutine execution in practice, -is because its execution time has exceeded some timeout. -While you can manually track the reference to the corresponding [Job] and launch a separate coroutine to cancel -the tracked one after delay, there is a ready to use [withTimeout] function that does it. -Look at the following example: - -```kotlin -fun main(args: Array) = runBlocking { - withTimeout(1300L) { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-06.kt) - -It produces the following output: - -```text -I'm sleeping 0 ... -I'm sleeping 1 ... -I'm sleeping 2 ... -Exception in thread "main" kotlinx.coroutines.experimental.TimeoutException: Timed out waiting for 1300 MILLISECONDS -``` - - - -The `TimeoutException` that is thrown by [withTimeout] is a private subclass of [CancellationException]. -We have not seen its stack trace printed on the console before. That is because -inside a cancelled coroutine `CancellationException` is considered to be a normal reason for coroutine completion. -However, in this example we have used `withTimeout` right inside the `main` function. - -Because cancellation is just an exception, all the resources will be closed in a usual way. -You can wrap the code with timeout in `try {...} catch (e: CancellationException) {...}` block if -you need to do some additional action specifically on timeout. - -## Composing suspending functions - -This section covers various approaches to composition of suspending functions. - -### Sequential by default - -Assume that we have two suspending functions defined elsewhere that do something useful like some kind of -remote service call or computation. We just pretend they are useful, but actually each one just -delays for a second for the purpose of this example: - - - -```kotlin -suspend fun doSomethingUsefulOne(): Int { - delay(1000L) // pretend we are doing something useful here - return 13 -} - -suspend fun doSomethingUsefulTwo(): Int { - delay(1000L) // pretend we are doing something useful here, too - return 29 -} -``` - - - -What do we do if need to invoke them _sequentially_ -- first `doSomethingUsefulOne` _and then_ -`doSomethingUsefulTwo` and compute the sum of their results? -In practise we do this if we use the results of the first function to make a decision on whether we need -to invoke the second one or to decide on how to invoke it. - -We just use a normal sequential invocation, because the code in the coroutine, just like in the regular -code, is _sequential_ by default. The following example demonstrates it by measuring the total -time it takes to execute both suspending functions: - -```kotlin -fun main(args: Array) = runBlocking { - val time = measureTimeMillis { - val one = doSomethingUsefulOne() - val two = doSomethingUsefulTwo() - println("The answer is ${one + two}") - } - println("Completed in $time ms") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-01.kt) - -It produces something like this: - -```text -The answer is 42 -Completed in 2017 ms -``` - - - -### Concurrent using async - -What if there are no dependencies between invocation of `doSomethingUsefulOne` and `doSomethingUsefulTwo` and -we want to get the answer faster, by doing both _concurrently_? This is where [async] comes to help. - -Conceptually, [async] is just like [launch]. It starts a separate coroutine which is a light-weight thread -that works concurrently with all the other coroutines. The difference is that `launch` returns a [Job] and -does not carry any resulting value, while `async` returns a [Deferred] -- a light-weight non-blocking future -that represents a promise to provide a result later. You can use `.await()` on a deferred value to get its eventual result, -but `Deferred` is also a `Job`, so you can cancel it if needed. - -```kotlin -fun main(args: Array) = runBlocking { - val time = measureTimeMillis { - val one = async(CommonPool) { doSomethingUsefulOne() } - val two = async(CommonPool) { doSomethingUsefulTwo() } - println("The answer is ${one.await() + two.await()}") - } - println("Completed in $time ms") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-02.kt) - -It produces something like this: - -```text -The answer is 42 -Completed in 1017 ms -``` - - - -This is twice as fast, because we have concurrent execution of two coroutines. -Note, that concurrency with coroutines is always explicit. - -### Lazily started async - -There is a laziness option to [async] with [CoroutineStart.LAZY] parameter. -It starts coroutine only when its result is needed by some -[await][Deferred.await] or if a [start][Job.start] function -is invoked. Run the following example that differs from the previous one only by this option: - -```kotlin -fun main(args: Array) = runBlocking { - val time = measureTimeMillis { - val one = async(CommonPool, CoroutineStart.LAZY) { doSomethingUsefulOne() } - val two = async(CommonPool, CoroutineStart.LAZY) { doSomethingUsefulTwo() } - println("The answer is ${one.await() + two.await()}") - } - println("Completed in $time ms") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-03.kt) - -It produces something like this: - -```text -The answer is 42 -Completed in 2017 ms -``` - - - -So, we are back to sequential execution, because we _first_ start and await for `one`, _and then_ start and await -for `two`. It is not the intended use-case for laziness. It is designed as a replacement for -the standard `lazy` function in cases when computation of the value involves suspending functions. - -### Async-style functions - -We can define async-style functions that invoke `doSomethingUsefulOne` and `doSomethingUsefulTwo` -_asynchronously_ using [async] coroutine builder. It is a good style to name such functions with -either "async" prefix of "Async" suffix to highlight the fact that they only start asynchronous -computation and one needs to use the resulting deferred value to get the result. - -```kotlin -// The result type of asyncSomethingUsefulOne is Deferred -fun asyncSomethingUsefulOne() = async(CommonPool) { - doSomethingUsefulOne() -} - -// The result type of asyncSomethingUsefulTwo is Deferred -fun asyncSomethingUsefulTwo() = async(CommonPool) { - doSomethingUsefulTwo() -} -``` - -Note, that these `asyncXXX` function are **not** _suspending_ functions. They can be used from anywhere. -However, their use always implies asynchronous (here meaning _concurrent_) execution of their action -with the invoking code. - -The following example shows their use outside of coroutine: - -```kotlin -// note, that we don't have `runBlocking` to the right of `main` in this example -fun main(args: Array) { - val time = measureTimeMillis { - // we can initiate async actions outside of a coroutine - val one = asyncSomethingUsefulOne() - val two = asyncSomethingUsefulTwo() - // but waiting for a result must involve either suspending or blocking. - // here we use `runBlocking { ... }` to block the main thread while waiting for the result - runBlocking { - println("The answer is ${one.await() + two.await()}") - } - } - println("Completed in $time ms") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-04.kt) - - - -## Coroutine context and dispatchers - -We've already seen `launch(CommonPool) {...}`, `async(CommonPool) {...}`, `run(NonCancellable) {...}`, etc. -In these code snippets [CommonPool] and [NonCancellable] are _coroutine contexts_. -This section covers other available choices. - -### Dispatchers and threads - -Coroutine context includes a [_coroutine dispatcher_][CoroutineDispatcher] which determines what thread or threads -the corresponding coroutine uses for its execution. Coroutine dispatcher can confine coroutine execution -to a specific thread, dispatch it to a thread pool, or let it run unconfined. Try the following example: - -```kotlin -fun main(args: Array) = runBlocking { - val jobs = arrayListOf() - jobs += launch(Unconfined) { // not confined -- will work with main thread - println(" 'Unconfined': I'm working in thread ${Thread.currentThread().name}") - } - jobs += launch(context) { // context of the parent, runBlocking coroutine - println(" 'context': I'm working in thread ${Thread.currentThread().name}") - } - jobs += launch(CommonPool) { // will get dispatched to ForkJoinPool.commonPool (or equivalent) - println(" 'CommonPool': I'm working in thread ${Thread.currentThread().name}") - } - jobs += launch(newSingleThreadContext("MyOwnThread")) { // will get its own new thread - println(" 'newSTC': I'm working in thread ${Thread.currentThread().name}") - } - jobs.forEach { it.join() } -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-context-01.kt) - -It produces the following output (maybe in different order): - -```text - 'Unconfined': I'm working in thread main - 'CommonPool': I'm working in thread ForkJoinPool.commonPool-worker-1 - 'newSTC': I'm working in thread MyOwnThread - 'context': I'm working in thread main -``` - - - -The difference between parent [context][CoroutineScope.context] and [Unconfined] context will be shown later. - -### Unconfined vs confined dispatcher - -The [Unconfined] coroutine dispatcher starts coroutine in the caller thread, but only until the -first suspension point. After suspension it resumes in the thread that is fully determined by the -suspending function that was invoked. Unconfined dispatcher is appropriate when coroutine does not -consume CPU time nor updates any shared data (like UI) that is confined to a specific thread. - -On the other side, [context][CoroutineScope.context] property that is available inside the block of any coroutine -via [CoroutineScope] interface, is a reference to a context of this particular coroutine. -This way, a parent context can be inherited. The default context of [runBlocking], in particular, -is confined to be invoker thread, so inheriting it has the effect of confining execution to -this thread with a predictable FIFO scheduling. - -```kotlin -fun main(args: Array) = runBlocking { - val jobs = arrayListOf() - jobs += launch(Unconfined) { // not confined -- will work with main thread - println(" 'Unconfined': I'm working in thread ${Thread.currentThread().name}") - delay(500) - println(" 'Unconfined': After delay in thread ${Thread.currentThread().name}") - } - jobs += launch(context) { // context of the parent, runBlocking coroutine - println(" 'context': I'm working in thread ${Thread.currentThread().name}") - delay(1000) - println(" 'context': After delay in thread ${Thread.currentThread().name}") - } - jobs.forEach { it.join() } -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-context-02.kt) - -Produces the output: - -```text - 'Unconfined': I'm working in thread main - 'context': I'm working in thread main - 'Unconfined': After delay in thread kotlinx.coroutines.ScheduledExecutor - 'context': After delay in thread main -``` - - - -So, the coroutine that had inherited `context` of `runBlocking {...}` continues to execute in the `main` thread, -while the unconfined one had resumed in the scheduler thread that [delay] function is using. - -### Debugging coroutines and threads - -Coroutines can suspend on one thread and resume on another thread with [Unconfined] dispatcher or -with a multi-threaded dispatcher like [CommonPool]. Even with a single-threaded dispatcher it might be hard to -figure out what coroutine was doing what, where, and when. The common approach to debugging applications with -threads is to print the thread name in the log file on each log statement. This feature is universally supported -by logging frameworks. When using coroutines, the thread name alone does not give much of a context, so -`kotlinx.coroutines` includes debugging facilities to make it easier. - -Run the following code with `-Dkotlinx.coroutines.debug` JVM option: - -```kotlin -fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") - -fun main(args: Array) = runBlocking { - val a = async(context) { - log("I'm computing a piece of the answer") - 6 - } - val b = async(context) { - log("I'm computing another piece of the answer") - 7 - } - log("The answer is ${a.await() * b.await()}") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-context-03.kt) - -There are three coroutines. The main coroutine (#1) -- `runBlocking` one, -and two coroutines computing deferred values `a` (#2) and `b` (#3). -They are all executing in the context of `runBlocking` and are confined to the main thread. -The output of this code is: - -```text -[main @coroutine#2] I'm computing a piece of the answer -[main @coroutine#3] I'm computing another piece of the answer -[main @coroutine#1] The answer is 42 -``` - - - -The `log` function prints the name of the thread in square brackets and you can see, that it is the `main` -thread, but the identifier of the currently executing coroutine is appended to it. This identifier -is consecutively assigned to all created coroutines when debugging mode is turned on. - -You can read more about debugging facilities in the documentation for [newCoroutineContext] function. - -### Jumping between threads - -Run the following code with `-Dkotlinx.coroutines.debug` JVM option: - -```kotlin -fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") - -fun main(args: Array) { - val ctx1 = newSingleThreadContext("Ctx1") - val ctx2 = newSingleThreadContext("Ctx2") - runBlocking(ctx1) { - log("Started in ctx1") - run(ctx2) { - log("Working in ctx2") - } - log("Back to ctx1") - } -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-context-04.kt) - -It demonstrates two new techniques. One is using [runBlocking] with an explicitly specified context, and -the second one is using [run] function to change a context of a coroutine while still staying in the -same coroutine as you can see in the output below: - -```text -[Ctx1 @coroutine#1] Started in ctx1 -[Ctx2 @coroutine#1] Working in ctx2 -[Ctx1 @coroutine#1] Back to ctx1 -``` - - - -### Job in the context - -The coroutine [Job] is part of its context. The coroutine can retrieve it from its own context -using `context[Job]` expression: - -```kotlin -fun main(args: Array) = runBlocking { - println("My job is ${context[Job]}") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-context-05.kt) - -It produces somethine like - -``` -My job is BlockingCoroutine{Active}@65ae6ba4 -``` - - - -So, [isActive][CoroutineScope.isActive] in [CoroutineScope] is just a convenient shortcut for `context[Job]!!.isActive`. - -### Children of a coroutine - -When [context][CoroutineScope.context] of a coroutine is used to launch another coroutine, -the [Job] of the new coroutine becomes -a _child_ of the parent coroutine's job. When the parent coroutine is cancelled, all its children -are recursively cancelled, too. - -```kotlin -fun main(args: Array) = runBlocking { - // start a coroutine to process some kind of incoming request - val request = launch(CommonPool) { - // it spawns two other jobs, one with its separate context - val job1 = launch(CommonPool) { - println("job1: I have my own context and execute independently!") - delay(1000) - println("job1: I am not affected by cancellation of the request") - } - // and the other inherits the parent context - val job2 = launch(context) { - println("job2: I am a child of the request coroutine") - delay(1000) - println("job2: I will not execute this line if my parent request is cancelled") - } - // request completes when both its sub-jobs complete: - job1.join() - job2.join() - } - delay(500) - request.cancel() // cancel processing of the request - delay(1000) // delay a second to see what happens - println("main: Who has survived request cancellation?") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-context-06.kt) - -The output of this code is: - -```text -job1: I have my own context and execute independently! -job2: I am a child of the request coroutine -job1: I am not affected by cancellation of the request -main: Who has survived request cancellation? -``` - - - -### Combining contexts - -Coroutine context can be combined using `+` operator. The context on the right-hand side replaces relevant entries -of the context on the left-hand side. For example, a [Job] of the parent coroutine can be inherited, while -its dispatcher replaced: - -```kotlin -fun main(args: Array) = runBlocking { - // start a coroutine to process some kind of incoming request - val request = launch(context) { // use the context of `runBlocking` - // spawns CPU-intensive child job in CommonPool !!! - val job = launch(context + CommonPool) { - println("job: I am a child of the request coroutine, but with a different dispatcher") - delay(1000) - println("job: I will not execute this line if my parent request is cancelled") - } - job.join() // request completes when its sub-job completes - } - delay(500) - request.cancel() // cancel processing of the request - delay(1000) // delay a second to see what happens - println("main: Who has survived request cancellation?") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-context-07.kt) - -The expected outcome of this code is: - -```text -job: I am a child of the request coroutine, but with a different dispatcher -main: Who has survived request cancellation? -``` - - - -### Naming coroutines for debugging - -Automatically assigned ids are good when coroutines log often and you just need to correlate log records -coming from the same coroutine. However, when coroutine is tied to the processing of a specific request -or doing some specific background task, it is better to name it explicitly for debugging purposes. -[CoroutineName] serves the same function as a thread name. It'll get displayed in the thread name that -is executing this coroutine when debugging more is turned on. - -The following example demonstrates this concept: - -```kotlin -fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") - -fun main(args: Array) = runBlocking(CoroutineName("main")) { - log("Started main coroutine") - // run two background value computations - val v1 = async(CommonPool + CoroutineName("v1coroutine")) { - log("Computing v1") - delay(500) - 252 - } - val v2 = async(CommonPool + CoroutineName("v2coroutine")) { - log("Computing v2") - delay(1000) - 6 - } - log("The answer for v1 / v2 = ${v1.await() / v2.await()}") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-context-08.kt) - -The output it produces with `-Dkotlinx.coroutines.debug` JVM option is similar to: - -```text -[main @main#1] Started main coroutine -[ForkJoinPool.commonPool-worker-1 @v1coroutine#2] Computing v1 -[ForkJoinPool.commonPool-worker-2 @v2coroutine#3] Computing v2 -[main @main#1] The answer for v1 / v2 = 42 -``` - - - -### Cancellation via explicit job - -Let us put our knowledge about contexts, children and jobs together. Assume that our application has -an object with a lifecycle, but that object is not a coroutine. For example, we are writing an Android application -and launch various coroutines in the context of an Android activity to perform asynchronous operations to fetch -and update data, do animations, etc. All of these coroutines must be cancelled when activity is destroyed -to avoid memory leaks. - -We can manage a lifecycle of our coroutines by creating an instance of [Job] that is tied to -the lifecycle of our activity. A job instance is created using [Job()][Job.invoke] factory function -as the following example shows. We need to make sure that all the coroutines are started -with this job in their context and then a single invocation of [Job.cancel] terminates them all. - -```kotlin -fun main(args: Array) = runBlocking { - val job = Job() // create a job object to manage our lifecycle - // now launch ten coroutines for a demo, each working for a different time - val coroutines = List(10) { i -> - // they are all children of our job object - launch(context + job) { // we use the context of main runBlocking thread, but with our own job object - delay(i * 200L) // variable delay 0ms, 200ms, 400ms, ... etc - println("Coroutine $i is done") - } - } - println("Launched ${coroutines.size} coroutines") - delay(500L) // delay for half a second - println("Cancelling job!") - job.cancel() // cancel our job.. !!! - delay(1000L) // delay for more to see if our coroutines are still working -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-context-09.kt) - -The output of this example is: - -```text -Launched 10 coroutines -Coroutine 0 is done -Coroutine 1 is done -Coroutine 2 is done -Cancelling job! -``` - - - -As you can see, only the first three coroutines had printed a message and the others were cancelled -by a single invocation of `job.cancel()`. So all we need to do in our hypothetical Android -application is to create a parent job object when activity is created, use it for child coroutines, -and cancel it when activity is destroyed. - -## Channels - -Deferred values provide a convenient way to transfer a single value between coroutines. -Channels provide a way to transfer a stream of values. - - - -### Channel basics - -A [Channel] is conceptually very similar to `BlockingQueue`. One key difference is that -instead of a blocking `put` operation it has a suspending [send][SendChannel.send], and instead of -a blocking `take` operation it has a suspending [receive][ReceiveChannel.receive]. - -```kotlin -fun main(args: Array) = runBlocking { - val channel = Channel() - launch(CommonPool) { - // this might be heavy CPU-consuming computation or async logic, we'll just send five squares - for (x in 1..5) channel.send(x * x) - } - // here we print five received integers: - repeat(5) { println(channel.receive()) } - println("Done!") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-01.kt) - -The output of this code is: - -```text -1 -4 -9 -16 -25 -Done! -``` - - - -### Closing and iteration over channels - -Unlike a queue, a channel can be closed to indicate that no more elements are coming. -On the receiver side it is convenient to use a regular `for` loop to receive elements -from the channel. - -Conceptually, a [close][SendChannel.close] is like sending a special close token to the channel. -The iteration stops as soon as this close token is received, so there is a guarantee -that all previously sent elements before the close are received: - -```kotlin -fun main(args: Array) = runBlocking { - val channel = Channel() - launch(CommonPool) { - for (x in 1..5) channel.send(x * x) - channel.close() // we're done sending - } - // here we print received values using `for` loop (until the channel is closed) - for (y in channel) println(y) - println("Done!") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-02.kt) - - - -### Building channel producers - -The pattern where a coroutine is producing a sequence of elements is quite common. -This is a part of _producer-consumer_ pattern that is often found in concurrent code. -You could abstract such a producer into a function that takes channel as its parameter, but this goes contrary -to common sense that results must be returned from functions. - -There is a convenience coroutine builder named [produce] that makes it easy to do it right on producer side, -and an extension function [consumeEach], that can replace a `for` loop on the consumer side: - -```kotlin -fun produceSquares() = produce(CommonPool) { - for (x in 1..5) send(x * x) -} - -fun main(args: Array) = runBlocking { - val squares = produceSquares() - squares.consumeEach { println(it) } - println("Done!") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-03.kt) - - - -### Pipelines - -Pipeline is a pattern where one coroutine is producing, possibly infinite, stream of values: - -```kotlin -fun produceNumbers() = produce(CommonPool) { - var x = 1 - while (true) send(x++) // infinite stream of integers starting from 1 -} -``` - -And another coroutine or coroutines are consuming that stream, doing some processing, and producing some other results. -In the below example the numbers are just squared: - -```kotlin -fun square(numbers: ReceiveChannel) = produce(CommonPool) { - for (x in numbers) send(x * x) -} -``` - -The main code starts and connects the whole pipeline: - -```kotlin -fun main(args: Array) = runBlocking { - val numbers = produceNumbers() // produces integers from 1 and on - val squares = square(numbers) // squares integers - for (i in 1..5) println(squares.receive()) // print first five - println("Done!") // we are done - squares.cancel() // need to cancel these coroutines in a larger app - numbers.cancel() -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-04.kt) - - - -We don't have to cancel these coroutines in this example app, because -[coroutines are like daemon threads](#coroutines-are-like-daemon-threads), -but in a larger app we'll need to stop our pipeline if we don't need it anymore. -Alternatively, we could have run pipeline coroutines as -[children of a coroutine](#children-of-a-coroutine). - -### Prime numbers with pipeline - -Let's take pipelines to the extreme with an example that generates prime numbers using a pipeline -of coroutines. We start with an infinite sequence of numbers. This time we introduce an -explicit context parameter, so that caller can control where our coroutines run: - - - -```kotlin -fun numbersFrom(context: CoroutineContext, start: Int) = produce(context) { - var x = start - while (true) send(x++) // infinite stream of integers from start -} -``` - -The following pipeline stage filters an incoming stream of numbers, removing all the numbers -that are divisible by the given prime number: - -```kotlin -fun filter(context: CoroutineContext, numbers: ReceiveChannel, prime: Int) = produce(context) { - for (x in numbers) if (x % prime != 0) send(x) -} -``` - -Now we build our pipeline by starting a stream of numbers from 2, taking a prime number from the current channel, -and launching new pipeline stage for each prime number found: - -``` -numbersFrom(2) -> filter(2) -> filter(3) -> filter(5) -> filter(7) ... -``` - -The following example prints the first ten prime numbers, -running the whole pipeline in the context of the main thread: - -```kotlin -fun main(args: Array) = runBlocking { - var cur = numbersFrom(context, 2) - for (i in 1..10) { - val prime = cur.receive() - println(prime) - cur = filter(context, cur, prime) - } -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-05.kt) - -The output of this code is: - -```text -2 -3 -5 -7 -11 -13 -17 -19 -23 -29 -``` - - - -Note, that you can build the same pipeline using `buildIterator` coroutine builder from the standard library. -Replace `produce` with `buildIterator`, `send` with `yield`, `receive` with `next`, -`ReceiveChannel` with `Iterator`, and get rid of the context. You will not need `runBlocking` either. -However, the benefit of a pipeline that uses channels as shown above is that it can actually use -multiple CPU cores if you run it in [CommonPool] context. - -Anyway, this is an extremely impractical way to find prime numbers. In practice, pipelines do involve some -other suspending invocations (like asynchronous calls to remote services) and these pipelines cannot be -built using `buildSeqeunce`/`buildIterator`, because they do not allow arbitrary suspension, unlike -`produce` which is fully asynchronous. - -### Fan-out - -Multiple coroutines may receive from the same channel, distributing work between themselves. -Let us start with a producer coroutine that is periodically producing integers -(ten numbers per second): - -```kotlin -fun produceNumbers() = produce(CommonPool) { - var x = 1 // start from 1 - while (true) { - send(x++) // produce next - delay(100) // wait 0.1s - } -} -``` - -Then we can have several processor coroutines. In this example, they just print their id and -received number: - -```kotlin -fun launchProcessor(id: Int, channel: ReceiveChannel) = launch(CommonPool) { - channel.consumeEach { - println("Processor #$id received $it") - } -} -``` - -Now let us launch five processors and let them work for a second. See what happens: - -```kotlin -fun main(args: Array) = runBlocking { - val producer = produceNumbers() - repeat(5) { launchProcessor(it, producer) } - delay(1000) - producer.cancel() // cancel producer coroutine and thus kill them all -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-06.kt) - -The output will be similar to the the following one, albeit the processor ids that receive -each specific integer may be different: - -``` -Processor #2 received 1 -Processor #4 received 2 -Processor #0 received 3 -Processor #1 received 4 -Processor #3 received 5 -Processor #2 received 6 -Processor #4 received 7 -Processor #0 received 8 -Processor #1 received 9 -Processor #3 received 10 -``` - - - -Note, that cancelling a producer coroutine closes its channel, thus eventually terminating iteration -over the channel that processor coroutines are doing. - -### Fan-in - -Multiple coroutines may send to the same channel. -For example, let us have a channel of strings, and a suspending function that -repeatedly sends a specified string to this channel with a specified delay: - -```kotlin -suspend fun sendString(channel: SendChannel, s: String, time: Long) { - while (true) { - delay(time) - channel.send(s) - } -} -``` - -Now, let us see what happens if we launch a couple of coroutines sending strings -(in this example we launch them in the context of the main thread): - -```kotlin -fun main(args: Array) = runBlocking { - val channel = Channel() - launch(context) { sendString(channel, "foo", 200L) } - launch(context) { sendString(channel, "BAR!", 500L) } - repeat(6) { // receive first six - println(channel.receive()) - } -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-07.kt) - -The output is: - -```text -foo -foo -BAR! -foo -foo -BAR! -``` - - - -### Buffered channels - -The channels shown so far had no buffer. Unbuffered channels transfer elements when sender and receiver -meet each other (aka rendezvous). If send is invoked first, then it is suspended until receive is invoked, -if receive is invoked first, it is suspended until send is invoked. - -Both [Channel()][Channel.invoke] factory function and [produce] builder take an optional `capacity` parameter to -specify _buffer size_. Buffer allows senders to send multiple elements before suspending, -similar to the `BlockingQueue` with a specified capacity, which blocks when buffer is full. - -Take a look at the behavior of the following code: - -```kotlin -fun main(args: Array) = runBlocking { - val channel = Channel(4) // create buffered channel - launch(context) { // launch sender coroutine - repeat(10) { - println("Sending $it") // print before sending each element - channel.send(it) // will suspend when buffer is full - } - } - // don't receive anything... just wait.... - delay(1000) -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-08.kt) - -It prints "sending" _five_ times using a buffered channel with capacity of _four_: - -```text -Sending 0 -Sending 1 -Sending 2 -Sending 3 -Sending 4 -``` - - - -The first four elements are added to the buffer and the sender suspends when trying to send the fifth one. - - -### Channels are fair - -Send and receive operations to channels are _fair_ with respect to the order of their invocation from -multiple coroutines. They are served in first-in first-out order, e.g. the first coroutine to invoke `receive` -gets the element. In the following example two coroutines "ping" and "pong" are -receiving the "ball" object from the shared "table" channel. - -```kotlin -data class Ball(var hits: Int) - -fun main(args: Array) = runBlocking { - val table = Channel() // a shared table - launch(context) { player("ping", table) } - launch(context) { player("pong", table) } - table.send(Ball(0)) // serve the ball - delay(1000) // delay 1 second - table.receive() // game over, grab the ball -} - -suspend fun player(name: String, table: Channel) { - for (ball in table) { // receive the ball in a loop - ball.hits++ - println("$name $ball") - delay(300) // wait a bit - table.send(ball) // send the ball back - } -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-09.kt) - -The "ping" coroutine is started first, so it is the first one to receive the ball. Even though "ping" -coroutine immediately starts receiving the ball again after sending it back to the table, the ball gets -received by the "pong" coroutine, because it was already waiting for it: - -```text -ping Ball(hits=1) -pong Ball(hits=2) -ping Ball(hits=3) -pong Ball(hits=4) -ping Ball(hits=5) -``` - - - -## Shared mutable state and concurrency - -Coroutines can be executed concurrently using a multi-threaded dispatcher like [CommonPool]. It presents -all the usual concurrency problems. The main problem being synchronization of access to **shared mutable state**. -Some solutions to this problem in the land of coroutines are similar to the solutions in the multi-threaded world, -but others are unique. - -### The problem - -Let us launch a thousand coroutines all doing the same action thousand times (for a total of a million executions). -We'll also measure their completion time for further comparisons: - - - - - - - - - -```kotlin -suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) { - val n = 1000 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - val jobs = List(n) { - launch(context) { - repeat(k) { action() } - } - } - jobs.forEach { it.join() } - } - println("Completed ${n * k} actions in $time ms") -} -``` - - - -We start with a very simple action that increments a shared mutable variable using -multi-threaded [CommonPool] context. - -```kotlin -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { - counter++ - } - println("Counter = $counter") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-01.kt) - - - -What does it print at the end? It is highly unlikely to ever print "Counter = 1000000", because a thousand coroutines -increment the `counter` concurrently from multiple threads without any synchronization. - -> Note: if you have an old system with 2 or fewer CPUs, then you _will_ consistently see 1000000, because -`CommonPool` is running in only one thread in this case. To reproduce the problem you'll need to make the -following change: - -```kotlin -val mtContext = newFixedThreadPoolContext(2, "mtPool") // explicitly define context with two threads -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(mtContext) { // use it instead of CommonPool in this sample and below - counter++ - } - println("Counter = $counter") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-01b.kt) - - - -### Volatiles are of no help - -There is common misconception that making a variable `volatile` solves concurrency problem. Let us try it: - -```kotlin -@Volatile // in Kotlin `volatile` is an annotation -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { - counter++ - } - println("Counter = $counter") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-02.kt) - - - -This code works slower, but we still don't get "Counter = 1000000" at the end, because volatile variables guarantee -linearizable (this is a technical term for "atomic") reads and writes to the corresponding variable, but -do not provide atomicity of larger actions (increment in our case). - -### Thread-safe data structures - -The general solution that works both for threads and for coroutines is to use a thread-safe (aka synchronized, -linearizable, or atomic) data structure that provides all the necessarily synchronization for the corresponding -operations that needs to be performed on a shared state. -In the case of a simple counter we can use `AtomicInteger` class which has atomic `incrementAndGet` operations: - -```kotlin -var counter = AtomicInteger() - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { - counter.incrementAndGet() - } - println("Counter = ${counter.get()}") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-03.kt) - - - -This is the fastest solution for this particular problem. It works for plain counters, collections, queues and other -standard data structures and basic operations on them. However, it does not easily scale to complex -state or to complex operations that do not have ready-to-use thread-safe implementations. - -### Thread confinement fine-grained - -_Thread confinement_ is an approach to the problem of shared mutable state where all access to the particular shared -state is confined to a single thread. It is typically used in UI applications, where all UI state is confined to -the single event-dispatch/application thread. It is easy to apply with coroutines by using a -single-threaded context: - -```kotlin -val counterContext = newSingleThreadContext("CounterContext") -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { // run each coroutine in CommonPool - run(counterContext) { // but confine each increment to the single-threaded context - counter++ - } - } - println("Counter = $counter") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-04.kt) - - - -This code works very slowly, because it does _fine-grained_ thread-confinement. Each individual increment switches -from multi-threaded `CommonPool` context to the single-threaded context using [run] block. - -### Thread confinement coarse-grained - -In practice, thread confinement is performed in large chunks, e.g. big pieces of state-updating business logic -are confined to the single thread. The following example does it like that, running each coroutine in -the single-threaded context to start with. - -```kotlin -val counterContext = newSingleThreadContext("CounterContext") -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(counterContext) { // run each coroutine in the single-threaded context - counter++ - } - println("Counter = $counter") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-05.kt) - - - -This now works much faster and produces correct result. - -### Mutual exclusion - -Mutual exclusion solution to the problem is to protect all modifications of the shared state with a _critical section_ -that is never executed concurrently. In a blocking world you'd typically use `synchronized` or `ReentrantLock` for that. -Coroutine's alternative is called [Mutex]. It has [lock][Mutex.lock] and [unlock][Mutex.unlock] functions to -delimit a critical section. The key difference is that `Mutex.lock` is a suspending function. It does not block a thread. - -```kotlin -val mutex = Mutex() -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { - mutex.lock() - try { counter++ } - finally { mutex.unlock() } - } - println("Counter = $counter") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-06.kt) - - - -The locking in this example is fine-grained, so it pays the price. However, it is a good choice for some situations -where you absolutely must modify some shared state periodically, but there is no natural thread that this state -is confined to. - -### Actors - -An actor is a combination of a coroutine, the state that is confined and is encapsulated into this coroutine, -and a channel to communicate with other coroutines. A simple actor can be written as a function, -but an actor with a complex state is better suited for a class. - -There is an [actor] coroutine builder that conveniently combines actor's mailbox channel into its -scope to receive messages from and combines the send channel into the resulting job object, so that a -single reference to the actor can be carried around as its handle. - -```kotlin -// Message types for counterActor -sealed class CounterMsg -object IncCounter : CounterMsg() // one-way message to increment counter -class GetCounter(val response: SendChannel) : CounterMsg() // a request with reply - -// This function launches a new counter actor -fun counterActor() = actor(CommonPool) { - var counter = 0 // actor state - for (msg in channel) { // iterate over incoming messages - when (msg) { - is IncCounter -> counter++ - is GetCounter -> msg.response.send(counter) - } - } -} - -fun main(args: Array) = runBlocking { - val counter = counterActor() // create the actor - massiveRun(CommonPool) { - counter.send(IncCounter) - } - val response = Channel() - counter.send(GetCounter(response)) - println("Counter = ${response.receive()}") - counter.close() // shutdown the actor -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-07.kt) - - - -It does not matter (for correctness) what context the actor itself is executed in. An actor is -a coroutine and a coroutine is executed sequentially, so confinement of the state to the specific coroutine -works as a solution to the problem of shared mutable state. - -Actor is more efficient than locking under load, because in this case it always has work to do and it does not -have to switch to a different context at all. - -> Note, that an [actor] coroutine builder is a dual of [produce] coroutine builder. An actor is associated - with the channel that it receives messages from, while a producer is associated with the channel that it - sends elements to. - -## Select expression - -Select expression makes it possible to await multiple suspending functions simultaneously and _select_ -the first one that becomes available. - - - -### Selecting from channels - -Let us have two producers of strings: `fizz` and `buzz`. The `fizz` produces "Fizz" string every 300 ms: - - - -```kotlin -fun fizz(context: CoroutineContext) = produce(context) { - while (true) { // sends "Fizz" every 300 ms - delay(300) - send("Fizz") - } -} -``` - -And the `buzz` produces "Buzz!" string every 500 ms: - -```kotlin -fun buzz(context: CoroutineContext) = produce(context) { - while (true) { // sends "Buzz!" every 500 ms - delay(500) - send("Buzz!") - } -} -``` - -Using [receive][ReceiveChannel.receive] suspending function we can receive _either_ from one channel or the -other. But [select] expression allows us to receive from _both_ simultaneously using its -[onReceive][SelectBuilder.onReceive] clauses: - -```kotlin -suspend fun selectFizzBuzz(fizz: ReceiveChannel, buzz: ReceiveChannel) { - select { // means that this select expression does not produce any result - fizz.onReceive { value -> // this is the first select clause - println("fizz -> '$value'") - } - buzz.onReceive { value -> // this is the second select clause - println("buzz -> '$value'") - } - } -} -``` - -Let us run it all seven times: - -```kotlin -fun main(args: Array) = runBlocking { - val fizz = fizz(context) - val buzz = buzz(context) - repeat(7) { - selectFizzBuzz(fizz, buzz) - } -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-select-01.kt) - -The result of this code is: - -```text -fizz -> 'Fizz' -buzz -> 'Buzz!' -fizz -> 'Fizz' -fizz -> 'Fizz' -buzz -> 'Buzz!' -fizz -> 'Fizz' -buzz -> 'Buzz!' -``` - - - -### Selecting on close - -The [onReceive][SelectBuilder.onReceive] clause in `select` fails when the channel is closed and the corresponding -`select` throws an exception. We can use [onReceiveOrNull][SelectBuilder.onReceiveOrNull] clause to perform a -specific action when the channel is closed. The following example also shows that `select` is an expression that returns -the result of its selected clause: - -```kotlin -suspend fun selectAorB(a: ReceiveChannel, b: ReceiveChannel): String = - select { - a.onReceiveOrNull { value -> - if (value == null) - "Channel 'a' is closed" - else - "a -> '$value'" - } - b.onReceiveOrNull { value -> - if (value == null) - "Channel 'b' is closed" - else - "b -> '$value'" - } - } -``` - -Let's use it with channel `a` that produces "Hello" string four times and -channel `b` that produces "World" four times: - -```kotlin -fun main(args: Array) = runBlocking { - // we are using the context of the main thread in this example for predictability ... - val a = produce(context) { - repeat(4) { send("Hello $it") } - } - val b = produce(context) { - repeat(4) { send("World $it") } - } - repeat(8) { // print first eight results - println(selectAorB(a, b)) - } -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-select-02.kt) - -The result of this code is quite interesting, so we'll analyze it in mode detail: - -```text -a -> 'Hello 0' -a -> 'Hello 1' -b -> 'World 0' -a -> 'Hello 2' -a -> 'Hello 3' -b -> 'World 1' -Channel 'a' is closed -Channel 'a' is closed -``` - - - -There are couple of observations to make out of it. - -First of all, `select` is _biased_ to the first clause. When several clauses are selectable at the same time, -the first one among them gets selected. Here, both channels are constantly producing strings, so `a` channel, -being the first clause in select, wins. However, because we are using unbuffered channel, the `a` gets suspended from -time to time on its [send][SendChannel.send] invocation and gives a chance for `b` to send, too. - -The second observation, is that [onReceiveOrNull][SelectBuilder.onReceiveOrNull] gets immediately selected when the -channel is already closed. - -### Selecting to send - -Select expression has [onSend][SelectBuilder.onSend] clause that can be used for a great good in combination -with a biased nature of selection. - -Let us write an example of producer of integers that sends its values to a `side` channel when -the consumers on its primary channel cannot keep up with it: - -```kotlin -fun produceNumbers(side: SendChannel) = produce(CommonPool) { - for (num in 1..10) { // produce 10 numbers from 1 to 10 - delay(100) // every 100 ms - select { - onSend(num) {} // Send to the primary channel - side.onSend(num) {} // or to the side channel - } - } -} -``` - -Consumer is going to be quite slow, taking 250 ms to process each number: - -```kotlin -fun main(args: Array) = runBlocking { - val side = Channel() // allocate side channel - launch(context) { // this is a very fast consumer for the side channel - side.consumeEach { println("Side channel has $it") } - } - produceNumbers(side).consumeEach { - println("Consuming $it") - delay(250) // let us digest the consumed number properly, do not hurry - } - println("Done consuming") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-select-03.kt) - -So let us see what happens: - -```text -Consuming 1 -Side channel has 2 -Side channel has 3 -Consuming 4 -Side channel has 5 -Side channel has 6 -Consuming 7 -Side channel has 8 -Side channel has 9 -Consuming 10 -Done consuming -``` - - - -### Selecting deferred values - -Deferred values can be selected using [onAwait][SelectBuilder.onAwait] clause. -Let us start with an async function that returns a deferred string value after -a random delay: - - - -```kotlin -fun asyncString(time: Int) = async(CommonPool) { - delay(time.toLong()) - "Waited for $time ms" -} -``` - -Let us start a dozen of them with a random delay. - -```kotlin -fun asyncStringsList(): List> { - val random = Random(3) - return List(12) { asyncString(random.nextInt(1000)) } -} -``` - -Now the main function awaits for the first of them to complete and counts the number of deferred values -that are still active. Note, that we've used here the fact that `select` expression is a Kotlin DSL, -so we can provide clauses for it using an arbitrary code. In this case we iterate over a list -of deferred values to provide `onAwait` clause for each deferred value. - -```kotlin -fun main(args: Array) = runBlocking { - val list = asyncStringsList() - val result = select { - list.withIndex().forEach { (index, deferred) -> - deferred.onAwait { answer -> - "Deferred $index produced answer '$answer'" - } - } - } - println(result) - val countActive = list.count { it.isActive } - println("$countActive coroutines are still active") -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-select-04.kt) - -The output is: - -```text -Deferred 4 produced answer 'Waited for 128 ms' -11 coroutines are still active -``` - - - -### Switch over a channel of deferred values - -Let us write a channel producer function that consumes a channel of deferred string values, waits for each received -deferred value, but only until the next deferred value comes over or the channel is closed. This example puts together -[onReceiveOrNull][SelectBuilder.onReceiveOrNull] and [onAwait][SelectBuilder.onAwait] clauses in the same `select`: - -```kotlin -fun switchMapDeferreds(input: ReceiveChannel>) = produce(CommonPool) { - var current = input.receive() // start with first received deferred value - while (isActive) { // loop while not cancelled/closed - val next = select?> { // return next deferred value from this select or null - input.onReceiveOrNull { update -> - update // replaces next value to wait - } - current.onAwait { value -> - send(value) // send value that current deferred has produced - input.receiveOrNull() // and use the next deferred from the input channel - } - } - if (next == null) { - println("Channel was closed") - break // out of loop - } else { - current = next - } - } -} -``` - -To test it, we'll use a simple async function that resolves to a specified string after a specified time: - -```kotlin -fun asyncString(str: String, time: Long) = async(CommonPool) { - delay(time) - str -} -``` - -The main function just launches a coroutine to print results of `switchMapDeferreds` and sends some test -data to it: - -```kotlin -fun main(args: Array) = runBlocking { - val chan = Channel>() // the channel for test - launch(context) { // launch printing coroutine - for (s in switchMapDeferreds(chan)) - println(s) // print each received string - } - chan.send(asyncString("BEGIN", 100)) - delay(200) // enough time for "BEGIN" to be produced - chan.send(asyncString("Slow", 500)) - delay(100) // not enough time to produce slow - chan.send(asyncString("Replace", 100)) - delay(500) // give it time before the last one - chan.send(asyncString("END", 500)) - delay(1000) // give it time to process - chan.close() // close the channel ... - delay(500) // and wait some time to let it finish -} -``` - -> You can get full code [here](kotlinx-coroutines-core/src/test/kotlin/guide/example-select-05.kt) - -The result of this code: - -```text -BEGIN -Replace -END -Channel was closed -``` - - - -## Further reading - -* [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md) -* [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md) -* [Coroutines design document (KEEP)](https://github.com/Kotlin/kotlin-coroutines/blob/master/kotlin-coroutines-informal.md) -* [Full kotlinx.coroutines API reference](http://kotlin.github.io/kotlinx.coroutines) - - - - -[launch]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/launch.html -[delay]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/delay.html -[runBlocking]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/run-blocking.html -[Job]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/index.html -[CancellationException]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-cancellation-exception.html -[yield]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/yield.html -[CoroutineScope.isActive]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-scope/is-active.html -[CoroutineScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-scope/index.html -[run]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/run.html -[NonCancellable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-non-cancellable/index.html -[withTimeout]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/with-timeout.html -[async]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/async.html -[Deferred]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-deferred/index.html -[CoroutineStart.LAZY]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-start/-l-a-z-y.html -[Deferred.await]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-deferred/await.html -[Job.start]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/start.html -[CommonPool]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-common-pool/index.html -[CoroutineDispatcher]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-dispatcher/index.html -[CoroutineScope.context]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-scope/context.html -[Unconfined]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-unconfined/index.html -[newCoroutineContext]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/new-coroutine-context.html -[CoroutineName]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-name/index.html -[Job.invoke]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/invoke.html -[Job.cancel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/cancel.html - -[Mutex]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.sync/-mutex/index.html -[Mutex.lock]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.sync/-mutex/lock.html -[Mutex.unlock]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.sync/-mutex/unlock.html - -[Channel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel/index.html -[SendChannel.send]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-send-channel/send.html -[ReceiveChannel.receive]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/receive.html -[SendChannel.close]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-send-channel/close.html -[produce]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/produce.html -[consumeEach]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/consume-each.html -[Channel.invoke]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel/invoke.html -[actor]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/actor.html - -[select]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/select.html -[SelectBuilder.onReceive]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-receive.html -[SelectBuilder.onReceiveOrNull]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-receive-or-null.html -[SelectBuilder.onSend]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-send.html -[SelectBuilder.onAwait]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-await.html - +It is recommended to read the guide on the [kotlinlang website](https://kotlinlang.org/docs/coroutines-guide.html), with proper HTML formatting and runnable samples. diff --git a/docs/basics.md b/docs/basics.md new file mode 100644 index 0000000000..a18bf3dddb --- /dev/null +++ b/docs/basics.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/coroutines-basics.html](https://kotlinlang.org/docs/coroutines-basics.html) page. + +To edit the documentation, open the [topics/coroutines-basics.md](topics/coroutines-basics.md) page. diff --git a/docs/cancellation-and-timeouts.md b/docs/cancellation-and-timeouts.md new file mode 100644 index 0000000000..ad6afca91f --- /dev/null +++ b/docs/cancellation-and-timeouts.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/cancellation-and-timeouts.html](https://kotlinlang.org/docs/cancellation-and-timeouts.html) page. + +To edit the documentation, open the [topics/cancellation-and-timeouts.md](topics/cancellation-and-timeouts.md) page. \ No newline at end of file diff --git a/docs/cfg/buildprofiles.xml b/docs/cfg/buildprofiles.xml new file mode 100644 index 0000000000..d1a081cad8 --- /dev/null +++ b/docs/cfg/buildprofiles.xml @@ -0,0 +1,9 @@ + + + + true + https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/ + true + + + \ No newline at end of file diff --git a/docs/channels.md b/docs/channels.md new file mode 100644 index 0000000000..73415ce973 --- /dev/null +++ b/docs/channels.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/channels.html](https://kotlinlang.org/docs/channels.html) page. + +To edit the documentation, open the [topics/channels.md](topics/channels.md) page. \ No newline at end of file diff --git a/docs/compatibility.md b/docs/compatibility.md new file mode 100644 index 0000000000..b61f629901 --- /dev/null +++ b/docs/compatibility.md @@ -0,0 +1 @@ +The documentation has been moved to the [topics/compatibility.md](topics/compatibility.md). \ No newline at end of file diff --git a/docs/composing-suspending-functions.md b/docs/composing-suspending-functions.md new file mode 100644 index 0000000000..4318c4ee39 --- /dev/null +++ b/docs/composing-suspending-functions.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/composing-suspending-functions.html](https://kotlinlang.org/docs/composing-suspending-functions.html) page. + +To edit the documentation, open the [topics/composing-suspending-functions.md](topics/composing-suspending-functions.md) page. \ No newline at end of file diff --git a/docs/coroutine-context-and-dispatchers.md b/docs/coroutine-context-and-dispatchers.md new file mode 100644 index 0000000000..f8ba1283c1 --- /dev/null +++ b/docs/coroutine-context-and-dispatchers.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/coroutine-context-and-dispatchers.html](https://kotlinlang.org/docs/coroutine-context-and-dispatchers.html) page. + +To edit the documentation, open the [topics/coroutine-context-and-dispatchers.md](topics/coroutine-context-and-dispatchers.md) page. \ No newline at end of file diff --git a/docs/coroutines-guide.md b/docs/coroutines-guide.md new file mode 100644 index 0000000000..e2caa750f6 --- /dev/null +++ b/docs/coroutines-guide.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/coroutines-guide.html](https://kotlinlang.org/docs/coroutines-guide.html) page. + +To edit the documentation, open the [topics/coroutines-guide.md](topics/coroutines-guide.md) page. \ No newline at end of file diff --git a/docs/debugging.md b/docs/debugging.md new file mode 100644 index 0000000000..a5dab630d3 --- /dev/null +++ b/docs/debugging.md @@ -0,0 +1 @@ +The documentation has been moved to the [topics/debugging.md](topics/debugging.md). \ No newline at end of file diff --git a/docs/exception-handling.md b/docs/exception-handling.md new file mode 100644 index 0000000000..77dbac9883 --- /dev/null +++ b/docs/exception-handling.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/exception-handling.html](https://kotlinlang.org/docs/exception-handling.html) page. + +To edit the documentation, open the [topics/exception-handling.md](topics/exception-handling.md) page. \ No newline at end of file diff --git a/docs/flow.md b/docs/flow.md new file mode 100644 index 0000000000..400a15f13a --- /dev/null +++ b/docs/flow.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/flow.html](https://kotlinlang.org/docs/flow.html) page. + +To edit the documentation, open the [topics/flow.md](topics/flow.md) page. \ No newline at end of file diff --git a/docs/images/after.png b/docs/images/after.png new file mode 100644 index 0000000000..b1e138c682 Binary files /dev/null and b/docs/images/after.png differ diff --git a/docs/images/before.png b/docs/images/before.png new file mode 100644 index 0000000000..7386ee213b Binary files /dev/null and b/docs/images/before.png differ diff --git a/docs/images/coroutine-breakpoint.png b/docs/images/coroutine-breakpoint.png new file mode 100644 index 0000000000..d0e34e89c0 Binary files /dev/null and b/docs/images/coroutine-breakpoint.png differ diff --git a/docs/images/coroutine-debug-1.png b/docs/images/coroutine-debug-1.png new file mode 100644 index 0000000000..d92d0ff11d Binary files /dev/null and b/docs/images/coroutine-debug-1.png differ diff --git a/docs/images/coroutine-debug-2.png b/docs/images/coroutine-debug-2.png new file mode 100644 index 0000000000..56f19c3def Binary files /dev/null and b/docs/images/coroutine-debug-2.png differ diff --git a/docs/images/coroutine-debug-3.png b/docs/images/coroutine-debug-3.png new file mode 100644 index 0000000000..43a1460aa6 Binary files /dev/null and b/docs/images/coroutine-debug-3.png differ diff --git a/docs/images/coroutine-idea-debugging-1.png b/docs/images/coroutine-idea-debugging-1.png new file mode 100644 index 0000000000..c824307290 Binary files /dev/null and b/docs/images/coroutine-idea-debugging-1.png differ diff --git a/docs/images/coroutines-and-channels/aggregate.png b/docs/images/coroutines-and-channels/aggregate.png new file mode 100644 index 0000000000..fed7b85861 Binary files /dev/null and b/docs/images/coroutines-and-channels/aggregate.png differ diff --git a/docs/images/coroutines-and-channels/background.png b/docs/images/coroutines-and-channels/background.png new file mode 100644 index 0000000000..0882eba35b Binary files /dev/null and b/docs/images/coroutines-and-channels/background.png differ diff --git a/docs/images/coroutines-and-channels/blocking.png b/docs/images/coroutines-and-channels/blocking.png new file mode 100644 index 0000000000..6f04c1a728 Binary files /dev/null and b/docs/images/coroutines-and-channels/blocking.png differ diff --git a/docs/images/coroutines-and-channels/buffered-channel.png b/docs/images/coroutines-and-channels/buffered-channel.png new file mode 100644 index 0000000000..009604b8d1 Binary files /dev/null and b/docs/images/coroutines-and-channels/buffered-channel.png differ diff --git a/docs/images/coroutines-and-channels/callbacks.png b/docs/images/coroutines-and-channels/callbacks.png new file mode 100644 index 0000000000..c02d683cbf Binary files /dev/null and b/docs/images/coroutines-and-channels/callbacks.png differ diff --git a/docs/images/coroutines-and-channels/concurrency.png b/docs/images/coroutines-and-channels/concurrency.png new file mode 100644 index 0000000000..e52e2ebacd Binary files /dev/null and b/docs/images/coroutines-and-channels/concurrency.png differ diff --git a/docs/images/coroutines-and-channels/conflated-channel.gif b/docs/images/coroutines-and-channels/conflated-channel.gif new file mode 100644 index 0000000000..bf118b666c Binary files /dev/null and b/docs/images/coroutines-and-channels/conflated-channel.gif differ diff --git a/docs/images/coroutines-and-channels/generating-token.png b/docs/images/coroutines-and-channels/generating-token.png new file mode 100644 index 0000000000..7c0aadc884 Binary files /dev/null and b/docs/images/coroutines-and-channels/generating-token.png differ diff --git a/docs/images/coroutines-and-channels/initial-window.png b/docs/images/coroutines-and-channels/initial-window.png new file mode 100644 index 0000000000..5e7e325760 Binary files /dev/null and b/docs/images/coroutines-and-channels/initial-window.png differ diff --git a/docs/images/coroutines-and-channels/loading.gif b/docs/images/coroutines-and-channels/loading.gif new file mode 100644 index 0000000000..8a862128cf Binary files /dev/null and b/docs/images/coroutines-and-channels/loading.gif differ diff --git a/docs/images/coroutines-and-channels/progress-and-concurrency.png b/docs/images/coroutines-and-channels/progress-and-concurrency.png new file mode 100644 index 0000000000..9d070c11e1 Binary files /dev/null and b/docs/images/coroutines-and-channels/progress-and-concurrency.png differ diff --git a/docs/images/coroutines-and-channels/progress.png b/docs/images/coroutines-and-channels/progress.png new file mode 100644 index 0000000000..778dfb10d4 Binary files /dev/null and b/docs/images/coroutines-and-channels/progress.png differ diff --git a/docs/images/coroutines-and-channels/rendezvous-channel.png b/docs/images/coroutines-and-channels/rendezvous-channel.png new file mode 100644 index 0000000000..23c7bc81a2 Binary files /dev/null and b/docs/images/coroutines-and-channels/rendezvous-channel.png differ diff --git a/docs/images/coroutines-and-channels/run-configuration.png b/docs/images/coroutines-and-channels/run-configuration.png new file mode 100644 index 0000000000..9661f8f619 Binary files /dev/null and b/docs/images/coroutines-and-channels/run-configuration.png differ diff --git a/docs/images/coroutines-and-channels/suspend-requests.png b/docs/images/coroutines-and-channels/suspend-requests.png new file mode 100644 index 0000000000..92f92f6bc8 Binary files /dev/null and b/docs/images/coroutines-and-channels/suspend-requests.png differ diff --git a/docs/images/coroutines-and-channels/suspension-process.gif b/docs/images/coroutines-and-channels/suspension-process.gif new file mode 100644 index 0000000000..f42dbe1e83 Binary files /dev/null and b/docs/images/coroutines-and-channels/suspension-process.gif differ diff --git a/docs/images/coroutines-and-channels/time-comparison.png b/docs/images/coroutines-and-channels/time-comparison.png new file mode 100644 index 0000000000..b1a52a3375 Binary files /dev/null and b/docs/images/coroutines-and-channels/time-comparison.png differ diff --git a/docs/images/coroutines-and-channels/unlimited-channel.png b/docs/images/coroutines-and-channels/unlimited-channel.png new file mode 100644 index 0000000000..7cdef2a975 Binary files /dev/null and b/docs/images/coroutines-and-channels/unlimited-channel.png differ diff --git a/docs/images/coroutines-and-channels/using-channel-many-coroutines.png b/docs/images/coroutines-and-channels/using-channel-many-coroutines.png new file mode 100644 index 0000000000..913a0ddc37 Binary files /dev/null and b/docs/images/coroutines-and-channels/using-channel-many-coroutines.png differ diff --git a/docs/images/coroutines-and-channels/using-channel.png b/docs/images/coroutines-and-channels/using-channel.png new file mode 100644 index 0000000000..85094bd873 Binary files /dev/null and b/docs/images/coroutines-and-channels/using-channel.png differ diff --git a/docs/images/flow-breakpoint.png b/docs/images/flow-breakpoint.png new file mode 100644 index 0000000000..a7a38cceaa Binary files /dev/null and b/docs/images/flow-breakpoint.png differ diff --git a/docs/images/flow-build-project.png b/docs/images/flow-build-project.png new file mode 100644 index 0000000000..12221c77a0 Binary files /dev/null and b/docs/images/flow-build-project.png differ diff --git a/docs/images/flow-debug-1.png b/docs/images/flow-debug-1.png new file mode 100644 index 0000000000..8a16984a83 Binary files /dev/null and b/docs/images/flow-debug-1.png differ diff --git a/docs/images/flow-debug-2.png b/docs/images/flow-debug-2.png new file mode 100644 index 0000000000..d06c86bf8a Binary files /dev/null and b/docs/images/flow-debug-2.png differ diff --git a/docs/images/flow-debug-3.png b/docs/images/flow-debug-3.png new file mode 100644 index 0000000000..af082c0595 Binary files /dev/null and b/docs/images/flow-debug-3.png differ diff --git a/docs/images/flow-debug-4.png b/docs/images/flow-debug-4.png new file mode 100644 index 0000000000..8d1c42df40 Binary files /dev/null and b/docs/images/flow-debug-4.png differ diff --git a/docs/images/flow-debug-project.png b/docs/images/flow-debug-project.png new file mode 100644 index 0000000000..f5b20bd9f2 Binary files /dev/null and b/docs/images/flow-debug-project.png differ diff --git a/docs/images/flow-resume-debug.png b/docs/images/flow-resume-debug.png new file mode 100644 index 0000000000..adb124947c Binary files /dev/null and b/docs/images/flow-resume-debug.png differ diff --git a/docs/images/new-gradle-project-jvm.png b/docs/images/new-gradle-project-jvm.png new file mode 100644 index 0000000000..108ce75832 Binary files /dev/null and b/docs/images/new-gradle-project-jvm.png differ diff --git a/docs/images/variable-optimised-out.png b/docs/images/variable-optimised-out.png new file mode 100644 index 0000000000..2db471e9ee Binary files /dev/null and b/docs/images/variable-optimised-out.png differ diff --git a/docs/kc.tree b/docs/kc.tree new file mode 100644 index 0000000000..0ee8a4ce10 --- /dev/null +++ b/docs/kc.tree @@ -0,0 +1,19 @@ + + + + + + + + + + + + + + + + + + + diff --git a/docs/knit.code.include b/docs/knit.code.include new file mode 100644 index 0000000000..8db8d745ed --- /dev/null +++ b/docs/knit.code.include @@ -0,0 +1,2 @@ +// This file was automatically generated from ${file.name} by Knit tool. Do not edit. +package ${knit.package}.${knit.name} \ No newline at end of file diff --git a/docs/knit.test.template b/docs/knit.test.template new file mode 100644 index 0000000000..b416374630 --- /dev/null +++ b/docs/knit.test.template @@ -0,0 +1,24 @@ +// This file was automatically generated from ${file.name} by Knit tool. Do not edit. +package ${test.package} + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class ${test.name} { +<#list cases as case><#assign method = test["mode.${case.param}"]!"custom"> + @Test + fun test${case.name}() { + test("${case.name}") { ${case.knit.package}.${case.knit.name}.main() }<#if method != "custom">.${method}( +<#list case.lines as line> + "${line?j_string}"<#sep>, + + ) +<#else>.also { lines -> + check(${case.param}) + } + + } +<#sep> + + +} \ No newline at end of file diff --git a/docs/select-expression.md b/docs/select-expression.md new file mode 100644 index 0000000000..d0451680a2 --- /dev/null +++ b/docs/select-expression.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/select-expression.html](https://kotlinlang.org/docs/select-expression.html) page. + +To edit the documentation, open the [topics/select-expression.md](topics/select-expression.md) page. \ No newline at end of file diff --git a/docs/shared-mutable-state-and-concurrency.md b/docs/shared-mutable-state-and-concurrency.md new file mode 100644 index 0000000000..ca05436823 --- /dev/null +++ b/docs/shared-mutable-state-and-concurrency.md @@ -0,0 +1,3 @@ +The documentation has been moved to the [https://kotlinlang.org/docs/shared-mutable-state-and-concurrency.html](https://kotlinlang.org/docs/shared-mutable-state-and-concurrency.html) page. + +To edit the documentation, open the [topics/shared-mutable-state-and-concurrency.md](topics/shared-mutable-state-and-concurrency.md) page. \ No newline at end of file diff --git a/docs/topics/cancellation-and-timeouts.md b/docs/topics/cancellation-and-timeouts.md new file mode 100644 index 0000000000..85190936f1 --- /dev/null +++ b/docs/topics/cancellation-and-timeouts.md @@ -0,0 +1,506 @@ + +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Cancellation and timeouts) + +This section covers coroutine cancellation and timeouts. + +## Cancelling coroutine execution + +In a long-running application, you might need fine-grained control on your background coroutines. +For example, a user might have closed the page that launched a coroutine, and now its result +is no longer needed and its operation can be cancelled. +The [launch] function returns a [Job] that can be used to cancel the running coroutine: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val job = launch { + repeat(1000) { i -> + println("job: I'm sleeping $i ...") + delay(500L) + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancel() // cancels the job + job.join() // waits for job's completion + println("main: Now I can quit.") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-01.kt). +> +{style="note"} + +It produces the following output: + +```text +job: I'm sleeping 0 ... +job: I'm sleeping 1 ... +job: I'm sleeping 2 ... +main: I'm tired of waiting! +main: Now I can quit. +``` + + + +As soon as main invokes `job.cancel`, we don't see any output from the other coroutine because it was cancelled. +There is also a [Job] extension function [cancelAndJoin] +that combines [cancel][Job.cancel] and [join][Job.join] invocations. + +## Cancellation is cooperative + +Coroutine cancellation is _cooperative_. A coroutine code has to cooperate to be cancellable. +All the suspending functions in `kotlinx.coroutines` are _cancellable_. They check for cancellation of +coroutine and throw [CancellationException] when cancelled. However, if a coroutine is working in +a computation and does not check for cancellation, then it cannot be cancelled, like the following +example shows: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val startTime = System.currentTimeMillis() + val job = launch(Dispatchers.Default) { + var nextPrintTime = startTime + var i = 0 + while (i < 5) { // computation loop, just wastes CPU + // print a message twice a second + if (System.currentTimeMillis() >= nextPrintTime) { + println("job: I'm sleeping ${i++} ...") + nextPrintTime += 500L + } + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-02.kt). +> +{style="note"} + +Run it to see that it continues to print "I'm sleeping" even after cancellation +until the job completes by itself after five iterations. + + + +The same problem can be observed by catching a [CancellationException] and not rethrowing it: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val job = launch(Dispatchers.Default) { + repeat(5) { i -> + try { + // print a message twice a second + println("job: I'm sleeping $i ...") + delay(500) + } catch (e: Exception) { + // log the exception + println(e) + } + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-03.kt). +> +{style="note"} + +While catching `Exception` is an anti-pattern, this issue may surface in more subtle ways, like when using the +[`runCatching`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/run-catching.html) function, +which does not rethrow [CancellationException]. + +## Making computation code cancellable + +There are two approaches to making computation code cancellable. +The first one is periodically invoking a suspending function that checks for cancellation. +There are the [yield] and [ensureActive] +functions, which are great choices for that purpose. +The other one is explicitly checking the cancellation status using [isActive]. +Let us try the latter approach. + +Replace `while (i < 5)` in the previous example with `while (isActive)` and rerun it. + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val startTime = System.currentTimeMillis() + val job = launch(Dispatchers.Default) { + var nextPrintTime = startTime + var i = 0 + while (isActive) { // cancellable computation loop + // prints a message twice a second + if (System.currentTimeMillis() >= nextPrintTime) { + println("job: I'm sleeping ${i++} ...") + nextPrintTime += 500L + } + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-04.kt). +> +{style="note"} + +As you can see, now this loop is cancelled. [isActive] is an extension property +available inside the coroutine via the [CoroutineScope] object. + + + +## Closing resources with finally + +Cancellable suspending functions throw [CancellationException] on cancellation, which can be handled in +the usual way. +For example, +the `try {...} finally {...}` expression and Kotlin's [use](https://kotlinlang.org/api/core/kotlin-stdlib/kotlin.io/use.html) +function execute their finalization actions normally when a coroutine is cancelled: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val job = launch { + try { + repeat(1000) { i -> + println("job: I'm sleeping $i ...") + delay(500L) + } + } finally { + println("job: I'm running finally") + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-05.kt). +> +{style="note"} + +Both [join][Job.join] and [cancelAndJoin] wait for all finalization actions to complete, +so the example above produces the following output: + +```text +job: I'm sleeping 0 ... +job: I'm sleeping 1 ... +job: I'm sleeping 2 ... +main: I'm tired of waiting! +job: I'm running finally +main: Now I can quit. +``` + + + +## Run non-cancellable block + +Any attempt to use a suspending function in the `finally` block of the previous example causes +[CancellationException], because the coroutine running this code is cancelled. Usually, this is not a +problem, since all well-behaved closing operations (closing a file, cancelling a job, or closing any kind of +communication channel) are usually non-blocking and do not involve any suspending functions. However, in the +rare case when you need to suspend in a cancelled coroutine you can wrap the corresponding code in +`withContext(NonCancellable) {...}` using [withContext] function and [NonCancellable] context as the following example shows: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val job = launch { + try { + repeat(1000) { i -> + println("job: I'm sleeping $i ...") + delay(500L) + } + } finally { + withContext(NonCancellable) { + println("job: I'm running finally") + delay(1000L) + println("job: And I've just delayed for 1 sec because I'm non-cancellable") + } + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-06.kt). +> +{style="note"} + + + +## Timeout + +The most obvious practical reason to cancel execution of a coroutine +is because its execution time has exceeded some timeout. +While you can manually track the reference to the corresponding [Job] and launch a separate coroutine to cancel +the tracked one after delay, there is a ready to use [withTimeout] function that does it. +Look at the following example: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + withTimeout(1300L) { + repeat(1000) { i -> + println("I'm sleeping $i ...") + delay(500L) + } + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-07.kt). +> +{style="note"} + +It produces the following output: + +```text +I'm sleeping 0 ... +I'm sleeping 1 ... +I'm sleeping 2 ... +Exception in thread "main" kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 1300 ms +``` + + + +The [TimeoutCancellationException] that is thrown by [withTimeout] is a subclass of [CancellationException]. +We have not seen its stack trace printed on the console before. That is because +inside a cancelled coroutine `CancellationException` is considered to be a normal reason for coroutine completion. +However, in this example we have used `withTimeout` right inside the `main` function. + +Since cancellation is just an exception, all resources are closed in the usual way. +You can wrap the code with timeout in a `try {...} catch (e: TimeoutCancellationException) {...}` block if +you need to do some additional action specifically on any kind of timeout or use the [withTimeoutOrNull] function +that is similar to [withTimeout] but returns `null` on timeout instead of throwing an exception: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val result = withTimeoutOrNull(1300L) { + repeat(1000) { i -> + println("I'm sleeping $i ...") + delay(500L) + } + "Done" // will get cancelled before it produces this result + } + println("Result is $result") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-08.kt). +> +{style="note"} + +There is no longer an exception when running this code: + +```text +I'm sleeping 0 ... +I'm sleeping 1 ... +I'm sleeping 2 ... +Result is null +``` + + + +## Asynchronous timeout and resources + + + +The timeout event in [withTimeout] is asynchronous with respect to the code running in its block and may happen at any time, +even right before the return from inside of the timeout block. Keep this in mind if you open or acquire some +resource inside the block that needs closing or release outside of the block. + +For example, here we imitate a closeable resource with the `Resource` class that simply keeps track of how many times +it was created by incrementing the `acquired` counter and decrementing the counter in its `close` function. +Now let us create a lot of coroutines, each of which creates a `Resource` at the end of the `withTimeout` block +and releases the resource outside the block. We add a small delay so that it is more likely that the timeout occurs +right when the `withTimeout` block is already finished, which will cause a resource leak. + +```kotlin +import kotlinx.coroutines.* + +//sampleStart +var acquired = 0 + +class Resource { + init { acquired++ } // Acquire the resource + fun close() { acquired-- } // Release the resource +} + +fun main() { + runBlocking { + repeat(10_000) { // Launch 10K coroutines + launch { + val resource = withTimeout(60) { // Timeout of 60 ms + delay(50) // Delay for 50 ms + Resource() // Acquire a resource and return it from withTimeout block + } + resource.close() // Release the resource + } + } + } + // Outside of runBlocking all coroutines have completed + println(acquired) // Print the number of resources still acquired +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-09.kt). +> +{style="note"} + + + +If you run the above code, you'll see that it does not always print zero, though it may depend on the timings +of your machine. You may need to tweak the timeout in this example to actually see non-zero values. + +> Note that incrementing and decrementing `acquired` counter here from 10K coroutines is completely thread-safe, +> since it always happens from the same thread, the one used by `runBlocking`. +> More on that will be explained in the chapter on coroutine context. +> +{style="note"} + +To work around this problem you can store a reference to the resource in a variable instead of returning it +from the `withTimeout` block. + +```kotlin +import kotlinx.coroutines.* + +var acquired = 0 + +class Resource { + init { acquired++ } // Acquire the resource + fun close() { acquired-- } // Release the resource +} + +fun main() { +//sampleStart + runBlocking { + repeat(10_000) { // Launch 10K coroutines + launch { + var resource: Resource? = null // Not acquired yet + try { + withTimeout(60) { // Timeout of 60 ms + delay(50) // Delay for 50 ms + resource = Resource() // Store a resource to the variable if acquired + } + // We can do something else with the resource here + } finally { + resource?.close() // Release the resource if it was acquired + } + } + } + } + // Outside of runBlocking all coroutines have completed + println(acquired) // Print the number of resources still acquired +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-cancel-10.kt). +> +{style="note"} + +This example always prints zero. Resources do not leak. + + + + + + +[launch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[cancelAndJoin]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/cancel-and-join.html +[Job.cancel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/cancel.html +[Job.join]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/join.html +[CancellationException]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-cancellation-exception/index.html +[yield]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/yield.html +[ensureActive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/ensure-active.html +[isActive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/is-active.html +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[withContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-context.html +[NonCancellable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-non-cancellable/index.html +[withTimeout]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-timeout.html +[TimeoutCancellationException]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-timeout-cancellation-exception/index.html +[withTimeoutOrNull]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-timeout-or-null.html + + diff --git a/docs/topics/channels.md b/docs/topics/channels.md new file mode 100644 index 0000000000..090f9e6990 --- /dev/null +++ b/docs/topics/channels.md @@ -0,0 +1,654 @@ + +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Channels) + +Deferred values provide a convenient way to transfer a single value between coroutines. +Channels provide a way to transfer a stream of values. + +## Channel basics + +A [Channel] is conceptually very similar to `BlockingQueue`. One key difference is that +instead of a blocking `put` operation it has a suspending [send][SendChannel.send], and instead of +a blocking `take` operation it has a suspending [receive][ReceiveChannel.receive]. + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { +//sampleStart + val channel = Channel() + launch { + // this might be heavy CPU-consuming computation or async logic, + // we'll just send five squares + for (x in 1..5) channel.send(x * x) + } + // here we print five received integers: + repeat(5) { println(channel.receive()) } + println("Done!") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-01.kt). +> +{style="note"} + +The output of this code is: + +```text +1 +4 +9 +16 +25 +Done! +``` + + + +## Closing and iteration over channels + +Unlike a queue, a channel can be closed to indicate that no more elements are coming. +On the receiver side it is convenient to use a regular `for` loop to receive elements +from the channel. + +Conceptually, a [close][SendChannel.close] is like sending a special close token to the channel. +The iteration stops as soon as this close token is received, so there is a guarantee +that all previously sent elements before the close are received: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { +//sampleStart + val channel = Channel() + launch { + for (x in 1..5) channel.send(x * x) + channel.close() // we're done sending + } + // here we print received values using `for` loop (until the channel is closed) + for (y in channel) println(y) + println("Done!") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-02.kt). +> +{style="note"} + + + +## Building channel producers + +The pattern where a coroutine is producing a sequence of elements is quite common. +This is a part of _producer-consumer_ pattern that is often found in concurrent code. +You could abstract such a producer into a function that takes channel as its parameter, but this goes contrary +to common sense that results must be returned from functions. + +There is a convenient coroutine builder named [produce] that makes it easy to do it right on producer side, +and an extension function [consumeEach], that replaces a `for` loop on the consumer side: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +//sampleStart +fun CoroutineScope.produceSquares(): ReceiveChannel = produce { + for (x in 1..5) send(x * x) +} + +fun main() = runBlocking { + val squares = produceSquares() + squares.consumeEach { println(it) } + println("Done!") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-03.kt). +> +{style="note"} + + + +## Pipelines + +A pipeline is a pattern where one coroutine is producing, possibly infinite, stream of values: + +```kotlin +fun CoroutineScope.produceNumbers() = produce { + var x = 1 + while (true) send(x++) // infinite stream of integers starting from 1 +} +``` + +And another coroutine or coroutines are consuming that stream, doing some processing, and producing some other results. +In the example below, the numbers are just squared: + +```kotlin +fun CoroutineScope.square(numbers: ReceiveChannel): ReceiveChannel = produce { + for (x in numbers) send(x * x) +} +``` + +The main code starts and connects the whole pipeline: + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { +//sampleStart + val numbers = produceNumbers() // produces integers from 1 and on + val squares = square(numbers) // squares integers + repeat(5) { + println(squares.receive()) // print first five + } + println("Done!") // we are done + coroutineContext.cancelChildren() // cancel children coroutines +//sampleEnd +} + +fun CoroutineScope.produceNumbers() = produce { + var x = 1 + while (true) send(x++) // infinite stream of integers starting from 1 +} + +fun CoroutineScope.square(numbers: ReceiveChannel): ReceiveChannel = produce { + for (x in numbers) send(x * x) +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-04.kt). +> +{style="note"} + + + +> All functions that create coroutines are defined as extensions on [CoroutineScope], +> so that we can rely on [structured concurrency](composing-suspending-functions.md#structured-concurrency-with-async) to make +> sure that we don't have lingering global coroutines in our application. +> +{style="note"} + +## Prime numbers with pipeline + +Let's take pipelines to the extreme with an example that generates prime numbers using a pipeline +of coroutines. We start with an infinite sequence of numbers. + +```kotlin +fun CoroutineScope.numbersFrom(start: Int) = produce { + var x = start + while (true) send(x++) // infinite stream of integers from start +} +``` + +The following pipeline stage filters an incoming stream of numbers, removing all the numbers +that are divisible by the given prime number: + +```kotlin +fun CoroutineScope.filter(numbers: ReceiveChannel, prime: Int) = produce { + for (x in numbers) if (x % prime != 0) send(x) +} +``` + +Now we build our pipeline by starting a stream of numbers from 2, taking a prime number from the current channel, +and launching new pipeline stage for each prime number found: + +``` +numbersFrom(2) -> filter(2) -> filter(3) -> filter(5) -> filter(7) ... +``` + +The following example prints the first ten prime numbers, +running the whole pipeline in the context of the main thread. Since all the coroutines are launched in +the scope of the main [runBlocking] coroutine +we don't have to keep an explicit list of all the coroutines we have started. +We use [cancelChildren][kotlin.coroutines.CoroutineContext.cancelChildren] +extension function to cancel all the children coroutines after we have printed +the first ten prime numbers. + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { +//sampleStart + var cur = numbersFrom(2) + repeat(10) { + val prime = cur.receive() + println(prime) + cur = filter(cur, prime) + } + coroutineContext.cancelChildren() // cancel all children to let main finish +//sampleEnd +} + +fun CoroutineScope.numbersFrom(start: Int) = produce { + var x = start + while (true) send(x++) // infinite stream of integers from start +} + +fun CoroutineScope.filter(numbers: ReceiveChannel, prime: Int) = produce { + for (x in numbers) if (x % prime != 0) send(x) +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-05.kt). +> +{style="note"} + +The output of this code is: + +```text +2 +3 +5 +7 +11 +13 +17 +19 +23 +29 +``` + + + +Note that you can build the same pipeline using +[`iterator`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.sequences/iterator.html) +coroutine builder from the standard library. +Replace `produce` with `iterator`, `send` with `yield`, `receive` with `next`, +`ReceiveChannel` with `Iterator`, and get rid of the coroutine scope. You will not need `runBlocking` either. +However, the benefit of a pipeline that uses channels as shown above is that it can actually use +multiple CPU cores if you run it in [Dispatchers.Default] context. + +Anyway, this is an extremely impractical way to find prime numbers. In practice, pipelines do involve some +other suspending invocations (like asynchronous calls to remote services) and these pipelines cannot be +built using `sequence`/`iterator`, because they do not allow arbitrary suspension, unlike +`produce`, which is fully asynchronous. + +## Fan-out + +Multiple coroutines may receive from the same channel, distributing work between themselves. +Let us start with a producer coroutine that is periodically producing integers +(ten numbers per second): + +```kotlin +fun CoroutineScope.produceNumbers() = produce { + var x = 1 // start from 1 + while (true) { + send(x++) // produce next + delay(100) // wait 0.1s + } +} +``` + +Then we can have several processor coroutines. In this example, they just print their id and +received number: + +```kotlin +fun CoroutineScope.launchProcessor(id: Int, channel: ReceiveChannel) = launch { + for (msg in channel) { + println("Processor #$id received $msg") + } +} +``` + +Now let us launch five processors and let them work for almost a second. See what happens: + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { +//sampleStart + val producer = produceNumbers() + repeat(5) { launchProcessor(it, producer) } + delay(950) + producer.cancel() // cancel producer coroutine and thus kill them all +//sampleEnd +} + +fun CoroutineScope.produceNumbers() = produce { + var x = 1 // start from 1 + while (true) { + send(x++) // produce next + delay(100) // wait 0.1s + } +} + +fun CoroutineScope.launchProcessor(id: Int, channel: ReceiveChannel) = launch { + for (msg in channel) { + println("Processor #$id received $msg") + } +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-06.kt). +> +{style="note"} + +The output will be similar to the following one, albeit the processor ids that receive +each specific integer may be different: + +```text +Processor #2 received 1 +Processor #4 received 2 +Processor #0 received 3 +Processor #1 received 4 +Processor #3 received 5 +Processor #2 received 6 +Processor #4 received 7 +Processor #0 received 8 +Processor #1 received 9 +Processor #3 received 10 +``` + + + +Note that cancelling a producer coroutine closes its channel, thus eventually terminating iteration +over the channel that processor coroutines are doing. + +Also, pay attention to how we explicitly iterate over channel with `for` loop to perform fan-out in `launchProcessor` code. +Unlike `consumeEach`, this `for` loop pattern is perfectly safe to use from multiple coroutines. If one of the processor +coroutines fails, then others would still be processing the channel, while a processor that is written via `consumeEach` +always consumes (cancels) the underlying channel on its normal or abnormal completion. + +## Fan-in + +Multiple coroutines may send to the same channel. +For example, let us have a channel of strings, and a suspending function that +repeatedly sends a specified string to this channel with a specified delay: + +```kotlin +suspend fun sendString(channel: SendChannel, s: String, time: Long) { + while (true) { + delay(time) + channel.send(s) + } +} +``` + +Now, let us see what happens if we launch a couple of coroutines sending strings +(in this example we launch them in the context of the main thread as main coroutine's children): + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { +//sampleStart + val channel = Channel() + launch { sendString(channel, "foo", 200L) } + launch { sendString(channel, "BAR!", 500L) } + repeat(6) { // receive first six + println(channel.receive()) + } + coroutineContext.cancelChildren() // cancel all children to let main finish +//sampleEnd +} + +suspend fun sendString(channel: SendChannel, s: String, time: Long) { + while (true) { + delay(time) + channel.send(s) + } +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-07.kt). +> +{style="note"} + +The output is: + +```text +foo +foo +BAR! +foo +foo +BAR! +``` + + + +## Buffered channels + +The channels shown so far had no buffer. Unbuffered channels transfer elements when sender and receiver +meet each other (aka rendezvous). If send is invoked first, then it is suspended until receive is invoked, +if receive is invoked first, it is suspended until send is invoked. + +Both [Channel()] factory function and [produce] builder take an optional `capacity` parameter to +specify _buffer size_. Buffer allows senders to send multiple elements before suspending, +similar to the `BlockingQueue` with a specified capacity, which blocks when buffer is full. + +Take a look at the behavior of the following code: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { +//sampleStart + val channel = Channel(4) // create buffered channel + val sender = launch { // launch sender coroutine + repeat(10) { + println("Sending $it") // print before sending each element + channel.send(it) // will suspend when buffer is full + } + } + // don't receive anything... just wait.... + delay(1000) + sender.cancel() // cancel sender coroutine +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-08.kt). +> +{style="note"} + +It prints "sending" _five_ times using a buffered channel with capacity of _four_: + +```text +Sending 0 +Sending 1 +Sending 2 +Sending 3 +Sending 4 +``` + + + +The first four elements are added to the buffer and the sender suspends when trying to send the fifth one. + +## Channels are fair + +Send and receive operations to channels are _fair_ with respect to the order of their invocation from +multiple coroutines. They are served in first-in first-out order, e.g. the first coroutine to invoke `receive` +gets the element. In the following example two coroutines "ping" and "pong" are +receiving the "ball" object from the shared "table" channel. + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +//sampleStart +data class Ball(var hits: Int) + +fun main() = runBlocking { + val table = Channel() // a shared table + launch { player("ping", table) } + launch { player("pong", table) } + table.send(Ball(0)) // serve the ball + delay(1000) // delay 1 second + coroutineContext.cancelChildren() // game over, cancel them +} + +suspend fun player(name: String, table: Channel) { + for (ball in table) { // receive the ball in a loop + ball.hits++ + println("$name $ball") + delay(300) // wait a bit + table.send(ball) // send the ball back + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-09.kt). +> +{style="note"} + +The "ping" coroutine is started first, so it is the first one to receive the ball. Even though "ping" +coroutine immediately starts receiving the ball again after sending it back to the table, the ball gets +received by the "pong" coroutine, because it was already waiting for it: + +```text +ping Ball(hits=1) +pong Ball(hits=2) +ping Ball(hits=3) +pong Ball(hits=4) +``` + + + +Note that sometimes channels may produce executions that look unfair due to the nature of the executor +that is being used. See [this issue](https://github.com/Kotlin/kotlinx.coroutines/issues/111) for details. + +## Ticker channels + +Ticker channel is a special rendezvous channel that produces `Unit` every time given delay passes since last consumption from this channel. +Though it may seem to be useless standalone, it is a useful building block to create complex time-based [produce] +pipelines and operators that do windowing and other time-dependent processing. +Ticker channel can be used in [select] to perform "on tick" action. + +To create such channel use a factory method [ticker]. +To indicate that no further elements are needed use [ReceiveChannel.cancel] method on it. + +Now let's see how it works in practice: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +//sampleStart +fun main() = runBlocking { + val tickerChannel = ticker(delayMillis = 200, initialDelayMillis = 0) // create a ticker channel + var nextElement = withTimeoutOrNull(1) { tickerChannel.receive() } + println("Initial element is available immediately: $nextElement") // no initial delay + + nextElement = withTimeoutOrNull(100) { tickerChannel.receive() } // all subsequent elements have 200ms delay + println("Next element is not ready in 100 ms: $nextElement") + + nextElement = withTimeoutOrNull(120) { tickerChannel.receive() } + println("Next element is ready in 200 ms: $nextElement") + + // Emulate large consumption delays + println("Consumer pauses for 300ms") + delay(300) + // Next element is available immediately + nextElement = withTimeoutOrNull(1) { tickerChannel.receive() } + println("Next element is available immediately after large consumer delay: $nextElement") + // Note that the pause between `receive` calls is taken into account and next element arrives faster + nextElement = withTimeoutOrNull(120) { tickerChannel.receive() } + println("Next element is ready in 100ms after consumer pause in 300ms: $nextElement") + + tickerChannel.cancel() // indicate that no more elements are needed +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-channel-10.kt). +> +{style="note"} + +It prints following lines: + +```text +Initial element is available immediately: kotlin.Unit +Next element is not ready in 100 ms: null +Next element is ready in 200 ms: kotlin.Unit +Consumer pauses for 300ms +Next element is available immediately after large consumer delay: kotlin.Unit +Next element is ready in 100ms after consumer pause in 300ms: kotlin.Unit +``` + + + +Note that [ticker] is aware of possible consumer pauses and, by default, adjusts next produced element +delay if a pause occurs, trying to maintain a fixed rate of produced elements. + +Optionally, a `mode` parameter equal to [TickerMode.FIXED_DELAY] can be specified to maintain a fixed +delay between elements. + + + + +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[runBlocking]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html +[kotlin.coroutines.CoroutineContext.cancelChildren]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/cancel-children.html +[Dispatchers.Default]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-default.html + + + +[Channel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-channel/index.html +[SendChannel.send]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/send.html +[ReceiveChannel.receive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/receive.html +[SendChannel.close]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/close.html +[produce]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/produce.html +[consumeEach]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/consume-each.html +[Channel()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-channel.html +[ticker]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/ticker.html +[ReceiveChannel.cancel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/cancel.html +[TickerMode.FIXED_DELAY]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-ticker-mode/-f-i-x-e-d_-d-e-l-a-y/index.html + + + +[select]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.selects/select.html + + diff --git a/docs/topics/compatibility.md b/docs/topics/compatibility.md new file mode 100644 index 0000000000..9570f1985b --- /dev/null +++ b/docs/topics/compatibility.md @@ -0,0 +1,129 @@ + + +* [Compatibility](#compatibility) +* [Public API types](#public-api-types) + * [Experimental API](#experimental-api) + * [Flow preview API](#flow-preview-api) + * [Obsolete API](#obsolete-api) + * [Internal API](#internal-api) + * [Stable API](#stable-api) + * [Deprecation cycle](#deprecation-cycle) +* [Using annotated API](#using-annotated-api) + * [Programmatically](#programmatically) + * [Gradle](#gradle) + * [Maven](#maven) + + + +## Compatibility +This document describes the compatibility policy of `kotlinx.coroutines` library since version 1.0.0 and semantics of compatibility-specific annotations. + + +## Public API types +`kotlinx.coroutines` public API comes in five flavours: stable, experimental, obsolete, internal and deprecated. +All public API except stable is marked with the corresponding annotation. + +### Experimental API +Experimental API is marked with [@ExperimentalCoroutinesApi][ExperimentalCoroutinesApi] annotation. +API is marked experimental when its design has potential open questions which may eventually lead to +either semantics changes of the API or its deprecation. + +By default, most of the new API is marked as experimental and becomes stable in one of the next major releases if no new issues arise. +Otherwise, either semantics is fixed without changes in ABI or API goes through deprecation cycle. + +When using experimental API may be dangerous: +* You are writing a library which depends on `kotlinx.coroutines` and want to use experimental coroutines API in a stable library API. +It may lead to undesired consequences when end users of your library update their `kotlinx.coroutines` version where experimental API +has slightly different semantics. +* You want to build core infrastructure of the application around experimental API. + +### Flow preview API +All [Flow]-related API is marked with [@FlowPreview][FlowPreview] annotation. +This annotation indicates that Flow API is in preview status. +We provide no compatibility guarantees between releases for preview features, including binary, source and semantics compatibility. + +When using preview API may be dangerous: +* You are writing a library/framework and want to use [Flow] API in a stable release or in a stable API. +* You want to use [Flow] in the core infrastructure of your application. +* You want to use [Flow] as "write-and-forget" solution and cannot afford additional maintenance cost when + it comes to `kotlinx.coroutines` updates. + + +### Obsolete API +Obsolete API is marked with [@ObsoleteCoroutinesApi][ObsoleteCoroutinesApi] annotation. +Obsolete API is similar to experimental, but already known to have serious design flaws and its potential replacement, +but replacement is not yet implemented. + +The semantics of this API won't be changed, but it will go through a deprecation cycle as soon as the replacement is ready. + +### Internal API +Internal API is marked with [@InternalCoroutinesApi][InternalCoroutinesApi] or is part of `kotlinx.coroutines.internal` package. +This API has no guarantees on its stability, can and will be changed and/or removed in the future releases. +If you can't avoid using internal API, please report it to [issue tracker](https://github.com/Kotlin/kotlinx.coroutines/issues/new). + +### Stable API +Stable API is guaranteed to preserve its ABI and documented semantics. If at some point unfixable design flaws will be discovered, +this API will go through a deprecation cycle and remain binary compatible as long as possible. + +### Deprecation cycle +When some API is deprecated, it goes through multiple stages and there is at least one major release between stages. +* Feature is deprecated with compilation warning. Most of the time, proper replacement +(and corresponding `replaceWith` declaration) is provided to automatically migrate deprecated usages with a help of IntelliJ IDEA. +* Deprecation level is increased to `error` or `hidden`. It is no longer possible to compile new code against deprecated API, + though it is still present in the ABI. +* API is completely removed. While we give our best efforts not to do so and have no plans of removing any API, we still are leaving +this option in case of unforeseen problems such as security holes. + +## Using annotated API +All API annotations are [kotlin.Experimental](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-experimental/). +It is done in order to produce compilation warning about using experimental or obsolete API. +Warnings can be disabled either programmatically for a specific call site or globally for the whole module. + +### Programmatically +For a specific call-site, warning can be disabled by using [OptIn](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-opt-in/) annotation: +```kotlin +@OptIn(ExperimentalCoroutinesApi::class) // Disables warning about experimental coroutines API +fun experimentalApiUsage() { + someKotlinxCoroutinesExperimentalMethod() +} +``` + +### Gradle +For the Gradle project, a warning can be disabled by passing a compiler flag in your `build.gradle` file: + +```groovy +tasks.withType(org.jetbrains.kotlin.gradle.tasks.AbstractKotlinCompile).all { + kotlinOptions.freeCompilerArgs += ["-Xuse-experimental=kotlinx.coroutines.ExperimentalCoroutinesApi"] +} + +``` + +### Maven +For the Maven project, a warning can be disabled by passing a compiler flag in your `pom.xml` file: +```xml + + kotlin-maven-plugin + org.jetbrains.kotlin + ... your configuration ... + + + -Xuse-experimental=kotlinx.coroutines.ExperimentalCoroutinesApi + + + +``` + + + + + +[Flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html + + + +[ExperimentalCoroutinesApi]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-experimental-coroutines-api/index.html +[FlowPreview]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-flow-preview/index.html +[ObsoleteCoroutinesApi]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-obsolete-coroutines-api/index.html +[InternalCoroutinesApi]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-internal-coroutines-api/index.html + + diff --git a/docs/topics/composing-suspending-functions.md b/docs/topics/composing-suspending-functions.md new file mode 100644 index 0000000000..1e6f590af2 --- /dev/null +++ b/docs/topics/composing-suspending-functions.md @@ -0,0 +1,419 @@ + +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Composing suspending functions) + +This section covers various approaches to composition of suspending functions. + +## Sequential by default + +Assume that we have two suspending functions defined elsewhere that do something useful like some kind of +remote service call or computation. We just pretend they are useful, but actually each one just +delays for a second for the purpose of this example: + +```kotlin +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} +``` + +What do we do if we need them to be invoked _sequentially_ — first `doSomethingUsefulOne` _and then_ +`doSomethingUsefulTwo`, and compute the sum of their results? +In practice, we do this if we use the result of the first function to make a decision on whether we need +to invoke the second one or to decide on how to invoke it. + +We use a normal sequential invocation, because the code in the coroutine, just like in the regular +code, is _sequential_ by default. The following example demonstrates it by measuring the total +time it takes to execute both suspending functions: + + + +```kotlin +import kotlinx.coroutines.* +import kotlin.system.* + +fun main() = runBlocking { +//sampleStart + val time = measureTimeMillis { + val one = doSomethingUsefulOne() + val two = doSomethingUsefulTwo() + println("The answer is ${one + two}") + } + println("Completed in $time ms") +//sampleEnd +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-compose-01.kt). +> +{style="note"} + +It produces something like this: + +```text +The answer is 42 +Completed in 2017 ms +``` + + + +## Concurrent using async + +What if there are no dependencies between invocations of `doSomethingUsefulOne` and `doSomethingUsefulTwo` and +we want to get the answer faster, by doing both _concurrently_? This is where [async] comes to help. + +Conceptually, [async] is just like [launch]. It starts a separate coroutine which is a light-weight thread +that works concurrently with all the other coroutines. The difference is that `launch` returns a [Job] and +does not carry any resulting value, while `async` returns a [Deferred] — a light-weight non-blocking future +that represents a promise to provide a result later. You can use `.await()` on a deferred value to get its eventual result, +but `Deferred` is also a `Job`, so you can cancel it if needed. + +```kotlin +import kotlinx.coroutines.* +import kotlin.system.* + +fun main() = runBlocking { +//sampleStart + val time = measureTimeMillis { + val one = async { doSomethingUsefulOne() } + val two = async { doSomethingUsefulTwo() } + println("The answer is ${one.await() + two.await()}") + } + println("Completed in $time ms") +//sampleEnd +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-compose-02.kt). +> +{style="note"} + +It produces something like this: + +```text +The answer is 42 +Completed in 1017 ms +``` + + + +This is twice as fast, because the two coroutines execute concurrently. +Note that concurrency with coroutines is always explicit. + +## Lazily started async + +Optionally, [async] can be made lazy by setting its `start` parameter to [CoroutineStart.LAZY]. +In this mode it only starts the coroutine when its result is required by +[await][Deferred.await], or if its `Job`'s [start][Job.start] function +is invoked. Run the following example: + +```kotlin +import kotlinx.coroutines.* +import kotlin.system.* + +fun main() = runBlocking { +//sampleStart + val time = measureTimeMillis { + val one = async(start = CoroutineStart.LAZY) { doSomethingUsefulOne() } + val two = async(start = CoroutineStart.LAZY) { doSomethingUsefulTwo() } + // some computation + one.start() // start the first one + two.start() // start the second one + println("The answer is ${one.await() + two.await()}") + } + println("Completed in $time ms") +//sampleEnd +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-compose-03.kt). +> +{style="note"} + +It produces something like this: + +```text +The answer is 42 +Completed in 1017 ms +``` + + + +So, here the two coroutines are defined but not executed as in the previous example, but the control is given to +the programmer on when exactly to start the execution by calling [start][Job.start]. We first +start `one`, then start `two`, and then await for the individual coroutines to finish. + +Note that if we just call [await][Deferred.await] in `println` without first calling [start][Job.start] on individual +coroutines, this will lead to sequential behavior, since [await][Deferred.await] starts the coroutine +execution and waits for its finish, which is not the intended use-case for laziness. +The use-case for `async(start = CoroutineStart.LAZY)` is a replacement for the +standard [lazy](https://kotlinlang.org/api/core/kotlin-stdlib/kotlin/lazy.html) function in cases +when computation of the value involves suspending functions. + + +## Async-style functions + +> This programming style with async functions is provided here only for illustration, because it is a popular style +> in other programming languages. Using this style with Kotlin coroutines is **strongly discouraged** for the +> reasons explained below. +> +{style="note"} + +We can define async-style functions that invoke `doSomethingUsefulOne` and `doSomethingUsefulTwo` +_asynchronously_ using the [async] coroutine builder using a [GlobalScope] reference to +opt-out of the structured concurrency. +We name such functions with the +"...Async" suffix to highlight the fact that they only start asynchronous computation and one needs +to use the resulting deferred value to get the result. + +> [GlobalScope] is a delicate API that can backfire in non-trivial ways, one of which will be explained +> below, so you must explicitly opt-in into using `GlobalScope` with `@OptIn(DelicateCoroutinesApi::class)`. +> +{style="note"} + +```kotlin +// The result type of somethingUsefulOneAsync is Deferred +@OptIn(DelicateCoroutinesApi::class) +fun somethingUsefulOneAsync() = GlobalScope.async { + doSomethingUsefulOne() +} + +// The result type of somethingUsefulTwoAsync is Deferred +@OptIn(DelicateCoroutinesApi::class) +fun somethingUsefulTwoAsync() = GlobalScope.async { + doSomethingUsefulTwo() +} +``` + +Note that these `xxxAsync` functions are **not** _suspending_ functions. They can be used from anywhere. +However, their use always implies asynchronous (here meaning _concurrent_) execution of their action +with the invoking code. + +The following example shows their use outside of coroutine: + + + +```kotlin +import kotlinx.coroutines.* +import kotlin.system.* + +//sampleStart +// note that we don't have `runBlocking` to the right of `main` in this example +fun main() { + val time = measureTimeMillis { + // we can initiate async actions outside of a coroutine + val one = somethingUsefulOneAsync() + val two = somethingUsefulTwoAsync() + // but waiting for a result must involve either suspending or blocking. + // here we use `runBlocking { ... }` to block the main thread while waiting for the result + runBlocking { + println("The answer is ${one.await() + two.await()}") + } + } + println("Completed in $time ms") +} +//sampleEnd + +@OptIn(DelicateCoroutinesApi::class) +fun somethingUsefulOneAsync() = GlobalScope.async { + doSomethingUsefulOne() +} + +@OptIn(DelicateCoroutinesApi::class) +fun somethingUsefulTwoAsync() = GlobalScope.async { + doSomethingUsefulTwo() +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-compose-04.kt). +> +{style="note"} + + + +Consider what happens if between the `val one = somethingUsefulOneAsync()` line and `one.await()` expression there is some logic +error in the code, and the program throws an exception, and the operation that was being performed by the program aborts. +Normally, a global error-handler could catch this exception, log and report the error for developers, but the program +could otherwise continue doing other operations. However, here we have `somethingUsefulOneAsync` still running in the background, +even though the operation that initiated it was aborted. This problem does not happen with structured +concurrency, as shown in the section below. + +## Structured concurrency with async + +Let's refactor the [Concurrent using async](#concurrent-using-async) example into a function that runs +`doSomethingUsefulOne` and `doSomethingUsefulTwo` concurrently and returns their combined results. +Since [async] is a [CoroutineScope] extension, +we'll use the [coroutineScope][_coroutineScope] function to provide the necessary scope: + +```kotlin +suspend fun concurrentSum(): Int = coroutineScope { + val one = async { doSomethingUsefulOne() } + val two = async { doSomethingUsefulTwo() } + one.await() + two.await() +} +``` + +This way, if something goes wrong inside the code of the `concurrentSum` function, and it throws an exception, +all the coroutines that were launched in its scope will be cancelled. + + + +```kotlin +import kotlinx.coroutines.* +import kotlin.system.* + +fun main() = runBlocking { +//sampleStart + val time = measureTimeMillis { + println("The answer is ${concurrentSum()}") + } + println("Completed in $time ms") +//sampleEnd +} + +suspend fun concurrentSum(): Int = coroutineScope { + val one = async { doSomethingUsefulOne() } + val two = async { doSomethingUsefulTwo() } + one.await() + two.await() +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-compose-05.kt). +> +{style="note"} + +We still have concurrent execution of both operations, as evident from the output of the above `main` function: + +```text +The answer is 42 +Completed in 1017 ms +``` + + + +Cancellation is always propagated through coroutines hierarchy: + + + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { + try { + failedConcurrentSum() + } catch(e: ArithmeticException) { + println("Computation failed with ArithmeticException") + } +} + +suspend fun failedConcurrentSum(): Int = coroutineScope { + val one = async { + try { + delay(Long.MAX_VALUE) // Emulates very long computation + 42 + } finally { + println("First child was cancelled") + } + } + val two = async { + println("Second child throws an exception") + throw ArithmeticException() + } + one.await() + two.await() +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-compose-06.kt). +> +{style="note"} + +Note how both the first `async` and the awaiting parent are cancelled on failure of one of the children +(namely, `two`): +```text +Second child throws an exception +First child was cancelled +Computation failed with ArithmeticException +``` + + + + + + +[async]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html +[launch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[Deferred]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/index.html +[CoroutineStart.LAZY]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-start/-l-a-z-y/index.html +[Deferred.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/await.html +[Job.start]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/start.html +[GlobalScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-global-scope/index.html +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[_coroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/coroutine-scope.html + + diff --git a/docs/topics/coroutine-context-and-dispatchers.md b/docs/topics/coroutine-context-and-dispatchers.md new file mode 100644 index 0000000000..ca5f54cd98 --- /dev/null +++ b/docs/topics/coroutine-context-and-dispatchers.md @@ -0,0 +1,697 @@ + +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Coroutine context and dispatchers) + +Coroutines always execute in some context represented by a value of the +[CoroutineContext](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.coroutines/-coroutine-context/) +type, defined in the Kotlin standard library. + +The coroutine context is a set of various elements. The main elements are the [Job] of the coroutine, +which we've seen before, and its dispatcher, which is covered in this section. + +## Dispatchers and threads + +The coroutine context includes a _coroutine dispatcher_ (see [CoroutineDispatcher]) that determines what thread or threads +the corresponding coroutine uses for its execution. The coroutine dispatcher can confine coroutine execution +to a specific thread, dispatch it to a thread pool, or let it run unconfined. + +All coroutine builders like [launch] and [async] accept an optional +[CoroutineContext](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.coroutines/-coroutine-context/) +parameter that can be used to explicitly specify the dispatcher for the new coroutine and other context elements. + +Try the following example: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + launch { // context of the parent, main runBlocking coroutine + println("main runBlocking : I'm working in thread ${Thread.currentThread().name}") + } + launch(Dispatchers.Unconfined) { // not confined -- will work with main thread + println("Unconfined : I'm working in thread ${Thread.currentThread().name}") + } + launch(Dispatchers.Default) { // will get dispatched to DefaultDispatcher + println("Default : I'm working in thread ${Thread.currentThread().name}") + } + launch(newSingleThreadContext("MyOwnThread")) { // will get its own new thread + println("newSingleThreadContext: I'm working in thread ${Thread.currentThread().name}") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-01.kt). +> +{style="note"} + +It produces the following output (maybe in different order): + +```text +Unconfined : I'm working in thread main +Default : I'm working in thread DefaultDispatcher-worker-1 +newSingleThreadContext: I'm working in thread MyOwnThread +main runBlocking : I'm working in thread main +``` + + + +When `launch { ... }` is used without parameters, it inherits the context (and thus dispatcher) +from the [CoroutineScope] it is being launched from. In this case, it inherits the +context of the main `runBlocking` coroutine which runs in the `main` thread. + +[Dispatchers.Unconfined] is a special dispatcher that also appears to run in the `main` thread, but it is, +in fact, a different mechanism that is explained later. + +The default dispatcher is used when no other dispatcher is explicitly specified in the scope. +It is represented by [Dispatchers.Default] and uses a shared background pool of threads. + +[newSingleThreadContext] creates a thread for the coroutine to run. +A dedicated thread is a very expensive resource. +In a real application it must be either released, when no longer needed, using the [close][ExecutorCoroutineDispatcher.close] +function, or stored in a top-level variable and reused throughout the application. + +## Unconfined vs confined dispatcher + +The [Dispatchers.Unconfined] coroutine dispatcher starts a coroutine in the caller thread, but only until the +first suspension point. After suspension it resumes the coroutine in the thread that is fully determined by the +suspending function that was invoked. The unconfined dispatcher is appropriate for coroutines which neither +consume CPU time nor update any shared data (like UI) confined to a specific thread. + +On the other side, the dispatcher is inherited from the outer [CoroutineScope] by default. +The default dispatcher for the [runBlocking] coroutine, in particular, +is confined to the invoker thread, so inheriting it has the effect of confining execution to +this thread with predictable FIFO scheduling. + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + launch(Dispatchers.Unconfined) { // not confined -- will work with main thread + println("Unconfined : I'm working in thread ${Thread.currentThread().name}") + delay(500) + println("Unconfined : After delay in thread ${Thread.currentThread().name}") + } + launch { // context of the parent, main runBlocking coroutine + println("main runBlocking: I'm working in thread ${Thread.currentThread().name}") + delay(1000) + println("main runBlocking: After delay in thread ${Thread.currentThread().name}") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-02.kt). +> +{style="note"} + +Produces the output: + +```text +Unconfined : I'm working in thread main +main runBlocking: I'm working in thread main +Unconfined : After delay in thread kotlinx.coroutines.DefaultExecutor +main runBlocking: After delay in thread main +``` + + + +So, the coroutine with the context inherited from `runBlocking {...}` continues to execute +in the `main` thread, while the unconfined one resumes in the default executor thread that the [delay] +function is using. + +> The unconfined dispatcher is an advanced mechanism that can be helpful in certain corner cases where +> dispatching of a coroutine for its execution later is not needed or produces undesirable side-effects, +> because some operation in a coroutine must be performed right away. +> The unconfined dispatcher should not be used in general code. +> +{style="note"} + +## Debugging coroutines and threads + +Coroutines can suspend on one thread and resume on another thread. +Even with a single-threaded dispatcher it might be hard to +figure out what the coroutine was doing, where, and when if you don't have special tooling. + +### Debugging with IDEA + +The Coroutine Debugger of the Kotlin plugin simplifies debugging coroutines in IntelliJ IDEA. + +> Debugging works for versions 1.3.8 or later of `kotlinx-coroutines-core`. +> +{style="note"} + +The **Debug** tool window contains the **Coroutines** tab. In this tab, you can find information about both currently running and suspended coroutines. +The coroutines are grouped by the dispatcher they are running on. + +![Debugging coroutines](coroutine-idea-debugging-1.png){width=700} + +With the coroutine debugger, you can: +* Check the state of each coroutine. +* See the values of local and captured variables for both running and suspended coroutines. +* See a full coroutine creation stack, as well as a call stack inside the coroutine. The stack includes all frames with +variable values, even those that would be lost during standard debugging. +* Get a full report that contains the state of each coroutine and its stack. To obtain it, right-click inside the **Coroutines** tab, and then click **Get Coroutines Dump**. + +To start coroutine debugging, you just need to set breakpoints and run the application in debug mode. + +Learn more about coroutines debugging in the [tutorial](https://kotlinlang.org/docs/tutorials/coroutines/debug-coroutines-with-idea.html). + +### Debugging using logging + +Another approach to debugging applications with +threads without Coroutine Debugger is to print the thread name in the log file on each log statement. This feature is universally supported +by logging frameworks. When using coroutines, the thread name alone does not give much of a context, so +`kotlinx.coroutines` includes debugging facilities to make it easier. + +Run the following code with `-Dkotlinx.coroutines.debug` JVM option: + +```kotlin +import kotlinx.coroutines.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +fun main() = runBlocking { +//sampleStart + val a = async { + log("I'm computing a piece of the answer") + 6 + } + val b = async { + log("I'm computing another piece of the answer") + 7 + } + log("The answer is ${a.await() * b.await()}") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-03.kt). +> +{style="note"} + +There are three coroutines. The main coroutine (#1) inside `runBlocking` +and two coroutines computing the deferred values `a` (#2) and `b` (#3). +They are all executing in the context of `runBlocking` and are confined to the main thread. +The output of this code is: + +```text +[main @coroutine#2] I'm computing a piece of the answer +[main @coroutine#3] I'm computing another piece of the answer +[main @coroutine#1] The answer is 42 +``` + + + +The `log` function prints the name of the thread in square brackets, and you can see that it is the `main` +thread with the identifier of the currently executing coroutine appended to it. This identifier +is consecutively assigned to all created coroutines when the debugging mode is on. + +> Debugging mode is also turned on when JVM is run with `-ea` option. +> You can read more about debugging facilities in the documentation of the [DEBUG_PROPERTY_NAME] property. +> +{style="note"} + +## Jumping between threads + +Run the following code with the `-Dkotlinx.coroutines.debug` JVM option (see [debug](#debugging-coroutines-and-threads)): + +```kotlin +import kotlinx.coroutines.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +fun main() { + newSingleThreadContext("Ctx1").use { ctx1 -> + newSingleThreadContext("Ctx2").use { ctx2 -> + runBlocking(ctx1) { + log("Started in ctx1") + withContext(ctx2) { + log("Working in ctx2") + } + log("Back to ctx1") + } + } + } +} +``` + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-04.kt). +> +{style="note"} + +The example above demonstrates new techniques in coroutine usage. + +The first technique shows how to use [runBlocking] with a specified context. +The second technique involves calling [withContext], +which may suspend the current coroutine and switch to a new context—provided the new context differs from the existing one. +Specifically, if you specify a different [CoroutineDispatcher], extra dispatches are required: +the block is scheduled on the new dispatcher, and once it finishes, execution returns to the original dispatcher. + +As a result, the output of the above code is: + +```text +[Ctx1 @coroutine#1] Started in ctx1 +[Ctx2 @coroutine#1] Working in ctx2 +[Ctx1 @coroutine#1] Back to ctx1 +``` + + + +The example above uses the `use` function from the Kotlin standard library +to properly release thread resources created by [newSingleThreadContext] when they're no longer needed. + +## Job in the context + +The coroutine's [Job] is part of its context, and can be retrieved from it +using the `coroutineContext[Job]` expression: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + println("My job is ${coroutineContext[Job]}") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-05.kt). +> +{style="note"} + +In [debug mode](#debugging-coroutines-and-threads), it outputs something like this: + +``` +My job is "coroutine#1":BlockingCoroutine{Active}@6d311334 +``` + + + +Note that [isActive] in [CoroutineScope] is just a convenient shortcut for +`coroutineContext[Job]?.isActive == true`. + +## Children of a coroutine + +When a coroutine is launched in the [CoroutineScope] of another coroutine, +it inherits its context via [CoroutineScope.coroutineContext] and +the [Job] of the new coroutine becomes +a _child_ of the parent coroutine's job. When the parent coroutine is cancelled, all its children +are recursively cancelled, too. + +However, this parent-child relation can be explicitly overridden in one of two ways: + +1. When a different scope is explicitly specified when launching a coroutine (for example, `GlobalScope.launch`), +it does not inherit a `Job` from the parent scope. +2. When a different `Job` object is passed as the context for the new coroutine (as shown in the example below), +it overrides the `Job` of the parent scope. + +In both cases, the launched coroutine is not tied to the scope it was launched from and operates independently. + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + // launch a coroutine to process some kind of incoming request + val request = launch { + // it spawns two other jobs + launch(Job()) { + println("job1: I run in my own Job and execute independently!") + delay(1000) + println("job1: I am not affected by cancellation of the request") + } + // and the other inherits the parent context + launch { + delay(100) + println("job2: I am a child of the request coroutine") + delay(1000) + println("job2: I will not execute this line if my parent request is cancelled") + } + } + delay(500) + request.cancel() // cancel processing of the request + println("main: Who has survived request cancellation?") + delay(1000) // delay the main thread for a second to see what happens +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-06.kt). +> +{style="note"} + +The output of this code is: + +```text +job1: I run in my own Job and execute independently! +job2: I am a child of the request coroutine +main: Who has survived request cancellation? +job1: I am not affected by cancellation of the request +``` + + + +## Parental responsibilities + +A parent coroutine always waits for the completion of all its children. +A parent does not have to explicitly track +all the children it launches, and it does not have to use [Job.join] to wait for them at the end: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + // launch a coroutine to process some kind of incoming request + val request = launch { + repeat(3) { i -> // launch a few children jobs + launch { + delay((i + 1) * 200L) // variable delay 200ms, 400ms, 600ms + println("Coroutine $i is done") + } + } + println("request: I'm done and I don't explicitly join my children that are still active") + } + request.join() // wait for completion of the request, including all its children + println("Now processing of the request is complete") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-07.kt). +> +{style="note"} + +The result is going to be: + +```text +request: I'm done and I don't explicitly join my children that are still active +Coroutine 0 is done +Coroutine 1 is done +Coroutine 2 is done +Now processing of the request is complete +``` + + + +## Naming coroutines for debugging + +Automatically assigned ids are good when coroutines log often and you just need to correlate log records +coming from the same coroutine. However, when a coroutine is tied to the processing of a specific request +or doing some specific background task, it is better to name it explicitly for debugging purposes. +The [CoroutineName] context element serves the same purpose as the thread name. It is included in the thread name that +is executing this coroutine when the [debugging mode](#debugging-coroutines-and-threads) is turned on. + +The following example demonstrates this concept: + +```kotlin +import kotlinx.coroutines.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +fun main() = runBlocking(CoroutineName("main")) { +//sampleStart + log("Started main coroutine") + // run two background value computations + val v1 = async(CoroutineName("v1coroutine")) { + delay(500) + log("Computing v1") + 6 + } + val v2 = async(CoroutineName("v2coroutine")) { + delay(1000) + log("Computing v2") + 7 + } + log("The answer for v1 * v2 = ${v1.await() * v2.await()}") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-08.kt). +> +{style="note"} + +The output it produces with `-Dkotlinx.coroutines.debug` JVM option is similar to: + +```text +[main @main#1] Started main coroutine +[main @v1coroutine#2] Computing v1 +[main @v2coroutine#3] Computing v2 +[main @main#1] The answer for v1 * v2 = 42 +``` + + + +## Combining context elements + +Sometimes we need to define multiple elements for a coroutine context. We can use the `+` operator for that. +For example, we can launch a coroutine with an explicitly specified dispatcher and an explicitly specified +name at the same time: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + launch(Dispatchers.Default + CoroutineName("test")) { + println("I'm working in thread ${Thread.currentThread().name}") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-09.kt). +> +{style="note"} + +The output of this code with the `-Dkotlinx.coroutines.debug` JVM option is: + +```text +I'm working in thread DefaultDispatcher-worker-1 @test#2 +``` + + + +## Coroutine scope + +Let us put our knowledge about contexts, children, and jobs together. +Assume that our application has an object with a lifecycle, but that object is not a coroutine. +For example, +we are writing an Android application, +and launching various coroutines in the context of an Android activity +to perform asynchronous operations to fetch and update data, +do animations, etc. These coroutines must be cancelled when the activity is destroyed +to avoid memory leaks. +We, of course, can manipulate contexts and jobs manually to tie the lifecycles of the activity +and its coroutines, but `kotlinx.coroutines` provides an abstraction encapsulating that: [CoroutineScope]. +You should be already familiar with the coroutine scope as all coroutine builders are declared as extensions on it. + +We manage the lifecycles of our coroutines by creating an instance of [CoroutineScope] tied to +the lifecycle of our activity. A `CoroutineScope` instance can be created by the [CoroutineScope()] or [MainScope()] +factory functions. The former creates a general-purpose scope, while the latter creates a scope for UI applications and uses +[Dispatchers.Main] as the default dispatcher: + +```kotlin +class Activity { + private val mainScope = MainScope() + + fun destroy() { + mainScope.cancel() + } + // to be continued ... +``` + +Now, we can launch coroutines in the scope of this `Activity` using the defined `mainScope`. +For the demo, we launch ten coroutines that delay for a different time: + +```kotlin + // class Activity continues + fun doSomething() { + // launch ten coroutines for a demo, each working for a different time + repeat(10) { i -> + mainScope.launch { + delay((i + 1) * 200L) // variable delay 200ms, 400ms, ... etc + println("Coroutine $i is done") + } + } + } +} // class Activity ends +``` + +In our main function we create the activity, call our test `doSomething` function, and destroy the activity after 500ms. +This cancels all the coroutines that were launched from `doSomething`. +We can see that because after the destruction +of the activity, no more messages are printed, even if we wait a little longer. + + + +```kotlin +import kotlinx.coroutines.* + +class Activity { + private val mainScope = CoroutineScope(Dispatchers.Default) // use Default for test purposes + + fun destroy() { + mainScope.cancel() + } + + fun doSomething() { + // launch ten coroutines for a demo, each working for a different time + repeat(10) { i -> + mainScope.launch { + delay((i + 1) * 200L) // variable delay 200ms, 400ms, ... etc + println("Coroutine $i is done") + } + } + } +} // class Activity ends + +fun main() = runBlocking { +//sampleStart + val activity = Activity() + activity.doSomething() // run test function + println("Launched coroutines") + delay(500L) // delay for half a second + println("Destroying activity!") + activity.destroy() // cancels all coroutines + delay(1000) // visually confirm that they don't work +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-10.kt). +> +{style="note"} + +The output of this example is: + +```text +Launched coroutines +Coroutine 0 is done +Coroutine 1 is done +Destroying activity! +``` + + + +As you can see, only the first two coroutines print a message and the others are cancelled +by a single invocation of [`mainScope.cancel()`][CoroutineScope.cancel] in `Activity.destroy()`. + +> Note that Android has first-party support for coroutine scope in all entities with the lifecycle. +> See [the corresponding documentation](https://developer.android.com/topic/libraries/architecture/coroutines#lifecyclescope). +> +{style="note"} + +### Thread-local data + +Sometimes it is convenient to be able to pass some thread-local data to or between coroutines. +However, since they are not bound to any particular thread, this will likely lead to boilerplate if done manually. + +For [`ThreadLocal`](https://docs.oracle.com/javase/8/docs/api/java/lang/ThreadLocal.html), +the [asContextElement] extension function is here for the rescue. It creates an additional context element +which keeps the value of the given `ThreadLocal` and restores it every time the coroutine switches its context. + +It is easy to demonstrate it in action: + +```kotlin +import kotlinx.coroutines.* + +val threadLocal = ThreadLocal() // declare thread-local variable + +fun main() = runBlocking { +//sampleStart + threadLocal.set("main") + println("Pre-main, current thread: ${Thread.currentThread()}, thread local value: '${threadLocal.get()}'") + val job = launch(Dispatchers.Default + threadLocal.asContextElement(value = "launch")) { + println("Launch start, current thread: ${Thread.currentThread()}, thread local value: '${threadLocal.get()}'") + yield() + println("After yield, current thread: ${Thread.currentThread()}, thread local value: '${threadLocal.get()}'") + } + job.join() + println("Post-main, current thread: ${Thread.currentThread()}, thread local value: '${threadLocal.get()}'") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-context-11.kt). +> +{style="note"} + +In this example, we launch a new coroutine in a background thread pool using [Dispatchers.Default], so +it works on different threads from the thread pool, but it still has the value of the thread local variable +that we specified using `threadLocal.asContextElement(value = "launch")`, +no matter which thread the coroutine is executed on. +Thus, the output (with [debug](#debugging-coroutines-and-threads)) is: + +```text +Pre-main, current thread: Thread[main @coroutine#1,5,main], thread local value: 'main' +Launch start, current thread: Thread[DefaultDispatcher-worker-1 @coroutine#2,5,main], thread local value: 'launch' +After yield, current thread: Thread[DefaultDispatcher-worker-2 @coroutine#2,5,main], thread local value: 'launch' +Post-main, current thread: Thread[main @coroutine#1,5,main], thread local value: 'main' +``` + + + +It's easy to forget to set the corresponding context element. The thread-local variable accessed from the coroutine may +then have an unexpected value if the thread running the coroutine is different. +To avoid such situations, it is recommended to use the [ensurePresent] method +and fail-fast on improper usages. + +`ThreadLocal` has first-class support and can be used with any primitive `kotlinx.coroutines` provides. +It has one key limitation, though: when a thread-local is mutated, a new value is not propagated to the coroutine caller +(because a context element cannot track all `ThreadLocal` object accesses), and the updated value is lost on the next suspension. +Use [withContext] to update the value of the thread-local in a coroutine, see [asContextElement] for more details. + +Alternatively, a value can be stored in a mutable box like `class Counter(var i: Int)`, which is, in turn, +stored in a thread-local variable. +However, in this case, you are fully responsible to synchronize +potentially concurrent modifications to the variable in this mutable box. + +For advanced usage, for example, for integration with logging MDC, transactional contexts or any other libraries +that internally use thread-locals for passing data, see the documentation of the [ThreadContextElement] interface +that should be implemented. + + + + +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[CoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/index.html +[launch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html +[async]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[Dispatchers.Unconfined]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-unconfined.html +[Dispatchers.Default]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-default.html +[newSingleThreadContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/new-single-thread-context.html +[ExecutorCoroutineDispatcher.close]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-executor-coroutine-dispatcher/close.html +[runBlocking]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html +[delay]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html +[DEBUG_PROPERTY_NAME]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-d-e-b-u-g_-p-r-o-p-e-r-t-y_-n-a-m-e.html +[withContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-context.html +[isActive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/is-active.html +[CoroutineScope.coroutineContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/coroutine-context.html +[Job.join]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/join.html +[CoroutineName]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-name/index.html +[CoroutineScope()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope.html +[MainScope()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-main-scope.html +[Dispatchers.Main]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-main.html +[CoroutineScope.cancel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/cancel.html +[asContextElement]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/as-context-element.html +[ensurePresent]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/ensure-present.html +[ThreadContextElement]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-thread-context-element/index.html + + diff --git a/docs/topics/coroutines-and-channels.md b/docs/topics/coroutines-and-channels.md new file mode 100644 index 0000000000..45081dc8bc --- /dev/null +++ b/docs/topics/coroutines-and-channels.md @@ -0,0 +1,1557 @@ +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Coroutines and channels − tutorial) + +In this tutorial, you'll learn how to use coroutines in IntelliJ IDEA to perform network requests without blocking the +underlying thread or callbacks. + +> No prior knowledge of coroutines is required, but you're expected to be familiar with basic Kotlin syntax. +> +{style="tip"} + +You'll learn: + +* Why and how to use suspending functions to perform network requests. +* How to send requests concurrently using coroutines. +* How to share information between different coroutines using channels. + +For network requests, you'll need the [Retrofit](https://square.github.io/retrofit/) library, but the approach shown in +this tutorial works similarly for any other libraries that support coroutines. + +> You can find solutions for all of the tasks on the `solutions` branch of the [project's repository](http://github.com/kotlin-hands-on/intro-coroutines). +> +{style="tip"} + +## Before you start + +1. Download and install the latest version of [IntelliJ IDEA](https://www.jetbrains.com/idea/download/index.html). +2. Clone the [project template](http://github.com/kotlin-hands-on/intro-coroutines) by choosing **Get from VCS** on the + Welcome screen or selecting **File | New | Project from Version Control**. + + You can also clone it from the command line: + + ```Bash + git clone https://github.com/kotlin-hands-on/intro-coroutines + ``` + +### Generate a GitHub developer token + +You'll be using the GitHub API in your project. To get access, provide your GitHub account name and either a password or a +token. If you have two-factor authentication enabled, a token will be enough. + +Generate a new GitHub token to use the GitHub API with [your account](https://github.com/settings/tokens/new): + +1. Specify the name of your token, for example, `coroutines-tutorial`: + + ![Generate a new GitHub token](generating-token.png){width=700} + +2. Do not select any scopes. Click **Generate token** at the bottom of the page. +3. Copy the generated token. + +### Run the code + +The program loads the contributors for all of the repositories under the given organization (named “kotlin” by default). +Later you'll add logic to sort the users by the number of their contributions. + +1. Open the `src/contributors/main.kt` file and run the `main()` function. You'll see the following window: + + ![First window](initial-window.png){width=500} + + If the font is too small, adjust it by changing the value of `setDefaultFontSize(18f)` in the `main()` function. + +2. Provide your GitHub username and token (or password) in the corresponding fields. +3. Make sure that the _BLOCKING_ option is selected in the _Variant_ dropdown menu. +4. Click _Load contributors_. The UI should freeze for some time and then show the list of contributors. +5. Open the program output to ensure the data has been loaded. The list of contributors is logged after each successful request. + +There are different ways of implementing this logic: by using [blocking requests](#blocking-requests) +or [callbacks](#callbacks). You'll compare these solutions with one that uses [coroutines](#coroutines) and see how +[channels](#channels) can be used to share information between different coroutines. + +## Blocking requests + +You will use the [Retrofit](https://square.github.io/retrofit/) library to perform HTTP requests to GitHub. It allows +requesting the list of repositories under the given organization and the list of contributors for each repository: + +```kotlin +interface GitHubService { + @GET("orgs/{org}/repos?per_page=100") + fun getOrgReposCall( + @Path("org") org: String + ): Call> + + @GET("repos/{owner}/{repo}/contributors?per_page=100") + fun getRepoContributorsCall( + @Path("owner") owner: String, + @Path("repo") repo: String + ): Call> +} +``` + +This API is used by the `loadContributorsBlocking()` function to fetch the list of contributors for the given organization. + +1. Open `src/tasks/Request1Blocking.kt` to see its implementation: + + ```kotlin + fun loadContributorsBlocking( + service: GitHubService, + req: RequestData + ): List { + val repos = service + .getOrgReposCall(req.org) // #1 + .execute() // #2 + .also { logRepos(req, it) } // #3 + .body() ?: emptyList() // #4 + + return repos.flatMap { repo -> + service + .getRepoContributorsCall(req.org, repo.name) // #1 + .execute() // #2 + .also { logUsers(repo, it) } // #3 + .bodyList() // #4 + }.aggregate() + } + ``` + + * At first, you get a list of the repositories under the given organization and store it in the `repos` list. Then for + each repository, the list of contributors is requested, and all of the lists are merged into one final list of + contributors. + * `getOrgReposCall()` and `getRepoContributorsCall()` both return an instance of the `*Call` class (`#1`). At this point, + no request is sent. + * `*Call.execute()` is then invoked to perform the request (`#2`). `execute()` is a synchronous call that blocks the + underlying thread. + * When you get the response, the result is logged by calling the specific `logRepos()` and `logUsers()` functions (`#3`). + If the HTTP response contains an error, this error will be logged here. + * Finally, get the response's body, which contains the data you need. For this tutorial, you'll use an empty list as a + result in case there is an error, and you'll log the corresponding error (`#4`). + +2. To avoid repeating `.body() ?: emptyList()`, an extension function `bodyList()` is declared: + + ```kotlin + fun Response>.bodyList(): List { + return body() ?: emptyList() + } + ``` + +3. Run the program again and take a look at the system output in IntelliJ IDEA. It should have something like this: + + ```text + 1770 [AWT-EventQueue-0] INFO Contributors - kotlin: loaded 40 repos + 2025 [AWT-EventQueue-0] INFO Contributors - kotlin-examples: loaded 23 contributors + 2229 [AWT-EventQueue-0] INFO Contributors - kotlin-koans: loaded 45 contributors + ... + ``` + + * The first item on each line is the number of milliseconds that have passed since the program started, then the thread + name in square brackets. You can see from which thread the loading request is called. + * The final item on each line is the actual message: how many repositories or contributors were loaded. + + This log output demonstrates that all of the results were logged from the main thread. When you run the code with a _BLOCKING_ + option, the window freezes and doesn't react to input until the loading is finished. All of the requests are executed from + the same thread as the one called `loadContributorsBlocking()` is from, which is the main UI thread (in Swing, it's an AWT + event dispatching thread). This main thread becomes blocked, and that's why the UI is frozen: + + ![The blocked main thread](blocking.png){width=700} + + After the list of contributors has loaded, the result is updated. + +4. In `src/contributors/Contributors.kt`, find the `loadContributors()` function responsible for choosing how + the contributors are loaded and look at how `loadContributorsBlocking()` is called: + + ```kotlin + when (getSelectedVariant()) { + BLOCKING -> { // Blocking UI thread + val users = loadContributorsBlocking(service, req) + updateResults(users, startTime) + } + } + ``` + + * The `updateResults()` call goes right after the `loadContributorsBlocking()` call. + * `updateResults()` updates the UI, so it must always be called from the UI thread. + * Since `loadContributorsBlocking()` is also called from the UI thread, the UI thread becomes blocked and the UI is + frozen. + +### Task 1 + +The first task helps you familiarize yourself with the task domain. Currently, each contributor's name is repeated +several times, once for every project they have taken part in. Implement the `aggregate()` function combining the users +so that each contributor is added only once. The `User.contributions` property should contain the total number of +contributions of the given user to _all_ the projects. The resulting list should be sorted in descending order according +to the number of contributions. + +Open `src/tasks/Aggregation.kt` and implement the `List.aggregate()` function. Users should be sorted by the total +number of their contributions. + +The corresponding test file `test/tasks/AggregationKtTest.kt` shows an example of the expected result. + +> You can jump between the source code and the test class automatically by using the [IntelliJ IDEA shortcut](https://www.jetbrains.com/help/idea/create-tests.html#test-code-navigation) +> `Ctrl+Shift+T` / `⇧ ⌘ T`. +> +{style="tip"} + +After implementing this task, the resulting list for the "kotlin" organization should be similar to the following: + +![The list for the "kotlin" organization](aggregate.png){width=500} + +#### Solution for task 1 {initial-collapse-state="collapsed" collapsible="true"} + +1. To group users by login, use [`groupBy()`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/group-by.html), + which returns a map from a login to all occurrences of the user with this login in different repositories. +2. For each map entry, count the total number of contributions for each user and create a new instance of the `User` class + by the given name and total of contributions. +3. Sort the resulting list in descending order: + + ```kotlin + fun List.aggregate(): List = + groupBy { it.login } + .map { (login, group) -> User(login, group.sumOf { it.contributions }) } + .sortedByDescending { it.contributions } + ``` + +An alternative solution is to use the [`groupingBy()`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/grouping-by.html) +function instead of `groupBy()`. + +## Callbacks + +The previous solution works, but it blocks the thread and therefore freezes the UI. A traditional approach that avoids this +is to use _callbacks_. + +Instead of calling the code that should be invoked right after the operation is completed, you can extract it +into a separate callback, often a lambda, and pass that lambda to the caller in order for it to be called later. + +To make the UI responsive, you can either move the whole computation to a separate thread or switch to the Retrofit API +which uses callbacks instead of blocking calls. + +### Use a background thread + +1. Open `src/tasks/Request2Background.kt` and see its implementation. First, the whole computation is moved to a different + thread. The `thread()` function starts a new thread: + + ```kotlin + thread { + loadContributorsBlocking(service, req) + } + ``` + + Now that all of the loading has been moved to a separate thread, the main thread is free and can be occupied by other + tasks: + + ![The freed main thread](background.png){width=700} + +2. The signature of the `loadContributorsBackground()` function changes. It takes an `updateResults()` + callback as the last argument to call it after all the loading completes: + + ```kotlin + fun loadContributorsBackground( + service: GitHubService, req: RequestData, + updateResults: (List) -> Unit + ) + ``` + +3. Now when the `loadContributorsBackground()` is called, the `updateResults()` call goes in the callback, not immediately + afterward as it did before: + + ```kotlin + loadContributorsBackground(service, req) { users -> + SwingUtilities.invokeLater { + updateResults(users, startTime) + } + } + ``` + + By calling `SwingUtilities.invokeLater`, you ensure that the `updateResults()` call, which updates the results, + happens on the main UI thread (AWT event dispatching thread). + +However, if you try to load the contributors via the `BACKGROUND` option, you can see that the list is updated but +nothing changes. + +### Task 2 + +Fix the `loadContributorsBackground()` function in `src/tasks/Request2Background.kt` so that the resulting list is shown +in the UI. + +#### Solution for task 2 {initial-collapse-state="collapsed" collapsible="true"} + +If you try to load the contributors, you can see in the log that the contributors are loaded but the result isn't displayed. +To fix this, call `updateResults()` on the resulting list of users: + +```kotlin +thread { + updateResults(loadContributorsBlocking(service, req)) +} +``` + +Make sure to call the logic passed in the callback explicitly. Otherwise, nothing will happen. + +### Use the Retrofit callback API + +In the previous solution, the whole loading logic is moved to the background thread, but that still isn't the best use of +resources. All of the loading requests go sequentially and the thread is blocked while waiting for the loading result, +while it could have been occupied by other tasks. Specifically, the thread could start loading another request to +receive the entire result earlier. + +Handling the data for each repository should then be divided into two parts: loading and processing the +resulting response. The second _processing_ part should be extracted into a callback. + +The loading for each repository can then be started before the result for the previous repository is received (and the +corresponding callback is called): + +![Using callback API](callbacks.png){width=700} + +The Retrofit callback API can help achieve this. The `Call.enqueue()` function starts an HTTP request and takes a +callback as an argument. In this callback, you need to specify what needs to be done after each request. + +Open `src/tasks/Request3Callbacks.kt` and see the implementation of `loadContributorsCallbacks()` that uses this API: + +```kotlin +fun loadContributorsCallbacks( + service: GitHubService, req: RequestData, + updateResults: (List) -> Unit +) { + service.getOrgReposCall(req.org).onResponse { responseRepos -> // #1 + logRepos(req, responseRepos) + val repos = responseRepos.bodyList() + + val allUsers = mutableListOf() + for (repo in repos) { + service.getRepoContributorsCall(req.org, repo.name) + .onResponse { responseUsers -> // #2 + logUsers(repo, responseUsers) + val users = responseUsers.bodyList() + allUsers += users + } + } + } + // TODO: Why doesn't this code work? How to fix that? + updateResults(allUsers.aggregate()) + } +``` + +* For convenience, this code fragment uses the `onResponse()` extension function declared in the same file. It takes a + lambda as an argument rather than an object expression. +* The logic for handling the responses is extracted into callbacks: the corresponding lambdas start at lines `#1` and `#2`. + +However, the provided solution doesn't work. If you run the program and load contributors by choosing the _CALLBACKS_ +option, you'll see that nothing is shown. However, the test from `Request3CallbacksKtTest` immediately returns the result +that it successfully passed. + +Think about why the given code doesn't work as expected and try to fix it, or see the solutions below. + +### Task 3 (optional) + +Rewrite the code in the `src/tasks/Request3Callbacks.kt` file so that the loaded list of contributors is shown. + +#### The first attempted solution for task 3 {initial-collapse-state="collapsed" collapsible="true"} + +In the current solution, many requests are started concurrently, which decreases the total loading time. However, +the result isn't loaded. This is because the `updateResults()` callback is called right after all of the loading requests are started, +before the `allUsers` list has been filled with the data. + +You could try to fix this with a change like the following: + +```kotlin +val allUsers = mutableListOf() +for ((index, repo) in repos.withIndex()) { // #1 + service.getRepoContributorsCall(req.org, repo.name) + .onResponse { responseUsers -> + logUsers(repo, responseUsers) + val users = responseUsers.bodyList() + allUsers += users + if (index == repos.lastIndex) { // #2 + updateResults(allUsers.aggregate()) + } + } +} +``` + +* First, you iterate over the list of repos with an index (`#1`). +* Then, from each callback, you check whether it's the last iteration (`#2`). +* And if that's the case, the result is updated. + +However, this code also fails to achieve our objective. Try to find the answer yourself, or see the solution below. + +#### The second attempted solution for task 3 {initial-collapse-state="collapsed" collapsible="true"} + +Since the loading requests are started concurrently, there's no guarantee that the result for the last one comes last. The +results can come in any order. + +Thus, if you compare the current index with the `lastIndex` as a condition for completion, you risk losing the results for +some repos. + +If the request that processes the last repo returns faster than some prior requests (which is likely to happen), all of the +results for requests that take more time will be lost. + +One way to fix this is to introduce an index and check whether all of the repositories have already been processed: + +```kotlin +val allUsers = Collections.synchronizedList(mutableListOf()) +val numberOfProcessed = AtomicInteger() +for (repo in repos) { + service.getRepoContributorsCall(req.org, repo.name) + .onResponse { responseUsers -> + logUsers(repo, responseUsers) + val users = responseUsers.bodyList() + allUsers += users + if (numberOfProcessed.incrementAndGet() == repos.size) { + updateResults(allUsers.aggregate()) + } + } +} +``` + +This code uses a synchronized version of the list and `AtomicInteger()` because, in general, there's no guarantee that +different callbacks that process `getRepoContributors()` requests will always be called from the same thread. + +#### The third attempted solution for task 3 {initial-collapse-state="collapsed" collapsible="true"} + +An even better solution is to use the `CountDownLatch` class. It stores a counter initialized with the number of +repositories. This counter is decremented after processing each repository. It then waits until the latch is counted +down to zero before updating the results: + +```kotlin +val countDownLatch = CountDownLatch(repos.size) +for (repo in repos) { + service.getRepoContributorsCall(req.org, repo.name) + .onResponse { responseUsers -> + // processing repository + countDownLatch.countDown() + } +} +countDownLatch.await() +updateResults(allUsers.aggregate()) +``` + +The result is then updated from the main thread. This is more direct than delegating the logic to the child threads. + +After reviewing these three attempts at a solution, you can see that writing correct code with callbacks is non-trivial +and error-prone, especially when several underlying threads and synchronization occur. + +> As an additional exercise, you can implement the same logic using a reactive approach with the RxJava library. All of the +> necessary dependencies and solutions for using RxJava can be found in a separate `rx` branch. It is also possible to +> complete this tutorial and implement or check the proposed Rx versions for a proper comparison. +> +{style="tip"} + +## Suspending functions + +You can implement the same logic using suspending functions. Instead of returning `Call>`, define the API +call as a [suspending function](composing-suspending-functions.md) as follows: + +```kotlin +interface GitHubService { + @GET("orgs/{org}/repos?per_page=100") + suspend fun getOrgRepos( + @Path("org") org: String + ): List +} +``` + +* `getOrgRepos()` is defined as a `suspend` function. When you use a suspending function to perform a request, the + underlying thread isn't blocked. More details about how this works will come in later sections. +* `getOrgRepos()` returns the result directly instead of returning a `Call`. If the result is unsuccessful, an + exception is thrown. + +Alternatively, Retrofit allows returning the result wrapped in `Response`. In this case, the result body is +provided, and it is possible to check for errors manually. This tutorial uses the versions that return `Response`. + +In `src/contributors/GitHubService.kt`, add the following declarations to the `GitHubService` interface: + +```kotlin +interface GitHubService { + // getOrgReposCall & getRepoContributorsCall declarations + + @GET("orgs/{org}/repos?per_page=100") + suspend fun getOrgRepos( + @Path("org") org: String + ): Response> + + @GET("repos/{owner}/{repo}/contributors?per_page=100") + suspend fun getRepoContributors( + @Path("owner") owner: String, + @Path("repo") repo: String + ): Response> +} +``` + +### Task 4 + +Your task is to change the code of the function that loads contributors to make use of two new suspending functions, +`getOrgRepos()` and `getRepoContributors()`. The new `loadContributorsSuspend()` function is marked as `suspend` to use the +new API. + +> Suspending functions can't be called everywhere. Calling a suspending function from `loadContributorsBlocking()` will +> result in an error with the message "Suspend function 'getOrgRepos' should be called only from a coroutine or another +> suspend function". +> +{style="note"} + +1. Copy the implementation of `loadContributorsBlocking()` that is defined in `src/tasks/Request1Blocking.kt` + into the `loadContributorsSuspend()` that is defined in `src/tasks/Request4Suspend.kt`. +2. Modify the code so that the new suspending functions are used instead of the ones that return `Call`s. +3. Run the program by choosing the _SUSPEND_ option and ensure that the UI is still responsive while the GitHub requests + are performed. + +#### Solution for task 4 {initial-collapse-state="collapsed" collapsible="true"} + +Replace `.getOrgReposCall(req.org).execute()` with `.getOrgRepos(req.org)` and repeat the same replacement for the +second "contributors" request: + +```kotlin +suspend fun loadContributorsSuspend(service: GitHubService, req: RequestData): List { + val repos = service + .getOrgRepos(req.org) + .also { logRepos(req, it) } + .bodyList() + + return repos.flatMap { repo -> + service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + }.aggregate() +} +``` + +* `loadContributorsSuspend()` should be defined as a `suspend` function. +* You no longer need to call `execute`, which returned the `Response` before, because now the API functions return + the `Response` directly. Note that this detail is specific to the Retrofit library. With other libraries, the API will be different, + but the concept is the same. + +## Coroutines + +The code with suspending functions looks similar to the "blocking" version. The major difference from the blocking version +is that instead of blocking the thread, the coroutine is suspended: + +```text +block -> suspend +thread -> coroutine +``` + +> Coroutines are often called lightweight threads because you can run code on coroutines, similar to how you run code on +> threads. The operations that were blocking before (and had to be avoided) can now suspend the coroutine instead. +> +{style="note"} + +### Starting a new coroutine + +If you look at how `loadContributorsSuspend()` is used in `src/contributors/Contributors.kt`, you can see that it's +called inside `launch`. `launch` is a library function that takes a lambda as an argument: + +```kotlin +launch { + val users = loadContributorsSuspend(req) + updateResults(users, startTime) +} +``` + +Here `launch` starts a new computation that is responsible for loading the data and showing the results. The computation +is suspendable – when performing network requests, it is suspended and releases the underlying thread. +When the network request returns the result, the computation is resumed. + +Such a suspendable computation is called a _coroutine_. So, in this case, `launch` _starts a new coroutine_ responsible +for loading data and showing the results. + +Coroutines run on top of threads and can be suspended. When a coroutine is suspended, the +corresponding computation is paused, removed from the thread, and stored in memory. Meanwhile, the thread is free to be +occupied by other tasks: + +![Suspending coroutines](suspension-process.gif){width=700} + +When the computation is ready to be continued, it is returned to a thread (not necessarily the same one). + +In the `loadContributorsSuspend()` example, each "contributors" request now waits for the result using the suspension +mechanism. First, the new request is sent. Then, while waiting for the response, the whole "load contributors" coroutine +that was started by the `launch` function is suspended. + +The coroutine resumes only after the corresponding response is received: + +![Suspending request](suspend-requests.png){width=700} + +While the response is waiting to be received, the thread is free to be occupied by other tasks. The UI stays responsive, +despite all the requests taking place on the main UI thread: + +1. Run the program using the _SUSPEND_ option. The log confirms that all of the requests are sent to the main UI thread: + + ```text + 2538 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin: loaded 30 repos + 2729 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - ts2kt: loaded 11 contributors + 3029 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin-koans: loaded 45 contributors + ... + 11252 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin-coroutines-workshop: loaded 1 contributors + ``` + +2. The log can show you which coroutine the corresponding code is running on. To enable it, open **Run | Edit configurations** + and add the `-Dkotlinx.coroutines.debug` VM option: + + ![Edit run configuration](run-configuration.png){width=500} + + The coroutine name will be attached to the thread name while `main()` is run with this option. You can also + modify the template for running all of the Kotlin files and enable this option by default. + +Now all of the code runs on one coroutine, the "load contributors" coroutine mentioned above, denoted as `@coroutine#1`. +While waiting for the result, you shouldn't reuse the thread for sending other requests because the code is +written sequentially. The new request is sent only when the previous result is received. + +Suspending functions treat the thread fairly and don't block it for "waiting". However, this doesn't yet bring any concurrency +into the picture. + +## Concurrency + +Kotlin coroutines are much less resource-intensive than threads. +Each time you want to start a new computation asynchronously, you can create a new coroutine instead. + +To start a new coroutine, use one of the main _coroutine builders_: `launch`, `async`, or `runBlocking`. Different +libraries can define additional coroutine builders. + +`async` starts a new coroutine and returns a `Deferred` object. `Deferred` represents a concept known by other names +such as `Future` or `Promise`. It stores a computation, but it _defers_ the moment you get the final result; +it _promises_ the result sometime in the _future_. + +The main difference between `async` and `launch` is that `launch` is used to start a computation that isn't expected to +return a specific result. `launch` returns a `Job` that represents the coroutine. It is possible to wait until it completes +by calling `Job.join()`. + +`Deferred` is a generic type that extends `Job`. An `async` call can return a `Deferred` or a `Deferred`, +depending on what the lambda returns (the last expression inside the lambda is the result). + +To get the result of a coroutine, you can call `await()` on the `Deferred` instance. While waiting for the result, +the coroutine that this `await()` is called from is suspended: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { + val deferred: Deferred = async { + loadData() + } + println("waiting...") + println(deferred.await()) +} + +suspend fun loadData(): Int { + println("loading...") + delay(1000L) + println("loaded!") + return 42 +} +``` + +`runBlocking` is used as a bridge between regular and suspending functions, or between the blocking and non-blocking worlds. It works +as an adaptor for starting the top-level main coroutine. It is intended primarily to be used in `main()` functions and +tests. + +> Watch [this video](https://www.youtube.com/watch?v=zEZc5AmHQhk) for a better understanding of coroutines. +> +{style="tip"} + +If there is a list of deferred objects, you can call `awaitAll()` to await the results of all of them: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { + val deferreds: List> = (1..3).map { + async { + delay(1000L * it) + println("Loading $it") + it + } + } + val sum = deferreds.awaitAll().sum() + println("$sum") +} +``` + +When each "contributors" request is started in a new coroutine, all of the requests are started asynchronously. A new request +can be sent before the result for the previous one is received: + +![Concurrent coroutines](concurrency.png){width=700} + +The total loading time is approximately the same as in the _CALLBACKS_ version, but it doesn't need any callbacks. +What's more, `async` explicitly emphasizes which parts run concurrently in the code. + +### Task 5 + +In the `Request5Concurrent.kt` file, implement a `loadContributorsConcurrent()` function by using the +previous `loadContributorsSuspend()` function. + +#### Tip for task 5 {initial-collapse-state="collapsed" collapsible="true"} + +You can only start a new coroutine inside a coroutine scope. Copy the content +from `loadContributorsSuspend()` to the `coroutineScope` call so that you can call `async` functions there: + +```kotlin +suspend fun loadContributorsConcurrent( + service: GitHubService, + req: RequestData +): List = coroutineScope { + // ... +} +``` + +Base your solution on the following scheme: + +```kotlin +val deferreds: List>> = repos.map { repo -> + async { + // load contributors for each repo + } +} +deferreds.awaitAll() // List> +``` + +#### Solution for task 5 {initial-collapse-state="collapsed" collapsible="true"} + +Wrap each "contributors" request with `async` to create as many coroutines as there are repositories. `async` +returns `Deferred>`. This is not an issue because creating new coroutines is not very resource-intensive, so you can +create as many as you need. + +1. You can no longer use `flatMap` because the `map` result is now a list of `Deferred` objects, not a list of lists. + `awaitAll()` returns `List>`, so call `flatten().aggregate()` to get the result: + + ```kotlin + suspend fun loadContributorsConcurrent( + service: GitHubService, + req: RequestData + ): List = coroutineScope { + val repos = service + .getOrgRepos(req.org) + .also { logRepos(req, it) } + .bodyList() + + val deferreds: List>> = repos.map { repo -> + async { + service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + } + } + deferreds.awaitAll().flatten().aggregate() + } + ``` + +2. Run the code and check the log. All of the coroutines still run on the main UI thread because + multithreading hasn't been employed yet, but you can already see the benefits of running coroutines concurrently. +3. To change this code to run "contributors" coroutines on different threads from the common thread pool, + specify `Dispatchers.Default` as the context argument for the `async` function: + + ```kotlin + async(Dispatchers.Default) { } + ``` + + * `CoroutineDispatcher` determines what thread or threads the corresponding coroutine should be run on. If you don't + specify one as an argument, `async` will use the dispatcher from the outer scope. + * `Dispatchers.Default` represents a shared pool of threads on the JVM. This pool provides a means for parallel execution. + It consists of as many threads as there are CPU cores available, but it will still have two threads if there's only one core. + +4. Modify the code in the `loadContributorsConcurrent()` function to start new coroutines on different threads from the + common thread pool. Also, add additional logging before sending the request: + + ```kotlin + async(Dispatchers.Default) { + log("starting loading for ${repo.name}") + service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + } + ``` + +5. Run the program once again. In the log, you can see that each coroutine can be started on one thread from the + thread pool and resumed on another: + + ```text + 1946 [DefaultDispatcher-worker-2 @coroutine#4] INFO Contributors - starting loading for kotlin-koans + 1946 [DefaultDispatcher-worker-3 @coroutine#5] INFO Contributors - starting loading for dokka + 1946 [DefaultDispatcher-worker-1 @coroutine#3] INFO Contributors - starting loading for ts2kt + ... + 2178 [DefaultDispatcher-worker-1 @coroutine#4] INFO Contributors - kotlin-koans: loaded 45 contributors + 2569 [DefaultDispatcher-worker-1 @coroutine#5] INFO Contributors - dokka: loaded 36 contributors + 2821 [DefaultDispatcher-worker-2 @coroutine#3] INFO Contributors - ts2kt: loaded 11 contributors + ``` + + For instance, in this log excerpt, `coroutine#4` is started on the `worker-2` thread and continued on the + `worker-1` thread. + +In `src/contributors/Contributors.kt`, check the implementation of the _CONCURRENT_ option: + +1. To run the coroutine only on the main UI thread, specify `Dispatchers.Main` as an argument: + + ```kotlin + launch(Dispatchers.Main) { + updateResults() + } + ``` + + * If the main thread is busy when you start a new coroutine on it, + the coroutine becomes suspended and scheduled for execution on this thread. The coroutine will only resume when the + thread becomes free. + * It's considered good practice to use the dispatcher from the outer scope rather than explicitly specifying it on each + end-point. If you define `loadContributorsConcurrent()` without passing `Dispatchers.Default` as an + argument, you can call this function in any context: with a `Default` dispatcher, with + the main UI thread, or with a custom dispatcher. + * As you'll see later, when calling `loadContributorsConcurrent()` from tests, you can call it in the context + with `TestDispatcher`, which simplifies testing. That makes this solution much more flexible. + +2. To specify the dispatcher on the caller side, apply the following change to the project while + letting `loadContributorsConcurrent` start coroutines in the inherited context: + + ```kotlin + launch(Dispatchers.Default) { + val users = loadContributorsConcurrent(service, req) + withContext(Dispatchers.Main) { + updateResults(users, startTime) + } + } + ``` + + * `updateResults()` should be called on the main UI thread, so you call it with the context of `Dispatchers.Main`. + * `withContext()` calls the given code with the specified coroutine context, is suspended until it completes, and returns + the result. An alternative but more verbose way to express this would be to start a new coroutine and explicitly + wait (by suspending) until it completes: `launch(context) { ... }.join()`. + +3. Run the code and ensure that the coroutines are executed on the threads from the thread pool. + +## Structured concurrency + +* The _coroutine scope_ is responsible for the structure and parent-child relationships between different coroutines. New + coroutines usually need to be started inside a scope. +* The _coroutine context_ stores additional technical information used to run a given coroutine, like the coroutine custom + name, or the dispatcher specifying the threads the coroutine should be scheduled on. + +When `launch`, `async`, or `runBlocking` are used to start a new coroutine, they automatically create the corresponding +scope. All of these functions take a lambda with a receiver as an argument, and `CoroutineScope` is the implicit receiver type: + +```kotlin +launch { /* this: CoroutineScope */ } +``` + +* New coroutines can only be started inside a scope. +* `launch` and `async` are declared as extensions to `CoroutineScope`, so an implicit or explicit receiver must always + be passed when you call them. +* The coroutine started by `runBlocking` is the only exception because `runBlocking` is defined as a top-level function. + But because it blocks the current thread, it's intended primarily to be used in `main()` functions and tests as a bridge + function. + +A new coroutine inside `runBlocking`, `launch`, or `async` is started automatically inside the scope: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { /* this: CoroutineScope */ + launch { /* ... */ } + // the same as: + this.launch { /* ... */ } +} +``` + +When you call `launch` inside `runBlocking`, it's called as an extension to the implicit receiver of +the `CoroutineScope` type. Alternatively, you could explicitly write `this.launch`. + +The nested coroutine (started by `launch` in this example) can be considered as a child of the outer coroutine (started +by `runBlocking`). This "parent-child" relationship works through scopes; the child coroutine is started from the scope +corresponding to the parent coroutine. + +It's possible to create a new scope without starting a new coroutine, by using the `coroutineScope` function. +To start new coroutines in a structured way inside a `suspend` function without access to the outer scope, you can create +a new coroutine scope that automatically becomes a child of the outer scope that this `suspend` function is called from. +`loadContributorsConcurrent()`is a good example. + +You can also start a new coroutine from the global scope using `GlobalScope.async` or `GlobalScope.launch`. +This will create a top-level "independent" coroutine. + +The mechanism behind the structure of the coroutines is called _structured concurrency_. It provides the following +benefits over global scopes: + +* The scope is generally responsible for child coroutines, whose lifetime is attached to the lifetime of the scope. +* The scope can automatically cancel child coroutines if something goes wrong or a user changes their mind and decides + to revoke the operation. +* The scope automatically waits for the completion of all child coroutines. + Therefore, if the scope corresponds to a coroutine, the parent coroutine does not complete until all the coroutines + launched in its scope have completed. + +When using `GlobalScope.async`, there is no structure that binds several coroutines to a smaller scope. +Coroutines started from the global scope are all independent – their lifetime is limited only by the lifetime of the +whole application. It's possible to store a reference to the coroutine started from the global scope and wait for its +completion or cancel it explicitly, but that won't happen automatically as it would with structured concurrency. + +### Canceling the loading of contributors + +Create two versions of the function that loads the list of contributors. Compare how both versions behave when you try to +cancel the parent coroutine. The first version will use `coroutineScope` to start all of the child coroutines, +whereas the second will use `GlobalScope`. + +1. In `Request5Concurrent.kt`, add a 3-second delay to the `loadContributorsConcurrent()` function: + + ```kotlin + suspend fun loadContributorsConcurrent( + service: GitHubService, + req: RequestData + ): List = coroutineScope { + // ... + async { + log("starting loading for ${repo.name}") + delay(3000) + // load repo contributors + } + // ... + } + ``` + + The delay affects all of the coroutines that send requests, so that there's enough time to cancel the loading + after the coroutines are started but before the requests are sent. + +2. Create the second version of the loading function: copy the implementation of `loadContributorsConcurrent()` to + `loadContributorsNotCancellable()` in `Request5NotCancellable.kt` and then remove the creation of a new `coroutineScope`. +3. The `async` calls now fail to resolve, so start them by using `GlobalScope.async`: + + ```kotlin + suspend fun loadContributorsNotCancellable( + service: GitHubService, + req: RequestData + ): List { // #1 + // ... + GlobalScope.async { // #2 + log("starting loading for ${repo.name}") + // load repo contributors + } + // ... + return deferreds.awaitAll().flatten().aggregate() // #3 + } + ``` + + * The function now returns the result directly, not as the last expression inside the lambda (lines `#1` and `#3`). + * All of the "contributors" coroutines are started inside the `GlobalScope`, not as children of the coroutine scope + (line `#2`). + +4. Run the program and choose the _CONCURRENT_ option to load the contributors. +5. Wait until all of the "contributors" coroutines are started, and then click _Cancel_. The log shows no new results, + which means that all of the requests were indeed canceled: + + ```text + 2896 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin: loaded 40 repos + 2901 [DefaultDispatcher-worker-2 @coroutine#4] INFO Contributors - starting loading for kotlin-koans + ... + 2909 [DefaultDispatcher-worker-5 @coroutine#36] INFO Contributors - starting loading for mpp-example + /* click on 'cancel' */ + /* no requests are sent */ + ``` + +6. Repeat step 5, but this time choose the `NOT_CANCELLABLE` option: + + ```text + 2570 [AWT-EventQueue-0 @coroutine#1] INFO Contributors - kotlin: loaded 30 repos + 2579 [DefaultDispatcher-worker-1 @coroutine#4] INFO Contributors - starting loading for kotlin-koans + ... + 2586 [DefaultDispatcher-worker-6 @coroutine#36] INFO Contributors - starting loading for mpp-example + /* click on 'cancel' */ + /* but all the requests are still sent: */ + 6402 [DefaultDispatcher-worker-5 @coroutine#4] INFO Contributors - kotlin-koans: loaded 45 contributors + ... + 9555 [DefaultDispatcher-worker-8 @coroutine#36] INFO Contributors - mpp-example: loaded 8 contributors + ``` + + In this case, no coroutines are canceled, and all the requests are still sent. + +7. Check how the cancellation is triggered in the "contributors" program. When the _Cancel_ button is clicked, + the main "loading" coroutine is explicitly canceled and the child coroutines are canceled automatically: + + ```kotlin + interface Contributors { + + fun loadContributors() { + // ... + when (getSelectedVariant()) { + CONCURRENT -> { + launch { + val users = loadContributorsConcurrent(service, req) + updateResults(users, startTime) + }.setUpCancellation() // #1 + } + } + } + + private fun Job.setUpCancellation() { + val loadingJob = this // #2 + + // cancel the loading job if the 'cancel' button was clicked: + val listener = ActionListener { + loadingJob.cancel() // #3 + updateLoadingStatus(CANCELED) + } + // add a listener to the 'cancel' button: + addCancelListener(listener) + + // update the status and remove the listener + // after the loading job is completed + } + } + ``` + +The `launch` function returns an instance of `Job`. `Job` stores a reference to the "loading coroutine", which loads +all of the data and updates the results. You can call the `setUpCancellation()` extension function on it (line `#1`), +passing an instance of `Job` as a receiver. + +Another way you could express this would be to explicitly write: + +```kotlin +val job = launch { } +job.setUpCancellation() +``` + +* For readability, you could refer to the `setUpCancellation()` function receiver inside the function with the + new `loadingJob` variable (line `#2`). +* Then you could add a listener to the _Cancel_ button so that when it's clicked, the `loadingJob` is canceled (line `#3`). + +With structured concurrency, you only need to cancel the parent coroutine and this automatically propagates cancellation +to all of the child coroutines. + +### Using the outer scope's context + +When you start new coroutines inside the given scope, it's much easier to ensure that all of them run with the same +context. It is also much easier to replace the context if needed. + +Now it's time to learn how using the dispatcher from the outer scope works. The new scope created by +the `coroutineScope` or by the coroutine builders always inherits the context from the outer scope. In this case, the +outer scope is the scope the `suspend loadContributorsConcurrent()` function was called from: + +```kotlin +launch(Dispatchers.Default) { // outer scope + val users = loadContributorsConcurrent(service, req) + // ... +} +``` + +All of the nested coroutines are automatically started with the inherited context. The dispatcher is a part of this +context. That's why all of the coroutines started by `async` are started with the context of the default dispatcher: + +```kotlin +suspend fun loadContributorsConcurrent( + service: GitHubService, req: RequestData +): List = coroutineScope { + // this scope inherits the context from the outer scope + // ... + async { // nested coroutine started with the inherited context + // ... + } + // ... +} +``` + +With structured concurrency, you can specify the major context elements (like dispatcher) once, when creating the +top-level coroutine. All the nested coroutines then inherit the context and modify it only if needed. + +> When you write code with coroutines for UI applications, for example Android ones, it's a common practice to +> use `CoroutineDispatchers.Main` by default for the top coroutine and then to explicitly put a different dispatcher when +> you need to run the code on a different thread. +> +{style="tip"} + +## Showing progress + +Despite the information for some repositories being loaded rather quickly, the user only sees the resulting list after all of +the data has been loaded. Until then, the loader icon runs showing the progress, but there's no information about the current +state or what contributors are already loaded. + +You can show the intermediate results earlier and display all of the contributors after loading the data for each of the +repositories: + +![Loading data](loading.gif){width=500} + +To implement this functionality, in the `src/tasks/Request6Progress.kt`, you'll need to pass the logic updating the UI +as a callback, so that it's called on each intermediate state: + +```kotlin +suspend fun loadContributorsProgress( + service: GitHubService, + req: RequestData, + updateResults: suspend (List, completed: Boolean) -> Unit +) { + // loading the data + // calling `updateResults()` on intermediate states +} +``` + +On the call site in `Contributors.kt`, the callback is passed to update the results from the `Main` thread for +the _PROGRESS_ option: + +```kotlin +launch(Dispatchers.Default) { + loadContributorsProgress(service, req) { users, completed -> + withContext(Dispatchers.Main) { + updateResults(users, startTime, completed) + } + } +} +``` + +* The `updateResults()` parameter is declared as `suspend` in `loadContributorsProgress()`. It's necessary to call + `withContext`, which is a `suspend` function inside the corresponding lambda argument. +* `updateResults()` callback takes an additional Boolean parameter as an argument specifying whether the loading has + completed and the results are final. + +### Task 6 + +In the `Request6Progress.kt` file, implement the `loadContributorsProgress()` function that shows the intermediate +progress. Base it on the `loadContributorsSuspend()` function from `Request4Suspend.kt`. + +* Use a simple version without concurrency; you'll add it later in the next section. +* The intermediate list of contributors should be shown in an "aggregated" state, not just the list of users loaded for + each repository. +* The total number of contributions for each user should be increased when the data for each new + repository is loaded. + +#### Solution for task 6 {initial-collapse-state="collapsed" collapsible="true"} + +To store the intermediate list of loaded contributors in the "aggregated" state, define an `allUsers` variable which +stores the list of users, and then update it after contributors for each new repository are loaded: + +```kotlin +suspend fun loadContributorsProgress( + service: GitHubService, + req: RequestData, + updateResults: suspend (List, completed: Boolean) -> Unit +) { + val repos = service + .getOrgRepos(req.org) + .also { logRepos(req, it) } + .bodyList() + + var allUsers = emptyList() + for ((index, repo) in repos.withIndex()) { + val users = service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + + allUsers = (allUsers + users).aggregate() + updateResults(allUsers, index == repos.lastIndex) + } +} +``` + +#### Consecutive vs concurrent + +An `updateResults()` callback is called after each request is completed: + +![Progress on requests](progress.png){width=700} + +This code doesn't include concurrency. It's sequential, so you don't need synchronization. + +The best option would be to send requests concurrently and update the intermediate results after getting the response +for each repository: + +![Concurrent requests](progress-and-concurrency.png){width=700} + +To add concurrency, use _channels_. + +## Channels + +Writing code with a shared mutable state is quite difficult and error-prone (like in the solution using callbacks). +A simpler way is to share information by communication rather than by using a common mutable state. +Coroutines can communicate with each other through _channels_. + +Channels are communication primitives that allow data to be passed between coroutines. One coroutine can _send_ +some information to a channel, while another can _receive_ that information from it: + +![Using channels](using-channel.png) + +A coroutine that sends (produces) information is often called a producer, and a coroutine that receives (consumes) +information is called a consumer. One or multiple coroutines can send information to the same channel, and one or multiple +coroutines can receive data from it: + +![Using channels with many coroutines](using-channel-many-coroutines.png) + +When many coroutines receive information from the same channel, each element is handled only once by one of the +consumers. Once an element is handled, it is immediately removed from the channel. + +You can think of a channel as similar to a collection of elements, or more precisely, a queue, in which elements are added +to one end and received from the other. However, there's an important difference: unlike collections, even in their +synchronized versions, a channel can _suspend_ `send()`and `receive()` operations. This happens when the channel is empty +or full. The channel can be full if the channel size has an upper bound. + +`Channel` is represented by three different interfaces: `SendChannel`, `ReceiveChannel`, and `Channel`, with the latter +extending the first two. You usually create a channel and give it to producers as a `SendChannel` instance so that only +they can send information to the channel. +You give a channel to consumers as a `ReceiveChannel` instance so that only they can receive from it. Both `send` +and `receive` methods are declared as `suspend`: + +```kotlin +interface SendChannel { + suspend fun send(element: E) + fun close(): Boolean +} + +interface ReceiveChannel { + suspend fun receive(): E +} + +interface Channel : SendChannel, ReceiveChannel +``` + +The producer can close a channel to indicate that no more elements are coming. + +Several types of channels are defined in the library. They differ in how many elements they can internally store and +whether the `send()` call can be suspended or not. +For all of the channel types, the `receive()` call behaves similarly: it receives an element if the channel is not empty; +otherwise, it is suspended. + + + +

An unlimited channel is the closest analog to a queue: producers can send elements to this channel and it will +keep growing indefinitely. The send() call will never be suspended. +If the program runs out of memory, you'll get an OutOfMemoryException. +The difference between an unlimited channel and a queue is that when a consumer tries to receive from an empty channel, +it becomes suspended until some new elements are sent.

+ Unlimited channel + + +

The size of a buffered channel is constrained by the specified number. +Producers can send elements to this channel until the size limit is reached. All of the elements are internally stored. +When the channel is full, the next `send` call on it is suspended until more free space becomes available.

+ Buffered channel + + +

The "Rendezvous" channel is a channel without a buffer, the same as a buffered channel with zero size. +One of the functions (send() or receive()) is always suspended until the other is called.

+

If the send() function is called and there's no suspended receive() call ready to process the element, then send() +is suspended. Similarly, if the receive() function is called and the channel is empty or, in other words, there's no +suspended send() call ready to send the element, the receive() call is suspended.

+

The "rendezvous" name ("a meeting at an agreed time and place") refers to the fact that send() and receive() +should "meet on time".

+ Rendezvous channel + + +

A new element sent to the conflated channel will overwrite the previously sent element, so the receiver will always +get only the latest element. The send() call is never suspended.

+ Conflated channel + + + +When you create a channel, specify its type or the buffer size (if you need a buffered one): + +```kotlin +val rendezvousChannel = Channel() +val bufferedChannel = Channel(10) +val conflatedChannel = Channel(CONFLATED) +val unlimitedChannel = Channel(UNLIMITED) +``` + +By default, a "Rendezvous" channel is created. + +In the following task, you'll create a "Rendezvous" channel, two producer coroutines, and a consumer coroutine: + +```kotlin +import kotlinx.coroutines.channels.Channel +import kotlinx.coroutines.* + +fun main() = runBlocking { + val channel = Channel() + launch { + channel.send("A1") + channel.send("A2") + log("A done") + } + launch { + channel.send("B1") + log("B done") + } + launch { + repeat(3) { + val x = channel.receive() + log(x) + } + } +} + +fun log(message: Any?) { + println("[${Thread.currentThread().name}] $message") +} +``` + +> Watch [this video](https://www.youtube.com/watch?v=HpWQUoVURWQ) for a better understanding of channels. +> +{style="tip"} + +### Task 7 + +In `src/tasks/Request7Channels.kt`, implement the function `loadContributorsChannels()` that requests all of the GitHub +contributors concurrently and shows intermediate progress at the same time. + +Use the previous functions, `loadContributorsConcurrent()` from `Request5Concurrent.kt` +and `loadContributorsProgress()` from `Request6Progress.kt`. + +#### Tip for task 7 {initial-collapse-state="collapsed" collapsible="true"} + +Different coroutines that concurrently receive contributor lists for different repositories can send all of the received +results to the same channel: + +```kotlin +val channel = Channel>() +for (repo in repos) { + launch { + val users = TODO() + // ... + channel.send(users) + } +} +``` + +Then the elements from this channel can be received one by one and processed: + +```kotlin +repeat(repos.size) { + val users = channel.receive() + // ... +} +``` + +Since the `receive()` calls are sequential, no additional synchronization is needed. + +#### Solution for task 7 {initial-collapse-state="collapsed" collapsible="true"} + +As with the `loadContributorsProgress()` function, you can create an `allUsers` variable to store the intermediate +states of the "all contributors" list. +Each new list received from the channel is added to the list of all users. You aggregate the result and update the state +using the `updateResults` callback: + +```kotlin +suspend fun loadContributorsChannels( + service: GitHubService, + req: RequestData, + updateResults: suspend (List, completed: Boolean) -> Unit +) = coroutineScope { + + val repos = service + .getOrgRepos(req.org) + .also { logRepos(req, it) } + .bodyList() + + val channel = Channel>() + for (repo in repos) { + launch { + val users = service.getRepoContributors(req.org, repo.name) + .also { logUsers(repo, it) } + .bodyList() + channel.send(users) + } + } + var allUsers = emptyList() + repeat(repos.size) { + val users = channel.receive() + allUsers = (allUsers + users).aggregate() + updateResults(allUsers, it == repos.lastIndex) + } +} +``` + +* Results for different repositories are added to the channel as soon as they are ready. At first, when all of the requests + are sent, and no data is received, the `receive()` call is suspended. In this case, the whole "load contributors" coroutine + is suspended. +* Then, when the list of users is sent to the channel, the "load contributors" coroutine resumes, the `receive()` call + returns this list, and the results are immediately updated. + +You can now run the program and choose the _CHANNELS_ option to load the contributors and see the result. + +Although neither coroutines nor channels completely remove the complexity that comes with concurrency, +they make life easier when you need to understand what's going on. + +## Testing coroutines + +Let's now test all solutions to check that the solution with concurrent coroutines is faster than the solution with +the `suspend` functions, and check that the solution with channels is faster than the simple "progress" one. + +In the following task, you'll compare the total running time of the solutions. You'll mock a GitHub service and make +this service return results after the given timeouts: + +```text +repos request - returns an answer within 1000 ms delay +repo-1 - 1000 ms delay +repo-2 - 1200 ms delay +repo-3 - 800 ms delay +``` + +The sequential solution with the `suspend` functions should take around 4000 ms (4000 = 1000 + (1000 + 1200 + 800)). +The concurrent solution should take around 2200 ms (2200 = 1000 + max(1000, 1200, 800)). + +For the solutions that show progress, you can also check the intermediate results with timestamps. + +The corresponding test data is defined in `test/contributors/testData.kt`, and the files `Request4SuspendKtTest`, +`Request7ChannelsKtTest`, and so on contain the straightforward tests that use mock service calls. + +However, there are two problems here: + +* These tests take too long to run. Each test takes around 2 to 4 seconds, and you need to wait for the results each + time. It's not very efficient. +* You can't rely on the exact time the solution runs because it still takes additional time to prepare and run the code. + You could add a constant, but then the time would differ from machine to machine. The mock service delays + should be higher than this constant so you can see a difference. If the constant is 0.5 sec, making the delays + 0.1 sec won't be enough. + +A better way would be to use special frameworks to test the timing while running the same code several times (which increases +the total time even more), but that is complicated to learn and set up. + +To solve these problems and make sure that solutions with provided test delays behave as expected, one faster than the other, +use _virtual_ time with a special test dispatcher. This dispatcher keeps track of the virtual time passed from +the start and runs everything immediately in real time. When you run coroutines on this dispatcher, +the `delay` will return immediately and advance the virtual time. + +Tests that use this mechanism run fast, but you can still check what happens at different moments in virtual time. The +total running time drastically decreases: + +![Comparison for total running time](time-comparison.png){width=700} + +To use virtual time, replace the `runBlocking` invocation with a `runTest`. `runTest` takes an +extension lambda to `TestScope` as an argument. +When you call `delay` in a `suspend` function inside this special scope, `delay` will increase the virtual time instead +of delaying in real time: + +```kotlin +@Test +fun testDelayInSuspend() = runTest { + val realStartTime = System.currentTimeMillis() + val virtualStartTime = currentTime + + foo() + println("${System.currentTimeMillis() - realStartTime} ms") // ~ 6 ms + println("${currentTime - virtualStartTime} ms") // 1000 ms +} + +suspend fun foo() { + delay(1000) // auto-advances without delay + println("foo") // executes eagerly when foo() is called +} +``` + +You can check the current virtual time using the `currentTime` property of `TestScope`. + +The actual running time in this example is several milliseconds, whereas virtual time equals the delay argument, which +is 1000 milliseconds. + +To get the full effect of "virtual" `delay` in child coroutines, +start all of the child coroutines with `TestDispatcher`. Otherwise, it won't work. This dispatcher is +automatically inherited from the other `TestScope`, unless you provide a different dispatcher: + +```kotlin +@Test +fun testDelayInLaunch() = runTest { + val realStartTime = System.currentTimeMillis() + val virtualStartTime = currentTime + + bar() + + println("${System.currentTimeMillis() - realStartTime} ms") // ~ 11 ms + println("${currentTime - virtualStartTime} ms") // 1000 ms +} + +suspend fun bar() = coroutineScope { + launch { + delay(1000) // auto-advances without delay + println("bar") // executes eagerly when bar() is called + } +} +``` + +If `launch` is called with the context of `Dispatchers.Default` in the example above, the test will fail. You'll get an +exception saying that the job has not been completed yet. + +You can test the `loadContributorsConcurrent()` function this way only if it starts the child coroutines with the +inherited context, without modifying it using the `Dispatchers.Default` dispatcher. + +You can specify the context elements like the dispatcher when _calling_ a function rather than when _defining_ it, +which allows for more flexibility and easier testing. + +> The testing API that supports virtual time is [Experimental](components-stability.md) and may change in the future. +> +{style="warning"} + +By default, the compiler shows warnings if you use the experimental testing API. To suppress these warnings, annotate +the test function or the whole class containing the tests with `@OptIn(ExperimentalCoroutinesApi::class)`. +Add the compiler argument instructing the compiler that you're using the experimental API: + +```kotlin +compileTestKotlin { + kotlinOptions { + freeCompilerArgs += "-Xuse-experimental=kotlin.Experimental" + } +} +``` + +In the project corresponding to this tutorial, the compiler argument has already been added to the Gradle script. + +### Task 8 + +Refactor the following tests in `tests/tasks/` to use virtual time instead of real time: + +* Request4SuspendKtTest.kt +* Request5ConcurrentKtTest.kt +* Request6ProgressKtTest.kt +* Request7ChannelsKtTest.kt + +Compare the total running times before and after applying your refactoring. + +#### Tip for task 8 {initial-collapse-state="collapsed" collapsible="true"} + +1. Replace the `runBlocking` invocation with `runTest`, and replace `System.currentTimeMillis()` with `currentTime`: + + ```kotlin + @Test + fun test() = runTest { + val startTime = currentTime + // action + val totalTime = currentTime - startTime + // testing result + } + ``` + +2. Uncomment the assertions that check the exact virtual time. +3. Don't forget to add `@UseExperimental(ExperimentalCoroutinesApi::class)`. + +#### Solution for task 8 {initial-collapse-state="collapsed" collapsible="true"} + +Here are the solutions for the concurrent and channels cases: + +```kotlin +fun testConcurrent() = runTest { + val startTime = currentTime + val result = loadContributorsConcurrent(MockGithubService, testRequestData) + Assert.assertEquals("Wrong result for 'loadContributorsConcurrent'", expectedConcurrentResults.users, result) + val totalTime = currentTime - startTime + + Assert.assertEquals( + "The calls run concurrently, so the total virtual time should be 2200 ms: " + + "1000 for repos request plus max(1000, 1200, 800) = 1200 for concurrent contributors requests)", + expectedConcurrentResults.timeFromStart, totalTime + ) +} +``` + +First, check that the results are available exactly at the expected virtual time, and then check the results +themselves: + +```kotlin +fun testChannels() = runTest { + val startTime = currentTime + var index = 0 + loadContributorsChannels(MockGithubService, testRequestData) { users, _ -> + val expected = concurrentProgressResults[index++] + val time = currentTime - startTime + Assert.assertEquals( + "Expected intermediate results after ${expected.timeFromStart} ms:", + expected.timeFromStart, time + ) + Assert.assertEquals("Wrong intermediate results after $time:", expected.users, users) + } +} +``` + +The first intermediate result for the last version with channels becomes available sooner than the progress version, and you +can see the difference in tests that use virtual time. + +> The tests for the remaining "suspend" and "progress" tasks are very similar – you can find them in the project's +> `solutions` branch. +> +{style="tip"} + +## What's next + +* Check out the [Asynchronous Programming with Kotlin](https://kotlinconf.com/workshops/) workshop at KotlinConf. +* Find out more about using [virtual time and the experimental testing package](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/). diff --git a/docs/topics/coroutines-basics.md b/docs/topics/coroutines-basics.md new file mode 100644 index 0000000000..5b467c58bf --- /dev/null +++ b/docs/topics/coroutines-basics.md @@ -0,0 +1,293 @@ + +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Coroutines basics) + +This section covers basic coroutine concepts. + +## Your first coroutine + +A _coroutine_ is an instance of a suspendable computation. It is conceptually similar to a thread, in the sense that it +takes a block of code to run that works concurrently with the rest of the code. +However, a coroutine is not bound to any particular thread. It may suspend its execution in one thread and resume in another one. + +Coroutines can be thought of as light-weight threads, but there is a number +of important differences that make their real-life usage very different from threads. + +Run the following code to get to your first working coroutine: + +```kotlin +import kotlinx.coroutines.* + +//sampleStart +fun main() = runBlocking { // this: CoroutineScope + launch { // launch a new coroutine and continue + delay(1000L) // non-blocking delay for 1 second (default time unit is ms) + println("World!") // print after delay + } + println("Hello") // main coroutine continues while a previous one is delayed +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-basic-01.kt). +> +{style="note"} + +You will see the following result: + +```text +Hello +World! +``` + + + +Let's dissect what this code does. + +[launch] is a _coroutine builder_. It launches a new coroutine concurrently with +the rest of the code, which continues to work independently. That's why `Hello` has been printed first. + +[delay] is a special _suspending function_. It _suspends_ the coroutine for a specific time. Suspending a coroutine +does not _block_ the underlying thread, but allows other coroutines to run and use the underlying thread for +their code. + +[runBlocking] is also a coroutine builder that bridges the non-coroutine world of a regular `fun main()` and +the code with coroutines inside of `runBlocking { ... }` curly braces. This is highlighted in an IDE by +`this: CoroutineScope` hint right after the `runBlocking` opening curly brace. + +If you remove or forget `runBlocking` in this code, you'll get an error on the [launch] call, since `launch` +is declared only on the [CoroutineScope]: + +``` +Unresolved reference: launch +``` + +The name of `runBlocking` means that the thread that runs it (in this case — the main thread) gets _blocked_ for +the duration of the call, until all the coroutines inside `runBlocking { ... }` complete their execution. You will +often see `runBlocking` used like that at the very top-level of the application and quite rarely inside the real code, +as threads are expensive resources and blocking them is inefficient and is often not desired. + +### Structured concurrency + +Coroutines follow a principle of +**structured concurrency** which means that new coroutines can only be launched in a specific [CoroutineScope] +which delimits the lifetime of the coroutine. The above example shows that [runBlocking] establishes the corresponding +scope and that is why the previous example waits until `World!` is printed after a second's delay and only then exits. + +In a real application, you will be launching a lot of coroutines. Structured concurrency ensures that they are not +lost and do not leak. An outer scope cannot complete until all its children coroutines complete. +Structured concurrency also ensures that any errors in the code are properly reported and are never lost. + +## Extract function refactoring + +Let's extract the block of code inside `launch { ... }` into a separate function. When you +perform "Extract function" refactoring on this code, you get a new function with the `suspend` modifier. +This is your first _suspending function_. Suspending functions can be used inside coroutines +just like regular functions, but their additional feature is that they can, in turn, +use other suspending functions (like `delay` in this example) to _suspend_ execution of a coroutine. + +```kotlin +import kotlinx.coroutines.* + +//sampleStart +fun main() = runBlocking { // this: CoroutineScope + launch { doWorld() } + println("Hello") +} + +// this is your first suspending function +suspend fun doWorld() { + delay(1000L) + println("World!") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-basic-02.kt). +> +{style="note"} + + + +## Scope builder + +In addition to the coroutine scope provided by different builders, it is possible to declare your own scope using the +[coroutineScope][_coroutineScope] builder. It creates a coroutine scope and does not complete until all launched children complete. + +[runBlocking] and [coroutineScope][_coroutineScope] builders may look similar because they both wait for their body and all its children to complete. +The main difference is that the [runBlocking] method _blocks_ the current thread for waiting, +while [coroutineScope][_coroutineScope] just suspends, releasing the underlying thread for other usages. +Because of that difference, [runBlocking] is a regular function and [coroutineScope][_coroutineScope] is a suspending function. + +You can use `coroutineScope` from any suspending function. +For example, you can move the concurrent printing of `Hello` and `World` into a `suspend fun doWorld()` function: + +```kotlin +import kotlinx.coroutines.* + +//sampleStart +fun main() = runBlocking { + doWorld() +} + +suspend fun doWorld() = coroutineScope { // this: CoroutineScope + launch { + delay(1000L) + println("World!") + } + println("Hello") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-basic-03.kt). +> +{style="note"} + +This code also prints: + +```text +Hello +World! +``` + + + +## Scope builder and concurrency + +A [coroutineScope][_coroutineScope] builder can be used inside any suspending function to perform multiple concurrent operations. +Let's launch two concurrent coroutines inside a `doWorld` suspending function: + +```kotlin +import kotlinx.coroutines.* + +//sampleStart +// Sequentially executes doWorld followed by "Done" +fun main() = runBlocking { + doWorld() + println("Done") +} + +// Concurrently executes both sections +suspend fun doWorld() = coroutineScope { // this: CoroutineScope + launch { + delay(2000L) + println("World 2") + } + launch { + delay(1000L) + println("World 1") + } + println("Hello") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-basic-04.kt). +> +{style="note"} + +Both pieces of code inside `launch { ... }` blocks execute _concurrently_, with +`World 1` printed first, after a second from start, and `World 2` printed next, after two seconds from start. +A [coroutineScope][_coroutineScope] in `doWorld` completes only after both are complete, so `doWorld` returns and +allows `Done` string to be printed only after that: + +```text +Hello +World 1 +World 2 +Done +``` + + + +## An explicit job + +A [launch] coroutine builder returns a [Job] object that is a handle to the launched coroutine and can be +used to wait for its completion explicitly. +For example, you can wait for the completion of the child coroutine and then print the "Done" string: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val job = launch { // launch a new coroutine and keep a reference to its Job + delay(1000L) + println("World!") + } + println("Hello") + job.join() // wait until child coroutine completes + println("Done") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-basic-05.kt). +> +{style="note"} + +This code produces: + +```text +Hello +World! +Done +``` + + + +## Coroutines are light-weight + +Coroutines are less resource-intensive than JVM threads. Code that exhausts the +JVM's available memory when using threads can be expressed using coroutines +without hitting resource limits. For example, the following code launches +50,000 distinct coroutines that each waits 5 seconds and then prints a period +('.') while consuming very little memory: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { + repeat(50_000) { // launch a lot of coroutines + launch { + delay(5000L) + print(".") + } + } +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-basic-06.kt). +> +{style="note"} + + + +If you write the same program using threads (remove `runBlocking`, replace +`launch` with `thread`, and replace `delay` with `Thread.sleep`), it will +consume a lot of memory. Depending on your operating system, JDK version, +and its settings, it will either throw an out-of-memory error or start threads slowly +so that there are never too many concurrently running threads. + + + + +[launch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html +[delay]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html +[runBlocking]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[_coroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/coroutine-scope.html +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html + + diff --git a/docs/topics/coroutines-guide.md b/docs/topics/coroutines-guide.md new file mode 100644 index 0000000000..ad0bfbd29a --- /dev/null +++ b/docs/topics/coroutines-guide.md @@ -0,0 +1,41 @@ +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + + +[//]: # (title: Coroutines guide) + +Kotlin provides only minimal low-level APIs in its standard library to enable other +libraries to utilize coroutines. Unlike many other languages with similar capabilities, `async` and `await` +are not keywords in Kotlin and are not even part of its standard library. Moreover, Kotlin's concept +of _suspending function_ provides a safer and less error-prone abstraction for asynchronous +operations than futures and promises. + +`kotlinx.coroutines` is a rich library for coroutines developed by JetBrains. It contains a number of high-level +coroutine-enabled primitives that this guide covers, including `launch`, `async`, and others. + +This is a guide about the core features of `kotlinx.coroutines` with a series of examples, divided up into different topics. + +In order to use coroutines as well as follow the examples in this guide, you need to add a dependency on the `kotlinx-coroutines-core` module as explained +[in the project README](https://github.com/Kotlin/kotlinx.coroutines/blob/master/README.md#using-in-your-projects). + +## Table of contents + +* [Coroutines basics](coroutines-basics.md) +* [Tutorial: Intro to coroutines and channels](coroutines-and-channels.md) +* [Cancellation and timeouts](cancellation-and-timeouts.md) +* [Composing suspending functions](composing-suspending-functions.md) +* [Coroutine context and dispatchers](coroutine-context-and-dispatchers.md) +* [Asynchronous Flow](flow.md) +* [Channels](channels.md) +* [Coroutine exceptions handling](exception-handling.md) +* [Shared mutable state and concurrency](shared-mutable-state-and-concurrency.md) +* [Select expression (experimental)](select-expression.md) +* [Tutorial: Debug coroutines using IntelliJ IDEA](debug-coroutines-with-idea.md) +* [Tutorial: Debug Kotlin Flow using IntelliJ IDEA](debug-flow-with-idea.md) + +## Additional references + +* [Guide to UI programming with coroutines](https://github.com/Kotlin/kotlinx.coroutines/blob/master/ui/coroutines-guide-ui.md) +* [Coroutines design document (KEEP)](https://github.com/Kotlin/KEEP/blob/master/proposals/coroutines.md) +* [Full kotlinx.coroutines API reference](https://kotlinlang.org/api/kotlinx.coroutines/) +* [Best practices for coroutines in Android](https://developer.android.com/kotlin/coroutines/coroutines-best-practices) +* [Additional Android resources for Kotlin coroutines and flow](https://developer.android.com/kotlin/coroutines/additional-resources) diff --git a/docs/topics/debug-coroutines-with-idea.md b/docs/topics/debug-coroutines-with-idea.md new file mode 100644 index 0000000000..1c48b558ff --- /dev/null +++ b/docs/topics/debug-coroutines-with-idea.md @@ -0,0 +1,119 @@ +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + + +[//]: # (title: Debug coroutines using IntelliJ IDEA – tutorial) + +This tutorial demonstrates how to create Kotlin coroutines and debug them using IntelliJ IDEA. + +The tutorial assumes you have prior knowledge of the [coroutines](coroutines-guide.md) concept. + +## Create coroutines + +1. Open a Kotlin project in IntelliJ IDEA. If you don't have a project, [create one](jvm-get-started.md#create-a-project). +2. To use the `kotlinx.coroutines` library in a Gradle project, add the following dependency to `build.gradle(.kts)`: + + + + + ```kotlin + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:%coroutinesVersion%") + } + ``` + + + + + ```groovy + dependencies { + implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:%coroutinesVersion%' + } + ``` + + + + + For other build systems, see instructions in the [`kotlinx.coroutines` README](https://github.com/Kotlin/kotlinx.coroutines#using-in-your-projects). + +3. Open the `Main.kt` file in `src/main/kotlin`. + + The `src` directory contains Kotlin source files and resources. The `Main.kt` file contains sample code that will print `Hello World!`. + +4. Change code in the `main()` function: + + * Use the [`runBlocking()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html) block to wrap a coroutine. + * Use the [`async()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html) function to create coroutines that compute deferred values `a` and `b`. + * Use the [`await()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/await.html) function to await the computation result. + * Use the [`println()`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/println.html) function to print computing status and the result of multiplication to the output. + + ```kotlin + import kotlinx.coroutines.* + + fun main() = runBlocking { + val a = async { + println("I'm computing part of the answer") + 6 + } + val b = async { + println("I'm computing another part of the answer") + 7 + } + println("The answer is ${a.await() * b.await()}") + } + ``` + +5. Build the code by clicking **Build Project**. + + ![Build an application](flow-build-project.png) + +## Debug coroutines + +1. Set breakpoints at the lines with the `println()` function call: + + ![Build a console application](coroutine-breakpoint.png) + +2. Run the code in debug mode by clicking **Debug** next to the run configuration at the top of the screen. + + ![Build a console application](flow-debug-project.png) + + The **Debug** tool window appears: + * The **Frames** tab contains the call stack. + * The **Variables** tab contains variables in the current context. + * The **Coroutines** tab contains information on running or suspended coroutines. It shows that there are three coroutines. + The first one has the **RUNNING** status, and the other two have the **CREATED** status. + + ![Debug the coroutine](coroutine-debug-1.png) + +3. Resume the debugger session by clicking **Resume Program** in the **Debug** tool window: + + ![Debug the coroutine](coroutine-debug-2.png) + + Now the **Coroutines** tab shows the following: + * The first coroutine has the **SUSPENDED** status – it is waiting for the values so it can multiply them. + * The second coroutine is calculating the `a` value – it has the **RUNNING** status. + * The third coroutine has the **CREATED** status and isn’t calculating the value of `b`. + +4. Resume the debugger session by clicking **Resume Program** in the **Debug** tool window: + + ![Build a console application](coroutine-debug-3.png) + + Now the **Coroutines** tab shows the following: + * The first coroutine has the **SUSPENDED** status – it is waiting for the values so it can multiply them. + * The second coroutine has computed its value and disappeared. + * The third coroutine is calculating the value of `b` – it has the **RUNNING** status. + +Using IntelliJ IDEA debugger, you can dig deeper into each coroutine to debug your code. + +### Optimized-out variables + +If you use `suspend` functions, in the debugger, you might see the "was optimized out" text next to a variable's name: + +![Variable "a" was optimized out](variable-optimised-out.png){width=480} + +This text means that the variable's lifetime was decreased, and the variable doesn't exist anymore. +It is difficult to debug code with optimized variables because you don't see their values. +You can disable this behavior with the `-Xdebug` compiler option. + +> __Never use this flag in production__: `-Xdebug` can [cause memory leaks](https://youtrack.jetbrains.com/issue/KT-48678/Coroutine-debugger-disable-was-optimised-out-compiler-feature#focus=Comments-27-6015585.0-0). +> +{style="warning"} diff --git a/docs/topics/debug-flow-with-idea.md b/docs/topics/debug-flow-with-idea.md new file mode 100644 index 0000000000..4e2541bc89 --- /dev/null +++ b/docs/topics/debug-flow-with-idea.md @@ -0,0 +1,163 @@ +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + + +[//]: # (title: Debug Kotlin Flow using IntelliJ IDEA – tutorial) + +This tutorial demonstrates how to create Kotlin Flow and debug it using IntelliJ IDEA. + +The tutorial assumes you have prior knowledge of the [coroutines](coroutines-guide.md) and [Kotlin Flow](flow.md#flows) concepts. + +## Create a Kotlin flow + +Create a Kotlin [flow](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flow.html) with a slow emitter and a slow collector: + +1. Open a Kotlin project in IntelliJ IDEA. If you don't have a project, [create one](jvm-get-started.md#create-a-project). +2. To use the `kotlinx.coroutines` library in a Gradle project, add the following dependency to `build.gradle(.kts)`: + + + + + ```kotlin + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:%coroutinesVersion%") + } + ``` + + + + + ```groovy + dependencies { + implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:%coroutinesVersion%' + } + ``` + + + + + For other build systems, see instructions in the [`kotlinx.coroutines` README](https://github.com/Kotlin/kotlinx.coroutines#using-in-your-projects). + +3. Open the `Main.kt` file in `src/main/kotlin`. + + The `src` directory contains Kotlin source files and resources. The `Main.kt` file contains sample code that will print `Hello World!`. + +4. Create the `simple()` function that returns a flow of three numbers: + + * Use the [`delay()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html) function to imitate CPU-consuming blocking code. It suspends the coroutine for 100 ms without blocking the thread. + * Produce the values in the `for` loop using the [`emit()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow-collector/emit.html) function. + + ```kotlin + import kotlinx.coroutines.* + import kotlinx.coroutines.flow.* + import kotlin.system.* + + fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) + emit(i) + } + } + ``` + +5. Change the code in the `main()` function: + + * Use the [`runBlocking()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html) block to wrap a coroutine. + * Collect the emitted values using the [`collect()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/collect.html) function. + * Use the [`delay()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html) function to imitate CPU-consuming code. It suspends the coroutine for 300 ms without blocking the thread. + * Print the collected value from the flow using the [`println()`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/println.html) function. + + ```kotlin + fun main() = runBlocking { + simple() + .collect { value -> + delay(300) + println(value) + } + } + ``` + +6. Build the code by clicking **Build Project**. + + ![Build an application](flow-build-project.png) + +## Debug the coroutine + +1. Set a breakpoint at the line where the `emit()` function is called: + + ![Build a console application](flow-breakpoint.png) + +2. Run the code in debug mode by clicking **Debug** next to the run configuration at the top of the screen. + + ![Build a console application](flow-debug-project.png) + + The **Debug** tool window appears: + * The **Frames** tab contains the call stack. + * The **Variables** tab contains variables in the current context. It tells us that the flow is emitting the first value. + * The **Coroutines** tab contains information on running or suspended coroutines. + + ![Debug the coroutine](flow-debug-1.png) + +3. Resume the debugger session by clicking **Resume Program** in the **Debug** tool window. The program stops at the same breakpoint. + + ![Debug the coroutine](flow-resume-debug.png) + + Now the flow emits the second value. + + ![Debug the coroutine](flow-debug-2.png) + +### Optimized-out variables + +If you use `suspend` functions, in the debugger, you might see the "was optimized out" text next to a variable's name: + +![Variable "a" was optimized out](variable-optimised-out.png) + +This text means that the variable's lifetime was decreased, and the variable doesn't exist anymore. +It is difficult to debug code with optimized variables because you don't see their values. +You can disable this behavior with the `-Xdebug` compiler option. + +> __Never use this flag in production__: `-Xdebug` can [cause memory leaks](https://youtrack.jetbrains.com/issue/KT-48678/Coroutine-debugger-disable-was-optimised-out-compiler-feature#focus=Comments-27-6015585.0-0). +> +{style="warning"} + +## Add a concurrently running coroutine + +1. Open the `Main.kt` file in `src/main/kotlin`. + +2. Enhance the code to run the emitter and collector concurrently: + + * Add a call to the [`buffer()`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/buffer.html) function to run the emitter and collector concurrently. `buffer()` stores emitted values and runs the flow collector in a separate coroutine. + + ```kotlin + fun main() = runBlocking { + simple() + .buffer() + .collect { value -> + delay(300) + println(value) + } + } + ``` + +3. Build the code by clicking **Build Project**. + +## Debug a Kotlin flow with two coroutines + +1. Set a new breakpoint at `println(value)`. + +2. Run the code in debug mode by clicking **Debug** next to the run configuration at the top of the screen. + + ![Build a console application](flow-debug-3.png) + + The **Debug** tool window appears. + + In the **Coroutines** tab, you can see that there are two coroutines running concurrently. The flow collector and emitter run in separate coroutines because of the `buffer()` function. + The `buffer()` function buffers emitted values from the flow. + The emitter coroutine has the **RUNNING** status, and the collector coroutine has the **SUSPENDED** status. + +3. Resume the debugger session by clicking **Resume Program** in the **Debug** tool window. + + ![Debugging coroutines](flow-debug-4.png) + + Now the collector coroutine has the **RUNNING** status, while the emitter coroutine has the **SUSPENDED** status. + + You can dig deeper into each coroutine to debug your code. diff --git a/docs/topics/debugging.md b/docs/topics/debugging.md new file mode 100644 index 0000000000..a2c32e0b18 --- /dev/null +++ b/docs/topics/debugging.md @@ -0,0 +1,108 @@ +**Table of contents** + + + +* [Debugging coroutines](#debugging-coroutines) +* [Debug mode](#debug-mode) +* [Stacktrace recovery](#stacktrace-recovery) + * [Stacktrace recovery machinery](#stacktrace-recovery-machinery) +* [Debug agent](#debug-agent) +* [Android optimization](#android-optimization) + + + +## Debugging coroutines + +Debugging asynchronous programs is challenging, because multiple concurrent coroutines are typically working at the same time. +To help with that, `kotlinx.coroutines` comes with additional features for debugging: debug mode, stacktrace recovery +and debug agent. + +## Debug mode + +The first debugging feature of `kotlinx.coroutines` is debug mode. +It can be enabled either by setting system property [DEBUG_PROPERTY_NAME] or by running Java with enabled assertions (`-ea` flag). +The latter is helpful to have debug mode enabled by default in unit tests. + +Debug mode attaches a unique [name][CoroutineName] to every launched coroutine. +Coroutine name can be seen in a regular Java debugger, +in a string representation of the coroutine or in the thread name executing named coroutine. +Overhead of this feature is negligible and it can be safely turned on by default to simplify logging and diagnostic. + +## Stacktrace recovery + +Stacktrace recovery is another useful feature of debug mode. It is enabled by default in the debug mode, +but can be separately disabled by setting `kotlinx.coroutines.stacktrace.recovery` system property to `false`. + +Stacktrace recovery tries to stitch asynchronous exception stacktrace with a stacktrace of the receiver by copying it, providing +not only information where an exception was thrown, but also where it was asynchronously rethrown or caught. + +It is easy to demonstrate with actual stacktraces of the same program that awaits asynchronous operation in `main` function +(runnable code is [here](../../kotlinx-coroutines-debug/test/RecoveryExample.kt)): + +| Without recovery | With recovery | +| - | - | +| ![before](../images/before.png "before") | ![after](../images/after.png "after") | + +The only downside of this approach is losing referential transparency of the exception. + +> Note that suppressed exceptions are not copied and are left intact in the cause +> in order to prevent cycles in the exceptions chain, obscure`[CIRCULAR REFERENCE]` messages +> and even [crashes](https://jira.qos.ch/browse/LOGBACK-1027) in some frameworks + +### Stacktrace recovery machinery + +This section explains the inner mechanism of stacktrace recovery and can be skipped. + +When an exception is rethrown between coroutines (e.g. through `withContext` or `Deferred.await` boundary), stacktrace recovery +machinery tries to create a copy of the original exception (with the original exception as the cause), then rewrite stacktrace +of the copy with coroutine-related stack frames (using [Throwable.setStackTrace](https://docs.oracle.com/javase/9/docs/api/java/lang/Throwable.html#setStackTrace-java.lang.StackTraceElement:A-)) +and then throws the resulting exception instead of the original one. + +Exception copy logic is straightforward: + 1) If the exception class implements [CopyableThrowable], [CopyableThrowable.createCopy] is used. + `null` can be returned from `createCopy` to opt-out specific exception from being recovered. + 2) If the exception class has class-specific fields not inherited from Throwable, the exception is not copied. + 3) Otherwise, one of the public exception's constructor is invoked reflectively with an optional `initCause` call. + 4) If the reflective copy has a changed message (exception constructor passed a modified `message` parameter to the superclass), + the exception is not copied in order to preserve a human-readable message. [CopyableThrowable] does not have such a limitation + and allows the copy to have a `message` different from that of the original. + +## Debug agent + +[kotlinx-coroutines-debug](../../kotlinx-coroutines-debug) module provides one of the most powerful debug capabilities in `kotlinx.coroutines`. + +This is a separate module with a JVM agent that keeps track of all alive coroutines, introspects and dumps them similar to thread dump command, +additionally enhancing stacktraces with information where coroutine was created. + +The full tutorial of how to use debug agent can be found in the corresponding [readme](../../kotlinx-coroutines-debug/README.md). + + + +## Android optimization + +In optimized (release) builds with R8 version 1.6.0 or later both +[Debugging mode](debugging.md#debug-mode) and +[Stacktrace recovery](debugging.md#stacktrace-recovery) +are permanently turned off. +For more details see ["Optimization" section for Android](../../ui/kotlinx-coroutines-android/README.md#optimization). + + + + +[DEBUG_PROPERTY_NAME]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-d-e-b-u-g_-p-r-o-p-e-r-t-y_-n-a-m-e.html +[CoroutineName]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-name/index.html +[CopyableThrowable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-copyable-throwable/index.html +[CopyableThrowable.createCopy]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-copyable-throwable/create-copy.html + + + diff --git a/docs/topics/exception-handling.md b/docs/topics/exception-handling.md new file mode 100644 index 0000000000..936688ebf3 --- /dev/null +++ b/docs/topics/exception-handling.md @@ -0,0 +1,526 @@ + +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Coroutine exceptions handling) + +This section covers exception handling and cancellation on exceptions. +We already know that a cancelled coroutine throws [CancellationException] in suspension points and that it +is ignored by the coroutines' machinery. Here we look at what happens if an exception is thrown during cancellation or multiple children of the same +coroutine throw an exception. + +## Exception propagation + +Coroutine builders come in two flavors: propagating exceptions automatically ([launch]) or +exposing them to users ([async] and [produce]). +When these builders are used to create a _root_ coroutine, that is not a _child_ of another coroutine, +the former builders treat exceptions as **uncaught** exceptions, similar to Java's `Thread.uncaughtExceptionHandler`, +while the latter are relying on the user to consume the final +exception, for example via [await][Deferred.await] or [receive][ReceiveChannel.receive] +([produce] and [receive][ReceiveChannel.receive] are covered in [Channels](https://github.com/Kotlin/kotlinx.coroutines/blob/master/docs/channels.md) section). + +It can be demonstrated by a simple example that creates root coroutines using the [GlobalScope]: + +> [GlobalScope] is a delicate API that can backfire in non-trivial ways. Creating a root coroutine for the +> whole application is one of the rare legitimate uses for `GlobalScope`, so you must explicitly opt-in into +> using `GlobalScope` with `@OptIn(DelicateCoroutinesApi::class)`. +> +{style="note"} + +```kotlin +import kotlinx.coroutines.* + +//sampleStart +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { + val job = GlobalScope.launch { // root coroutine with launch + println("Throwing exception from launch") + throw IndexOutOfBoundsException() // Will be printed to the console by Thread.defaultUncaughtExceptionHandler + } + job.join() + println("Joined failed job") + val deferred = GlobalScope.async { // root coroutine with async + println("Throwing exception from async") + throw ArithmeticException() // Nothing is printed, relying on user to call await + } + try { + deferred.await() + println("Unreached") + } catch (e: ArithmeticException) { + println("Caught ArithmeticException") + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-01.kt). +> +{style="note"} + +The output of this code is (with [debug](https://github.com/Kotlin/kotlinx.coroutines/blob/master/docs/coroutine-context-and-dispatchers.md#debugging-coroutines-and-threads)): + +```text +Throwing exception from launch +Exception in thread "DefaultDispatcher-worker-1 @coroutine#2" java.lang.IndexOutOfBoundsException +Joined failed job +Throwing exception from async +Caught ArithmeticException +``` + + + +## CoroutineExceptionHandler + +It is possible to customize the default behavior of printing **uncaught** exceptions to the console. +[CoroutineExceptionHandler] context element on a _root_ coroutine can be used as a generic `catch` block for +this root coroutine and all its children where custom exception handling may take place. +It is similar to [`Thread.uncaughtExceptionHandler`](https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#setUncaughtExceptionHandler-java.lang.Thread.UncaughtExceptionHandler-). +You cannot recover from the exception in the `CoroutineExceptionHandler`. The coroutine had already completed +with the corresponding exception when the handler is called. Normally, the handler is used to +log the exception, show some kind of error message, terminate, and/or restart the application. + + +`CoroutineExceptionHandler` is invoked only on **uncaught** exceptions — exceptions that were not handled in any other way. +In particular, all _children_ coroutines (coroutines created in the context of another [Job]) delegate handling of +their exceptions to their parent coroutine, which also delegates to the parent, and so on until the root, +so the `CoroutineExceptionHandler` installed in their context is never used. +In addition to that, [async] builder always catches all exceptions and represents them in the resulting [Deferred] object, +so its `CoroutineExceptionHandler` has no effect either. + +> Coroutines running in supervision scope do not propagate exceptions to their parent and are +> excluded from this rule. A further [Supervision](#supervision) section of this document gives more details. +> +{style="note"} + +```kotlin +import kotlinx.coroutines.* + +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { +//sampleStart + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception") + } + val job = GlobalScope.launch(handler) { // root coroutine, running in GlobalScope + throw AssertionError() + } + val deferred = GlobalScope.async(handler) { // also root, but async instead of launch + throw ArithmeticException() // Nothing will be printed, relying on user to call deferred.await() + } + joinAll(job, deferred) +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-02.kt). +> +{style="note"} + +The output of this code is: + +```text +CoroutineExceptionHandler got java.lang.AssertionError +``` + + + +## Cancellation and exceptions + +Cancellation is closely related to exceptions. Coroutines internally use `CancellationException` for cancellation, these +exceptions are ignored by all handlers, so they should be used only as the source of additional debug information, which can +be obtained by `catch` block. +When a coroutine is cancelled using [Job.cancel], it terminates, but it does not cancel its parent. + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val job = launch { + val child = launch { + try { + delay(Long.MAX_VALUE) + } finally { + println("Child is cancelled") + } + } + yield() + println("Cancelling child") + child.cancel() + child.join() + yield() + println("Parent is not cancelled") + } + job.join() +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-03.kt). +> +{style="note"} + +The output of this code is: + +```text +Cancelling child +Child is cancelled +Parent is not cancelled +``` + + + +If a coroutine encounters an exception other than `CancellationException`, it cancels its parent with that exception. +This behaviour cannot be overridden and is used to provide stable coroutines hierarchies for +[structured concurrency](https://github.com/Kotlin/kotlinx.coroutines/blob/master/docs/composing-suspending-functions.md#structured-concurrency-with-async). +[CoroutineExceptionHandler] implementation is not used for child coroutines. + +> In these examples, [CoroutineExceptionHandler] is always installed to a coroutine +> that is created in [GlobalScope]. It does not make sense to install an exception handler to a coroutine that +> is launched in the scope of the main [runBlocking], since the main coroutine is going to be always cancelled +> when its child completes with exception despite the installed handler. +> +{style="note"} + +The original exception is handled by the parent only when all its children terminate, +which is demonstrated by the following example. + +```kotlin +import kotlinx.coroutines.* + +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { +//sampleStart + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception") + } + val job = GlobalScope.launch(handler) { + launch { // the first child + try { + delay(Long.MAX_VALUE) + } finally { + withContext(NonCancellable) { + println("Children are cancelled, but exception is not handled until all children terminate") + delay(100) + println("The first child finished its non cancellable block") + } + } + } + launch { // the second child + delay(10) + println("Second child throws an exception") + throw ArithmeticException() + } + } + job.join() +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-04.kt). +> +{style="note"} + +The output of this code is: + +```text +Second child throws an exception +Children are cancelled, but exception is not handled until all children terminate +The first child finished its non cancellable block +CoroutineExceptionHandler got java.lang.ArithmeticException +``` + + + +## Exceptions aggregation + +When multiple children of a coroutine fail with an exception, the +general rule is "the first exception wins", so the first exception gets handled. +All additional exceptions that happen after the first one are attached to the first exception as suppressed ones. + + + +```kotlin +import kotlinx.coroutines.* +import java.io.* + +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception with suppressed ${exception.suppressed.contentToString()}") + } + val job = GlobalScope.launch(handler) { + launch { + try { + delay(Long.MAX_VALUE) // it gets cancelled when another sibling fails with IOException + } finally { + throw ArithmeticException() // the second exception + } + } + launch { + delay(100) + throw IOException() // the first exception + } + delay(Long.MAX_VALUE) + } + job.join() +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-05.kt). +> +{style="note"} + +The output of this code is: + +```text +CoroutineExceptionHandler got java.io.IOException with suppressed [java.lang.ArithmeticException] +``` + + + +> Note that this mechanism currently only works on Java version 1.7+. +> The JS and Native restrictions are temporary and will be lifted in the future. +> +{style="note"} + +Cancellation exceptions are transparent and are unwrapped by default: + +```kotlin +import kotlinx.coroutines.* +import java.io.* + +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { +//sampleStart + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception") + } + val job = GlobalScope.launch(handler) { + val innerJob = launch { // all this stack of coroutines will get cancelled + launch { + launch { + throw IOException() // the original exception + } + } + } + try { + innerJob.join() + } catch (e: CancellationException) { + println("Rethrowing CancellationException with original cause") + throw e // cancellation exception is rethrown, yet the original IOException gets to the handler + } + } + job.join() +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-06.kt). +> +{style="note"} + +The output of this code is: + +```text +Rethrowing CancellationException with original cause +CoroutineExceptionHandler got java.io.IOException +``` + + + +## Supervision + +As we have studied before, cancellation is a bidirectional relationship propagating through the whole +hierarchy of coroutines. Let us take a look at the case when unidirectional cancellation is required. + +A good example of such a requirement is a UI component with the job defined in its scope. If any of the UI's child tasks +have failed, it is not always necessary to cancel (effectively kill) the whole UI component, +but if the UI component is destroyed (and its job is cancelled), then it is necessary to cancel all child jobs as their results are no longer needed. + +Another example is a server process that spawns multiple child jobs and needs to _supervise_ +their execution, tracking their failures and only restarting the failed ones. + +### Supervision job + +The [SupervisorJob][SupervisorJob()] can be used for these purposes. +It is similar to a regular [Job][Job()] with the only exception that cancellation is propagated +only downwards. This can easily be demonstrated using the following example: + +```kotlin +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val supervisor = SupervisorJob() + with(CoroutineScope(coroutineContext + supervisor)) { + // launch the first child -- its exception is ignored for this example (don't do this in practice!) + val firstChild = launch(CoroutineExceptionHandler { _, _ -> }) { + println("The first child is failing") + throw AssertionError("The first child is cancelled") + } + // launch the second child + val secondChild = launch { + firstChild.join() + // Cancellation of the first child is not propagated to the second child + println("The first child is cancelled: ${firstChild.isCancelled}, but the second one is still active") + try { + delay(Long.MAX_VALUE) + } finally { + // But cancellation of the supervisor is propagated + println("The second child is cancelled because the supervisor was cancelled") + } + } + // wait until the first child fails & completes + firstChild.join() + println("Cancelling the supervisor") + supervisor.cancel() + secondChild.join() + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-supervision-01.kt). +> +{style="note"} + +The output of this code is: + +```text +The first child is failing +The first child is cancelled: true, but the second one is still active +Cancelling the supervisor +The second child is cancelled because the supervisor was cancelled +``` + + + +### Supervision scope + +Instead of [coroutineScope][_coroutineScope], we can use [supervisorScope][_supervisorScope] for _scoped_ concurrency. It propagates the cancellation +in one direction only and cancels all its children only if it failed itself. It also waits for all children before completion +just like [coroutineScope][_coroutineScope] does. + +```kotlin +import kotlin.coroutines.* +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + try { + supervisorScope { + val child = launch { + try { + println("The child is sleeping") + delay(Long.MAX_VALUE) + } finally { + println("The child is cancelled") + } + } + // Give our child a chance to execute and print using yield + yield() + println("Throwing an exception from the scope") + throw AssertionError() + } + } catch(e: AssertionError) { + println("Caught an assertion error") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-supervision-02.kt). +> +{style="note"} + +The output of this code is: + +```text +The child is sleeping +Throwing an exception from the scope +The child is cancelled +Caught an assertion error +``` + + + +#### Exceptions in supervised coroutines + +Another crucial difference between regular and supervisor jobs is exception handling. +Every child should handle its exceptions by itself via the exception handling mechanism. +This difference comes from the fact that child's failure does not propagate to the parent. +It means that coroutines launched directly inside the [supervisorScope][_supervisorScope] _do_ use the [CoroutineExceptionHandler] +that is installed in their scope in the same way as root coroutines do +(see the [CoroutineExceptionHandler](#coroutineexceptionhandler) section for details). + +```kotlin +import kotlin.coroutines.* +import kotlinx.coroutines.* + +fun main() = runBlocking { +//sampleStart + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception") + } + supervisorScope { + val child = launch(handler) { + println("The child throws an exception") + throw AssertionError() + } + println("The scope is completing") + } + println("The scope is completed") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-supervision-03.kt). +> +{style="note"} + +The output of this code is: + +```text +The scope is completing +The child throws an exception +CoroutineExceptionHandler got java.lang.AssertionError +The scope is completed +``` + + + + + + +[CancellationException]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-cancellation-exception/index.html +[launch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html +[async]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html +[Deferred.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/await.html +[GlobalScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-global-scope/index.html +[CoroutineExceptionHandler]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-exception-handler/index.html +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[Deferred]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/index.html +[Job.cancel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/cancel.html +[runBlocking]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html +[SupervisorJob()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-supervisor-job.html +[Job()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job.html +[_coroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/coroutine-scope.html +[_supervisorScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/supervisor-scope.html + + + +[produce]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/produce.html +[ReceiveChannel.receive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/receive.html + + diff --git a/docs/topics/flow.md b/docs/topics/flow.md new file mode 100644 index 0000000000..3f8c694943 --- /dev/null +++ b/docs/topics/flow.md @@ -0,0 +1,1902 @@ + +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Asynchronous Flow) + +A suspending function asynchronously returns a single value, but how can we return +multiple asynchronously computed values? This is where Kotlin Flows come in. + +## Representing multiple values + +Multiple values can be represented in Kotlin using [collections]. +For example, we can have a `simple` function that returns a [List] +of three numbers and then print them all using [forEach]: + +```kotlin +fun simple(): List = listOf(1, 2, 3) + +fun main() { + simple().forEach { value -> println(value) } +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-01.kt). +> +{style="note"} + +This code outputs: + +```text +1 +2 +3 +``` + + + +### Sequences + +If we are computing the numbers with some CPU-consuming blocking code +(each computation taking 100ms), then we can represent the numbers using a [Sequence]: + +```kotlin +fun simple(): Sequence = sequence { // sequence builder + for (i in 1..3) { + Thread.sleep(100) // pretend we are computing it + yield(i) // yield next value + } +} + +fun main() { + simple().forEach { value -> println(value) } +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-02.kt). +> +{style="note"} + +This code outputs the same numbers, but it waits 100ms before printing each one. + + + +### Suspending functions + +However, this computation blocks the main thread that is running the code. +When these values are computed by asynchronous code we can mark the `simple` function with a `suspend` modifier, +so that it can perform its work without blocking and return the result as a list: + +```kotlin +import kotlinx.coroutines.* + +//sampleStart +suspend fun simple(): List { + delay(1000) // pretend we are doing something asynchronous here + return listOf(1, 2, 3) +} + +fun main() = runBlocking { + simple().forEach { value -> println(value) } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-03.kt). +> +{style="note"} + +This code prints the numbers after waiting for a second. + + + +### Flows + +Using the `List` result type, means we can only return all the values at once. To represent +the stream of values that are being computed asynchronously, we can use a [`Flow`][Flow] type just like we would use a `Sequence` type for synchronously computed values: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = flow { // flow builder + for (i in 1..3) { + delay(100) // pretend we are doing something useful here + emit(i) // emit next value + } +} + +fun main() = runBlocking { + // Launch a concurrent coroutine to check if the main thread is blocked + launch { + for (k in 1..3) { + println("I'm not blocked $k") + delay(100) + } + } + // Collect the flow + simple().collect { value -> println(value) } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-04.kt). +> +{style="note"} + +This code waits 100ms before printing each number without blocking the main thread. This is verified +by printing "I'm not blocked" every 100ms from a separate coroutine that is running in the main thread: + +```text +I'm not blocked 1 +1 +I'm not blocked 2 +2 +I'm not blocked 3 +3 +``` + + + +Notice the following differences in the code with the [Flow] from the earlier examples: + +* A builder function of [Flow] type is called [flow][_flow]. +* Code inside a `flow { ... }` builder block can suspend. +* The `simple` function is no longer marked with a `suspend` modifier. +* Values are _emitted_ from the flow using an [emit][FlowCollector.emit] function. +* Values are _collected_ from the flow using a [collect][collect] function. + +> We can replace [delay] with `Thread.sleep` in the body of `simple`'s `flow { ... }` and see that the main +> thread is blocked in this case. +> +{style="note"} + +## Flows are cold + +Flows are _cold_ streams similar to sequences — the code inside a [flow][_flow] builder does not +run until the flow is collected. This becomes clear in the following example: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = flow { + println("Flow started") + for (i in 1..3) { + delay(100) + emit(i) + } +} + +fun main() = runBlocking { + println("Calling simple function...") + val flow = simple() + println("Calling collect...") + flow.collect { value -> println(value) } + println("Calling collect again...") + flow.collect { value -> println(value) } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-05.kt). +> +{style="note"} + +Which prints: + +```text +Calling simple function... +Calling collect... +Flow started +1 +2 +3 +Calling collect again... +Flow started +1 +2 +3 +``` + + + +This is a key reason the `simple` function (which returns a flow) is not marked with `suspend` modifier. +The `simple()` call itself returns quickly and does not wait for anything. The flow starts afresh every time it is +collected and that is why we see "Flow started" every time we call `collect` again. + +## Flow cancellation basics + +Flows adhere to the general cooperative cancellation of coroutines. As usual, flow collection can be +cancelled when the flow is suspended in a cancellable suspending function (like [delay]). +The following example shows how the flow gets cancelled on a timeout when running in a [withTimeoutOrNull] block +and stops executing its code: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) + println("Emitting $i") + emit(i) + } +} + +fun main() = runBlocking { + withTimeoutOrNull(250) { // Timeout after 250ms + simple().collect { value -> println(value) } + } + println("Done") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-06.kt). +> +{style="note"} + +Notice how only two numbers get emitted by the flow in the `simple` function, producing the following output: + +```text +Emitting 1 +1 +Emitting 2 +2 +Done +``` + + + +See [Flow cancellation checks](#flow-cancellation-checks) section for more details. + +## Flow builders + +The `flow { ... }` builder from the previous examples is the most basic one. There are other builders +that allow flows to be declared: + +* The [flowOf] builder defines a flow that emits a fixed set of values. +* Various collections and sequences can be converted to flows using the `.asFlow()` extension function. + +For example, the snippet that prints the numbers 1 to 3 from a flow can be rewritten as follows: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { +//sampleStart + // Convert an integer range to a flow + (1..3).asFlow().collect { value -> println(value) } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-07.kt). +> +{style="note"} + + + +## Intermediate flow operators + +Flows can be transformed using operators, in the same way as you would transform collections and +sequences. +Intermediate operators are applied to an upstream flow and return a downstream flow. +These operators are cold, just like flows are. A call to such an operator is not +a suspending function itself. It works quickly, returning the definition of a new transformed flow. + +The basic operators have familiar names like [map] and [filter]. +An important difference of these operators from sequences is that blocks of +code inside these operators can call suspending functions. + +For example, a flow of incoming requests can be +mapped to its results with a [map] operator, even when performing a request is a long-running +operation that is implemented by a suspending function: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +suspend fun performRequest(request: Int): String { + delay(1000) // imitate long-running asynchronous work + return "response $request" +} + +fun main() = runBlocking { + (1..3).asFlow() // a flow of requests + .map { request -> performRequest(request) } + .collect { response -> println(response) } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-08.kt). +> +{style="note"} + +It produces the following three lines, each appearing one second after the previous: + +```text +response 1 +response 2 +response 3 +``` + + + +### Transform operator + +Among the flow transformation operators, the most general one is called [transform]. It can be used to imitate +simple transformations like [map] and [filter], as well as implement more complex transformations. +Using the `transform` operator, we can [emit][FlowCollector.emit] arbitrary values an arbitrary number of times. + +For example, using `transform` we can emit a string before performing a long-running asynchronous request +and follow it with a response: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +suspend fun performRequest(request: Int): String { + delay(1000) // imitate long-running asynchronous work + return "response $request" +} + +fun main() = runBlocking { +//sampleStart + (1..3).asFlow() // a flow of requests + .transform { request -> + emit("Making request $request") + emit(performRequest(request)) + } + .collect { response -> println(response) } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-09.kt). +> +{style="note"} + +The output of this code is: + +```text +Making request 1 +response 1 +Making request 2 +response 2 +Making request 3 +response 3 +``` + + + +### Size-limiting operators + +Size-limiting intermediate operators like [take] cancel the execution of the flow when the corresponding limit +is reached. Cancellation in coroutines is always performed by throwing an exception, so that all the resource-management +functions (like `try { ... } finally { ... }` blocks) operate normally in case of cancellation: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun numbers(): Flow = flow { + try { + emit(1) + emit(2) + println("This line will not execute") + emit(3) + } finally { + println("Finally in numbers") + } +} + +fun main() = runBlocking { + numbers() + .take(2) // take only the first two + .collect { value -> println(value) } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-10.kt). +> +{style="note"} + +The output of this code clearly shows that the execution of the `flow { ... }` body in the `numbers()` function +stopped after emitting the second number: + +```text +1 +2 +Finally in numbers +``` + + + +## Terminal flow operators + +Terminal operators on flows are _suspending functions_ that start a collection of the flow. +The [collect] operator is the most basic one, but there are other terminal operators, which can make it easier: + +* Conversion to various collections like [toList] and [toSet]. +* Operators to get the [first] value and to ensure that a flow emits a [single] value. +* Reducing a flow to a value with [reduce] and [fold]. + +For example: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { +//sampleStart + val sum = (1..5).asFlow() + .map { it * it } // squares of numbers from 1 to 5 + .reduce { a, b -> a + b } // sum them (terminal operator) + println(sum) +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-11.kt). +> +{style="note"} + +Prints a single number: + +```text +55 +``` + + + +## Flows are sequential + +Each individual collection of a flow is performed sequentially unless special operators that operate +on multiple flows are used. The collection works directly in the coroutine that calls a terminal operator. +No new coroutines are launched by default. +Each emitted value is processed by all the intermediate operators from +upstream to downstream and is then delivered to the terminal operator after. + +See the following example that filters the even integers and maps them to strings: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { +//sampleStart + (1..5).asFlow() + .filter { + println("Filter $it") + it % 2 == 0 + } + .map { + println("Map $it") + "string $it" + }.collect { + println("Collect $it") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-12.kt). +> +{style="note"} + +Producing: + +```text +Filter 1 +Filter 2 +Map 2 +Collect string 2 +Filter 3 +Filter 4 +Map 4 +Collect string 4 +Filter 5 +``` + + + +## Flow context + +Collection of a flow always happens in the context of the calling coroutine. For example, if there is +a `simple` flow, then the following code runs in the context specified +by the author of this code, regardless of the implementation details of the `simple` flow: + +```kotlin +withContext(context) { + simple().collect { value -> + println(value) // run in the specified context + } +} +``` + + + +This property of a flow is called _context preservation_. + +So, by default, code in the `flow { ... }` builder runs in the context that is provided by a collector +of the corresponding flow. For example, consider the implementation of a `simple` function that prints the thread +it is called on and emits three numbers: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +//sampleStart +fun simple(): Flow = flow { + log("Started simple flow") + for (i in 1..3) { + emit(i) + } +} + +fun main() = runBlocking { + simple().collect { value -> log("Collected $value") } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-13.kt). +> +{style="note"} + +Running this code produces: + +```text +[main @coroutine#1] Started simple flow +[main @coroutine#1] Collected 1 +[main @coroutine#1] Collected 2 +[main @coroutine#1] Collected 3 +``` + + + +Since `simple().collect` is called from the main thread, the body of `simple`'s flow is also called in the main thread. +This is the perfect default for fast-running or asynchronous code that does not care about the execution context and +does not block the caller. + +### A common pitfall when using withContext + +However, the long-running CPU-consuming code might need to be executed in the context of [Dispatchers.Default] and UI-updating +code might need to be executed in the context of [Dispatchers.Main]. Usually, [withContext] is used +to change the context in the code using Kotlin coroutines, but code in the `flow { ... }` builder has to honor the context +preservation property and is not allowed to [emit][FlowCollector.emit] from a different context. + +Try running the following code: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = flow { + // The WRONG way to change context for CPU-consuming code in flow builder + kotlinx.coroutines.withContext(Dispatchers.Default) { + for (i in 1..3) { + Thread.sleep(100) // pretend we are computing it in CPU-consuming way + emit(i) // emit next value + } + } +} + +fun main() = runBlocking { + simple().collect { value -> println(value) } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-14.kt). +> +{style="note"} + +This code produces the following exception: + +```text +Exception in thread "main" java.lang.IllegalStateException: Flow invariant is violated: + Flow was collected in [CoroutineId(1), "coroutine#1":BlockingCoroutine{Active}@5511c7f8, BlockingEventLoop@2eac3323], + but emission happened in [CoroutineId(1), "coroutine#1":DispatchedCoroutine{Active}@2dae0000, Dispatchers.Default]. + Please refer to 'flow' documentation or use 'flowOn' instead + at ... +``` + + + +### flowOn operator + +The exception refers to the [flowOn] function that shall be used to change the context of the flow emission. +The correct way to change the context of a flow is shown in the example below, which also prints the +names of the corresponding threads to show how it all works: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +//sampleStart +fun simple(): Flow = flow { + for (i in 1..3) { + Thread.sleep(100) // pretend we are computing it in CPU-consuming way + log("Emitting $i") + emit(i) // emit next value + } +}.flowOn(Dispatchers.Default) // RIGHT way to change context for CPU-consuming code in flow builder + +fun main() = runBlocking { + simple().collect { value -> + log("Collected $value") + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-15.kt). +> +{style="note"} + +Notice how `flow { ... }` works in the background thread, while collection happens in the main thread: + +```text +[DefaultDispatcher-worker-1 @coroutine#2] Emitting 1 +[main @coroutine#1] Collected 1 +[DefaultDispatcher-worker-1 @coroutine#2] Emitting 2 +[main @coroutine#1] Collected 2 +[DefaultDispatcher-worker-1 @coroutine#2] Emitting 3 +[main @coroutine#1] Collected 3 +``` + + + +Another thing to observe here is that the [flowOn] operator has changed the default sequential nature of the flow. +Now collection happens in one coroutine ("coroutine#1") and emission happens in another coroutine +("coroutine#2") that is running in another thread concurrently with the collecting coroutine. The [flowOn] operator +creates another coroutine for an upstream flow when it has to change the [CoroutineDispatcher] in its context. + +## Buffering + +Running different parts of a flow in different coroutines can be helpful from the standpoint of the overall time it takes +to collect the flow, especially when long-running asynchronous operations are involved. For example, consider a case when +the emission by a `simple` flow is slow, taking 100 ms to produce an element; and collector is also slow, +taking 300 ms to process an element. Let's see how long it takes to collect such a flow with three numbers: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.system.* + +//sampleStart +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) // pretend we are asynchronously waiting 100 ms + emit(i) // emit next value + } +} + +fun main() = runBlocking { + val time = measureTimeMillis { + simple().collect { value -> + delay(300) // pretend we are processing it for 300 ms + println(value) + } + } + println("Collected in $time ms") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-16.kt). +> +{style="note"} + +It produces something like this, with the whole collection taking around 1200 ms (three numbers, 400 ms for each): + +```text +1 +2 +3 +Collected in 1220 ms +``` + + + +We can use a [buffer] operator on a flow to run emitting code of the `simple` flow concurrently with collecting code, +as opposed to running them sequentially: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.system.* + +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) // pretend we are asynchronously waiting 100 ms + emit(i) // emit next value + } +} + +fun main() = runBlocking { +//sampleStart + val time = measureTimeMillis { + simple() + .buffer() // buffer emissions, don't wait + .collect { value -> + delay(300) // pretend we are processing it for 300 ms + println(value) + } + } + println("Collected in $time ms") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-17.kt). +> +{style="note"} + +It produces the same numbers just faster, as we have effectively created a processing pipeline, +having to only wait 100 ms for the first number and then spending only 300 ms to process +each number. This way it takes around 1000 ms to run: + +```text +1 +2 +3 +Collected in 1071 ms +``` + + + +> Note that the [flowOn] operator uses the same buffering mechanism when it has to change a [CoroutineDispatcher], +> but here we explicitly request buffering without changing the execution context. +> +{style="note"} + +### Conflation + +When a flow represents partial results of the operation or operation status updates, it may not be necessary +to process each value, but instead, only most recent ones. In this case, the [conflate] operator can be used to skip +intermediate values when a collector is too slow to process them. Building on the previous example: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.system.* + +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) // pretend we are asynchronously waiting 100 ms + emit(i) // emit next value + } +} + +fun main() = runBlocking { +//sampleStart + val time = measureTimeMillis { + simple() + .conflate() // conflate emissions, don't process each one + .collect { value -> + delay(300) // pretend we are processing it for 300 ms + println(value) + } + } + println("Collected in $time ms") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-18.kt). +> +{style="note"} + +We see that while the first number was still being processed the second, and third were already produced, so +the second one was _conflated_ and only the most recent (the third one) was delivered to the collector: + +```text +1 +3 +Collected in 758 ms +``` + + + +### Processing the latest value + +Conflation is one way to speed up processing when both the emitter and collector are slow. It does it by dropping emitted values. +The other way is to cancel a slow collector and restart it every time a new value is emitted. There is +a family of `xxxLatest` operators that perform the same essential logic of a `xxx` operator, but cancel the +code in their block on a new value. Let's try changing [conflate] to [collectLatest] in the previous example: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.system.* + +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) // pretend we are asynchronously waiting 100 ms + emit(i) // emit next value + } +} + +fun main() = runBlocking { +//sampleStart + val time = measureTimeMillis { + simple() + .collectLatest { value -> // cancel & restart on the latest value + println("Collecting $value") + delay(300) // pretend we are processing it for 300 ms + println("Done $value") + } + } + println("Collected in $time ms") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-19.kt). +> +{style="note"} + +Since the body of [collectLatest] takes 300 ms, but new values are emitted every 100 ms, we see that the block +is run on every value, but completes only for the last value: + +```text +Collecting 1 +Collecting 2 +Collecting 3 +Done 3 +Collected in 741 ms +``` + + + +## Composing multiple flows + +There are lots of ways to compose multiple flows. + +### Zip + +Just like the [Sequence.zip] extension function in the Kotlin standard library, +flows have a [zip] operator that combines the corresponding values of two flows: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { +//sampleStart + val nums = (1..3).asFlow() // numbers 1..3 + val strs = flowOf("one", "two", "three") // strings + nums.zip(strs) { a, b -> "$a -> $b" } // compose a single string + .collect { println(it) } // collect and print +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-20.kt). +> +{style="note"} + +This example prints: + +```text +1 -> one +2 -> two +3 -> three +``` + + + +### Combine + +When flow represents the most recent value of a variable or operation (see also the related +section on [conflation](#conflation)), it might be needed to perform a computation that depends on +the most recent values of the corresponding flows and to recompute it whenever any of the upstream +flows emit a value. The corresponding family of operators is called [combine]. + +For example, if the numbers in the previous example update every 300ms, but strings update every 400 ms, +then zipping them using the [zip] operator will still produce the same result, +albeit results that are printed every 400 ms: + +> We use a [onEach] intermediate operator in this example to delay each element and make the code +> that emits sample flows more declarative and shorter. +> +{style="note"} + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { +//sampleStart + val nums = (1..3).asFlow().onEach { delay(300) } // numbers 1..3 every 300 ms + val strs = flowOf("one", "two", "three").onEach { delay(400) } // strings every 400 ms + val startTime = System.currentTimeMillis() // remember the start time + nums.zip(strs) { a, b -> "$a -> $b" } // compose a single string with "zip" + .collect { value -> // collect and print + println("$value at ${System.currentTimeMillis() - startTime} ms from start") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-21.kt). +> +{style="note"} + + + +However, when using a [combine] operator here instead of a [zip]: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { +//sampleStart + val nums = (1..3).asFlow().onEach { delay(300) } // numbers 1..3 every 300 ms + val strs = flowOf("one", "two", "three").onEach { delay(400) } // strings every 400 ms + val startTime = System.currentTimeMillis() // remember the start time + nums.combine(strs) { a, b -> "$a -> $b" } // compose a single string with "combine" + .collect { value -> // collect and print + println("$value at ${System.currentTimeMillis() - startTime} ms from start") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-22.kt). +> +{style="note"} + +We get quite a different output, where a line is printed at each emission from either `nums` or `strs` flows: + +```text +1 -> one at 452 ms from start +2 -> one at 651 ms from start +2 -> two at 854 ms from start +3 -> two at 952 ms from start +3 -> three at 1256 ms from start +``` + + + +## Flattening flows + +Flows represent asynchronously received sequences of values, and so it is quite easy to get into a situation +where each value triggers a request for another sequence of values. For example, we can have the following +function that returns a flow of two strings 500 ms apart: + +```kotlin +fun requestFlow(i: Int): Flow = flow { + emit("$i: First") + delay(500) // wait 500 ms + emit("$i: Second") +} +``` + + + +Now if we have a flow of three integers and call `requestFlow` on each of them like this: + +```kotlin +(1..3).asFlow().map { requestFlow(it) } +``` + + + +Then we will end up with a flow of flows (`Flow>`) that needs to be _flattened_ into a single flow for +further processing. Collections and sequences have [flatten][Sequence.flatten] and [flatMap][Sequence.flatMap] +operators for this. However, due to the asynchronous nature of flows they call for different _modes_ of flattening, +and hence, a family of flattening operators on flows exists. + +### flatMapConcat + +Concatenation of flows of flows is provided by the [flatMapConcat] and [flattenConcat] operators. They are the +most direct analogues of the corresponding sequence operators. They wait for the inner flow to complete before +starting to collect the next one as the following example shows: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun requestFlow(i: Int): Flow = flow { + emit("$i: First") + delay(500) // wait 500 ms + emit("$i: Second") +} + +fun main() = runBlocking { +//sampleStart + val startTime = System.currentTimeMillis() // remember the start time + (1..3).asFlow().onEach { delay(100) } // emit a number every 100 ms + .flatMapConcat { requestFlow(it) } + .collect { value -> // collect and print + println("$value at ${System.currentTimeMillis() - startTime} ms from start") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-23.kt). +> +{style="note"} + +The sequential nature of [flatMapConcat] is clearly seen in the output: + +```text +1: First at 121 ms from start +1: Second at 622 ms from start +2: First at 727 ms from start +2: Second at 1227 ms from start +3: First at 1328 ms from start +3: Second at 1829 ms from start +``` + + + +### flatMapMerge + +Another flattening operation is to concurrently collect all the incoming flows and merge their values into +a single flow so that values are emitted as soon as possible. +It is implemented by [flatMapMerge] and [flattenMerge] operators. They both accept an optional +`concurrency` parameter that limits the number of concurrent flows that are collected at the same time +(it is equal to [DEFAULT_CONCURRENCY] by default). + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun requestFlow(i: Int): Flow = flow { + emit("$i: First") + delay(500) // wait 500 ms + emit("$i: Second") +} + +fun main() = runBlocking { +//sampleStart + val startTime = System.currentTimeMillis() // remember the start time + (1..3).asFlow().onEach { delay(100) } // a number every 100 ms + .flatMapMerge { requestFlow(it) } + .collect { value -> // collect and print + println("$value at ${System.currentTimeMillis() - startTime} ms from start") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-24.kt). +> +{style="note"} + +The concurrent nature of [flatMapMerge] is obvious: + +```text +1: First at 136 ms from start +2: First at 231 ms from start +3: First at 333 ms from start +1: Second at 639 ms from start +2: Second at 732 ms from start +3: Second at 833 ms from start +``` + + + +> Note that the [flatMapMerge] calls its block of code (`{ requestFlow(it) }` in this example) sequentially, but +> collects the resulting flows concurrently, it is the equivalent of performing a sequential +> `map { requestFlow(it) }` first and then calling [flattenMerge] on the result. +> +{style="note"} + +### flatMapLatest + +In a similar way to the [collectLatest] operator, that was described in the section +["Processing the latest value"](#processing-the-latest-value), there is the corresponding "Latest" +flattening mode where the collection of the previous flow is cancelled as soon as new flow is emitted. +It is implemented by the [flatMapLatest] operator. + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun requestFlow(i: Int): Flow = flow { + emit("$i: First") + delay(500) // wait 500 ms + emit("$i: Second") +} + +fun main() = runBlocking { +//sampleStart + val startTime = System.currentTimeMillis() // remember the start time + (1..3).asFlow().onEach { delay(100) } // a number every 100 ms + .flatMapLatest { requestFlow(it) } + .collect { value -> // collect and print + println("$value at ${System.currentTimeMillis() - startTime} ms from start") + } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-25.kt). +> +{style="note"} + +The output here in this example is a good demonstration of how [flatMapLatest] works: + +```text +1: First at 142 ms from start +2: First at 322 ms from start +3: First at 425 ms from start +3: Second at 931 ms from start +``` + + + +> Note that [flatMapLatest] cancels all the code in its block (`{ requestFlow(it) }` in this example) when a new value +> is received. +> It makes no difference in this particular example, because the call to `requestFlow` itself is fast, not-suspending, +> and cannot be cancelled. However, a differnce in output would be visible if we were to use suspending functions +> like `delay` in `requestFlow`. +> +{style="note"} + +## Flow exceptions + +Flow collection can complete with an exception when an emitter or code inside the operators throw an exception. +There are several ways to handle these exceptions. + +### Collector try and catch + +A collector can use Kotlin's [`try/catch`][exceptions] block to handle exceptions: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) // emit next value + } +} + +fun main() = runBlocking { + try { + simple().collect { value -> + println(value) + check(value <= 1) { "Collected $value" } + } + } catch (e: Throwable) { + println("Caught $e") + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-26.kt). +> +{style="note"} + +This code successfully catches an exception in [collect] terminal operator and, +as we see, no more values are emitted after that: + +```text +Emitting 1 +1 +Emitting 2 +2 +Caught java.lang.IllegalStateException: Collected 2 +``` + + + +### Everything is caught + +The previous example actually catches any exception happening in the emitter or in any intermediate or terminal operators. +For example, let's change the code so that emitted values are [mapped][map] to strings, +but the corresponding code produces an exception: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = + flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) // emit next value + } + } + .map { value -> + check(value <= 1) { "Crashed on $value" } + "string $value" + } + +fun main() = runBlocking { + try { + simple().collect { value -> println(value) } + } catch (e: Throwable) { + println("Caught $e") + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-27.kt). +> +{style="note"} + +This exception is still caught and collection is stopped: + +```text +Emitting 1 +string 1 +Emitting 2 +Caught java.lang.IllegalStateException: Crashed on 2 +``` + + + +## Exception transparency + +But how can code of the emitter encapsulate its exception handling behavior? + +Flows must be _transparent to exceptions_ and it is a violation of the exception transparency to [emit][FlowCollector.emit] values in the +`flow { ... }` builder from inside of a `try/catch` block. This guarantees that a collector throwing an exception +can always catch it using `try/catch` as in the previous example. + +The emitter can use a [catch] operator that preserves this exception transparency and allows encapsulation +of its exception handling. The body of the `catch` operator can analyze an exception +and react to it in different ways depending on which exception was caught: + +* Exceptions can be rethrown using `throw`. +* Exceptions can be turned into emission of values using [emit][FlowCollector.emit] from the body of [catch]. +* Exceptions can be ignored, logged, or processed by some other code. + +For example, let us emit the text on catching an exception: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = + flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) // emit next value + } + } + .map { value -> + check(value <= 1) { "Crashed on $value" } + "string $value" + } + +fun main() = runBlocking { +//sampleStart + simple() + .catch { e -> emit("Caught $e") } // emit on exception + .collect { value -> println(value) } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-28.kt). +> +{style="note"} + +The output of the example is the same, even though we do not have `try/catch` around the code anymore. + + + +### Transparent catch + +The [catch] intermediate operator, honoring exception transparency, catches only upstream exceptions +(that is an exception from all the operators above `catch`, but not below it). +If the block in `collect { ... }` (placed below `catch`) throws an exception then it escapes: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) + } +} + +fun main() = runBlocking { + simple() + .catch { e -> println("Caught $e") } // does not catch downstream exceptions + .collect { value -> + check(value <= 1) { "Collected $value" } + println(value) + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-29.kt). +> +{style="note"} + +A "Caught ..." message is not printed despite there being a `catch` operator: + +```text +Emitting 1 +1 +Emitting 2 +Exception in thread "main" java.lang.IllegalStateException: Collected 2 + at ... +``` + + + +### Catching declaratively + +We can combine the declarative nature of the [catch] operator with a desire to handle all the exceptions, by moving the body +of the [collect] operator into [onEach] and putting it before the `catch` operator. Collection of this flow must +be triggered by a call to `collect()` without parameters: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) + } +} + +fun main() = runBlocking { +//sampleStart + simple() + .onEach { value -> + check(value <= 1) { "Collected $value" } + println(value) + } + .catch { e -> println("Caught $e") } + .collect() +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-30.kt). +> +{style="note"} + +Now we can see that a "Caught ..." message is printed and so we can catch all the exceptions without explicitly +using a `try/catch` block: + +```text +Emitting 1 +1 +Emitting 2 +Caught java.lang.IllegalStateException: Collected 2 +``` + + + +## Flow completion + +When flow collection completes (normally or exceptionally) it may need to execute an action. +As you may have already noticed, it can be done in two ways: imperative or declarative. + +### Imperative finally block + +In addition to `try`/`catch`, a collector can also use a `finally` block to execute an action +upon `collect` completion. + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = (1..3).asFlow() + +fun main() = runBlocking { + try { + simple().collect { value -> println(value) } + } finally { + println("Done") + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-31.kt). +> +{style="note"} + +This code prints three numbers produced by the `simple` flow followed by a "Done" string: + +```text +1 +2 +3 +Done +``` + + + +### Declarative handling + +For the declarative approach, flow has [onCompletion] intermediate operator that is invoked +when the flow has completely collected. + +The previous example can be rewritten using an [onCompletion] operator and produces the same output: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = (1..3).asFlow() + +fun main() = runBlocking { +//sampleStart + simple() + .onCompletion { println("Done") } + .collect { value -> println(value) } +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-32.kt). +> +{style="note"} + + + +The key advantage of [onCompletion] is a nullable `Throwable` parameter of the lambda that can be used +to determine whether the flow collection was completed normally or exceptionally. In the following +example the `simple` flow throws an exception after emitting the number 1: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = flow { + emit(1) + throw RuntimeException() +} + +fun main() = runBlocking { + simple() + .onCompletion { cause -> if (cause != null) println("Flow completed exceptionally") } + .catch { cause -> println("Caught exception") } + .collect { value -> println(value) } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-33.kt). +> +{style="note"} + +As you may expect, it prints: + +```text +1 +Flow completed exceptionally +Caught exception +``` + + + +The [onCompletion] operator, unlike [catch], does not handle the exception. As we can see from the above +example code, the exception still flows downstream. It will be delivered to further `onCompletion` operators +and can be handled with a `catch` operator. + +### Successful completion + +Another difference with [catch] operator is that [onCompletion] sees all exceptions and receives +a `null` exception only on successful completion of the upstream flow (without cancellation or failure). + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun simple(): Flow = (1..3).asFlow() + +fun main() = runBlocking { + simple() + .onCompletion { cause -> println("Flow completed with $cause") } + .collect { value -> + check(value <= 1) { "Collected $value" } + println(value) + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-34.kt). +> +{style="note"} + +We can see the completion cause is not null, because the flow was aborted due to downstream exception: + +```text +1 +Flow completed with java.lang.IllegalStateException: Collected 2 +Exception in thread "main" java.lang.IllegalStateException: Collected 2 +``` + + + +## Imperative versus declarative + +Now we know how to collect flow, and handle its completion and exceptions in both imperative and declarative ways. +The natural question here is, which approach is preferred and why? +As a library, we do not advocate for any particular approach and believe that both options +are valid and should be selected according to your own preferences and code style. + +## Launching flow + +It is easy to use flows to represent asynchronous events that are coming from some source. +In this case, we need an analogue of the `addEventListener` function that registers a piece of code with a reaction +for incoming events and continues further work. The [onEach] operator can serve this role. +However, `onEach` is an intermediate operator. We also need a terminal operator to collect the flow. +Otherwise, just calling `onEach` has no effect. + +If we use the [collect] terminal operator after `onEach`, then the code after it will wait until the flow is collected: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +// Imitate a flow of events +fun events(): Flow = (1..3).asFlow().onEach { delay(100) } + +fun main() = runBlocking { + events() + .onEach { event -> println("Event: $event") } + .collect() // <--- Collecting the flow waits + println("Done") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-35.kt). +> +{style="note"} + +As you can see, it prints: + +```text +Event: 1 +Event: 2 +Event: 3 +Done +``` + + + +The [launchIn] terminal operator comes in handy here. By replacing `collect` with `launchIn` we can +launch a collection of the flow in a separate coroutine, so that execution of further code +immediately continues: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +// Imitate a flow of events +fun events(): Flow = (1..3).asFlow().onEach { delay(100) } + +//sampleStart +fun main() = runBlocking { + events() + .onEach { event -> println("Event: $event") } + .launchIn(this) // <--- Launching the flow in a separate coroutine + println("Done") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-36.kt). +> +{style="note"} + +It prints: + +```text +Done +Event: 1 +Event: 2 +Event: 3 +``` + + + +The required parameter to `launchIn` must specify a [CoroutineScope] in which the coroutine to collect the flow is +launched. In the above example this scope comes from the [runBlocking] +coroutine builder, so while the flow is running, this [runBlocking] scope waits for completion of its child coroutine +and keeps the main function from returning and terminating this example. + +In actual applications a scope will come from an entity with a limited +lifetime. As soon as the lifetime of this entity is terminated the corresponding scope is cancelled, cancelling +the collection of the corresponding flow. This way the pair of `onEach { ... }.launchIn(scope)` works +like the `addEventListener`. However, there is no need for the corresponding `removeEventListener` function, +as cancellation and structured concurrency serve this purpose. + +Note that [launchIn] also returns a [Job], which can be used to [cancel][Job.cancel] the corresponding flow collection +coroutine only without cancelling the whole scope or to [join][Job.join] it. + +### Flow cancellation checks + +For convenience, the [flow][_flow] builder performs additional [ensureActive] checks for cancellation on each emitted value. +It means that a busy loop emitting from a `flow { ... }` is cancellable: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun foo(): Flow = flow { + for (i in 1..5) { + println("Emitting $i") + emit(i) + } +} + +fun main() = runBlocking { + foo().collect { value -> + if (value == 3) cancel() + println(value) + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-37.kt). +> +{style="note"} + +We get only numbers up to 3 and a [CancellationException] after trying to emit number 4: + +```text +Emitting 1 +1 +Emitting 2 +2 +Emitting 3 +3 +Emitting 4 +Exception in thread "main" kotlinx.coroutines.JobCancellationException: BlockingCoroutine was cancelled; job="coroutine#1":BlockingCoroutine{Cancelled}@6d7b4f4c +``` + + + +However, most other flow operators do not do additional cancellation checks on their own for performance reasons. +For example, if you use [IntRange.asFlow] extension to write the same busy loop and don't suspend anywhere, +then there are no checks for cancellation: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun main() = runBlocking { + (1..5).asFlow().collect { value -> + if (value == 3) cancel() + println(value) + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-38.kt). +> +{style="note"} + +All numbers from 1 to 5 are collected and cancellation gets detected only before return from `runBlocking`: + +```text +1 +2 +3 +4 +5 +Exception in thread "main" kotlinx.coroutines.JobCancellationException: BlockingCoroutine was cancelled; job="coroutine#1":BlockingCoroutine{Cancelled}@3327bd23 +``` + + + +#### Making busy flow cancellable + +In the case where you have a busy loop with coroutines you must explicitly check for cancellation. +You can add `.onEach { currentCoroutineContext().ensureActive() }`, but there is a ready-to-use +[cancellable] operator provided to do that: + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +//sampleStart +fun main() = runBlocking { + (1..5).asFlow().cancellable().collect { value -> + if (value == 3) cancel() + println(value) + } +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-flow-39.kt). +> +{style="note"} + +With the `cancellable` operator only the numbers from 1 to 3 are collected: + +```text +1 +2 +3 +Exception in thread "main" kotlinx.coroutines.JobCancellationException: BlockingCoroutine was cancelled; job="coroutine#1":BlockingCoroutine{Cancelled}@5ec0a365 +``` + + + +## Flow and Reactive Streams + +For those who are familiar with [Reactive Streams](https://www.reactive-streams.org/) or reactive frameworks such as RxJava and project Reactor, +design of the Flow may look very familiar. + +Indeed, its design was inspired by Reactive Streams and its various implementations. But Flow main goal is to have as simple design as possible, +be Kotlin and suspension friendly and respect structured concurrency. Achieving this goal would be impossible without reactive pioneers and their tremendous work. You can read the complete story in [Reactive Streams and Kotlin Flows](https://medium.com/@elizarov/reactive-streams-and-kotlin-flows-bfd12772cda4) article. + +While being different, conceptually, Flow *is* a reactive stream and it is possible to convert it to the reactive (spec and TCK compliant) Publisher and vice versa. +Such converters are provided by `kotlinx.coroutines` out-of-the-box and can be found in corresponding reactive modules (`kotlinx-coroutines-reactive` for Reactive Streams, `kotlinx-coroutines-reactor` for Project Reactor and `kotlinx-coroutines-rx2`/`kotlinx-coroutines-rx3` for RxJava2/RxJava3). +Integration modules include conversions from and to `Flow`, integration with Reactor's `Context` and suspension-friendly ways to work with various reactive entities. + + + +[collections]: https://kotlinlang.org/docs/reference/collections-overview.html +[List]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/-list/ +[forEach]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/for-each.html +[Sequence]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.sequences/ +[Sequence.zip]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.sequences/zip.html +[Sequence.flatten]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.sequences/flatten.html +[Sequence.flatMap]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.sequences/flat-map.html +[exceptions]: https://kotlinlang.org/docs/reference/exceptions.html + + + + +[delay]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html +[withTimeoutOrNull]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-timeout-or-null.html +[Dispatchers.Default]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-default.html +[Dispatchers.Main]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-main.html +[withContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-context.html +[CoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/index.html +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[runBlocking]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[Job.cancel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/cancel.html +[Job.join]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/join.html +[ensureActive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/ensure-active.html +[CancellationException]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-cancellation-exception/index.html + + + +[Flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html +[_flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flow.html +[FlowCollector.emit]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow-collector/emit.html +[collect]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/collect.html +[flowOf]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flow-of.html +[map]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/map.html +[filter]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/filter.html +[transform]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/transform.html +[take]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/take.html +[toList]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/to-list.html +[toSet]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/to-set.html +[first]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/first.html +[single]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/single.html +[reduce]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/reduce.html +[fold]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/fold.html +[flowOn]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flow-on.html +[buffer]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/buffer.html +[conflate]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/conflate.html +[collectLatest]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/collect-latest.html +[zip]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/zip.html +[combine]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/combine.html +[onEach]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/on-each.html +[flatMapConcat]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flat-map-concat.html +[flattenConcat]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flatten-concat.html +[flatMapMerge]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flat-map-merge.html +[flattenMerge]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flatten-merge.html +[DEFAULT_CONCURRENCY]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-d-e-f-a-u-l-t_-c-o-n-c-u-r-r-e-n-c-y.html +[flatMapLatest]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flat-map-latest.html +[catch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/catch.html +[onCompletion]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/on-completion.html +[launchIn]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/launch-in.html +[IntRange.asFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/as-flow.html +[cancellable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/cancellable.html + + diff --git a/docs/topics/knit.properties b/docs/topics/knit.properties new file mode 100644 index 0000000000..f71ccf2ae6 --- /dev/null +++ b/docs/topics/knit.properties @@ -0,0 +1,5 @@ +knit.package=kotlinx.coroutines.guide +knit.dir=../../kotlinx-coroutines-core/jvm/test/guide/ + +test.package=kotlinx.coroutines.guide.test +test.dir=../../kotlinx-coroutines-core/jvm/test/guide/test/ diff --git a/docs/topics/select-expression.md b/docs/topics/select-expression.md new file mode 100644 index 0000000000..ded445b260 --- /dev/null +++ b/docs/topics/select-expression.md @@ -0,0 +1,508 @@ + +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Select expression \(experimental\)) + +Select expression makes it possible to await multiple suspending functions simultaneously and _select_ +the first one that becomes available. + +> Select expressions are an experimental feature of `kotlinx.coroutines`. Their API is expected to +> evolve in the upcoming updates of the `kotlinx.coroutines` library with potentially +> breaking changes. +> +{style="note"} + +## Selecting from channels + +Let us have two producers of strings: `fizz` and `buzz`. The `fizz` produces "Fizz" string every 500 ms: + +```kotlin +fun CoroutineScope.fizz() = produce { + while (true) { // sends "Fizz" every 500 ms + delay(500) + send("Fizz") + } +} +``` + +And the `buzz` produces "Buzz!" string every 1000 ms: + +```kotlin +fun CoroutineScope.buzz() = produce { + while (true) { // sends "Buzz!" every 1000 ms + delay(1000) + send("Buzz!") + } +} +``` + +Using [receive][ReceiveChannel.receive] suspending function we can receive _either_ from one channel or the +other. But [select] expression allows us to receive from _both_ simultaneously using its +[onReceive][ReceiveChannel.onReceive] clauses: + +```kotlin +suspend fun selectFizzBuzz(fizz: ReceiveChannel, buzz: ReceiveChannel) { + select { // means that this select expression does not produce any result + fizz.onReceive { value -> // this is the first select clause + println("fizz -> '$value'") + } + buzz.onReceive { value -> // this is the second select clause + println("buzz -> '$value'") + } + } +} +``` + +Let us run it all seven times: + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* + +fun CoroutineScope.fizz() = produce { + while (true) { // sends "Fizz" every 500 ms + delay(500) + send("Fizz") + } +} + +fun CoroutineScope.buzz() = produce { + while (true) { // sends "Buzz!" every 1000 ms + delay(1000) + send("Buzz!") + } +} + +suspend fun selectFizzBuzz(fizz: ReceiveChannel, buzz: ReceiveChannel) { + select { // means that this select expression does not produce any result + fizz.onReceive { value -> // this is the first select clause + println("fizz -> '$value'") + } + buzz.onReceive { value -> // this is the second select clause + println("buzz -> '$value'") + } + } +} + +fun main() = runBlocking { +//sampleStart + val fizz = fizz() + val buzz = buzz() + repeat(7) { + selectFizzBuzz(fizz, buzz) + } + coroutineContext.cancelChildren() // cancel fizz & buzz coroutines +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-select-01.kt). +> +{style="note"} + +The result of this code is: + +```text +fizz -> 'Fizz' +buzz -> 'Buzz!' +fizz -> 'Fizz' +fizz -> 'Fizz' +buzz -> 'Buzz!' +fizz -> 'Fizz' +fizz -> 'Fizz' +``` + + + +## Selecting on close + +The [onReceive][ReceiveChannel.onReceive] clause in `select` fails when the channel is closed causing the corresponding +`select` to throw an exception. We can use [onReceiveCatching][ReceiveChannel.onReceiveCatching] clause to perform a +specific action when the channel is closed. The following example also shows that `select` is an expression that returns +the result of its selected clause: + +```kotlin +suspend fun selectAorB(a: ReceiveChannel, b: ReceiveChannel): String = + select { + a.onReceiveCatching { it -> + val value = it.getOrNull() + if (value != null) { + "a -> '$value'" + } else { + "Channel 'a' is closed" + } + } + b.onReceiveCatching { it -> + val value = it.getOrNull() + if (value != null) { + "b -> '$value'" + } else { + "Channel 'b' is closed" + } + } + } +``` + + +Let's use it with channel `a` that produces "Hello" string four times and +channel `b` that produces "World" four times: + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* + +suspend fun selectAorB(a: ReceiveChannel, b: ReceiveChannel): String = + select { + a.onReceiveCatching { it -> + val value = it.getOrNull() + if (value != null) { + "a -> '$value'" + } else { + "Channel 'a' is closed" + } + } + b.onReceiveCatching { it -> + val value = it.getOrNull() + if (value != null) { + "b -> '$value'" + } else { + "Channel 'b' is closed" + } + } + } + +fun main() = runBlocking { +//sampleStart + val a = produce { + repeat(4) { send("Hello $it") } + } + val b = produce { + repeat(4) { send("World $it") } + } + repeat(8) { // print first eight results + println(selectAorB(a, b)) + } + coroutineContext.cancelChildren() +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-select-02.kt). +> +{style="note"} + +The result of this code is quite interesting, so we'll analyze it in more detail: + +```text +a -> 'Hello 0' +a -> 'Hello 1' +b -> 'World 0' +a -> 'Hello 2' +a -> 'Hello 3' +b -> 'World 1' +Channel 'a' is closed +Channel 'a' is closed +``` + + + +There are a couple of observations to make out of it. + +First of all, `select` is _biased_ to the first clause. When several clauses are selectable at the same time, +the first one among them gets selected. Here, both channels are constantly producing strings, so `a` channel, +being the first clause in select, wins. However, because we are using unbuffered channel, the `a` gets suspended from +time to time on its [send][SendChannel.send] invocation and gives a chance for `b` to send, too. + +The second observation, is that [onReceiveCatching][ReceiveChannel.onReceiveCatching] gets immediately selected when the +channel is already closed. + +## Selecting to send + +Select expression has [onSend][SendChannel.onSend] clause that can be used for a great good in combination +with a biased nature of selection. + +Let us write an example of a producer of integers that sends its values to a `side` channel when +the consumers on its primary channel cannot keep up with it: + +```kotlin +fun CoroutineScope.produceNumbers(side: SendChannel) = produce { + for (num in 1..10) { // produce 10 numbers from 1 to 10 + delay(100) // every 100 ms + select { + onSend(num) {} // Send to the primary channel + side.onSend(num) {} // or to the side channel + } + } +} +``` + +Consumer is going to be quite slow, taking 250 ms to process each number: + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* + +fun CoroutineScope.produceNumbers(side: SendChannel) = produce { + for (num in 1..10) { // produce 10 numbers from 1 to 10 + delay(100) // every 100 ms + select { + onSend(num) {} // Send to the primary channel + side.onSend(num) {} // or to the side channel + } + } +} + +fun main() = runBlocking { +//sampleStart + val side = Channel() // allocate side channel + launch { // this is a very fast consumer for the side channel + side.consumeEach { println("Side channel has $it") } + } + produceNumbers(side).consumeEach { + println("Consuming $it") + delay(250) // let us digest the consumed number properly, do not hurry + } + println("Done consuming") + coroutineContext.cancelChildren() +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-select-03.kt). +> +{style="note"} + +So let us see what happens: + +```text +Consuming 1 +Side channel has 2 +Side channel has 3 +Consuming 4 +Side channel has 5 +Side channel has 6 +Consuming 7 +Side channel has 8 +Side channel has 9 +Consuming 10 +Done consuming +``` + + + +## Selecting deferred values + +Deferred values can be selected using [onAwait][Deferred.onAwait] clause. +Let us start with an async function that returns a deferred string value after +a random delay: + +```kotlin +fun CoroutineScope.asyncString(time: Int) = async { + delay(time.toLong()) + "Waited for $time ms" +} +``` + +Let us start a dozen of them with a random delay. + +```kotlin +fun CoroutineScope.asyncStringsList(): List> { + val random = Random(3) + return List(12) { asyncString(random.nextInt(1000)) } +} +``` + +Now the main function awaits for the first of them to complete and counts the number of deferred values +that are still active. Note that we've used here the fact that `select` expression is a Kotlin DSL, +so we can provide clauses for it using an arbitrary code. In this case we iterate over a list +of deferred values to provide `onAwait` clause for each deferred value. + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import java.util.* + +fun CoroutineScope.asyncString(time: Int) = async { + delay(time.toLong()) + "Waited for $time ms" +} + +fun CoroutineScope.asyncStringsList(): List> { + val random = Random(3) + return List(12) { asyncString(random.nextInt(1000)) } +} + +fun main() = runBlocking { +//sampleStart + val list = asyncStringsList() + val result = select { + list.withIndex().forEach { (index, deferred) -> + deferred.onAwait { answer -> + "Deferred $index produced answer '$answer'" + } + } + } + println(result) + val countActive = list.count { it.isActive } + println("$countActive coroutines are still active") +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-select-04.kt). +> +{style="note"} + +The output is: + +```text +Deferred 4 produced answer 'Waited for 128 ms' +11 coroutines are still active +``` + + + +## Switch over a channel of deferred values + +Let us write a channel producer function that consumes a channel of deferred string values, waits for each received +deferred value, but only until the next deferred value comes over or the channel is closed. This example puts together +[onReceiveCatching][ReceiveChannel.onReceiveCatching] and [onAwait][Deferred.onAwait] clauses in the same `select`: + +```kotlin +fun CoroutineScope.switchMapDeferreds(input: ReceiveChannel>) = produce { + var current = input.receive() // start with first received deferred value + while (isActive) { // loop while not cancelled/closed + val next = select?> { // return next deferred value from this select or null + input.onReceiveCatching { update -> + update.getOrNull() + } + current.onAwait { value -> + send(value) // send value that current deferred has produced + input.receiveCatching().getOrNull() // and use the next deferred from the input channel + } + } + if (next == null) { + println("Channel was closed") + break // out of loop + } else { + current = next + } + } +} +``` + +To test it, we'll use a simple async function that resolves to a specified string after a specified time: + +```kotlin +fun CoroutineScope.asyncString(str: String, time: Long) = async { + delay(time) + str +} +``` + +The main function just launches a coroutine to print results of `switchMapDeferreds` and sends some test +data to it: + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* + +fun CoroutineScope.switchMapDeferreds(input: ReceiveChannel>) = produce { + var current = input.receive() // start with first received deferred value + while (isActive) { // loop while not cancelled/closed + val next = select?> { // return next deferred value from this select or null + input.onReceiveCatching { update -> + update.getOrNull() + } + current.onAwait { value -> + send(value) // send value that current deferred has produced + input.receiveCatching().getOrNull() // and use the next deferred from the input channel + } + } + if (next == null) { + println("Channel was closed") + break // out of loop + } else { + current = next + } + } +} + +fun CoroutineScope.asyncString(str: String, time: Long) = async { + delay(time) + str +} + +fun main() = runBlocking { +//sampleStart + val chan = Channel>() // the channel for test + launch { // launch printing coroutine + for (s in switchMapDeferreds(chan)) + println(s) // print each received string + } + chan.send(asyncString("BEGIN", 100)) + delay(200) // enough time for "BEGIN" to be produced + chan.send(asyncString("Slow", 500)) + delay(100) // not enough time to produce slow + chan.send(asyncString("Replace", 100)) + delay(500) // give it time before the last one + chan.send(asyncString("END", 500)) + delay(1000) // give it time to process + chan.close() // close the channel ... + delay(500) // and wait some time to let it finish +//sampleEnd +} +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-select-05.kt). +> +{style="note"} + +The result of this code: + +```text +BEGIN +Replace +END +Channel was closed +``` + + + + + + +[Deferred.onAwait]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/on-await.html + + + +[ReceiveChannel.receive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/receive.html +[ReceiveChannel.onReceive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/on-receive.html +[ReceiveChannel.onReceiveCatching]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/on-receive-catching.html +[SendChannel.send]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/send.html +[SendChannel.onSend]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/on-send.html + + + +[select]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.selects/select.html + + diff --git a/docs/topics/shared-mutable-state-and-concurrency.md b/docs/topics/shared-mutable-state-and-concurrency.md new file mode 100644 index 0000000000..133c9e2cfe --- /dev/null +++ b/docs/topics/shared-mutable-state-and-concurrency.md @@ -0,0 +1,386 @@ + +https://github.com/Kotlin/kotlinx.coroutines/edit/master/docs/topics/ + +[//]: # (title: Shared mutable state and concurrency) + +Coroutines can be executed parallelly using a multi-threaded dispatcher like the [Dispatchers.Default]. It presents +all the usual parallelism problems. The main problem being synchronization of access to **shared mutable state**. +Some solutions to this problem in the land of coroutines are similar to the solutions in the multi-threaded world, +but others are unique. + +## The problem + +Let us launch a hundred coroutines all doing the same action a thousand times. +We'll also measure their completion time for further comparisons: + +```kotlin +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} +``` + +We start with a very simple action that increments a shared mutable variable using +multi-threaded [Dispatchers.Default]. + + + +```kotlin +import kotlinx.coroutines.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +//sampleStart +var counter = 0 + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + counter++ + } + } + println("Counter = $counter") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-sync-01.kt). +> +{style="note"} + + + +What does it print at the end? It is highly unlikely to ever print "Counter = 100000", because a hundred coroutines +increment the `counter` concurrently from multiple threads without any synchronization. + +## Volatiles are of no help + +There is a common misconception that making a variable `volatile` solves concurrency problem. Let us try it: + + + +```kotlin +import kotlinx.coroutines.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +//sampleStart +@Volatile // in Kotlin `volatile` is an annotation +var counter = 0 + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + counter++ + } + } + println("Counter = $counter") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-sync-02.kt). +> +{style="note"} + + + +This code works slower, but we still don't always get "Counter = 100000" at the end, because volatile variables guarantee +linearizable (this is a technical term for "atomic") reads and writes to the corresponding variable, but +do not provide atomicity of larger actions (increment in our case). + +## Thread-safe data structures + +The general solution that works both for threads and for coroutines is to use a thread-safe (aka synchronized, +linearizable, or atomic) data structure that provides all the necessary synchronization for the corresponding +operations that needs to be performed on a shared state. +In the case of a simple counter we can use `AtomicInteger` class which has atomic `incrementAndGet` operations: + + + +```kotlin +import kotlinx.coroutines.* +import java.util.concurrent.atomic.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +//sampleStart +val counter = AtomicInteger() + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + counter.incrementAndGet() + } + } + println("Counter = $counter") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-sync-03.kt). +> +{style="note"} + + + +This is the fastest solution for this particular problem. It works for plain counters, collections, queues and other +standard data structures and basic operations on them. However, it does not easily scale to complex +state or to complex operations that do not have ready-to-use thread-safe implementations. + +## Thread confinement fine-grained + +_Thread confinement_ is an approach to the problem of shared mutable state where all access to the particular shared +state is confined to a single thread. It is typically used in UI applications, where all UI state is confined to +the single event-dispatch/application thread. It is easy to apply with coroutines by using a single-threaded context. + + + +```kotlin +import kotlinx.coroutines.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +//sampleStart +val counterContext = newSingleThreadContext("CounterContext") +var counter = 0 + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + // confine each increment to a single-threaded context + withContext(counterContext) { + counter++ + } + } + } + println("Counter = $counter") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-sync-04.kt). +> +{style="note"} + + + +This code works very slowly, because it does _fine-grained_ thread-confinement. Each individual increment switches +from multi-threaded [Dispatchers.Default] context to the single-threaded context using +[withContext(counterContext)][withContext] block. + +## Thread confinement coarse-grained + +In practice, thread confinement is performed in large chunks, e.g. big pieces of state-updating business logic +are confined to the single thread. The following example does it like that, running each coroutine in +the single-threaded context to start with. + + + +```kotlin +import kotlinx.coroutines.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +//sampleStart +val counterContext = newSingleThreadContext("CounterContext") +var counter = 0 + +fun main() = runBlocking { + // confine everything to a single-threaded context + withContext(counterContext) { + massiveRun { + counter++ + } + } + println("Counter = $counter") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-sync-05.kt). +> +{style="note"} + + + +This now works much faster and produces correct result. + +## Mutual exclusion + +Mutual exclusion solution to the problem is to protect all modifications of the shared state with a _critical section_ +that is never executed concurrently. In a blocking world you'd typically use `synchronized` or `ReentrantLock` for that. +Coroutine's alternative is called [Mutex]. It has [lock][Mutex.lock] and [unlock][Mutex.unlock] functions to +delimit a critical section. The key difference is that `Mutex.lock()` is a suspending function. It does not block a thread. + +There is also [withLock] extension function that conveniently represents +`mutex.lock(); try { ... } finally { mutex.unlock() }` pattern: + + + +```kotlin +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +//sampleStart +val mutex = Mutex() +var counter = 0 + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + // protect each increment with lock + mutex.withLock { + counter++ + } + } + } + println("Counter = $counter") +} +//sampleEnd +``` +{kotlin-runnable="true" kotlin-min-compiler-version="1.3"} + +> You can get the full code [here](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/test/guide/example-sync-06.kt). +> +{style="note"} + + + +The locking in this example is fine-grained, so it pays the price. However, it is a good choice for some situations +where you absolutely must modify some shared state periodically, but there is no natural thread that this state +is confined to. + + + + +[Dispatchers.Default]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-default.html +[withContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-context.html + + + +[Mutex]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/index.html +[Mutex.lock]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/lock.html +[Mutex.unlock]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/unlock.html +[withLock]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/with-lock.html + + diff --git a/docs/writerside.cfg b/docs/writerside.cfg new file mode 100644 index 0000000000..7efe584730 --- /dev/null +++ b/docs/writerside.cfg @@ -0,0 +1,14 @@ + + + + + + + + + + + + + + diff --git a/dokka-templates/README.md b/dokka-templates/README.md new file mode 100644 index 0000000000..0891177785 --- /dev/null +++ b/dokka-templates/README.md @@ -0,0 +1,4 @@ +# Customize Dokka's HTML. +To customize Dokka's HTML output, place a file in this folder. +Dokka will find a template file there. If the file is not found, a default one will be used. +This folder is defined by the templatesDir property. \ No newline at end of file diff --git a/gradle.properties b/gradle.properties new file mode 100644 index 0000000000..7af5770c35 --- /dev/null +++ b/gradle.properties @@ -0,0 +1,51 @@ +# Kotlin +version=1.10.2-SNAPSHOT +group=org.jetbrains.kotlinx +kotlin_version=2.1.0 +# DO NOT rename this property without adapting kotlinx.train build chain: +atomicfu_version=0.26.1 +benchmarks_version=0.4.13 +benchmarks_jmh_version=1.37 + +# Dependencies +junit_version=4.12 +junit5_version=5.7.0 +knit_version=0.5.0 +lincheck_version=2.18.1 +dokka_version=2.0.0 +byte_buddy_version=1.10.9 +reactor_version=3.4.1 +reactor_docs_version=3.4.5 +reactive_streams_version=1.0.3 +rxjava2_version=2.2.8 +rxjava3_version=3.0.2 +javafx_version=17.0.2 +javafx_plugin_version=0.0.8 +binary_compatibility_validator_version=0.16.2 +kover_version=0.8.0-Beta2 +blockhound_version=1.0.8.RELEASE +jna_version=5.9.0 + +# Gradle +jdk_toolchain_version=11 +animalsniffer_version=2.0.0 + +# Android versions +android_version=4.1.1.4 +androidx_annotation_version=1.1.0 +robolectric_version=4.9 +baksmali_version=2.2.7 + +# Settings +kotlin.incremental.multiplatform=true +kotlin.native.ignoreDisabledTargets=true + +# JS IR backend sometimes crashes with out-of-memory +# TODO: Remove once KT-37187 is fixed +org.gradle.jvmargs=-Xmx3g + +kotlinx.atomicfu.enableJvmIrTransformation=true +# When the flag below is set to `true`, AtomicFU cannot process +# usages of `moveForward` in `ConcurrentLinkedList.kt` correctly. +kotlinx.atomicfu.enableJsIrTransformation=false +kotlinx.atomicfu.enableNativeIrTransformation=true diff --git a/gradle/wrapper/gradle-wrapper.jar b/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 0000000000..9bbc975c74 Binary files /dev/null and b/gradle/wrapper/gradle-wrapper.jar differ diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 0000000000..2a6e21b2ba --- /dev/null +++ b/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,8 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +distributionSha256Sum=fba8464465835e74f7270bbf43d6d8a8d7709ab0a43ce1aa3323f73e9aa0c612 +distributionUrl=https\://services.gradle.org/distributions/gradle-8.13-all.zip +networkTimeout=10000 +validateDistributionUrl=true +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists diff --git a/gradlew b/gradlew new file mode 100755 index 0000000000..faf93008b7 --- /dev/null +++ b/gradlew @@ -0,0 +1,251 @@ +#!/bin/sh + +# +# Copyright © 2015-2021 the original authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +############################################################################## +# +# Gradle start up script for POSIX generated by Gradle. +# +# Important for running: +# +# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is +# noncompliant, but you have some other compliant shell such as ksh or +# bash, then to run this script, type that shell name before the whole +# command line, like: +# +# ksh Gradle +# +# Busybox and similar reduced shells will NOT work, because this script +# requires all of these POSIX shell features: +# * functions; +# * expansions «$var», «${var}», «${var:-default}», «${var+SET}», +# «${var#prefix}», «${var%suffix}», and «$( cmd )»; +# * compound commands having a testable exit status, especially «case»; +# * various built-in commands including «command», «set», and «ulimit». +# +# Important for patching: +# +# (2) This script targets any POSIX shell, so it avoids extensions provided +# by Bash, Ksh, etc; in particular arrays are avoided. +# +# The "traditional" practice of packing multiple parameters into a +# space-separated string is a well documented source of bugs and security +# problems, so this is (mostly) avoided, by progressively accumulating +# options in "$@", and eventually passing that to Java. +# +# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, +# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly; +# see the in-line comments for details. +# +# There are tweaks for specific operating systems such as AIX, CygWin, +# Darwin, MinGW, and NonStop. +# +# (3) This script is generated from the Groovy template +# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt +# within the Gradle project. +# +# You can find Gradle at https://github.com/gradle/gradle/. +# +############################################################################## + +# Attempt to set APP_HOME + +# Resolve links: $0 may be a link +app_path=$0 + +# Need this for daisy-chained symlinks. +while + APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path + [ -h "$app_path" ] +do + ls=$( ls -ld "$app_path" ) + link=${ls#*' -> '} + case $link in #( + /*) app_path=$link ;; #( + *) app_path=$APP_HOME$link ;; + esac +done + +# This is normally unused +# shellcheck disable=SC2034 +APP_BASE_NAME=${0##*/} +# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036) +APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD=maximum + +warn () { + echo "$*" +} >&2 + +die () { + echo + echo "$*" + echo + exit 1 +} >&2 + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "$( uname )" in #( + CYGWIN* ) cygwin=true ;; #( + Darwin* ) darwin=true ;; #( + MSYS* | MINGW* ) msys=true ;; #( + NONSTOP* ) nonstop=true ;; +esac + +CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD=$JAVA_HOME/jre/sh/java + else + JAVACMD=$JAVA_HOME/bin/java + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD=java + if ! command -v java >/dev/null 2>&1 + then + die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +fi + +# Increase the maximum file descriptors if we can. +if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then + case $MAX_FD in #( + max*) + # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + MAX_FD=$( ulimit -H -n ) || + warn "Could not query maximum file descriptor limit" + esac + case $MAX_FD in #( + '' | soft) :;; #( + *) + # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + ulimit -n "$MAX_FD" || + warn "Could not set maximum file descriptor limit to $MAX_FD" + esac +fi + +# Collect all arguments for the java command, stacking in reverse order: +# * args from the command line +# * the main class name +# * -classpath +# * -D...appname settings +# * --module-path (only if needed) +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. + +# For Cygwin or MSYS, switch paths to Windows format before running java +if "$cygwin" || "$msys" ; then + APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) + CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" ) + + JAVACMD=$( cygpath --unix "$JAVACMD" ) + + # Now convert the arguments - kludge to limit ourselves to /bin/sh + for arg do + if + case $arg in #( + -*) false ;; # don't mess with options #( + /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath + [ -e "$t" ] ;; #( + *) false ;; + esac + then + arg=$( cygpath --path --ignore --mixed "$arg" ) + fi + # Roll the args list around exactly as many times as the number of + # args, so each arg winds up back in the position where it started, but + # possibly modified. + # + # NB: a `for` loop captures its iteration list before it begins, so + # changing the positional parameters here affects neither the number of + # iterations, nor the values presented in `arg`. + shift # remove old arg + set -- "$@" "$arg" # push replacement arg + done +fi + + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' + +# Collect all arguments for the java command: +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments, +# and any embedded shellness will be escaped. +# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be +# treated as '${Hostname}' itself on the command line. + +set -- \ + "-Dorg.gradle.appname=$APP_BASE_NAME" \ + -classpath "$CLASSPATH" \ + org.gradle.wrapper.GradleWrapperMain \ + "$@" + +# Stop when "xargs" is not available. +if ! command -v xargs >/dev/null 2>&1 +then + die "xargs is not available" +fi + +# Use "xargs" to parse quoted args. +# +# With -n1 it outputs one arg per line, with the quotes and backslashes removed. +# +# In Bash we could simply go: +# +# readarray ARGS < <( xargs -n1 <<<"$var" ) && +# set -- "${ARGS[@]}" "$@" +# +# but POSIX shell has neither arrays nor command substitution, so instead we +# post-process each arg (as a line of input to sed) to backslash-escape any +# character that might be a shell metacharacter, then use eval to reverse +# that process (while maintaining the separation between arguments), and wrap +# the whole thing up as a single "set" statement. +# +# This will of course break if any of these variables contains a newline or +# an unmatched quote. +# + +eval "set -- $( + printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | + xargs -n1 | + sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | + tr '\n' ' ' + )" '"$@"' + +exec "$JAVACMD" "$@" diff --git a/gradlew.bat b/gradlew.bat new file mode 100644 index 0000000000..9b42019c79 --- /dev/null +++ b/gradlew.bat @@ -0,0 +1,94 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem +@rem SPDX-License-Identifier: Apache-2.0 +@rem + +@if "%DEBUG%"=="" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%"=="" set DIRNAME=. +@rem This is normally unused +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if %ERRORLEVEL% equ 0 goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +goto fail + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %* + +:end +@rem End local scope for the variables with windows NT shell +if %ERRORLEVEL% equ 0 goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +set EXIT_CODE=%ERRORLEVEL% +if %EXIT_CODE% equ 0 set EXIT_CODE=1 +if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE% +exit /b %EXIT_CODE% + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/integration-testing/.gitignore b/integration-testing/.gitignore new file mode 100644 index 0000000000..24d00437ce --- /dev/null +++ b/integration-testing/.gitignore @@ -0,0 +1,2 @@ +.kotlin +kotlin-js-store \ No newline at end of file diff --git a/integration-testing/README.md b/integration-testing/README.md new file mode 100644 index 0000000000..040ca684d3 --- /dev/null +++ b/integration-testing/README.md @@ -0,0 +1,17 @@ +# Integration tests + +This is a supplementary project that provides integration tests. + +The tests are the following: +* `mavenTest` depends on the published artifacts and tests artifacts binary content for absence of atomicfu in the classpath. +* `jvmCoreTest` miscellaneous tests that check the behaviour of `kotlinx-coroutines-core` dependency in a smoke manner. +* `coreAgentTest` checks that `kotlinx-coroutines-core` can be run as a Java agent. +* `debugAgentTest` checks that the coroutine debugger can be run as a Java agent. +* `debugDynamicAgentTest` checks that `kotlinx-coroutines-debug` agent can self-attach dynamically to JVM as a standalone dependency. +* `debugDynamicAgentJpmsTest` checks that `kotlinx-coroutines-debug` agent can self-attach dynamically to JVM as a standalone dependency (with JPMS) +* `smokeTest` builds the multiplatform test project that depends on coroutines. +* `java8Test` checks that some APIs built with Java 9+ can be used with Java 8. + +The `integration-testing` project is expected to be in a subdirectory of the main `kotlinx.coroutines` project. + +To run all the available tests: `./gradlew publishToMavenLocal` + `cd integration-testing` + `./gradlew check`. diff --git a/integration-testing/build.gradle.kts b/integration-testing/build.gradle.kts new file mode 100644 index 0000000000..dc68f14d36 --- /dev/null +++ b/integration-testing/build.gradle.kts @@ -0,0 +1,223 @@ +import org.jetbrains.kotlin.gradle.dsl.JvmTarget +import org.jetbrains.kotlin.gradle.tasks.KotlinCompile +import org.jetbrains.kotlin.gradle.tasks.KotlinJvmCompile +import org.jetbrains.kotlin.gradle.targets.js.npm.tasks.KotlinNpmInstallTask +import org.jetbrains.kotlin.gradle.dsl.jvm.JvmTargetValidationMode + +buildscript { + /* + These property group is used to build kotlinx.coroutines against Kotlin compiler snapshot. + How does it work: + When build_snapshot_train is set to true, kotlin_version property is overridden with kotlin_snapshot_version, + atomicfu_version is overwritten by TeamCity environment (AFU is built with snapshot and published to mavenLocal + as previous step or the snapshot build). + Additionally, mavenLocal and Sonatype snapshots are added to repository list and stress tests are disabled. + DO NOT change the name of these properties without adapting kotlinx.train build chain. + */ + fun checkIsSnapshotTrainProperty(): Boolean { + val buildSnapshotTrain = rootProject.properties["build_snapshot_train"]?.toString() + return !buildSnapshotTrain.isNullOrEmpty() + } + + fun checkIsSnapshotVersion(): Boolean { + var usingSnapshotVersion = checkIsSnapshotTrainProperty() + rootProject.properties.forEach { (key, value) -> + if (key.endsWith("_version") && value is String && value.endsWith("-SNAPSHOT")) { + println("NOTE: USING SNAPSHOT VERSION: $key=$value") + usingSnapshotVersion = true + } + } + return usingSnapshotVersion + } + + val usingSnapshotVersion = checkIsSnapshotVersion() + val hasSnapshotTrainProperty = checkIsSnapshotTrainProperty() + + extra.apply { + set("using_snapshot_version", usingSnapshotVersion) + set("build_snapshot_train", hasSnapshotTrainProperty) + } + + if (usingSnapshotVersion) { + repositories { + mavenLocal() + maven("https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev") + } + } +} + +plugins { + id("org.jetbrains.kotlin.jvm") version extra["kotlin_version"].toString() +} + +repositories { + if (extra["using_snapshot_version"] == true) { + maven("https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev") + } + mavenLocal() + mavenCentral() +} + +java { + sourceCompatibility = JavaVersion.VERSION_1_8 + targetCompatibility = JavaVersion.VERSION_1_8 +} + +val kotlinVersion = if (extra["build_snapshot_train"] == true) { + rootProject.properties["kotlin_snapshot_version"]?.toString() + ?: throw IllegalArgumentException("'kotlin_snapshot_version' should be defined when building with snapshot compiler") +} else { + rootProject.properties["kotlin_version"].toString() +} + +val asmVersion = property("asm_version") + +dependencies { + testImplementation("org.jetbrains.kotlin:kotlin-test:$kotlinVersion") + testImplementation("org.ow2.asm:asm:$asmVersion") + implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8") +} + +val coroutinesVersion = property("coroutines_version").toString() + +sourceSets { + // An assortment of tests for behavior of the core coroutines module on JVM + create("jvmCoreTest") { + compileClasspath += sourceSets.test.get().runtimeClasspath + runtimeClasspath += sourceSets.test.get().runtimeClasspath + + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutinesVersion") + implementation("com.google.guava:guava:31.1-jre") + } + } + + // Checks correctness of Maven publication (JAR resources) and absence of atomicfu symbols + create("mavenTest") { + compileClasspath += sourceSets.test.get().runtimeClasspath + runtimeClasspath += sourceSets.test.get().runtimeClasspath + + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutinesVersion") + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:$coroutinesVersion") + } + } + + // Checks that kotlinx-coroutines-debug can be used as -javaagent parameter + create("debugAgentTest") { + compileClasspath += sourceSets.test.get().runtimeClasspath + runtimeClasspath += sourceSets.test.get().runtimeClasspath + + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutinesVersion") + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-debug:$coroutinesVersion") + } + } + + // Checks that kotlinx-coroutines-debug agent can self-attach dynamically to JVM as a standalone dependency + create("debugDynamicAgentTest") { + compileClasspath += sourceSets.test.get().runtimeClasspath + runtimeClasspath += sourceSets.test.get().runtimeClasspath + + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutinesVersion") + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-debug:$coroutinesVersion") + } + } + + // Checks that kotlinx-coroutines-core can be used as -javaagent parameter + create("coreAgentTest") { + compileClasspath += sourceSets.test.get().runtimeClasspath + runtimeClasspath += sourceSets.test.get().runtimeClasspath + + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutinesVersion") + } + } +} + +kotlin { + jvmToolchain(17) + + compilerOptions { + jvmTarget.set(JvmTarget.JVM_17) + } +} + +tasks { + named("compileDebugAgentTestKotlin") { + compilerOptions { + freeCompilerArgs.add("-Xallow-kotlin-package") + jvmTarget.set(JvmTarget.JVM_1_8) + } + } + + named("compileTestKotlin") { + compilerOptions { + jvmTarget.set(JvmTarget.JVM_17) + } + } + + create("jvmCoreTest") { + environment("version", coroutinesVersion) + val sourceSet = sourceSets[name] + testClassesDirs = sourceSet.output.classesDirs + classpath = sourceSet.runtimeClasspath + } + + create("mavenTest") { + environment("version", coroutinesVersion) + val sourceSet = sourceSets[name] + testClassesDirs = sourceSet.output.classesDirs + classpath = sourceSet.runtimeClasspath + } + + create("debugAgentTest") { + val sourceSet = sourceSets[name] + val coroutinesDebugJar = sourceSet.runtimeClasspath.filter { + it.name == "kotlinx-coroutines-debug-$coroutinesVersion.jar" + }.singleFile + jvmArgs("-javaagent:$coroutinesDebugJar") + testClassesDirs = sourceSet.output.classesDirs + classpath = sourceSet.runtimeClasspath + systemProperties["overwrite.probes"] = project.properties["overwrite.probes"] + } + + create("debugDynamicAgentTest") { + val sourceSet = sourceSets[name] + testClassesDirs = sourceSet.output.classesDirs + classpath = sourceSet.runtimeClasspath + } + + create("coreAgentTest") { + val sourceSet = sourceSets[name] + val coroutinesDebugJar = sourceSet.runtimeClasspath.filter { + it.name == "kotlinx-coroutines-core-jvm-$coroutinesVersion.jar" + }.singleFile + jvmArgs("-javaagent:$coroutinesDebugJar") + testClassesDirs = sourceSet.output.classesDirs + classpath = sourceSet.runtimeClasspath + } + + check { + dependsOn( + "jvmCoreTest", + "debugDynamicAgentTest", + "mavenTest", + "debugAgentTest", + "coreAgentTest", + ":jpmsTest:check", + "smokeTest:build", + "java8Test:check" + ) + } + + // Drop this when node js version become stable + withType(KotlinNpmInstallTask::class.java).configureEach { + args.add("--ignore-engines") + } + + withType(KotlinJvmCompile::class.java).configureEach { + jvmTargetValidationMode = JvmTargetValidationMode.WARNING + } +} diff --git a/integration-testing/gradle.properties b/integration-testing/gradle.properties new file mode 100644 index 0000000000..56cac5b0ec --- /dev/null +++ b/integration-testing/gradle.properties @@ -0,0 +1,7 @@ +kotlin_version=2.1.0 +coroutines_version=1.10.2-SNAPSHOT +asm_version=9.3 +junit5_version=5.7.0 + +kotlin.code.style=official +kotlin.mpp.stability.nowarn=true diff --git a/integration-testing/gradle/wrapper/gradle-wrapper.jar b/integration-testing/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 0000000000..a4b76b9530 Binary files /dev/null and b/integration-testing/gradle/wrapper/gradle-wrapper.jar differ diff --git a/integration-testing/gradle/wrapper/gradle-wrapper.properties b/integration-testing/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 0000000000..e6aba2515d --- /dev/null +++ b/integration-testing/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,7 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-8.5-all.zip +networkTimeout=10000 +validateDistributionUrl=true +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists diff --git a/integration-testing/gradlew b/integration-testing/gradlew new file mode 100755 index 0000000000..f5feea6d6b --- /dev/null +++ b/integration-testing/gradlew @@ -0,0 +1,252 @@ +#!/bin/sh + +# +# Copyright © 2015-2021 the original authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +############################################################################## +# +# Gradle start up script for POSIX generated by Gradle. +# +# Important for running: +# +# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is +# noncompliant, but you have some other compliant shell such as ksh or +# bash, then to run this script, type that shell name before the whole +# command line, like: +# +# ksh Gradle +# +# Busybox and similar reduced shells will NOT work, because this script +# requires all of these POSIX shell features: +# * functions; +# * expansions «$var», «${var}», «${var:-default}», «${var+SET}», +# «${var#prefix}», «${var%suffix}», and «$( cmd )»; +# * compound commands having a testable exit status, especially «case»; +# * various built-in commands including «command», «set», and «ulimit». +# +# Important for patching: +# +# (2) This script targets any POSIX shell, so it avoids extensions provided +# by Bash, Ksh, etc; in particular arrays are avoided. +# +# The "traditional" practice of packing multiple parameters into a +# space-separated string is a well documented source of bugs and security +# problems, so this is (mostly) avoided, by progressively accumulating +# options in "$@", and eventually passing that to Java. +# +# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, +# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly; +# see the in-line comments for details. +# +# There are tweaks for specific operating systems such as AIX, CygWin, +# Darwin, MinGW, and NonStop. +# +# (3) This script is generated from the Groovy template +# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt +# within the Gradle project. +# +# You can find Gradle at https://github.com/gradle/gradle/. +# +############################################################################## + +# Attempt to set APP_HOME + +# Resolve links: $0 may be a link +app_path=$0 + +# Need this for daisy-chained symlinks. +while + APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path + [ -h "$app_path" ] +do + ls=$( ls -ld "$app_path" ) + link=${ls#*' -> '} + case $link in #( + /*) app_path=$link ;; #( + *) app_path=$APP_HOME$link ;; + esac +done + +# This is normally unused +# shellcheck disable=SC2034 +APP_BASE_NAME=${0##*/} +# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036) +APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s +' "$PWD" ) || exit + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD=maximum + +warn () { + echo "$*" +} >&2 + +die () { + echo + echo "$*" + echo + exit 1 +} >&2 + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "$( uname )" in #( + CYGWIN* ) cygwin=true ;; #( + Darwin* ) darwin=true ;; #( + MSYS* | MINGW* ) msys=true ;; #( + NONSTOP* ) nonstop=true ;; +esac + +CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD=$JAVA_HOME/jre/sh/java + else + JAVACMD=$JAVA_HOME/bin/java + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD=java + if ! command -v java >/dev/null 2>&1 + then + die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +fi + +# Increase the maximum file descriptors if we can. +if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then + case $MAX_FD in #( + max*) + # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + MAX_FD=$( ulimit -H -n ) || + warn "Could not query maximum file descriptor limit" + esac + case $MAX_FD in #( + '' | soft) :;; #( + *) + # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + ulimit -n "$MAX_FD" || + warn "Could not set maximum file descriptor limit to $MAX_FD" + esac +fi + +# Collect all arguments for the java command, stacking in reverse order: +# * args from the command line +# * the main class name +# * -classpath +# * -D...appname settings +# * --module-path (only if needed) +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. + +# For Cygwin or MSYS, switch paths to Windows format before running java +if "$cygwin" || "$msys" ; then + APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) + CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" ) + + JAVACMD=$( cygpath --unix "$JAVACMD" ) + + # Now convert the arguments - kludge to limit ourselves to /bin/sh + for arg do + if + case $arg in #( + -*) false ;; # don't mess with options #( + /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath + [ -e "$t" ] ;; #( + *) false ;; + esac + then + arg=$( cygpath --path --ignore --mixed "$arg" ) + fi + # Roll the args list around exactly as many times as the number of + # args, so each arg winds up back in the position where it started, but + # possibly modified. + # + # NB: a `for` loop captures its iteration list before it begins, so + # changing the positional parameters here affects neither the number of + # iterations, nor the values presented in `arg`. + shift # remove old arg + set -- "$@" "$arg" # push replacement arg + done +fi + + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' + +# Collect all arguments for the java command: +# * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments, +# and any embedded shellness will be escaped. +# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be +# treated as '${Hostname}' itself on the command line. + +set -- \ + "-Dorg.gradle.appname=$APP_BASE_NAME" \ + -classpath "$CLASSPATH" \ + org.gradle.wrapper.GradleWrapperMain \ + "$@" + +# Stop when "xargs" is not available. +if ! command -v xargs >/dev/null 2>&1 +then + die "xargs is not available" +fi + +# Use "xargs" to parse quoted args. +# +# With -n1 it outputs one arg per line, with the quotes and backslashes removed. +# +# In Bash we could simply go: +# +# readarray ARGS < <( xargs -n1 <<<"$var" ) && +# set -- "${ARGS[@]}" "$@" +# +# but POSIX shell has neither arrays nor command substitution, so instead we +# post-process each arg (as a line of input to sed) to backslash-escape any +# character that might be a shell metacharacter, then use eval to reverse +# that process (while maintaining the separation between arguments), and wrap +# the whole thing up as a single "set" statement. +# +# This will of course break if any of these variables contains a newline or +# an unmatched quote. +# + +eval "set -- $( + printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | + xargs -n1 | + sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | + tr '\n' ' ' + )" '"$@"' + +exec "$JAVACMD" "$@" diff --git a/integration-testing/gradlew.bat b/integration-testing/gradlew.bat new file mode 100644 index 0000000000..9d21a21834 --- /dev/null +++ b/integration-testing/gradlew.bat @@ -0,0 +1,94 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem +@rem SPDX-License-Identifier: Apache-2.0 +@rem + +@if "%DEBUG%"=="" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%"=="" set DIRNAME=. +@rem This is normally unused +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if %ERRORLEVEL% equ 0 goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +goto fail + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %* + +:end +@rem End local scope for the variables with windows NT shell +if %ERRORLEVEL% equ 0 goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +set EXIT_CODE=%ERRORLEVEL% +if %EXIT_CODE% equ 0 set EXIT_CODE=1 +if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE% +exit /b %EXIT_CODE% + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/integration-testing/java8Test/build.gradle.kts b/integration-testing/java8Test/build.gradle.kts new file mode 100644 index 0000000000..ada90adaab --- /dev/null +++ b/integration-testing/java8Test/build.gradle.kts @@ -0,0 +1,25 @@ +plugins { + kotlin("jvm") +} + +repositories { + mavenCentral() + maven("https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev") + // Coroutines from the outer project are published by previous CI buils step + mavenLocal() +} + +tasks.test { + useJUnitPlatform() +} + +val coroutinesVersion = property("coroutines_version") +val junit5Version = property("junit5_version") + +kotlin { + jvmToolchain(8) + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-debug:$coroutinesVersion") + testImplementation("org.junit.jupiter:junit-jupiter-engine:$junit5Version") + } +} diff --git a/integration-testing/java8Test/src/test/kotlin/JUnit5TimeoutCompilation.kt b/integration-testing/java8Test/src/test/kotlin/JUnit5TimeoutCompilation.kt new file mode 100644 index 0000000000..573a7a5c7d --- /dev/null +++ b/integration-testing/java8Test/src/test/kotlin/JUnit5TimeoutCompilation.kt @@ -0,0 +1,9 @@ +import kotlinx.coroutines.debug.junit5.CoroutinesTimeout +import org.junit.jupiter.api.* + +class JUnit5TimeoutCompilation { + @CoroutinesTimeout(1000) + @Test + fun testCoroutinesTimeoutNotFailing() { + } +} diff --git a/integration-testing/jpmsTest/build.gradle.kts b/integration-testing/jpmsTest/build.gradle.kts new file mode 100644 index 0000000000..f96f99822f --- /dev/null +++ b/integration-testing/jpmsTest/build.gradle.kts @@ -0,0 +1,47 @@ +@file:Suppress("PropertyName") +plugins { + kotlin("jvm") +} + +val coroutines_version: String by project + +repositories { + if (project.properties["build_snapshot_train"]?.toString()?.toBoolean() == true) { + maven("https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev") + } + mavenLocal() + mavenCentral() +} + +java { + modularity.inferModulePath.set(true) +} + +kotlin { + jvmToolchain(17) + + val test = target.compilations.getByName("test") + target.compilations.create("debugDynamicAgentJpmsTest") { + associateWith(test) + + + defaultSourceSet.dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutines_version") + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-debug:$coroutines_version") + } + + tasks.register("debugDynamicAgentJpmsTest") { + testClassesDirs = output.classesDirs + classpath = javaSourceSet.runtimeClasspath + } + } +} + +tasks.named("check") { + dependsOn(tasks.withType()) +} + +dependencies { + testImplementation(kotlin("test-junit")) +} + diff --git a/integration-testing/jpmsTest/src/debugDynamicAgentJpmsTest/java/module-info.java b/integration-testing/jpmsTest/src/debugDynamicAgentJpmsTest/java/module-info.java new file mode 100644 index 0000000000..180d85c36a --- /dev/null +++ b/integration-testing/jpmsTest/src/debugDynamicAgentJpmsTest/java/module-info.java @@ -0,0 +1,7 @@ +module debug.dynamic.agent.jpms.test { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.coroutines.debug; + requires junit; + requires kotlin.test; +} \ No newline at end of file diff --git a/integration-testing/jpmsTest/src/debugDynamicAgentJpmsTest/kotlin/DynamicAttachDebugJpmsTest.kt b/integration-testing/jpmsTest/src/debugDynamicAgentJpmsTest/kotlin/DynamicAttachDebugJpmsTest.kt new file mode 100644 index 0000000000..dcfcd0e106 --- /dev/null +++ b/integration-testing/jpmsTest/src/debugDynamicAgentJpmsTest/kotlin/DynamicAttachDebugJpmsTest.kt @@ -0,0 +1,54 @@ +@file:OptIn(ExperimentalCoroutinesApi::class) + +import org.junit.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* +import org.junit.Ignore +import org.junit.Test +import java.io.* +import java.lang.IllegalStateException +import kotlin.test.* + +class DynamicAttachDebugJpmsTest { + + /** + * This test is disabled because: + * Dynamic Attach with JPMS is not yet supported. + * + * Here is the state of experiments: + * When launching this test with additional workarounds like + * ``` + * jvmArgs("--add-exports=kotlinx.coroutines.debug/kotlinx.coroutines.repackaged.net.bytebuddy=com.sun.jna") + * jvmArgs("--add-exports=kotlinx.coroutines.debug/kotlinx.coroutines.repackaged.net.bytebuddy.agent=com.sun.jna") + *``` + * + * Then we see issues like + * + * ``` + * Caused by: java.lang.IllegalStateException: The Byte Buddy agent is not loaded or this method is not called via the system class loader + * at kotlinx.coroutines.debug/kotlinx.coroutines.repackaged.net.bytebuddy.agent.Installer.getInstrumentation(Installer.java:61) + * ... 54 more + * ``` + */ + @Ignore("shaded byte-buddy does not work with JPMS") + @Test + fun testAgentDumpsCoroutines() = + DebugProbes.withDebugProbes { + runBlocking { + val baos = ByteArrayOutputStream() + DebugProbes.dumpCoroutines(PrintStream(baos)) + // if the agent works, then dumps should contain something, + // at least the fact that this test is running. + Assert.assertTrue(baos.toString().contains("testAgentDumpsCoroutines")) + } + } + + @Test + fun testAgentIsNotInstalled() { + assertEquals(false, DebugProbes.isInstalled) + assertFailsWith { + DebugProbes.dumpCoroutines(PrintStream(ByteArrayOutputStream())) + } + } + +} diff --git a/integration-testing/settings.gradle.kts b/integration-testing/settings.gradle.kts new file mode 100644 index 0000000000..f6711b86c0 --- /dev/null +++ b/integration-testing/settings.gradle.kts @@ -0,0 +1,14 @@ +pluginManagement { + repositories { + mavenCentral() + maven("https://plugins.gradle.org/m2/") + maven("https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev") + mavenLocal() + } +} + +include("smokeTest") +include("java8Test") +include(":jpmsTest") + +rootProject.name = "kotlinx-coroutines-integration-testing" diff --git a/integration-testing/smokeTest/build.gradle.kts b/integration-testing/smokeTest/build.gradle.kts new file mode 100644 index 0000000000..3739c7209e --- /dev/null +++ b/integration-testing/smokeTest/build.gradle.kts @@ -0,0 +1,89 @@ +import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl +import org.jetbrains.kotlin.gradle.dsl.HasConfigurableKotlinCompilerOptions +import org.jetbrains.kotlin.gradle.dsl.JvmTarget +import org.jetbrains.kotlin.gradle.dsl.KotlinJvmCompilerOptions +import org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootExtension + +plugins { + id("org.jetbrains.kotlin.multiplatform") +} + +repositories { + mavenCentral() + maven("https://maven.pkg.jetbrains.space/kotlin/p/kotlin/dev") + // Coroutines from the outer project are published by previous CI builds step + mavenLocal() +} + +kotlin { + jvm() + js(IR) { + nodejs() + } + @OptIn(ExperimentalWasmDsl::class) + wasmJs { + nodejs() + } + + macosArm64() + macosX64() + linuxArm64() + linuxX64() + mingwX64() + + val coroutinesVersion = property("coroutines_version") + + sourceSets { + commonMain { + dependencies { + implementation(kotlin("stdlib-common")) + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:$coroutinesVersion") + } + } + commonTest { + dependencies { + implementation(kotlin("test-common")) + implementation(kotlin("test-annotations-common")) + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:$coroutinesVersion") + } + } + jsTest { + dependencies { + implementation(kotlin("test-js")) + } + } + wasmJsTest { + dependencies { + implementation(kotlin("test-wasm-js")) + } + } + wasmWasiTest { + dependencies { + implementation(kotlin("test-wasm-wasi")) + } + } + jvmTest { + dependencies { + implementation(kotlin("test")) + implementation(kotlin("test-junit")) + } + } + } + + targets.all { + val kotlinCompilerTaskName = compilations.getByName("main").compileKotlinTaskName + @Suppress("UNCHECKED_CAST") + val kotlinCompilerTask = tasks.getByName(kotlinCompilerTaskName) as? HasConfigurableKotlinCompilerOptions + kotlinCompilerTask?.compilerOptions { + jvmTarget.set(JvmTarget.JVM_1_8) + } + } +} + +// Drop this configuration when the Node.JS version in KGP will support wasm gc milestone 4 +// check it here: +// https://github.com/JetBrains/kotlin/blob/master/libraries/tools/kotlin-gradle-plugin/src/common/kotlin/org/jetbrains/kotlin/gradle/targets/js/nodejs/NodeJsRootExtension.kt +rootProject.extensions.findByType(NodeJsRootExtension::class.java)?.apply { + nodeVersion = "21.0.0-v8-canary202309167e82ab1fa2" + nodeDownloadBaseUrl = "https://nodejs.org/download/v8-canary" +} \ No newline at end of file diff --git a/integration-testing/smokeTest/src/commonMain/kotlin/Sample.kt b/integration-testing/smokeTest/src/commonMain/kotlin/Sample.kt new file mode 100644 index 0000000000..c5da677bb1 --- /dev/null +++ b/integration-testing/smokeTest/src/commonMain/kotlin/Sample.kt @@ -0,0 +1,9 @@ +import kotlinx.coroutines.* + +suspend fun doWorld() = coroutineScope { + launch { + delay(1000L) + println("World!") + } + println("Hello") +} diff --git a/integration-testing/smokeTest/src/commonTest/kotlin/SampleTest.kt b/integration-testing/smokeTest/src/commonTest/kotlin/SampleTest.kt new file mode 100644 index 0000000000..98366d87a8 --- /dev/null +++ b/integration-testing/smokeTest/src/commonTest/kotlin/SampleTest.kt @@ -0,0 +1,9 @@ +import kotlinx.coroutines.test.* +import kotlin.test.* + +class SampleTest { + @Test + fun test() = runTest { + doWorld() + } +} diff --git a/integration-testing/src/coreAgentTest/kotlin/CoreAgentTest.kt b/integration-testing/src/coreAgentTest/kotlin/CoreAgentTest.kt new file mode 100644 index 0000000000..3b4dc3468f --- /dev/null +++ b/integration-testing/src/coreAgentTest/kotlin/CoreAgentTest.kt @@ -0,0 +1,19 @@ +import org.junit.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* +import org.junit.Test +import java.io.* + +class CoreAgentTest { + + @Test + fun testAgentDumpsCoroutines() = runBlocking { + val baos = ByteArrayOutputStream() + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") + DebugProbesImpl.dumpCoroutines(PrintStream(baos)) + // if the agent works, then dumps should contain something, + // at least the fact that this test is running. + Assert.assertTrue(baos.toString().contains("testAgentDumpsCoroutines")) + } + +} diff --git a/integration-testing/src/debugAgentTest/kotlin/DebugAgentTest.kt b/integration-testing/src/debugAgentTest/kotlin/DebugAgentTest.kt new file mode 100644 index 0000000000..64ed981e7e --- /dev/null +++ b/integration-testing/src/debugAgentTest/kotlin/DebugAgentTest.kt @@ -0,0 +1,20 @@ +@file:OptIn(ExperimentalCoroutinesApi::class) + +import org.junit.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* +import org.junit.Test +import java.io.* + +class DebugAgentTest { + + @Test + fun testAgentDumpsCoroutines() = runBlocking { + val baos = ByteArrayOutputStream() + DebugProbes.dumpCoroutines(PrintStream(baos)) + // if the agent works, then dumps should contain something, + // at least the fact that this test is running. + Assert.assertTrue(baos.toString().contains("testAgentDumpsCoroutines")) + } + +} diff --git a/integration-testing/src/debugAgentTest/kotlin/DebugProbes.kt b/integration-testing/src/debugAgentTest/kotlin/DebugProbes.kt new file mode 100644 index 0000000000..849d4d4e3c --- /dev/null +++ b/integration-testing/src/debugAgentTest/kotlin/DebugProbes.kt @@ -0,0 +1,11 @@ +@file:Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +package kotlin.coroutines.jvm.internal + +import kotlinx.coroutines.debug.internal.* +import kotlin.coroutines.* + +internal fun probeCoroutineCreated(completion: Continuation): Continuation = DebugProbesImpl.probeCoroutineCreated(completion) + +internal fun probeCoroutineResumed(frame: Continuation<*>) = DebugProbesImpl.probeCoroutineResumed(frame) + +internal fun probeCoroutineSuspended(frame: Continuation<*>) = DebugProbesImpl.probeCoroutineSuspended(frame) diff --git a/integration-testing/src/debugAgentTest/kotlin/PrecompiledDebugProbesTest.kt b/integration-testing/src/debugAgentTest/kotlin/PrecompiledDebugProbesTest.kt new file mode 100644 index 0000000000..2a63dffb41 --- /dev/null +++ b/integration-testing/src/debugAgentTest/kotlin/PrecompiledDebugProbesTest.kt @@ -0,0 +1,46 @@ +import org.junit.Test +import java.io.* +import kotlin.test.* + +/* + * This is intentionally put here instead of coreAgentTest to avoid accidental classpath replacing + * and ruining core agent test. + */ +class PrecompiledDebugProbesTest { + + private val overwrite = java.lang.Boolean.getBoolean("overwrite.probes") + + @Test + fun testClassFileContent() { + val clz = Class.forName("kotlin.coroutines.jvm.internal.DebugProbesKt") + val classFileResourcePath = clz.name.replace(".", "/") + ".class" + val array = clz.classLoader.getResourceAsStream(classFileResourcePath).use { it.readBytes() } + assertJava8Compliance(array) + // we expect the integration testing project to be in a subdirectory of the main kotlinx.coroutines project + val base = File("").absoluteFile.parentFile + val probes = File(base, "kotlinx-coroutines-core/jvm/resources/DebugProbesKt.bin") + val binContent = probes.readBytes() + if (overwrite) { + FileOutputStream(probes).use { it.write(array) } + println("Content was successfully overwritten!") + } else { + assertTrue( + array.contentEquals(binContent), + "Compiled DebugProbesKt.class does not match the file shipped as a resource in kotlinx-coroutines-core. " + + "Typically it happens because of the Kotlin version update (-> binary metadata). " + + "In that case, run the same test with -Poverwrite.probes=true." + ) + } + } + + private fun assertJava8Compliance(classBytes: ByteArray) { + DataInputStream(classBytes.inputStream()).use { + val magic: Int = it.readInt() + if (magic != -0x35014542) throw IllegalArgumentException("Not a valid class!") + val minor: Int = it.readUnsignedShort() + val major: Int = it.readUnsignedShort() + assertEquals(52, major) + assertEquals(0, minor) + } + } +} diff --git a/integration-testing/src/debugDynamicAgentTest/kotlin/DynamicAttachDebugTest.kt b/integration-testing/src/debugDynamicAgentTest/kotlin/DynamicAttachDebugTest.kt new file mode 100644 index 0000000000..046951955e --- /dev/null +++ b/integration-testing/src/debugDynamicAgentTest/kotlin/DynamicAttachDebugTest.kt @@ -0,0 +1,26 @@ +import org.junit.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* +import org.junit.Test +import java.io.* +import java.lang.IllegalStateException + +class DynamicAttachDebugTest { + + @Test + fun testAgentDumpsCoroutines() = + DebugProbes.withDebugProbes { + runBlocking { + val baos = ByteArrayOutputStream() + DebugProbes.dumpCoroutines(PrintStream(baos)) + // if the agent works, then dumps should contain something, + // at least the fact that this test is running. + Assert.assertTrue(baos.toString().contains("testAgentDumpsCoroutines")) + } + } + + @Test(expected = IllegalStateException::class) + fun testAgentIsNotInstalled() { + DebugProbes.dumpCoroutines(PrintStream(ByteArrayOutputStream())) + } +} diff --git a/integration-testing/src/jvmCoreTest/kotlin/Jdk8InCoreIntegration.kt b/integration-testing/src/jvmCoreTest/kotlin/Jdk8InCoreIntegration.kt new file mode 100644 index 0000000000..a733f0630e --- /dev/null +++ b/integration-testing/src/jvmCoreTest/kotlin/Jdk8InCoreIntegration.kt @@ -0,0 +1,18 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.future.* +import org.junit.Test +import kotlin.test.* + +/* + * Integration test that ensures signatures from both the jdk8 and the core source sets of the kotlinx-coroutines-core subproject are used. + */ +class Jdk8InCoreIntegration { + + @Test + fun testFuture() = runBlocking { + val future = future { yield(); 42 } + future.whenComplete { r, _ -> assertEquals(42, r) } + assertEquals(42, future.await()) + } +} diff --git a/integration-testing/src/jvmCoreTest/kotlin/ListAllCoroutineThrowableSubclassesTest.kt b/integration-testing/src/jvmCoreTest/kotlin/ListAllCoroutineThrowableSubclassesTest.kt new file mode 100644 index 0000000000..65fec8c0b1 --- /dev/null +++ b/integration-testing/src/jvmCoreTest/kotlin/ListAllCoroutineThrowableSubclassesTest.kt @@ -0,0 +1,55 @@ +package kotlinx.coroutines + +import com.google.common.reflect.* +import kotlinx.coroutines.* +import org.junit.Test +import java.io.Serializable +import java.lang.reflect.Modifier +import kotlin.test.* + +class ListAllCoroutineThrowableSubclassesTest { + + /* + * These are all the known throwables in kotlinx.coroutines. + * If you add one, this test will fail to make + * you ensure your exception type is java.io.Serializable. + * + * We do not have means to check it automatically, so checks are delegated to humans. + * + * See #3328 for serialization rationale. + */ + private val knownThrowables = setOf( + "kotlinx.coroutines.TimeoutCancellationException", + "kotlinx.coroutines.JobCancellationException", + "kotlinx.coroutines.internal.UndeliveredElementException", + "kotlinx.coroutines.CompletionHandlerException", + "kotlinx.coroutines.internal.DiagnosticCoroutineContextException", + "kotlinx.coroutines.internal.ExceptionSuccessfullyProcessed", + "kotlinx.coroutines.CoroutinesInternalError", + "kotlinx.coroutines.DispatchException", + "kotlinx.coroutines.channels.ClosedSendChannelException", + "kotlinx.coroutines.channels.ClosedReceiveChannelException", + "kotlinx.coroutines.flow.internal.ChildCancelledException", + "kotlinx.coroutines.flow.internal.AbortFlowException", + "kotlinx.coroutines.debug.junit5.CoroutinesTimeoutException", + ) + + @Test + fun testThrowableSubclassesAreSerializable() { + val classes = ClassPath.from(this.javaClass.classLoader) + .getTopLevelClassesRecursive("kotlinx.coroutines") + // Not in the classpath: requires explicit dependency + .filter { it.name != "kotlinx.coroutines.debug.CoroutinesBlockHoundIntegration" + && it.name != "kotlinx.coroutines.debug.junit5.CoroutinesTimeoutExtension" }; + val throwables = classes.filter { Throwable::class.java.isAssignableFrom(it.load()) }.map { it.toString() } + for (throwable in throwables) { + for (field in throwable.javaClass.declaredFields) { + if (Modifier.isStatic(field.modifiers)) continue + val type = field.type + assertTrue(type.isPrimitive || Serializable::class.java.isAssignableFrom(type), + "Throwable $throwable has non-serializable field $field") + } + } + assertEquals(knownThrowables.sorted(), throwables.sorted()) + } +} diff --git a/integration-testing/src/mavenTest/kotlin/MavenPublicationAtomicfuValidator.kt b/integration-testing/src/mavenTest/kotlin/MavenPublicationAtomicfuValidator.kt new file mode 100644 index 0000000000..a1449c1d42 --- /dev/null +++ b/integration-testing/src/mavenTest/kotlin/MavenPublicationAtomicfuValidator.kt @@ -0,0 +1,73 @@ +package kotlinx.coroutines.validator + +import org.junit.Test +import org.objectweb.asm.* +import org.objectweb.asm.ClassReader.* +import org.objectweb.asm.ClassWriter.* +import org.objectweb.asm.Opcodes.* +import java.util.jar.* +import kotlin.test.* + +class MavenPublicationAtomicfuValidator { + private val ATOMIC_FU_REF = "Lkotlinx/atomicfu/".toByteArray() + private val KOTLIN_METADATA_DESC = "Lkotlin/Metadata;" + + @Test + fun testNoAtomicfuInClasspath() { + val result = runCatching { Class.forName("kotlinx.atomicfu.AtomicInt") } + assertTrue(result.exceptionOrNull() is ClassNotFoundException) + } + + @Test + fun testNoAtomicfuInMppJar() { + val clazz = Class.forName("kotlinx.coroutines.Job") + JarFile(clazz.protectionDomain.codeSource.location.file).checkForAtomicFu() + } + + @Test + fun testNoAtomicfuInAndroidJar() { + val clazz = Class.forName("kotlinx.coroutines.android.HandlerDispatcher") + JarFile(clazz.protectionDomain.codeSource.location.file).checkForAtomicFu() + } + + private fun JarFile.checkForAtomicFu() { + val foundClasses = mutableListOf() + for (e in entries()) { + if (!e.name.endsWith(".class")) continue + val bytes = getInputStream(e).use { it.readBytes() } + // The atomicfu compiler plugin does not remove atomic properties from metadata, + // so for now we check that there are no ATOMIC_FU_REF left in the class bytecode excluding metadata. + // This may be reverted after the fix in the compiler plugin transformer (for Kotlin 1.8.0). + val outBytes = bytes.eraseMetadata() + if (outBytes.checkBytes()) { + foundClasses += e.name // report error at the end with all class names + } + } + if (foundClasses.isNotEmpty()) { + error("Found references to atomicfu in jar file $name in the following class files: ${ + foundClasses.joinToString("") { "\n\t\t" + it } + }") + } + close() + } + + private fun ByteArray.checkBytes(): Boolean { + loop@for (i in 0 until this.size - ATOMIC_FU_REF.size) { + for (j in 0 until ATOMIC_FU_REF.size) { + if (this[i + j] != ATOMIC_FU_REF[j]) continue@loop + } + return true + } + return false + } + + private fun ByteArray.eraseMetadata(): ByteArray { + val cw = ClassWriter(COMPUTE_MAXS or COMPUTE_FRAMES) + ClassReader(this).accept(object : ClassVisitor(ASM9, cw) { + override fun visitAnnotation(descriptor: String?, visible: Boolean): AnnotationVisitor? { + return if (descriptor == KOTLIN_METADATA_DESC) null else super.visitAnnotation(descriptor, visible) + } + }, SKIP_FRAMES) + return cw.toByteArray() + } +} diff --git a/integration-testing/src/mavenTest/kotlin/MavenPublicationMetaInfValidator.kt b/integration-testing/src/mavenTest/kotlin/MavenPublicationMetaInfValidator.kt new file mode 100644 index 0000000000..acbc77376c --- /dev/null +++ b/integration-testing/src/mavenTest/kotlin/MavenPublicationMetaInfValidator.kt @@ -0,0 +1,66 @@ +package kotlinx.coroutines.validator + +import org.junit.Test +import org.objectweb.asm.* +import org.objectweb.asm.ClassReader.* +import org.objectweb.asm.ClassWriter.* +import org.objectweb.asm.Opcodes.* +import java.util.jar.* +import kotlin.test.* + +class MavenPublicationMetaInfValidator { + + @Test + fun testMetaInfCoreStructure() { + val clazz = Class.forName("kotlinx.coroutines.Job") + JarFile(clazz.protectionDomain.codeSource.location.file).checkMetaInfStructure( + setOf( + "MANIFEST.MF", + "kotlinx-coroutines-core.kotlin_module", + "com.android.tools/proguard/coroutines.pro", + "com.android.tools/r8/coroutines.pro", + "proguard/coroutines.pro", + "versions/9/module-info.class", + "kotlinx_coroutines_core.version" + ) + ) + } + + @Test + fun testMetaInfAndroidStructure() { + val clazz = Class.forName("kotlinx.coroutines.android.HandlerDispatcher") + JarFile(clazz.protectionDomain.codeSource.location.file).checkMetaInfStructure( + setOf( + "MANIFEST.MF", + "kotlinx-coroutines-android.kotlin_module", + "services/kotlinx.coroutines.CoroutineExceptionHandler", + "services/kotlinx.coroutines.internal.MainDispatcherFactory", + "com.android.tools/r8-from-1.6.0/coroutines.pro", + "com.android.tools/r8-upto-3.0.0/coroutines.pro", + "com.android.tools/proguard/coroutines.pro", + "proguard/coroutines.pro", + "versions/9/module-info.class", + "kotlinx_coroutines_android.version" + ) + ) + } + + private fun JarFile.checkMetaInfStructure(expected: Set) { + val actual = HashSet() + for (e in entries()) { + if (e.isDirectory() || !e.name.contains("META-INF")) { + continue + } + val partialName = e.name.substringAfter("META-INF/") + actual.add(partialName) + } + + if (actual != expected) { + val intersection = actual.intersect(expected) + val mismatch = actual.subtract(intersection) + expected.subtract(intersection) + fail("Mismatched files: " + mismatch) + } + + close() + } +} diff --git a/integration-testing/src/mavenTest/kotlin/MavenPublicationVersionValidator.kt b/integration-testing/src/mavenTest/kotlin/MavenPublicationVersionValidator.kt new file mode 100644 index 0000000000..0728c41604 --- /dev/null +++ b/integration-testing/src/mavenTest/kotlin/MavenPublicationVersionValidator.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.validator + +import org.junit.Test +import java.util.jar.* +import kotlin.test.* + +class MavenPublicationVersionValidator { + + @Test + fun testMppJar() { + val clazz = Class.forName("kotlinx.coroutines.Job") + JarFile(clazz.protectionDomain.codeSource.location.file).checkForVersion("kotlinx_coroutines_core.version") + } + + @Test + fun testAndroidJar() { + val clazz = Class.forName("kotlinx.coroutines.android.HandlerDispatcher") + JarFile(clazz.protectionDomain.codeSource.location.file).checkForVersion("kotlinx_coroutines_android.version") + } + + private fun JarFile.checkForVersion(file: String) { + val actualFile = "META-INF/$file" + val version = System.getenv("version") + use { + for (e in entries()) { + if (e.name == actualFile) { + val string = getInputStream(e).readAllBytes().decodeToString() + assertEquals(version, string) + return + } + } + error("File $file not found") + } + } +} diff --git a/integration/README.md b/integration/README.md new file mode 100644 index 0000000000..1d61ba4d6e --- /dev/null +++ b/integration/README.md @@ -0,0 +1,10 @@ +# Coroutines integration + +This directory contains modules that provide integration with various asynchronous callback- and future-based libraries. +Module names below correspond to the artifact names in Maven/Gradle. + +## Modules + +* [kotlinx-coroutines-guava](kotlinx-coroutines-guava/README.md) -- integration with Guava [ListenableFuture](https://github.com/google/guava/wiki/ListenableFutureExplained). +* [kotlinx-coroutines-slf4j](kotlinx-coroutines-slf4j/README.md) -- integration with SLF4J [MDC](https://logback.qos.ch/manual/mdc.html). +* [kotlinx-coroutines-play-services](kotlinx-coroutines-play-services) -- integration with Google Play Services [Tasks API](https://developers.google.com/android/guides/tasks). diff --git a/integration/kotlinx-coroutines-guava/README.md b/integration/kotlinx-coroutines-guava/README.md new file mode 100644 index 0000000000..c84fb8610c --- /dev/null +++ b/integration/kotlinx-coroutines-guava/README.md @@ -0,0 +1,67 @@ +# Module kotlinx-coroutines-guava + +Integration with Guava [ListenableFuture](https://github.com/google/guava/wiki/ListenableFutureExplained). + +Coroutine builders: + +| **Name** | **Result** | **Scope** | **Description** +| -------- | ---------- | ---------- | --------------- +| [future] | [ListenableFuture][com.google.common.util.concurrent.ListenableFuture] | [CoroutineScope] | Returns a single value with the future result + +Extension functions: + +| **Name** | **Description** +| -------- | --------------- +| [ListenableFuture.await][com.google.common.util.concurrent.ListenableFuture.await] | Awaits for completion of the future (cancellable) +| [Deferred.asListenableFuture][kotlinx.coroutines.Deferred.asListenableFuture] | Converts a deferred value to the future + +## Example + +Given the following functions defined in some Java API based on Guava: + +```java +public ListenableFuture loadImageAsync(String name); // starts async image loading +public Image combineImages(Image image1, Image image2); // synchronously combines two images using some algorithm +``` + +We can consume this API from Kotlin coroutine to load two images and combine then asynchronously. +The resulting function returns `ListenableFuture` for ease of use back from Guava-based Java code. + +```kotlin +fun combineImagesAsync(name1: String, name2: String): ListenableFuture = future { + val future1 = loadImageAsync(name1) // start loading first image + val future2 = loadImageAsync(name2) // start loading second image + combineImages(future1.await(), future2.await()) // wait for both, combine, and return result +} +``` + +Note that this module should be used only for integration with existing Java APIs based on `ListenableFuture`. +Writing pure-Kotlin code that uses `ListenableFuture` is highly not recommended, since the resulting APIs based +on the futures are quite error-prone. See the discussion on +[Asynchronous Programming Styles](https://github.com/Kotlin/kotlin-coroutines/blob/master/kotlin-coroutines-informal.md#asynchronous-programming-styles) +for details on general problems pertaining to any future-based API and keep in mind that `ListenableFuture` exposes +a _blocking_ method +[get](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Future.html#get--) +that makes it especially bad choice for coroutine-based Kotlin code. + +# Package kotlinx.coroutines.future + +Integration with Guava [ListenableFuture](https://github.com/google/guava/wiki/ListenableFutureExplained). + + + + +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html + + + + +[future]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-guava/kotlinx.coroutines.guava/future.html +[com.google.common.util.concurrent.ListenableFuture.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-guava/kotlinx.coroutines.guava/await.html +[kotlinx.coroutines.Deferred.asListenableFuture]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-guava/kotlinx.coroutines.guava/as-listenable-future.html + + + +[com.google.common.util.concurrent.ListenableFuture]: https://kotlinlang.org/api/kotlinx.coroutines/https://google.github.io/guava/releases/31.0.1-jre/api/docs/com/google/common/util/concurrent/ListenableFuture.html + + diff --git a/integration/kotlinx-coroutines-guava/api/kotlinx-coroutines-guava.api b/integration/kotlinx-coroutines-guava/api/kotlinx-coroutines-guava.api new file mode 100644 index 0000000000..26bc229e9d --- /dev/null +++ b/integration/kotlinx-coroutines-guava/api/kotlinx-coroutines-guava.api @@ -0,0 +1,8 @@ +public final class kotlinx/coroutines/guava/ListenableFutureKt { + public static final fun asDeferred (Lcom/google/common/util/concurrent/ListenableFuture;)Lkotlinx/coroutines/Deferred; + public static final fun asListenableFuture (Lkotlinx/coroutines/Deferred;)Lcom/google/common/util/concurrent/ListenableFuture; + public static final fun await (Lcom/google/common/util/concurrent/ListenableFuture;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun future (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;)Lcom/google/common/util/concurrent/ListenableFuture; + public static synthetic fun future$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lcom/google/common/util/concurrent/ListenableFuture; +} + diff --git a/integration/kotlinx-coroutines-guava/build.gradle.kts b/integration/kotlinx-coroutines-guava/build.gradle.kts new file mode 100644 index 0000000000..72f34e9ddd --- /dev/null +++ b/integration/kotlinx-coroutines-guava/build.gradle.kts @@ -0,0 +1,14 @@ +val guavaVersion = "31.0.1-jre" + +dependencies { + api("com.google.guava:guava:$guavaVersion") +} + +java { + targetCompatibility = JavaVersion.VERSION_1_8 + sourceCompatibility = JavaVersion.VERSION_1_8 +} + +externalDocumentationLink( + url = "https://google.github.io/guava/releases/$guavaVersion/api/docs/" +) diff --git a/integration/kotlinx-coroutines-guava/package.list b/integration/kotlinx-coroutines-guava/package.list new file mode 100644 index 0000000000..9ad26f4474 --- /dev/null +++ b/integration/kotlinx-coroutines-guava/package.list @@ -0,0 +1,16 @@ +com.google.common.annotations +com.google.common.base +com.google.common.cache +com.google.common.collect +com.google.common.escape +com.google.common.eventbus +com.google.common.graph +com.google.common.hash +com.google.common.html +com.google.common.io +com.google.common.math +com.google.common.net +com.google.common.primitives +com.google.common.reflect +com.google.common.util.concurrent +com.google.common.xml diff --git a/integration/kotlinx-coroutines-guava/src/ListenableFuture.kt b/integration/kotlinx-coroutines-guava/src/ListenableFuture.kt new file mode 100644 index 0000000000..ea9addc68a --- /dev/null +++ b/integration/kotlinx-coroutines-guava/src/ListenableFuture.kt @@ -0,0 +1,505 @@ +package kotlinx.coroutines.guava + +import com.google.common.util.concurrent.* +import com.google.common.util.concurrent.internal.* +import kotlinx.coroutines.* +import java.util.concurrent.* +import java.util.concurrent.CancellationException +import kotlin.coroutines.* + +/** + * Starts [block] in a new coroutine and returns a [ListenableFuture] pointing to its result. + * + * The coroutine is started immediately. Passing [CoroutineStart.LAZY] to [start] throws + * [IllegalArgumentException], because Futures don't have a way to start lazily. + * + * When the created coroutine [isCompleted][Job.isCompleted], it will try to + * *synchronously* complete the returned Future with the same outcome. This will + * succeed, barring a race with external cancellation of returned [ListenableFuture]. + * + * Cancellation is propagated bidirectionally. + * + * `CoroutineContext` is inherited from this [CoroutineScope]. Additional context elements can be + * added/overlaid by passing [context]. + * + * If the context does not have a [CoroutineDispatcher], nor any other [ContinuationInterceptor] + * member, [Dispatchers.Default] is used. + * + * The parent job is inherited from this [CoroutineScope], and can be overridden by passing + * a [Job] in [context]. + * + * See [newCoroutineContext][CoroutineScope.newCoroutineContext] for a description of debugging + * facilities. + * + * Note that the error and cancellation semantics of [future] are _different_ than [async]'s. + * In contrast to [Deferred], [Future] doesn't have an intermediate `Cancelling` state. If + * the returned `Future` is successfully cancelled, and `block` throws afterward, the thrown + * error is dropped, and getting the `Future`'s value will throw a `CancellationException` with + * no cause. This is to match the specification and behavior of + * `java.util.concurrent.FutureTask`. + * + * @param context added overlaying [CoroutineScope.coroutineContext] to form the new context. + * @param start coroutine start option. The default value is [CoroutineStart.DEFAULT]. + * @param block the code to execute. + */ +public fun CoroutineScope.future( + context: CoroutineContext = EmptyCoroutineContext, + start: CoroutineStart = CoroutineStart.DEFAULT, + block: suspend CoroutineScope.() -> T +): ListenableFuture { + require(!start.isLazy) { "$start start is not supported" } + val newContext = newCoroutineContext(context) + val coroutine = ListenableFutureCoroutine(newContext) + coroutine.start(start, coroutine, block) + return coroutine.future +} + +/** + * Returns a [Deferred] that is completed or failed by `this` [ListenableFuture]. + * + * Completion is non-atomic between the two promises. + * + * Cancellation is propagated bidirectionally. + * + * When `this` `ListenableFuture` completes (either successfully or exceptionally) it will try to + * complete the returned `Deferred` with the same value or exception. This will succeed, barring a + * race with cancellation of the `Deferred`. + * + * When `this` `ListenableFuture` is [successfully cancelled][java.util.concurrent.Future.cancel], + * it will cancel the returned `Deferred`. + * + * When the returned `Deferred` is [cancelled][Deferred.cancel], it will try to propagate the + * cancellation to `this` `ListenableFuture`. Propagation will succeed, barring a race with the + * `ListenableFuture` completing normally. This is the only case in which the returned `Deferred` + * will complete with a different outcome than `this` `ListenableFuture`. + */ +public fun ListenableFuture.asDeferred(): Deferred { + /* This method creates very specific behaviour as it entangles the `Deferred` and + * `ListenableFuture`. This behaviour is the best discovered compromise between the possible + * states and interface contracts of a `Future` and the states of a `Deferred`. The specific + * behaviour is described here. + * + * When `this` `ListenableFuture` is successfully cancelled - meaning + * `ListenableFuture.cancel()` returned `true` - it will synchronously cancel the returned + * `Deferred`. This can only race with cancellation of the returned `Deferred`, so the + * `Deferred` will always be put into its "cancelling" state and (barring uncooperative + * cancellation) _eventually_ reach its "cancelled" state when either promise is successfully + * cancelled. + * + * When the returned `Deferred` is cancelled, `ListenableFuture.cancel()` will be synchronously + * called on `this` `ListenableFuture`. This will attempt to cancel the `Future`, though + * cancellation may not succeed and the `ListenableFuture` may complete in a non-cancelled + * terminal state. + * + * The returned `Deferred` may receive and suppress the `true` return value from + * `ListenableFuture.cancel()` when the task is cancelled via the `Deferred` reference to it. + * This is unavoidable, so make sure no idempotent cancellation work is performed by a + * reference-holder of the `ListenableFuture` task. The idempotent work won't get done if + * cancellation was from the `Deferred` representation of the task. + * + * This is inherently a race. See `Future.cancel()` for a description of `Future` cancellation + * semantics. See `Job` for a description of coroutine cancellation semantics. + */ + // First, try the fast-fast error path for Guava ListenableFutures. This will save allocating an + // Exception by using the same instance the Future created. + if (this is InternalFutureFailureAccess) { + val t: Throwable? = InternalFutures.tryInternalFastPathGetFailure(this) + if (t != null) { + return CompletableDeferred().also { + it.completeExceptionally(t) + } + } + } + + // Second, try the fast path for a completed Future. The Future is known to be done, so get() + // will not block, and thus it won't be interrupted. Calling getUninterruptibly() instead of + // getDone() in this known-non-interruptible case saves the volatile read that getDone() uses to + // handle interruption. + if (isDone) { + return try { + CompletableDeferred(Uninterruptibles.getUninterruptibly(this)) + } catch (e: CancellationException) { + CompletableDeferred().also { it.cancel(e) } + } catch (e: ExecutionException) { + // ExecutionException is the only kind of exception that can be thrown from a gotten + // Future. Anything else showing up here indicates a very fundamental bug in a + // Future implementation. + CompletableDeferred().also { it.completeExceptionally(e.nonNullCause()) } + } + } + + // Finally, if this isn't done yet, attach a Listener that will complete the Deferred. + val deferred = CompletableDeferred() + Futures.addCallback(this, object : FutureCallback { + override fun onSuccess(result: T) { + runCatching { deferred.complete(result) } + .onFailure { handleCoroutineException(EmptyCoroutineContext, it) } + } + + override fun onFailure(t: Throwable) { + runCatching { deferred.completeExceptionally(t) } + .onFailure { handleCoroutineException(EmptyCoroutineContext, it) } + } + }, MoreExecutors.directExecutor()) + + // ... And cancel the Future when the deferred completes. Since the return type of this method + // is Deferred, the only interaction point from the caller is to cancel the Deferred. If this + // completion handler runs before the Future is completed, the Deferred must have been + // cancelled and should propagate its cancellation. If it runs after the Future is completed, + // this is a no-op. + deferred.invokeOnCompletion { + cancel(false) + } + // Return hides the CompletableDeferred. This should prevent casting. + @OptIn(InternalForInheritanceCoroutinesApi::class) + return object : Deferred by deferred {} +} + +/** + * Returns the cause from an [ExecutionException] thrown by a [Future.get] or similar. + * + * [ExecutionException] _always_ wraps a non-null cause when Future.get() throws. A Future cannot + * fail without a non-null `cause`, because the only way a Future _can_ fail is an uncaught + * [Exception]. + * + * If this !! throws [NullPointerException], a Future is breaking its interface contract and losing + * state - a serious fundamental bug. + */ +private fun ExecutionException.nonNullCause(): Throwable { + return this.cause!! +} + +/** + * Returns a [ListenableFuture] that is completed or failed by `this` [Deferred]. + * + * Completion is non-atomic between the two promises. + * + * When either promise successfully completes, it will attempt to synchronously complete its + * counterpart with the same value. This will succeed barring a race with cancellation. + * + * When either promise completes with an Exception, it will attempt to synchronously complete its + * counterpart with the same Exception. This will succeed barring a race with cancellation. + * + * Cancellation is propagated bidirectionally. + * + * When the returned [Future] is successfully cancelled - meaning [Future.cancel] returned true - + * [Deferred.cancel] will be synchronously called on `this` [Deferred]. This will attempt to cancel + * the `Deferred`, though cancellation may not succeed and the `Deferred` may complete in a + * non-cancelled terminal state. + * + * When `this` `Deferred` reaches its "cancelled" state with a successful cancellation - meaning it + * completes with [kotlinx.coroutines.CancellationException] - `this` `Deferred` will synchronously + * cancel the returned `Future`. This can only race with cancellation of the returned `Future`, so + * the returned `Future` will always _eventually_ reach its cancelled state when either promise is + * successfully cancelled, for their different meanings of "successfully cancelled". + * + * This is inherently a race. See [Future.cancel] for a description of `Future` cancellation + * semantics. See [Job] for a description of coroutine cancellation semantics. See + * [JobListenableFuture.cancel] for greater detail on the overlapped cancellation semantics and + * corner cases of this method. + */ +public fun Deferred.asListenableFuture(): ListenableFuture { + val listenableFuture = JobListenableFuture(this) + // This invokeOnCompletion completes the JobListenableFuture with the same result as `this` Deferred. + // The JobListenableFuture may have completed earlier if it got cancelled! See JobListenableFuture.cancel(). + invokeOnCompletion { throwable -> + if (throwable == null) { + listenableFuture.complete(getCompleted()) + } else { + listenableFuture.completeExceptionallyOrCancel(throwable) + } + } + return listenableFuture +} + +/** + * Awaits completion of `this` [ListenableFuture] without blocking a thread. + * + * This suspend function is cancellable. + * + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this function + * stops waiting for the future and immediately resumes with [CancellationException][kotlinx.coroutines.CancellationException]. + * + * This method is intended to be used with one-shot Futures, so on coroutine cancellation, the Future is cancelled as well. + * If cancelling the given future is undesired, use [Futures.nonCancellationPropagating] or + * [kotlinx.coroutines.NonCancellable]. + */ +public suspend fun ListenableFuture.await(): T { + try { + if (isDone) return Uninterruptibles.getUninterruptibly(this) + } catch (e: ExecutionException) { + // ExecutionException is the only kind of exception that can be thrown from a gotten + // Future, other than CancellationException. Cancellation is propagated upward so that + // the coroutine running this suspend function may process it. + // Any other Exception showing up here indicates a very fundamental bug in a + // Future implementation. + throw e.nonNullCause() + } + + return suspendCancellableCoroutine { cont: CancellableContinuation -> + addListener( + ToContinuation(this, cont), + MoreExecutors.directExecutor()) + cont.invokeOnCancellation { + cancel(false) + } + } +} + +/** + * Propagates the outcome of [futureToObserve] to [continuation] on completion. + * + * Cancellation is propagated as cancelling the continuation. If [futureToObserve] completes + * and fails, the cause of the Future will be propagated without a wrapping + * [ExecutionException] when thrown. + */ +private class ToContinuation( + val futureToObserve: ListenableFuture, + val continuation: CancellableContinuation +): Runnable { + override fun run() { + if (futureToObserve.isCancelled) { + continuation.cancel() + } else { + try { + continuation.resume(Uninterruptibles.getUninterruptibly(futureToObserve)) + } catch (e: ExecutionException) { + // ExecutionException is the only kind of exception that can be thrown from a gotten + // Future. Anything else showing up here indicates a very fundamental bug in a + // Future implementation. + continuation.resumeWithException(e.nonNullCause()) + } + } + } +} + +/** + * An [AbstractCoroutine] intended for use directly creating a [ListenableFuture] handle to + * completion. + * + * If [future] is successfully cancelled, cancellation is propagated to `this` `Coroutine`. + * By documented contract, a [Future] has been cancelled if + * and only if its `isCancelled()` method returns true. + * + * Any error that occurs after successfully cancelling a [ListenableFuture] is lost. + * The contract of [Future] does not permit it to return an error after it is successfully cancelled. + * On the other hand, we can't report an unhandled exception to [CoroutineExceptionHandler], + * otherwise [Future.cancel] can lead to an app crash which arguably is a contract violation. + * In contrast to [Future] which can't change its outcome after a successful cancellation, + * cancelling a [Deferred] places that [Deferred] in the cancelling/cancelled states defined by [Job], + * which _can_ show the error. + * + * This may be counterintuitive, but it maintains the error and cancellation contracts of both + * the [Deferred] and [ListenableFuture] types, while permitting both kinds of promise to point + * to the same running task. + */ +private class ListenableFutureCoroutine( + context: CoroutineContext +) : AbstractCoroutine(context, initParentJob = true, active = true) { + + // JobListenableFuture propagates external cancellation to `this` coroutine. See JobListenableFuture. + @JvmField + val future = JobListenableFuture(this) + + override fun onCompleted(value: T) { + future.complete(value) + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + // Note: if future was cancelled in a race with a cancellation of this + // coroutine, and the future was successfully cancelled first, the cause of coroutine + // cancellation is dropped in this promise. A Future can only be completed once. + // + // This is consistent with FutureTask behaviour. A race between a Future.cancel() and + // a FutureTask.setException() for the same Future will similarly drop the + // cause of a failure-after-cancellation. + future.completeExceptionallyOrCancel(cause) + } +} + +/** + * A [ListenableFuture] that delegates to an internal [SettableFuture], collaborating with it. + * + * This setup allows the returned [ListenableFuture] to maintain the following properties: + * + * - Correct implementation of [Future]'s happens-after semantics documented for [get], [isDone] + * and [isCancelled] methods + * - Cancellation propagation both to and from [Deferred] + * - Correct cancellation and completion semantics even when this [ListenableFuture] is combined + * with different concrete implementations of [ListenableFuture] + * - Fully correct cancellation and listener happens-after obeying [Future] and + * [ListenableFuture]'s documented and implicit contracts is surprisingly difficult to achieve. + * The best way to be correct, especially given the fun corner cases from + * [AbstractFuture.setFuture], is to just use an [AbstractFuture]. + * - To maintain sanity, this class implements [ListenableFuture] and uses an auxiliary [SettableFuture] + * around coroutine's result as a state engine to establish happens-after-completion. This + * could probably be compressed into one subclass of [AbstractFuture] to save an allocation, at the + * cost of the implementation's readability. + */ +private class JobListenableFuture(private val jobToCancel: Job): ListenableFuture { + /** + * Serves as a state machine for [Future] cancellation. + * + * [AbstractFuture] has a highly-correct atomic implementation of `Future`'s completion and + * cancellation semantics. By using that type, the [JobListenableFuture] can delegate its semantics to + * `auxFuture.get()` the result in such a way that the `Deferred` is always complete when returned. + * + * To preserve Coroutine's [CancellationException], this future points to either `T` or [Cancelled]. + */ + private val auxFuture = SettableFuture.create() + + /** + * `true` if [auxFuture.get][ListenableFuture.get] throws [ExecutionException]. + * + * Note: this is eventually consistent with the state of [auxFuture]. + * + * Unfortunately, there's no API to figure out if [ListenableFuture] throws [ExecutionException] + * apart from calling [ListenableFuture.get] on it. To avoid unnecessary [ExecutionException] allocation + * we use this field as an optimization. + */ + private var auxFutureIsFailed: Boolean = false + + /** + * When the attached coroutine [isCompleted][Job.isCompleted] successfully + * its outcome should be passed to this method. + * + * This should succeed barring a race with external cancellation. + */ + fun complete(result: T): Boolean = auxFuture.set(result) + + /** + * When the attached coroutine [isCompleted][Job.isCompleted] [exceptionally][Job.isCancelled] + * its outcome should be passed to this method. + * + * This method will map coroutine's exception into corresponding Future's exception. + * + * This should succeed barring a race with external cancellation. + */ + // CancellationException is wrapped into `Cancelled` to preserve original cause and message. + // All the other exceptions are delegated to SettableFuture.setException. + fun completeExceptionallyOrCancel(t: Throwable): Boolean = + if (t is CancellationException) auxFuture.set(Cancelled(t)) + else auxFuture.setException(t).also { if (it) auxFutureIsFailed = true } + + /** + * Returns cancellation _in the sense of [Future]_. This is _not_ equivalent to + * [Job.isCancelled]. + * + * When done, this Future is cancelled if its [auxFuture] is cancelled, or if [auxFuture] + * contains [CancellationException]. + * + * See [cancel]. + */ + override fun isCancelled(): Boolean { + // This expression ensures that isCancelled() will *never* return true when isDone() returns false. + // In the case that the deferred has completed with cancellation, completing `this`, its + // reaching the "cancelled" state with a cause of CancellationException is treated as the + // same thing as auxFuture getting cancelled. If the Job is in the "cancelling" state and + // this Future hasn't itself been successfully cancelled, the Future will return + // isCancelled() == false. This is the only discovered way to reconcile the two different + // cancellation contracts. + return auxFuture.isCancelled || isDone && !auxFutureIsFailed && try { + Uninterruptibles.getUninterruptibly(auxFuture) is Cancelled + } catch (e: CancellationException) { + // `auxFuture` got cancelled right after `auxFuture.isCancelled` returned false. + true + } catch (e: ExecutionException) { + // `auxFutureIsFailed` hasn't been updated yet. + auxFutureIsFailed = true + false + } + } + + /** + * Waits for [auxFuture] to complete by blocking, then uses its `result` + * to get the `T` value `this` [ListenableFuture] is pointing to or throw a [CancellationException]. + * This establishes happens-after ordering for completion of the entangled coroutine. + * + * [SettableFuture.get] can only throw [CancellationException] if it was cancelled externally. + * Otherwise it returns [Cancelled] that encapsulates outcome of the entangled coroutine. + * + * [auxFuture] _must be complete_ in order for the [isDone] and [isCancelled] happens-after + * contract of [Future] to be correctly followed. + */ + override fun get(): T { + return getInternal(auxFuture.get()) + } + + /** See [get()]. */ + override fun get(timeout: Long, unit: TimeUnit): T { + return getInternal(auxFuture.get(timeout, unit)) + } + + /** See [get()]. */ + private fun getInternal(result: Any?): T = if (result is Cancelled) { + throw CancellationException().initCause(result.exception) + } else { + // We know that `auxFuture` can contain either `T` or `Cancelled`. + @Suppress("UNCHECKED_CAST") + result as T + } + + override fun addListener(listener: Runnable, executor: Executor) { + auxFuture.addListener(listener, executor) + } + + override fun isDone(): Boolean { + return auxFuture.isDone + } + + /** + * Tries to cancel [jobToCancel] if `this` future was cancelled. This is fundamentally racy. + * + * The call to `cancel()` will try to cancel [auxFuture]: if and only if cancellation of [auxFuture] + * succeeds, [jobToCancel] will have its [Job.cancel] called. + * + * This arrangement means that [jobToCancel] _might not successfully cancel_, if the race resolves + * in a particular way. [jobToCancel] may also be in its "cancelling" state while this + * ListenableFuture is complete and cancelled. + */ + override fun cancel(mayInterruptIfRunning: Boolean): Boolean { + // TODO: call jobToCancel.cancel() _before_ running the listeners. + // `auxFuture.cancel()` will execute auxFuture's listeners. This delays cancellation of + // `jobToCancel` until after auxFuture's listeners have already run. + // Consider moving `jobToCancel.cancel()` into [AbstractFuture.afterDone] when the API is finalized. + return if (auxFuture.cancel(mayInterruptIfRunning)) { + jobToCancel.cancel() + true + } else { + false + } + } + + override fun toString(): String = buildString { + append(super.toString()) + append("[status=") + if (isDone) { + try { + when (val result = Uninterruptibles.getUninterruptibly(auxFuture)) { + is Cancelled -> append("CANCELLED, cause=[${result.exception}]") + else -> append("SUCCESS, result=[$result]") + } + } catch (e: CancellationException) { + // `this` future was cancelled by `Future.cancel`. In this case there's no cause or message. + append("CANCELLED") + } catch (e: ExecutionException) { + append("FAILURE, cause=[${e.cause}]") + } catch (t: Throwable) { + // Violation of Future's contract, should never happen. + append("UNKNOWN, cause=[${t.javaClass} thrown from get()]") + } + } else { + append("PENDING, delegate=[$auxFuture]") + } + append(']') + } +} + +/** + * A wrapper for `Coroutine`'s [CancellationException]. + * + * If the coroutine is _cancelled normally_, we want to show the reason of cancellation to the user. Unfortunately, + * [SettableFuture] can't store the reason of cancellation. To mitigate this, we wrap cancellation exception into this + * class and pass it into [SettableFuture.complete]. See implementation of [JobListenableFuture]. + */ +private class Cancelled(@JvmField val exception: CancellationException) diff --git a/integration/kotlinx-coroutines-guava/src/module-info.java b/integration/kotlinx-coroutines-guava/src/module-info.java new file mode 100644 index 0000000000..0b8ccafd68 --- /dev/null +++ b/integration/kotlinx-coroutines-guava/src/module-info.java @@ -0,0 +1,7 @@ +module kotlinx.coroutines.guava { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires com.google.common; + + exports kotlinx.coroutines.guava; +} diff --git a/integration/kotlinx-coroutines-guava/test/FutureAsDeferredUnhandledCompletionExceptionTest.kt b/integration/kotlinx-coroutines-guava/test/FutureAsDeferredUnhandledCompletionExceptionTest.kt new file mode 100644 index 0000000000..5637493dfb --- /dev/null +++ b/integration/kotlinx-coroutines-guava/test/FutureAsDeferredUnhandledCompletionExceptionTest.kt @@ -0,0 +1,43 @@ +package kotlinx.coroutines.guava + +import kotlinx.coroutines.testing.* +import com.google.common.util.concurrent.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class FutureAsDeferredUnhandledCompletionExceptionTest : TestBase() { + + // This is a separate test in order to avoid interference with uncaught exception handlers in other tests + private val exceptionHandler = Thread.getDefaultUncaughtExceptionHandler() + private lateinit var caughtException: Throwable + + @Before + fun setUp() { + Thread.setDefaultUncaughtExceptionHandler { _, e -> caughtException = e } + } + + @After + fun tearDown() { + Thread.setDefaultUncaughtExceptionHandler(exceptionHandler) + } + + @Test + fun testLostExceptionOnSuccess() = runTest { + val future = SettableFuture.create() + val deferred = future.asDeferred() + deferred.invokeOnCompletion { throw TestException() } + future.set(1) + assertTrue { caughtException is CompletionHandlerException && caughtException.cause is TestException } + } + + @Test + fun testLostExceptionOnFailure() = runTest { + val future = SettableFuture.create() + val deferred = future.asDeferred() + deferred.invokeOnCompletion { throw TestException() } + future.setException(TestException2()) + assertTrue { caughtException is CompletionHandlerException && caughtException.cause is TestException } + } +} diff --git a/integration/kotlinx-coroutines-guava/test/ListenableFutureExceptionsTest.kt b/integration/kotlinx-coroutines-guava/test/ListenableFutureExceptionsTest.kt new file mode 100644 index 0000000000..374f698979 --- /dev/null +++ b/integration/kotlinx-coroutines-guava/test/ListenableFutureExceptionsTest.kt @@ -0,0 +1,88 @@ +package kotlinx.coroutines.guava + +import kotlinx.coroutines.testing.* +import com.google.common.base.* +import com.google.common.util.concurrent.* +import kotlinx.coroutines.* +import org.junit.Test +import java.io.* +import java.util.concurrent.* +import kotlin.test.* + +class ListenableFutureExceptionsTest : TestBase() { + + @Test + fun testAwait() { + testException(IOException(), { it is IOException }) + } + + @Test + fun testAwaitChained() { + testException(IOException(), { it is IOException }, { i -> i!! + 1 }) + } + + @Test + fun testAwaitCompletionException() { + testException(CompletionException("test", IOException()), { it is CompletionException }) + } + + @Test + fun testAwaitChainedCompletionException() { + testException( + CompletionException("test", IOException()), + { it is CompletionException }, + { i -> i!! + 1 }) + } + + @Test + fun testAwaitTestException() { + testException(TestException(), { it is TestException }) + } + + @Test + fun testAwaitChainedTestException() { + testException(TestException(), { it is TestException }, { i -> i!! + 1 }) + } + + private fun testException( + exception: Throwable, + expected: ((Throwable) -> Boolean), + transformer: ((Int?) -> Int?)? = null + ) { + + // Fast path + runTest { + val future = SettableFuture.create() + val chained = if (transformer == null) { + future + } else { + Futures.transform(future, Function(transformer), MoreExecutors.directExecutor()) + } + future.setException(exception) + try { + chained.await() + } catch (e: Throwable) { + assertTrue(expected(e)) + } + } + + // Slow path + runTest { + val future = SettableFuture.create() + val chained = if (transformer == null) { + future + } else { + Futures.transform(future, Function(transformer), MoreExecutors.directExecutor()) + } + launch { + future.setException(exception) + } + + try { + chained.await() + } catch (e: Throwable) { + assertTrue(expected(e)) + } + } + } +} diff --git a/integration/kotlinx-coroutines-guava/test/ListenableFutureTest.kt b/integration/kotlinx-coroutines-guava/test/ListenableFutureTest.kt new file mode 100644 index 0000000000..442543840d --- /dev/null +++ b/integration/kotlinx-coroutines-guava/test/ListenableFutureTest.kt @@ -0,0 +1,809 @@ +package kotlinx.coroutines.guava + +import kotlinx.coroutines.testing.* +import com.google.common.util.concurrent.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Ignore +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.CancellationException +import java.util.concurrent.atomic.* +import kotlin.test.* + +class ListenableFutureTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("ForkJoinPool.commonPool-worker-") + } + + @Test + fun testSimpleAwait() { + val service = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + val future = GlobalScope.future { + service.submit(Callable { + "O" + }).await() + "K" + } + assertEquals("OK", future.get()) + } + + @Test + fun testAwaitWithContext() = runTest { + val future = SettableFuture.create() + val deferred = async { + withContext(Dispatchers.Default) { + future.await() + } + } + + future.set(1) + assertEquals(1, deferred.await()) + } + + @Test + fun testAwaitWithCancellation() = runTest(expected = {it is TestCancellationException}) { + val future = SettableFuture.create() + val deferred = async { + withContext(Dispatchers.Default) { + future.await() + } + } + + deferred.cancel(TestCancellationException()) + deferred.await() // throws TCE + expectUnreached() + } + + @Test + fun testCompletedFuture() { + val toAwait = SettableFuture.create() + toAwait.set("O") + val future = GlobalScope.future { + toAwait.await() + "K" + } + assertEquals("OK", future.get()) + } + + @Test + fun testWaitForFuture() { + val toAwait = SettableFuture.create() + val future = GlobalScope.future { + toAwait.await() + "K" + } + assertFalse(future.isDone) + toAwait.set("O") + assertEquals("OK", future.get()) + } + + @Test + fun testCompletedFutureExceptionally() { + val toAwait = SettableFuture.create() + toAwait.setException(IllegalArgumentException("O")) + val future = GlobalScope.future { + try { + toAwait.await() + } catch (e: RuntimeException) { + assertIs(e) + e.message!! + } + "K" + } + assertEquals("OK", future.get()) + } + + @Test + fun testWaitForFutureWithException() { + val toAwait = SettableFuture.create() + val future = GlobalScope.future { + try { + toAwait.await() + } catch (e: RuntimeException) { + assertIs(e) + e.message!! + } + "K" + } + assertFalse(future.isDone) + toAwait.setException(IllegalArgumentException("O")) + assertEquals("OK", future.get()) + } + + @Test + fun testExceptionInsideCoroutine() { + val service = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + val future = GlobalScope.future { + if (service.submit(Callable { true }).await()) { + throw IllegalStateException("OK") + } + "fail" + } + try { + future.get() + fail("'get' should've throw an exception") + } catch (e: ExecutionException) { + assertIs(e.cause) + assertEquals("OK", e.cause!!.message) + } + } + + @Test + fun testFutureLazyStartThrows() { + expect(1) + val e = assertFailsWith { + GlobalScope.future(start = CoroutineStart.LAZY) {} + } + + assertEquals("LAZY start is not supported", e.message) + finish(2) + } + + @Test + fun testCompletedDeferredAsListenableFuture() = runBlocking { + expect(1) + val deferred = async(start = CoroutineStart.UNDISPATCHED) { + expect(2) // completed right away + "OK" + } + expect(3) + val future = deferred.asListenableFuture() + assertEquals("OK", future.await()) + finish(4) + } + + @Test + fun testWaitForDeferredAsListenableFuture() = runBlocking { + expect(1) + val deferred = async { + expect(3) // will complete later + "OK" + } + expect(2) + val future = deferred.asListenableFuture() + assertEquals("OK", future.await()) // await yields main thread to deferred coroutine + finish(4) + } + + @Test + fun testAsListenableFutureThrowable() { + val deferred = GlobalScope.async { + throw OutOfMemoryError() + } + + val future = deferred.asListenableFuture() + try { + future.get() + } catch (e: ExecutionException) { + assertTrue(future.isDone) + assertIs(e.cause) + } + } + + @Test + fun testCancellableAwait() = runBlocking { + expect(1) + val toAwait = SettableFuture.create() + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + toAwait.await() // suspends + } catch (e: CancellationException) { + expect(5) // should throw cancellation exception + throw e + } + } + expect(3) + job.cancel() // cancel the job + toAwait.set("fail") // too late, the waiting job was already cancelled + expect(4) // job processing of cancellation was scheduled, not executed yet + yield() // yield main thread to job + finish(6) + } + + @Test + fun testFutureAwaitCancellationPropagatingToDeferred() = runTest { + + val latch = CountDownLatch(1) + val executor = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + val future = executor.submit(Callable { latch.await(); 42 }) + val deferred = async { + expect(2) + future.await() + } + expect(1) + yield() + future.cancel(/*mayInterruptIfRunning=*/true) + expect(3) + latch.countDown() + deferred.join() + assertTrue(future.isCancelled) + assertTrue(deferred.isCancelled) + assertFailsWith { future.get() } + finish(4) + } + + @Test + fun testFutureAwaitCancellationPropagatingToDeferredNoInterruption() = runTest { + + val latch = CountDownLatch(1) + val executor = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + val future = executor.submit(Callable { latch.await(); 42 }) + val deferred = async { + expect(2) + future.await() + } + expect(1) + yield() + future.cancel(/*mayInterruptIfRunning=*/false) + expect(3) + latch.countDown() + deferred.join() + assertTrue(future.isCancelled) + assertTrue(deferred.isCancelled) + assertFailsWith { future.get() } + finish(4) + } + + @Test + fun testAsListenableFutureCancellationPropagatingToDeferred() = runTest { + val latch = CountDownLatch(1) + val executor = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + val future = executor.submit(Callable { latch.await(); 42 }) + val deferred = async { + expect(2) + future.await() + } + val asListenableFuture = deferred.asListenableFuture() + expect(1) + yield() + asListenableFuture.cancel(/*mayInterruptIfRunning=*/true) + expect(3) + latch.countDown() + deferred.join() + assertTrue(future.isCancelled) + assertTrue(deferred.isCancelled) + assertTrue(asListenableFuture.isCancelled) + assertFailsWith { future.get() } + finish(4) + } + + @Test + fun testAsListenableFutureCancellationPropagatingToDeferredNoInterruption() = runTest { + val latch = CountDownLatch(1) + val executor = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + val future = executor.submit(Callable { latch.await(); 42 }) + val deferred = async { + expect(2) + future.await() + } + val asListenableFuture = deferred.asListenableFuture() + expect(1) + yield() + asListenableFuture.cancel(/*mayInterruptIfRunning=*/false) + expect(3) + latch.countDown() + deferred.join() + assertFailsWith { asListenableFuture.get() } + assertTrue(future.isCancelled) + assertTrue(asListenableFuture.isCancelled) + assertTrue(deferred.isCancelled) + assertFailsWith { future.get() } + finish(4) + } + + @Test + fun testAsListenableFutureCancellationThroughSetFuture() = runTest { + val latch = CountDownLatch(1) + val future = SettableFuture.create() + val deferred = async { + expect(2) + future.await() + } + val asListenableFuture = deferred.asListenableFuture() + expect(1) + yield() + future.setFuture(Futures.immediateCancelledFuture()) + expect(3) + latch.countDown() + deferred.join() + assertFailsWith { asListenableFuture.get() } + // Future was not interrupted, but also wasn't blocking, so it will be successfully + // cancelled by its parent Coroutine. + assertTrue(future.isCancelled) + assertTrue(asListenableFuture.isCancelled) + assertTrue(deferred.isCancelled) + assertFailsWith { future.get() } + finish(4) + } + + @Test + @Ignore // TODO: propagate cancellation before running listeners. + fun testAsListenableFuturePropagatesCancellationBeforeRunningListeners() = runTest { + expect(1) + val deferred = async(context = Dispatchers.Unconfined) { + try { + delay(Long.MAX_VALUE) + } finally { + expect(3) // Cancelled. + } + } + val asFuture = deferred.asListenableFuture() + asFuture.addListener(Runnable { expect(4) }, MoreExecutors.directExecutor()) + assertFalse(asFuture.isDone) + expect(2) + asFuture.cancel(false) + assertTrue(asFuture.isDone) + assertTrue(asFuture.isCancelled) + assertFailsWith { deferred.await() } + finish(5) + } + + @Test + fun testFutureCancellation() = runTest { + val future = awaitFutureWithCancel(true) + assertTrue(future.isCancelled) + assertFailsWith { future.get() } + finish(4) + } + + @Test + fun testAsListenableDeferredCancellationCauseAndMessagePropagate() = runTest { + val deferred = CompletableDeferred() + val inputCancellationException = CancellationException("Foobar") + inputCancellationException.initCause(OutOfMemoryError("Foobaz")) + deferred.cancel(inputCancellationException) + val asFuture = deferred.asListenableFuture() + + val outputCancellationException = + assertFailsWith { asFuture.get() } + val cause = outputCancellationException.cause + assertNotNull(cause) + assertEquals(cause.message, "Foobar") + assertIs(cause.cause) + assertEquals(cause.cause?.message, "Foobaz") + } + + @Test + fun testNoFutureCancellation() = runTest { + val future = awaitFutureWithCancel(false) + assertFalse(future.isCancelled) + @Suppress("BlockingMethodInNonBlockingContext") + assertEquals(42, future.get()) + finish(4) + } + + @Test + fun testCancelledDeferredAsListenableFutureAwaitThrowsCancellation() = runTest { + val future = Futures.immediateCancelledFuture() + val asDeferred = future.asDeferred() + val asDeferredAsFuture = asDeferred.asListenableFuture() + + assertTrue(asDeferredAsFuture.isCancelled) + assertFailsWith { + asDeferredAsFuture.await() + } + } + + @Test + fun testCancelledDeferredAsListenableFutureAsDeferredPassesCancellationAlong() = runTest { + val deferred = CompletableDeferred() + deferred.completeExceptionally(CancellationException()) + val asFuture = deferred.asListenableFuture() + val asFutureAsDeferred = asFuture.asDeferred() + + assertTrue(asFutureAsDeferred.isCancelled) + assertTrue(asFutureAsDeferred.isCompleted) + // By documentation, join() shouldn't throw when asDeferred is already complete. + asFutureAsDeferred.join() + assertIs(asFutureAsDeferred.getCompletionExceptionOrNull()) + } + + @Test + fun testCancelledFutureAsDeferredAwaitThrowsCancellation() = runTest { + val future = Futures.immediateCancelledFuture() + val asDeferred = future.asDeferred() + + assertTrue(asDeferred.isCancelled) + assertFailsWith { + asDeferred.await() + } + } + + @Test + fun testCancelledFutureAsDeferredJoinDoesNotThrow() = runTest { + val future = Futures.immediateCancelledFuture() + val asDeferred = future.asDeferred() + + assertTrue(asDeferred.isCancelled) + assertTrue(asDeferred.isCompleted) + // By documentation, join() shouldn't throw when asDeferred is already complete. + asDeferred.join() + assertIs(asDeferred.getCompletionExceptionOrNull()) + } + + @Test + fun testCompletedFutureAsDeferred() = runTest { + val future = SettableFuture.create() + val task = async { + expect(2) + assertEquals(42, future.asDeferred().await()) + expect(4) + } + + expect(1) + yield() + expect(3) + future.set(42) + task.join() + finish(5) + } + + @Test + fun testFailedFutureAsDeferred() = runTest { + val future = SettableFuture.create().apply { + setException(TestException()) + } + val deferred = future.asDeferred() + assertTrue(deferred.isCancelled && deferred.isCompleted) + val completionException = deferred.getCompletionExceptionOrNull()!! + assertIs(completionException) + + try { + deferred.await() + expectUnreached() + } catch (e: Throwable) { + assertIs(e) + } + } + + @Test + fun testFutureCompletedWithNullFastPathAsDeferred() = runTest { + val executor = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + val future = executor.submit(Callable { null }).also { + @Suppress("BlockingMethodInNonBlockingContext") + it.get() + } + assertNull(future.asDeferred().await()) + } + + @Test + fun testFutureCompletedWithNullSlowPathAsDeferred() = runTest { + val latch = CountDownLatch(1) + val executor = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + + val future = executor.submit(Callable { + latch.await() + null + }) + + val awaiter = async(start = CoroutineStart.UNDISPATCHED) { + future.asDeferred().await() + } + + latch.countDown() + assertNull(awaiter.await()) + } + + @Test + fun testThrowingFutureAsDeferred() = runTest { + val executor = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + val future = executor.submit(Callable { throw TestException() }) + try { + future.asDeferred().await() + expectUnreached() + } catch (e: Throwable) { + assertIs(e) + } + } + + @Test + fun testStructuredException() = runTest( + expected = { it is TestException } // exception propagates to parent with structured concurrency + ) { + val result = future(Dispatchers.Unconfined) { + throw TestException("FAIL") + } + result.checkFutureException() + } + + @Test + fun testChildException() = runTest( + expected = { it is TestException } // exception propagates to parent with structured concurrency + ) { + val result = future(Dispatchers.Unconfined) { + // child crashes + launch { throw TestException("FAIL") } + 42 + } + result.checkFutureException() + } + + @Test + fun testExternalCancellation() = runTest { + val future = future(Dispatchers.Unconfined) { + try { + delay(Long.MAX_VALUE) + expectUnreached() + } catch (e: CancellationException) { + expect(2) + throw e + } + } + + yield() + expect(1) + future.cancel(true) + finish(3) + } + + @Test + fun testExceptionOnExternalCancellation() = runTest(expected = {it is TestException}) { + val result = future(Dispatchers.Unconfined) { + try { + expect(1) + delay(Long.MAX_VALUE) + expectUnreached() + } catch (e: CancellationException) { + expect(3) + throw TestException() + } + } + expect(2) + result.cancel(true) + finish(4) + } + + @Test + fun testUnhandledExceptionOnExternalCancellation() = runTest { + expect(1) + // No parent here (NonCancellable), so nowhere to propagate exception + val result = future(NonCancellable + Dispatchers.Unconfined) { + try { + delay(Long.MAX_VALUE) + } finally { + expect(2) + throw TestException() // this exception cannot be handled and is set to be lost. + } + } + result.cancel(true) + finish(3) + } + + /** This test ensures that we never pass [CancellationException] to [CoroutineExceptionHandler]. */ + @Test + fun testCancellationExceptionOnExternalCancellation() = runTest { + expect(1) + // No parent here (NonCancellable), so nowhere to propagate exception + val result = future(NonCancellable + Dispatchers.Unconfined) { + try { + delay(Long.MAX_VALUE) + } finally { + expect(2) + throw TestCancellationException() // this exception cannot be handled + } + } + assertTrue(result.cancel(true)) + finish(3) + } + + @Test + fun testCancellingFutureContextJobCancelsFuture() = runTest { + expect(1) + val supervisorJob = SupervisorJob() + val future = future(context = supervisorJob) { + expect(2) + try { + delay(Long.MAX_VALUE) + expectUnreached() + } catch (e: CancellationException) { + expect(4) + throw e + } + } + yield() + expect(3) + supervisorJob.cancel(CancellationException("Parent cancelled", TestException())) + supervisorJob.join() + assertTrue(future.isDone) + assertTrue(future.isCancelled) + val thrown = assertFailsWith { future.get() } + val cause = thrown.cause + assertNotNull(cause) + assertIs(cause) + assertEquals("Parent cancelled", cause.message) + assertIs(cause.cause) + finish(5) + } + + @Test + fun testFutureChildException() = runTest { + val future = future(context = NonCancellable + Dispatchers.Unconfined) { + val foo = async { delay(Long.MAX_VALUE); 42 } + val bar = async { throw TestException() } + foo.await() + bar.await() + } + future.checkFutureException() + } + + @Test + fun testFutureIsDoneAfterChildrenCompleted() = runTest { + expect(1) + val testException = TestException() + val futureIsAllowedToFinish = CountDownLatch(1) + // Don't propagate exception to the test and use different dispatchers as we are going to block test thread. + val future = future(context = NonCancellable + Dispatchers.Default) { + val foo = async(start = CoroutineStart.UNDISPATCHED) { + try { + delay(Long.MAX_VALUE) + 42 + } finally { + futureIsAllowedToFinish.await() + expect(3) + } + } + val bar = async { throw testException } + foo.await() + bar.await() + } + yield() + expect(2) + futureIsAllowedToFinish.countDown() + // Blocking get should succeed after internal coroutine completes. + val thrown = assertFailsWith { future.get() } + expect(4) + assertEquals(testException, thrown.cause) + finish(5) + } + + @Test + @Ignore // TODO: propagate cancellation before running listeners. + fun testFuturePropagatesCancellationBeforeRunningListeners() = runTest { + expect(1) + val future = future(context = Dispatchers.Unconfined) { + try { + delay(Long.MAX_VALUE) + } finally { + expect(3) // Cancelled. + } + } + future.addListener(Runnable { expect(4) }, MoreExecutors.directExecutor()) + assertFalse(future.isDone) + expect(2) + future.cancel(false) + assertTrue(future.isDone) + assertTrue(future.isCancelled) + finish(5) + } + + @Test + fun testFutureCompletedExceptionally() = runTest { + val testException = TestException() + // NonCancellable to not propagate error to this scope. + val future = future(context = NonCancellable) { + throw testException + } + yield() + assertTrue(future.isDone) + assertFalse(future.isCancelled) + val thrown = assertFailsWith { future.get() } + assertEquals(testException, thrown.cause) + } + + @Test + fun testAsListenableFutureCompletedExceptionally() = runTest { + val testException = TestException() + val deferred = CompletableDeferred().apply { + completeExceptionally(testException) + } + val asListenableFuture = deferred.asListenableFuture() + assertTrue(asListenableFuture.isDone) + assertFalse(asListenableFuture.isCancelled) + val thrown = assertFailsWith { asListenableFuture.get() } + assertEquals(testException, thrown.cause) + } + + private inline fun ListenableFuture<*>.checkFutureException() { + val e = assertFailsWith { get() } + val cause = e.cause!! + assertIs(cause) + } + + @Suppress("SuspendFunctionOnCoroutineScope") + private suspend fun CoroutineScope.awaitFutureWithCancel(cancellable: Boolean): ListenableFuture { + val latch = CountDownLatch(1) + val executor = MoreExecutors.listeningDecorator(ForkJoinPool.commonPool()) + val future = executor.submit(Callable { latch.await(); 42 }) + val deferred = async { + expect(2) + if (cancellable) future.await() + else future.asDeferred().await() + } + expect(1) + yield() + deferred.cancel() + expect(3) + latch.countDown() + return future + } + + @Test + fun testCancelledParent() = runTest({ it is CancellationException }) { + cancel() + future { expectUnreached() } + future(start = CoroutineStart.ATOMIC) { } + future(start = CoroutineStart.UNDISPATCHED) { } + } + + @Test + fun testStackOverflow() = runTest { + val future = SettableFuture.create() + val completed = AtomicLong() + val count = 10000L + val children = ArrayList() + for (i in 0 until count) { + children += launch(Dispatchers.Default) { + future.asDeferred().await() + completed.incrementAndGet() + } + } + future.set(1) + withTimeout(60_000) { + children.forEach { it.join() } + assertEquals(count, completed.get()) + } + } + + @Test + fun testFuturePropagatesExceptionToParentAfterCancellation() = runTest { + val throwLatch = CompletableDeferred() + val cancelLatch = CompletableDeferred() + val parent = Job() + val scope = CoroutineScope(parent) + val exception = TestException("propagated to parent") + val future = scope.future { + cancelLatch.complete(true) + withContext(NonCancellable) { + throwLatch.await() + throw exception + } + } + cancelLatch.await() + future.cancel(true) + throwLatch.complete(true) + parent.join() + assertTrue(parent.isCancelled) + assertEquals(exception, parent.getCancellationException().cause) + } + + // Stress tests. + + @Test + fun testFutureDoesNotReportToCoroutineExceptionHandler() = runTest { + repeat(1000) { + supervisorScope { // Don't propagate failures in children to parent and other children. + val innerFuture = SettableFuture.create() + val outerFuture = async { innerFuture.await() } + + withContext(Dispatchers.Default) { + launch { innerFuture.setException(TestException("can be lost")) } + launch { outerFuture.cancel() } + // nothing should be reported to CoroutineExceptionHandler, otherwise `Future.cancel` contract violation. + } + } + } + } + + @Test + fun testJobListenableFutureIsCancelledDoesNotThrow() = runTest { + repeat(1000) { + val deferred = CompletableDeferred() + val asListenableFuture = deferred.asListenableFuture() + // We heed two threads to test a race condition. + withContext(Dispatchers.Default) { + val cancellationJob = launch { + asListenableFuture.cancel(false) + } + while (!cancellationJob.isCompleted) { + asListenableFuture.isCancelled // Shouldn't throw. + } + } + } + } +} diff --git a/integration/kotlinx-coroutines-guava/test/ListenableFutureToStringTest.kt b/integration/kotlinx-coroutines-guava/test/ListenableFutureToStringTest.kt new file mode 100644 index 0000000000..c70c7608d4 --- /dev/null +++ b/integration/kotlinx-coroutines-guava/test/ListenableFutureToStringTest.kt @@ -0,0 +1,66 @@ +package kotlinx.coroutines.guava + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class ListenableFutureToStringTest : TestBase() { + @Test + fun testSuccessfulFuture() = runTest { + val deferred = CompletableDeferred("OK") + val succeededFuture = deferred.asListenableFuture() + val toString = succeededFuture.toString() + assertTrue(message = "Unexpected format: $toString") { + toString.matches(Regex("""kotlinx\.coroutines\.guava\.JobListenableFuture@[^\[]*\[status=SUCCESS, result=\[OK]]""")) + } + } + + @Test + fun testFailedFuture() = runTest { + val exception = TestRuntimeException("test") + val deferred = CompletableDeferred().apply { + completeExceptionally(exception) + } + val failedFuture = deferred.asListenableFuture() + val toString = failedFuture.toString() + assertTrue(message = "Unexpected format: $toString") { + toString.matches(Regex("""kotlinx\.coroutines\.guava\.JobListenableFuture@[^\[]*\[status=FAILURE, cause=\[$exception]]""")) + } + } + + @Test + fun testPendingFuture() = runTest { + val deferred = CompletableDeferred() + val pendingFuture = deferred.asListenableFuture() + val toString = pendingFuture.toString() + assertTrue(message = "Unexpected format: $toString") { + toString.matches(Regex("""kotlinx\.coroutines\.guava\.JobListenableFuture@[^\[]*\[status=PENDING, delegate=\[.*]]""")) + } + } + + @Test + fun testCancelledCoroutineAsListenableFuture() = runTest { + val exception = CancellationException("test") + val deferred = CompletableDeferred().apply { + cancel(exception) + } + val cancelledFuture = deferred.asListenableFuture() + val toString = cancelledFuture.toString() + assertTrue(message = "Unexpected format: $toString") { + toString.matches(Regex("""kotlinx\.coroutines\.guava\.JobListenableFuture@[^\[]*\[status=CANCELLED, cause=\[$exception]]""")) + } + } + + @Test + fun testCancelledFuture() = runTest { + val deferred = CompletableDeferred() + val cancelledFuture = deferred.asListenableFuture().apply { + cancel(false) + } + val toString = cancelledFuture.toString() + assertTrue(message = "Unexpected format: $toString") { + toString.matches(Regex("""kotlinx\.coroutines\.guava\.JobListenableFuture@[^\[]*\[status=CANCELLED]""")) + } + } +} diff --git a/integration/kotlinx-coroutines-jdk8/README.md b/integration/kotlinx-coroutines-jdk8/README.md new file mode 100644 index 0000000000..56e145fc4e --- /dev/null +++ b/integration/kotlinx-coroutines-jdk8/README.md @@ -0,0 +1,3 @@ +# Stub module + +Stub module for backwards compatibility. Since 1.7.0, this module was merged with core. diff --git a/integration/kotlinx-coroutines-jdk8/api/kotlinx-coroutines-jdk8.api b/integration/kotlinx-coroutines-jdk8/api/kotlinx-coroutines-jdk8.api new file mode 100644 index 0000000000..e69de29bb2 diff --git a/integration/kotlinx-coroutines-jdk8/build.gradle.kts b/integration/kotlinx-coroutines-jdk8/build.gradle.kts new file mode 100644 index 0000000000..e69de29bb2 diff --git a/integration/kotlinx-coroutines-jdk8/src/module-info.java b/integration/kotlinx-coroutines-jdk8/src/module-info.java new file mode 100644 index 0000000000..d83596c2fa --- /dev/null +++ b/integration/kotlinx-coroutines-jdk8/src/module-info.java @@ -0,0 +1,3 @@ +@SuppressWarnings("JavaModuleNaming") +module kotlinx.coroutines.jdk8 { +} diff --git a/integration/kotlinx-coroutines-play-services/README.md b/integration/kotlinx-coroutines-play-services/README.md new file mode 100644 index 0000000000..17b6500a3e --- /dev/null +++ b/integration/kotlinx-coroutines-play-services/README.md @@ -0,0 +1,45 @@ +# Module kotlinx-coroutines-play-services + +Integration with Google Play Services [Tasks API](https://developers.google.com/android/guides/tasks). + +Extension functions: + +| **Name** | **Description** +| -------- | --------------- +| [Task.asDeferred][asDeferred] | Converts a Task into a Deferred +| [Task.await][await] | Awaits for completion of the Task (cancellable) +| [Deferred.asTask][asTask] | Converts a deferred value to a Task + +## Example + +Using Firebase APIs becomes simple: + +```kotlin +FirebaseAuth.getInstance().signInAnonymously().await() +val snapshot = try { + FirebaseFirestore.getInstance().document("users/$id").get().await() // Cancellable await +} catch (e: FirebaseFirestoreException) { + // Handle exception + return@async +} + +// Do stuff +``` + +If the `Task` supports cancellation via passing a `CancellationToken`, pass the corresponding `CancellationTokenSource` to `asDeferred` or `await` to support bi-directional cancellation: + +```kotlin +val cancellationTokenSource = CancellationTokenSource() +val currentLocationTask = fusedLocationProviderClient.getCurrentLocation(PRIORITY_HIGH_ACCURACY, cancellationTokenSource.token) +val currentLocation = currentLocationTask.await(cancellationTokenSource) // cancelling `await` also cancels `currentLocationTask`, and vice versa +``` + + + + + +[asDeferred]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-play-services/kotlinx.coroutines.tasks/as-deferred.html +[await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-play-services/kotlinx.coroutines.tasks/await.html +[asTask]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-play-services/kotlinx.coroutines.tasks/as-task.html + + diff --git a/integration/kotlinx-coroutines-play-services/api/kotlinx-coroutines-play-services.api b/integration/kotlinx-coroutines-play-services/api/kotlinx-coroutines-play-services.api new file mode 100644 index 0000000000..cc23e8db2e --- /dev/null +++ b/integration/kotlinx-coroutines-play-services/api/kotlinx-coroutines-play-services.api @@ -0,0 +1,8 @@ +public final class kotlinx/coroutines/tasks/TasksKt { + public static final fun asDeferred (Lcom/google/android/gms/tasks/Task;)Lkotlinx/coroutines/Deferred; + public static final fun asDeferred (Lcom/google/android/gms/tasks/Task;Lcom/google/android/gms/tasks/CancellationTokenSource;)Lkotlinx/coroutines/Deferred; + public static final fun asTask (Lkotlinx/coroutines/Deferred;)Lcom/google/android/gms/tasks/Task; + public static final fun await (Lcom/google/android/gms/tasks/Task;Lcom/google/android/gms/tasks/CancellationTokenSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun await (Lcom/google/android/gms/tasks/Task;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + diff --git a/integration/kotlinx-coroutines-play-services/build.gradle.kts b/integration/kotlinx-coroutines-play-services/build.gradle.kts new file mode 100644 index 0000000000..2a03440140 --- /dev/null +++ b/integration/kotlinx-coroutines-play-services/build.gradle.kts @@ -0,0 +1,18 @@ +val tasksVersion = "16.0.1" + +project.configureAar() + +dependencies { + configureAarUnpacking() + api("com.google.android.gms:play-services-tasks:$tasksVersion") { + exclude(group="com.android.support") + } + + // Required by robolectric + testImplementation("androidx.test:core:1.2.0") + testImplementation("androidx.test:monitor:1.2.0") +} + +externalDocumentationLink( + url = "https://developers.google.com/android/reference/" +) diff --git a/integration/kotlinx-coroutines-play-services/package.list b/integration/kotlinx-coroutines-play-services/package.list new file mode 100644 index 0000000000..4b22ba1482 --- /dev/null +++ b/integration/kotlinx-coroutines-play-services/package.list @@ -0,0 +1 @@ +com.google.android.gms.tasks \ No newline at end of file diff --git a/integration/kotlinx-coroutines-play-services/src/Tasks.kt b/integration/kotlinx-coroutines-play-services/src/Tasks.kt new file mode 100644 index 0000000000..946449b9df --- /dev/null +++ b/integration/kotlinx-coroutines-play-services/src/Tasks.kt @@ -0,0 +1,163 @@ +@file:Suppress("RedundantVisibilityModifier") + +package kotlinx.coroutines.tasks + +import com.google.android.gms.tasks.* +import kotlinx.coroutines.* +import java.lang.Runnable +import java.util.concurrent.Executor +import kotlin.coroutines.* + +/** + * Converts this deferred to the instance of [Task]. + * If deferred is cancelled then resulting task will be cancelled as well. + */ +public fun Deferred.asTask(): Task { + val cancellation = CancellationTokenSource() + val source = TaskCompletionSource(cancellation.token) + + invokeOnCompletion callback@{ + if (it is CancellationException) { + cancellation.cancel() + return@callback + } + + val t = getCompletionExceptionOrNull() + if (t == null) { + source.setResult(getCompleted()) + } else { + source.setException(t as? Exception ?: RuntimeExecutionException(t)) + } + } + + return source.task +} + +/** + * Converts this task to an instance of [Deferred]. + * If task is cancelled then resulting deferred will be cancelled as well. + * However, the opposite is not true: if the deferred is cancelled, the [Task] will not be cancelled. + * For bi-directional cancellation, an overload that accepts [CancellationTokenSource] can be used. + */ +public fun Task.asDeferred(): Deferred = asDeferredImpl(null) + +/** + * Converts this task to an instance of [Deferred] with a [CancellationTokenSource] to control cancellation. + * The cancellation of this function is bi-directional: + * - If the given task is cancelled, the resulting deferred will be cancelled. + * - If the resulting deferred is cancelled, the provided [cancellationTokenSource] will be cancelled. + * + * Providing a [CancellationTokenSource] that is unrelated to the receiving [Task] is not supported and + * leads to an unspecified behaviour. + */ +@ExperimentalCoroutinesApi // Since 1.5.1, tentatively until 1.6.0 +public fun Task.asDeferred(cancellationTokenSource: CancellationTokenSource): Deferred = + asDeferredImpl(cancellationTokenSource) + +private fun Task.asDeferredImpl(cancellationTokenSource: CancellationTokenSource?): Deferred { + val deferred = CompletableDeferred() + if (isComplete) { + val e = exception + if (e == null) { + if (isCanceled) { + deferred.cancel() + } else { + @Suppress("UNCHECKED_CAST") + deferred.complete(result as T) + } + } else { + deferred.completeExceptionally(e) + } + } else { + // Run the callback directly to avoid unnecessarily scheduling on the main thread. + addOnCompleteListener(DirectExecutor) { + val e = it.exception + if (e == null) { + @Suppress("UNCHECKED_CAST") + if (it.isCanceled) deferred.cancel() else deferred.complete(it.result as T) + } else { + deferred.completeExceptionally(e) + } + } + } + + if (cancellationTokenSource != null) { + deferred.invokeOnCompletion { + cancellationTokenSource.cancel() + } + } + // Prevent casting to CompletableDeferred and manual completion. + @OptIn(InternalForInheritanceCoroutinesApi::class) + return object : Deferred by deferred {} +} + +/** + * Awaits the completion of the task without blocking a thread. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this function + * stops waiting for the completion stage and immediately resumes with [CancellationException]. + * + * For bi-directional cancellation, an overload that accepts [CancellationTokenSource] can be used. + */ +public suspend fun Task.await(): T = awaitImpl(null) + +/** + * Awaits the completion of the task that is linked to the given [CancellationTokenSource] to control cancellation. + * + * This suspending function is cancellable and cancellation is bi-directional: + * - If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this function + * cancels the [cancellationTokenSource] and throws a [CancellationException]. + * - If the task is cancelled, then this function will throw a [CancellationException]. + * + * Providing a [CancellationTokenSource] that is unrelated to the receiving [Task] is not supported and + * leads to an unspecified behaviour. + */ +@ExperimentalCoroutinesApi // Since 1.5.1, tentatively until 1.6.0 +public suspend fun Task.await(cancellationTokenSource: CancellationTokenSource): T = + awaitImpl(cancellationTokenSource) + +private suspend fun Task.awaitImpl(cancellationTokenSource: CancellationTokenSource?): T { + // fast path + if (isComplete) { + val e = exception + return if (e == null) { + if (isCanceled) { + throw CancellationException("Task $this was cancelled normally.") + } else { + @Suppress("UNCHECKED_CAST") + result as T + } + } else { + throw e + } + } + + return suspendCancellableCoroutine { cont -> + // Run the callback directly to avoid unnecessarily scheduling on the main thread. + addOnCompleteListener(DirectExecutor) { + val e = it.exception + if (e == null) { + @Suppress("UNCHECKED_CAST") + if (it.isCanceled) cont.cancel() else cont.resume(it.result as T) + } else { + cont.resumeWithException(e) + } + } + + if (cancellationTokenSource != null) { + cont.invokeOnCancellation { + cancellationTokenSource.cancel() + } + } + } +} + +/** + * An [Executor] that just directly executes the [Runnable]. + */ +private object DirectExecutor : Executor { + override fun execute(r: Runnable) { + r.run() + } +} diff --git a/integration/kotlinx-coroutines-play-services/test/FakeAndroid.kt b/integration/kotlinx-coroutines-play-services/test/FakeAndroid.kt new file mode 100644 index 0000000000..e286ee197b --- /dev/null +++ b/integration/kotlinx-coroutines-play-services/test/FakeAndroid.kt @@ -0,0 +1,25 @@ +package android.os + +import kotlinx.coroutines.GlobalScope +import kotlinx.coroutines.launch +import java.util.concurrent.* + +class Handler(val looper: Looper) { + fun post(r: Runnable): Boolean { + try { + GlobalScope.launch { r.run() } + } catch (e: RejectedExecutionException) { + // Execute leftover callbacks in place for tests + r.run() + } + + return true + } +} + +class Looper { + companion object { + @JvmStatic + fun getMainLooper() = Looper() + } +} diff --git a/integration/kotlinx-coroutines-play-services/test/TaskTest.kt b/integration/kotlinx-coroutines-play-services/test/TaskTest.kt new file mode 100644 index 0000000000..e364383709 --- /dev/null +++ b/integration/kotlinx-coroutines-play-services/test/TaskTest.kt @@ -0,0 +1,416 @@ +package kotlinx.coroutines.tasks + +import kotlinx.coroutines.testing.* +import com.google.android.gms.tasks.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.locks.* +import kotlin.concurrent.* +import kotlin.test.* + +class TaskTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("ForkJoinPool.commonPool-worker-") + } + + @Test + fun testCompletedDeferredAsTask() = runTest { + expect(1) + val deferred = async(start = CoroutineStart.UNDISPATCHED) { + expect(2) // Completed immediately + "OK" + } + expect(3) + val task = deferred.asTask() + assertEquals("OK", task.await()) + finish(4) + } + + @Test + fun testDeferredAsTask() = runTest { + expect(1) + val deferred = async { + expect(3) // Completed later + "OK" + } + expect(2) + val task = deferred.asTask() + assertEquals("OK", task.await()) + finish(4) + } + + @Test + fun testCancelledAsTask() = runTest { + val deferred = async(Dispatchers.Default) { + delay(100) + }.apply { cancel() } + + val task = deferred.asTask() + try { + runTest { task.await() } + } catch (e: Exception) { + assertIs(e) + assertTrue(task.isCanceled) + } + } + + @Test + fun testThrowingAsTask() = runTest({ e -> e is TestException }) { + val deferred = async(Dispatchers.Default) { + throw TestException("Fail") + } + + val task = deferred.asTask() + runTest(expected = { it is TestException }) { + task.await() + } + } + + @Test + fun testStateAsTask() = runTest { + val lock = ReentrantLock().apply { lock() } + + val deferred: Deferred = Tasks.call { + lock.withLock { 42 } + }.asDeferred() + + assertFalse(deferred.isCompleted) + lock.unlock() + + assertEquals(42, deferred.await()) + assertTrue(deferred.isCompleted) + } + + @Test + fun testTaskAsDeferred() = runTest { + val deferred = Tasks.forResult(42).asDeferred() + assertEquals(42, deferred.await()) + } + + @Test + fun testNullResultTaskAsDeferred() = runTest { + assertNull(Tasks.forResult(null).asDeferred().await()) + } + + @Test + fun testCancelledTaskAsDeferred() = runTest { + val deferred = Tasks.forCanceled().asDeferred() + + assertTrue(deferred.isCancelled) + try { + deferred.await() + fail("deferred.await() should be cancelled") + } catch (e: Exception) { + assertIs(e) + } + } + + @Test + fun testFailedTaskAsDeferred() = runTest { + val deferred = Tasks.forException(TestException("something went wrong")).asDeferred() + + assertTrue(deferred.isCancelled && deferred.isCompleted) + val completionException = deferred.getCompletionExceptionOrNull()!! + assertIs(completionException) + assertEquals("something went wrong", completionException.message) + + try { + deferred.await() + fail("deferred.await() should throw an exception") + } catch (e: Exception) { + assertIs(e) + assertEquals("something went wrong", e.message) + } + } + + @Test + fun testFailingTaskAsDeferred() = runTest { + val lock = ReentrantLock().apply { lock() } + + val deferred: Deferred = Tasks.call { + lock.withLock { throw TestException("something went wrong") } + }.asDeferred() + + assertFalse(deferred.isCompleted) + lock.unlock() + + try { + deferred.await() + fail("deferred.await() should throw an exception") + } catch (e: Exception) { + assertIs(e) + assertEquals("something went wrong", e.message) + assertSame(e.cause, deferred.getCompletionExceptionOrNull()) // debug mode stack augmentation + } + } + + @Test + fun testCancellableTaskAsDeferred() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val deferred = Tasks.forResult(42).asDeferred(cancellationTokenSource) + assertEquals(42, deferred.await()) + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testNullResultCancellableTaskAsDeferred() = runTest { + val cancellationTokenSource = CancellationTokenSource() + assertNull(Tasks.forResult(null).asDeferred(cancellationTokenSource).await()) + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testCancelledCancellableTaskAsDeferred() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val deferred = Tasks.forCanceled().asDeferred(cancellationTokenSource) + + assertTrue(deferred.isCancelled) + try { + deferred.await() + fail("deferred.await() should be cancelled") + } catch (e: Exception) { + assertIs(e) + } + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testCancellingCancellableTaskAsDeferred() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val task = TaskCompletionSource(cancellationTokenSource.token).task + val deferred = task.asDeferred(cancellationTokenSource) + + deferred.cancel() + try { + deferred.await() + fail("deferred.await() should be cancelled") + } catch (e: Exception) { + assertIs(e) + } + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testExternallyCancelledCancellableTaskAsDeferred() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val task = TaskCompletionSource(cancellationTokenSource.token).task + val deferred = task.asDeferred(cancellationTokenSource) + + cancellationTokenSource.cancel() + + try { + deferred.await() + fail("deferred.await() should be cancelled") + } catch (e: Exception) { + assertIs(e) + } + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testSeparatelyCancelledCancellableTaskAsDeferred() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val task = TaskCompletionSource().task + task.asDeferred(cancellationTokenSource) + + cancellationTokenSource.cancel() + + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testFailedCancellableTaskAsDeferred() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val deferred = + Tasks.forException(TestException("something went wrong")).asDeferred(cancellationTokenSource) + + assertTrue(deferred.isCancelled && deferred.isCompleted) + val completionException = deferred.getCompletionExceptionOrNull()!! + assertIs(completionException) + assertEquals("something went wrong", completionException.message) + + try { + deferred.await() + fail("deferred.await() should throw an exception") + } catch (e: Exception) { + assertIs(e) + assertEquals("something went wrong", e.message) + } + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testFailingCancellableTaskAsDeferred() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val lock = ReentrantLock().apply { lock() } + + val deferred: Deferred = Tasks.call { + lock.withLock { throw TestException("something went wrong") } + }.asDeferred(cancellationTokenSource) + + assertFalse(deferred.isCompleted) + lock.unlock() + + try { + deferred.await() + fail("deferred.await() should throw an exception") + } catch (e: Exception) { + assertIs(e) + assertEquals("something went wrong", e.message) + assertSame(e.cause, deferred.getCompletionExceptionOrNull()) // debug mode stack augmentation + } + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testFastPathCompletedTaskWithCancelledTokenSourceAsDeferred() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val deferred = Tasks.forResult(42).asDeferred(cancellationTokenSource) + cancellationTokenSource.cancel() + assertEquals(42, deferred.await()) + } + + @Test + fun testAwaitCancellableTask() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val taskCompletionSource = TaskCompletionSource(cancellationTokenSource.token) + + val deferred: Deferred = async(start = CoroutineStart.UNDISPATCHED) { + taskCompletionSource.task.await(cancellationTokenSource) + } + + assertFalse(deferred.isCompleted) + taskCompletionSource.setResult(42) + + assertEquals(42, deferred.await()) + assertTrue(deferred.isCompleted) + } + + @Test + fun testFailedAwaitTask() = runTest(expected = { it is TestException }) { + val cancellationTokenSource = CancellationTokenSource() + val taskCompletionSource = TaskCompletionSource(cancellationTokenSource.token) + + val deferred: Deferred = async(start = CoroutineStart.UNDISPATCHED) { + taskCompletionSource.task.await(cancellationTokenSource) + } + + assertFalse(deferred.isCompleted) + taskCompletionSource.setException(TestException("something went wrong")) + + deferred.await() + } + + @Test + fun testCancelledAwaitCancellableTask() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val taskCompletionSource = TaskCompletionSource(cancellationTokenSource.token) + + val deferred: Deferred = async(start = CoroutineStart.UNDISPATCHED) { + taskCompletionSource.task.await(cancellationTokenSource) + } + + assertFalse(deferred.isCompleted) + // Cancel the deferred + deferred.cancel() + + try { + deferred.await() + fail("deferred.await() should be cancelled") + } catch (e: Exception) { + assertIs(e) + } + + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testExternallyCancelledAwaitCancellableTask() = runTest { + val cancellationTokenSource = CancellationTokenSource() + val taskCompletionSource = TaskCompletionSource(cancellationTokenSource.token) + + val deferred: Deferred = async(start = CoroutineStart.UNDISPATCHED) { + taskCompletionSource.task.await(cancellationTokenSource) + } + + assertFalse(deferred.isCompleted) + // Cancel the cancellation token source + cancellationTokenSource.cancel() + + try { + deferred.await() + fail("deferred.await() should be cancelled") + } catch (e: Exception) { + assertIs(e) + } + + assertTrue(cancellationTokenSource.token.isCancellationRequested) + } + + @Test + fun testFastPathCancellationTokenSourceCancelledAwaitCancellableTask() = runTest { + val cancellationTokenSource = CancellationTokenSource() + // Construct a task without the cancellation token source + val taskCompletionSource = TaskCompletionSource() + + val deferred: Deferred = async(start = CoroutineStart.LAZY) { + taskCompletionSource.task.await(cancellationTokenSource) + } + + assertFalse(deferred.isCompleted) + cancellationTokenSource.cancel() + + // Cancelling the token doesn't cancel the deferred + assertTrue(cancellationTokenSource.token.isCancellationRequested) + assertFalse(deferred.isCompleted) + + // Cleanup + deferred.cancel() + } + + @Test + fun testSlowPathCancellationTokenSourceCancelledAwaitCancellableTask() = runTest { + val cancellationTokenSource = CancellationTokenSource() + // Construct a task without the cancellation token source + val taskCompletionSource = TaskCompletionSource() + + val deferred: Deferred = async(start = CoroutineStart.UNDISPATCHED) { + taskCompletionSource.task.await(cancellationTokenSource) + } + + assertFalse(deferred.isCompleted) + cancellationTokenSource.cancel() + + // Cancelling the token doesn't cancel the deferred + assertTrue(cancellationTokenSource.token.isCancellationRequested) + assertFalse(deferred.isCompleted) + + // Cleanup + deferred.cancel() + } + + @Test + fun testFastPathWithCompletedTaskAndCanceledTokenSourceAwaitTask() = runTest { + val firstCancellationTokenSource = CancellationTokenSource() + val secondCancellationTokenSource = CancellationTokenSource() + // Construct a task with a different cancellation token source + val taskCompletionSource = TaskCompletionSource(firstCancellationTokenSource.token) + + val deferred: Deferred = async(start = CoroutineStart.LAZY) { + taskCompletionSource.task.await(secondCancellationTokenSource) + } + + assertFalse(deferred.isCompleted) + secondCancellationTokenSource.cancel() + + assertFalse(deferred.isCompleted) + taskCompletionSource.setResult(42) + + assertEquals(42, deferred.await()) + assertTrue(deferred.isCompleted) + } + + class TestException(message: String) : Exception(message) +} diff --git a/integration/kotlinx-coroutines-slf4j/README.md b/integration/kotlinx-coroutines-slf4j/README.md new file mode 100644 index 0000000000..37f87008d6 --- /dev/null +++ b/integration/kotlinx-coroutines-slf4j/README.md @@ -0,0 +1,26 @@ +# Module kotlinx-coroutines-slf4j + +Integration with SLF4J [MDC](https://logback.qos.ch/manual/mdc.html). + +## Example + +Add [MDCContext] to the coroutine context so that the SLF4J MDC context is captured and passed into the coroutine. + +```kotlin +MDC.put("kotlin", "rocks") // put a value into the MDC context + +launch(MDCContext()) { + logger.info { "..." } // the MDC context will contain the mapping here +} +``` + +# Package kotlinx.coroutines.slf4j + +Integration with SLF4J [MDC](https://logback.qos.ch/manual/mdc.html). + + + + +[MDCContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-slf4j/kotlinx.coroutines.slf4j/-m-d-c-context/index.html + + diff --git a/integration/kotlinx-coroutines-slf4j/api/kotlinx-coroutines-slf4j.api b/integration/kotlinx-coroutines-slf4j/api/kotlinx-coroutines-slf4j.api new file mode 100644 index 0000000000..6b565d4c1a --- /dev/null +++ b/integration/kotlinx-coroutines-slf4j/api/kotlinx-coroutines-slf4j.api @@ -0,0 +1,15 @@ +public final class kotlinx/coroutines/slf4j/MDCContext : kotlin/coroutines/AbstractCoroutineContextElement, kotlinx/coroutines/ThreadContextElement { + public static final field Key Lkotlinx/coroutines/slf4j/MDCContext$Key; + public fun ()V + public fun (Ljava/util/Map;)V + public synthetic fun (Ljava/util/Map;ILkotlin/jvm/internal/DefaultConstructorMarker;)V + public final fun getContextMap ()Ljava/util/Map; + public synthetic fun restoreThreadContext (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Object;)V + public fun restoreThreadContext (Lkotlin/coroutines/CoroutineContext;Ljava/util/Map;)V + public synthetic fun updateThreadContext (Lkotlin/coroutines/CoroutineContext;)Ljava/lang/Object; + public fun updateThreadContext (Lkotlin/coroutines/CoroutineContext;)Ljava/util/Map; +} + +public final class kotlinx/coroutines/slf4j/MDCContext$Key : kotlin/coroutines/CoroutineContext$Key { +} + diff --git a/integration/kotlinx-coroutines-slf4j/build.gradle.kts b/integration/kotlinx-coroutines-slf4j/build.gradle.kts new file mode 100644 index 0000000000..aaf0a42360 --- /dev/null +++ b/integration/kotlinx-coroutines-slf4j/build.gradle.kts @@ -0,0 +1,10 @@ +dependencies { + implementation("org.slf4j:slf4j-api:1.7.32") + testImplementation("io.github.microutils:kotlin-logging:2.1.0") + testRuntimeOnly("ch.qos.logback:logback-classic:1.2.7") + testRuntimeOnly("ch.qos.logback:logback-core:1.2.7") +} + +externalDocumentationLink( + url = "https://www.slf4j.org/apidocs/" +) diff --git a/integration/kotlinx-coroutines-slf4j/package.list b/integration/kotlinx-coroutines-slf4j/package.list new file mode 100644 index 0000000000..bfea07f520 --- /dev/null +++ b/integration/kotlinx-coroutines-slf4j/package.list @@ -0,0 +1,21 @@ +org.apache.commons.logging +org.apache.commons.logging.impl +org.apache.log4j +org.apache.log4j.helpers +org.apache.log4j.spi +org.apache.log4j.xml +org.slf4j +org.slf4j.agent +org.slf4j.bridge +org.slf4j.cal10n +org.slf4j.event +org.slf4j.ext +org.slf4j.helpers +org.slf4j.instrumentation +org.slf4j.jul +org.slf4j.log4j12 +org.slf4j.nop +org.slf4j.osgi.logservice.impl +org.slf4j.profiler +org.slf4j.simple +org.slf4j.spi diff --git a/integration/kotlinx-coroutines-slf4j/src/MDCContext.kt b/integration/kotlinx-coroutines-slf4j/src/MDCContext.kt new file mode 100644 index 0000000000..32830fc818 --- /dev/null +++ b/integration/kotlinx-coroutines-slf4j/src/MDCContext.kt @@ -0,0 +1,108 @@ +package kotlinx.coroutines.slf4j + +import kotlinx.coroutines.* +import org.slf4j.MDC +import kotlin.coroutines.AbstractCoroutineContextElement +import kotlin.coroutines.CoroutineContext + +/** + * The value of [MDC] context map. + * See [MDC.getCopyOfContextMap]. + */ +public typealias MDCContextMap = Map? + +/** + * [MDC] context element for [CoroutineContext]. + * + * Example: + * + * ``` + * MDC.put("kotlin", "rocks") // Put a value into the MDC context + * + * launch(MDCContext()) { + * logger.info { "..." } // The MDC context contains the mapping here + * } + * ``` + * + * Note that you cannot update MDC context from inside the coroutine simply + * using [MDC.put]. These updates are going to be lost on the next suspension and + * reinstalled to the MDC context that was captured or explicitly specified in + * [contextMap] when this object was created on the next resumption. + * + * For example, the following code will not work as expected: + * + * ``` + * launch(MDCContext()) { + * MDC.put("key", "value") // This update will be lost + * delay(100) + * println(MDC.get("key")) // This will print null + * } + * ``` + * + * Instead, you should use [withContext] to capture the updated MDC context: + * + * ``` + * launch(MDCContext()) { + * MDC.put("key", "value") // This update will be captured + * withContext(MDCContext()) { + * delay(100) + * println(MDC.get("key")) // This will print "value" + * } + * } + * ``` + * + * There is no way to implicitly propagate MDC context updates from inside the coroutine to the outer scope. + * You have to capture the updated MDC context and restore it explicitly. For example: + * + * ``` + * MDC.put("a", "b") + * val contextMap = withContext(MDCContext()) { + * MDC.put("key", "value") + * withContext(MDCContext()) { + * MDC.put("key2", "value2") + * withContext(MDCContext()) { + * yield() + * MDC.getCopyOfContextMap() + * } + * } + * } + * // contextMap contains: {"a"="b", "key"="value", "key2"="value2"} + * MDC.setContextMap(contextMap) + * ``` + * + * @param contextMap the value of [MDC] context map. + * Default value is the copy of the current thread's context map that is acquired via + * [MDC.getCopyOfContextMap]. + */ +public class MDCContext( + /** + * The value of [MDC] context map. + */ + @Suppress("MemberVisibilityCanBePrivate") + public val contextMap: MDCContextMap = MDC.getCopyOfContextMap() +) : ThreadContextElement, AbstractCoroutineContextElement(Key) { + /** + * Key of [MDCContext] in [CoroutineContext]. + */ + public companion object Key : CoroutineContext.Key + + /** @suppress */ + override fun updateThreadContext(context: CoroutineContext): MDCContextMap { + val oldState = MDC.getCopyOfContextMap() + setCurrent(contextMap) + return oldState + } + + /** @suppress */ + override fun restoreThreadContext(context: CoroutineContext, oldState: MDCContextMap) { + setCurrent(oldState) + } + + private fun setCurrent(contextMap: MDCContextMap) { + if (contextMap == null) { + MDC.clear() + } else { + MDC.setContextMap(contextMap) + } + } +} diff --git a/integration/kotlinx-coroutines-slf4j/src/module-info.java b/integration/kotlinx-coroutines-slf4j/src/module-info.java new file mode 100644 index 0000000000..57e5aae4d0 --- /dev/null +++ b/integration/kotlinx-coroutines-slf4j/src/module-info.java @@ -0,0 +1,7 @@ +module kotlinx.coroutines.slf4j { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires org.slf4j; + + exports kotlinx.coroutines.slf4j; +} diff --git a/integration/kotlinx-coroutines-slf4j/test-resources/logback-test.xml b/integration/kotlinx-coroutines-slf4j/test-resources/logback-test.xml new file mode 100644 index 0000000000..8051011490 --- /dev/null +++ b/integration/kotlinx-coroutines-slf4j/test-resources/logback-test.xml @@ -0,0 +1,15 @@ + + + + + + %X{first} %X{last} - %m%n + + + + + + + + + diff --git a/integration/kotlinx-coroutines-slf4j/test/MDCContextTest.kt b/integration/kotlinx-coroutines-slf4j/test/MDCContextTest.kt new file mode 100644 index 0000000000..dd3b34cdee --- /dev/null +++ b/integration/kotlinx-coroutines-slf4j/test/MDCContextTest.kt @@ -0,0 +1,144 @@ +package kotlinx.coroutines.slf4j + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import org.slf4j.* +import kotlin.coroutines.* +import kotlin.test.* + +class MDCContextTest : TestBase() { + @Before + fun setUp() { + MDC.clear() + } + + @After + fun tearDown() { + MDC.clear() + } + + @Test + fun testContextIsNotPassedByDefaultBetweenCoroutines() = runTest { + expect(1) + MDC.put("myKey", "myValue") + // Standalone launch + GlobalScope.launch { + assertNull(MDC.get("myKey")) + expect(2) + }.join() + finish(3) + } + + @Test + fun testContextCanBePassedBetweenCoroutines() = runTest { + expect(1) + MDC.put("myKey", "myValue") + // Scoped launch with MDCContext element + launch(MDCContext()) { + assertEquals("myValue", MDC.get("myKey")) + expect(2) + }.join() + + finish(3) + } + + @Test + fun testContextInheritance() = runTest { + expect(1) + MDC.put("myKey", "myValue") + withContext(MDCContext()) { + MDC.put("myKey", "myValue2") + // Scoped launch with inherited MDContext element + launch(Dispatchers.Default) { + assertEquals("myValue", MDC.get("myKey")) + expect(2) + }.join() + + finish(3) + } + assertEquals("myValue", MDC.get("myKey")) + } + + @Test + fun testContextPassedWhileOnMainThread() { + MDC.put("myKey", "myValue") + // No MDCContext element + runBlocking { + assertEquals("myValue", MDC.get("myKey")) + } + } + + @Test + fun testContextCanBePassedWhileOnMainThread() { + MDC.put("myKey", "myValue") + runBlocking(MDCContext()) { + assertEquals("myValue", MDC.get("myKey")) + } + } + + @Test + fun testContextNeededWithOtherContext() { + MDC.put("myKey", "myValue") + runBlocking(MDCContext()) { + assertEquals("myValue", MDC.get("myKey")) + } + } + + @Test + fun testContextMayBeEmpty() { + runBlocking(MDCContext()) { + assertNull(MDC.get("myKey")) + } + } + + @Test + fun testContextWithContext() = runTest { + MDC.put("myKey", "myValue") + val mainDispatcher = kotlin.coroutines.coroutineContext[ContinuationInterceptor]!! + withContext(Dispatchers.Default + MDCContext()) { + assertEquals("myValue", MDC.get("myKey")) + assertEquals("myValue", coroutineContext[MDCContext]?.contextMap?.get("myKey")) + withContext(mainDispatcher) { + assertEquals("myValue", MDC.get("myKey")) + } + } + } + + /** Tests that the initially captured MDC context gets restored after suspension. */ + @Test + fun testSuspensionsUndoingMdcContextUpdates() = runTest { + MDC.put("a", "b") + withContext(MDCContext()) { + MDC.put("key", "value") + assertEquals("b", MDC.get("a")) + yield() + assertNull(MDC.get("key")) + assertEquals("b", MDC.get("a")) + } + } + + /** Tests capturing and restoring the MDC context. */ + @Test + fun testRestoringMdcContext() = runTest { + MDC.put("a", "b") + val contextMap = withContext(MDCContext()) { + MDC.put("key", "value") + assertEquals("b", MDC.get("a")) + withContext(MDCContext()) { + assertEquals("value", MDC.get("key")) + MDC.put("key2", "value2") + assertEquals("value2", MDC.get("key2")) + withContext(MDCContext()) { + yield() + MDC.getCopyOfContextMap() + } + } + } + MDC.setContextMap(contextMap) + assertEquals("value2", MDC.get("key2")) + assertEquals("value", MDC.get("key")) + assertEquals("b", MDC.get("a")) + } +} diff --git a/knit.properties b/knit.properties new file mode 100644 index 0000000000..486204bb1f --- /dev/null +++ b/knit.properties @@ -0,0 +1,12 @@ +knit.include=docs/knit.code.include +test.template=docs/knit.test.template + +# Various test validation modes and their corresponding methods from TestUtil +test.mode.=verifyLines +test.mode.STARTS_WITH=verifyLinesStartWith +test.mode.ARBITRARY_TIME=verifyLinesArbitraryTime +test.mode.FLEXIBLE_TIME=verifyLinesFlexibleTime +test.mode.FLEXIBLE_THREAD=verifyLinesFlexibleThread +test.mode.LINES_START_UNORDERED=verifyLinesStartUnordered +test.mode.LINES_START=verifyLinesStart +test.mode.EXCEPTION=verifyExceptions diff --git a/knit/README.md b/knit/README.md deleted file mode 100644 index a6a31b00f4..0000000000 --- a/knit/README.md +++ /dev/null @@ -1,15 +0,0 @@ -# Knit - -This is a very simple tool that produces Kotlin source example files from a markdown document that includes -snippets of Kotlin code in its body. It is used to produce examples for -[coroutines guide](../coroutines-guide.md). - -## Updating guide - -* In project root directory do: - * Run `mvn clean` - * Run `mvn compile` - * Run `mvn pre-site` (or `mvn site` if you have Jekyll) - * Run `Knit coroutines-guide.md` (from IDEA, mark `knit/src` as source root first) -* Commit updated `coroutines-guide.md` and examples - diff --git a/knit/src/Knit.kt b/knit/src/Knit.kt deleted file mode 100644 index 829a7e1433..0000000000 --- a/knit/src/Knit.kt +++ /dev/null @@ -1,457 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -import java.io.* - -const val DIRECTIVE_START = "" - -const val TOC_DIRECTIVE = "TOC" -const val KNIT_DIRECTIVE = "KNIT" -const val INCLUDE_DIRECTIVE = "INCLUDE" -const val CLEAR_DIRECTIVE = "CLEAR" -const val TEST_DIRECTIVE = "TEST" - -const val TEST_OUT_DIRECTIVE = "TEST_OUT" - -const val SITE_ROOT_DIRECTIVE = "SITE_ROOT" -const val DOCS_ROOT_DIRECTIVE = "DOCS_ROOT" -const val INDEX_DIRECTIVE = "INDEX" - -const val CODE_START = "```kotlin" -const val CODE_END = "```" - -const val TEST_START = "```text" -const val TEST_END = "```" - -const val SECTION_START = "##" - -const val PACKAGE_PREFIX = "package " -const val STARTS_WITH_PREDICATE = "STARTS_WITH" -const val ARBITRARY_TIME_PREDICATE = "ARBITRARY_TIME" -const val FLEXIBLE_TIME_PREDICATE = "FLEXIBLE_TIME" -const val FLEXIBLE_THREAD_PREDICATE = "FLEXIBLE_THREAD" -const val LINES_START_UNORDERED_PREDICATE = "LINES_START_UNORDERED" -const val LINES_START_PREDICATE = "LINES_START" - -val API_REF_REGEX = Regex("(^|[ \\]])\\[([A-Za-z0-9_.]+)\\]($|[^\\[\\(])") - -fun main(args: Array) { - if (args.isEmpty()) { - println("Usage: Knit ") - return - } - args.forEach(::knit) -} - -fun knit(markdownFileName: String) { - println("*** Reading $markdownFileName") - val markdownFile = File(markdownFileName) - val tocLines = arrayListOf() - var knitRegex: Regex? = null - val includes = arrayListOf() - val codeLines = arrayListOf() - val testLines = arrayListOf() - var testOut: String? = null - val testOutLines = arrayListOf() - var lastPgk: String? = null - val files = mutableSetOf() - val allApiRefs = arrayListOf() - val remainingApiRefNames = mutableSetOf() - var siteRoot: String? = null - var docsRoot: String? = null - // read markdown file - var putBackLine: String? = null - val markdown = markdownFile.withMarkdownTextReader { - mainLoop@ while (true) { - val inLine = putBackLine ?: readLine() ?: break - putBackLine = null - val directive = directive(inLine) - if (directive != null && markdownPart == MarkdownPart.TOC) { - markdownPart = MarkdownPart.POST_TOC - postTocText += inLine - } - when (directive?.name) { - TOC_DIRECTIVE -> { - requireSingleLine(directive) - require(directive.param.isEmpty()) { "$TOC_DIRECTIVE directive must not have parameters" } - require(markdownPart == MarkdownPart.PRE_TOC) { "Only one TOC directive is supported" } - markdownPart = MarkdownPart.TOC - } - KNIT_DIRECTIVE -> { - requireSingleLine(directive) - require(!directive.param.isEmpty()) { "$KNIT_DIRECTIVE directive must include regex parameter" } - require(knitRegex == null) { "Only one KNIT directive is supported"} - knitRegex = Regex("\\((" + directive.param + ")\\)") - continue@mainLoop - } - INCLUDE_DIRECTIVE -> { - if (directive.param.isEmpty()) { - require(!directive.singleLine) { "$INCLUDE_DIRECTIVE directive without parameters must not be single line" } - readUntilTo(DIRECTIVE_END, codeLines) - } else { - val include = Include(Regex(directive.param)) - if (directive.singleLine) { - include.lines += codeLines - codeLines.clear() - } else { - readUntilTo(DIRECTIVE_END, include.lines) - } - includes += include - } - continue@mainLoop - } - CLEAR_DIRECTIVE -> { - requireSingleLine(directive) - require(directive.param.isEmpty()) { "$CLEAR_DIRECTIVE directive must not have parameters" } - codeLines.clear() - continue@mainLoop - } - TEST_OUT_DIRECTIVE -> { - require(!directive.param.isEmpty()) { "$TEST_OUT_DIRECTIVE directive must include file name parameter" } - flushTestOut(markdownFile.parentFile, testOut, testOutLines) - testOut = directive.param - readUntil(DIRECTIVE_END).forEach { testOutLines += it } - } - TEST_DIRECTIVE -> { - require(lastPgk != null) { "'$PACKAGE_PREFIX' prefix was not found in emitted code"} - require(testOut != null) { "$TEST_OUT_DIRECTIVE directive was not specified" } - var predicate = directive.param - if (testLines.isEmpty()) { - if (directive.singleLine) { - require(!predicate.isEmpty()) { "$TEST_OUT_DIRECTIVE must be preceded by $TEST_START block or contain test predicate"} - } else - testLines += readUntil(DIRECTIVE_END) - } else { - requireSingleLine(directive) - } - makeTest(testOutLines, lastPgk!!, testLines, predicate) - testLines.clear() - } - SITE_ROOT_DIRECTIVE -> { - requireSingleLine(directive) - siteRoot = directive.param - } - DOCS_ROOT_DIRECTIVE -> { - requireSingleLine(directive) - docsRoot = directive.param - } - INDEX_DIRECTIVE -> { - requireSingleLine(directive) - require(siteRoot != null) { "$SITE_ROOT_DIRECTIVE must be specified" } - require(docsRoot != null) { "$DOCS_ROOT_DIRECTIVE must be specified" } - val indexLines = processApiIndex(siteRoot!!, docsRoot!!, directive.param, remainingApiRefNames) - skip = true - while (true) { - val skipLine = readLine() ?: break@mainLoop - if (directive(skipLine) != null) { - putBackLine = skipLine - break - } - } - skip = false - outText += indexLines - outText += putBackLine!! - } - } - if (inLine.startsWith(CODE_START)) { - require(testOut == null || testLines.isEmpty()) { "Previous test was not emitted with $TEST_DIRECTIVE" } - codeLines += "" - readUntilTo(CODE_END, codeLines) - continue@mainLoop - } - if (inLine.startsWith(TEST_START)) { - require(testOut == null || testLines.isEmpty()) { "Previous test was not emitted with $TEST_DIRECTIVE" } - readUntilTo(TEST_END, testLines) - continue@mainLoop - } - if (inLine.startsWith(SECTION_START) && markdownPart == MarkdownPart.POST_TOC) { - val i = inLine.indexOf(' ') - require(i >= 2) { "Invalid section start" } - val name = inLine.substring(i + 1).trim() - tocLines += " ".repeat(i - 2) + "* [$name](#${makeSectionRef(name)})" - continue@mainLoop - } - for (match in API_REF_REGEX.findAll(inLine)) { - val apiRef = ApiRef(lineNumber, match.groups[2]!!.value) - allApiRefs += apiRef - remainingApiRefNames += apiRef.name - } - knitRegex?.find(inLine)?.let { knitMatch -> - val fileName = knitMatch.groups[1]!!.value - val file = File(markdownFile.parentFile, fileName) - require(files.add(file)) { "Duplicate file: $file"} - println("Knitting $file ...") - val outLines = arrayListOf() - for (include in includes) { - val includeMatch = include.regex.matchEntire(fileName) ?: continue - include.lines.forEach { includeLine -> - val line = makeReplacements(includeLine, includeMatch) - if (line.startsWith(PACKAGE_PREFIX)) - lastPgk = line.substring(PACKAGE_PREFIX.length).trim() - outLines += line - } - } - outLines += codeLines - codeLines.clear() - writeLinesIfNeeded(file, outLines) - } - } - } - // update markdown file with toc - val newLines = buildList { - addAll(markdown.preTocText) - if (!tocLines.isEmpty()) { - add("") - addAll(tocLines) - add("") - } - addAll(markdown.postTocText) - } - if (newLines != markdown.inText) writeLines(markdownFile, newLines) - // check apiRefs - for (apiRef in allApiRefs) { - if (apiRef.name in remainingApiRefNames) { - println("WARNING: $markdownFile: ${apiRef.line}: Broken reference to [${apiRef.name}]") - } - } - // write test output - flushTestOut(markdownFile.parentFile, testOut, testOutLines) -} - -fun makeTest(testOutLines: MutableList, pgk: String, test: List, predicate: String) { - val funName = buildString { - var cap = true - for (c in pgk) { - if (c == '.') { - cap = true - } else { - append(if (cap) c.toUpperCase() else c) - cap = false - } - } - } - testOutLines += "" - testOutLines += " @Test" - testOutLines += " fun test$funName() {" - val prefix = " test { $pgk.main(emptyArray()) }" - when (predicate) { - "" -> makeTestLines(testOutLines, prefix, "verifyLines", test) - STARTS_WITH_PREDICATE -> makeTestLines(testOutLines, prefix, "verifyLinesStartWith", test) - ARBITRARY_TIME_PREDICATE -> makeTestLines(testOutLines, prefix, "verifyLinesArbitraryTime", test) - FLEXIBLE_TIME_PREDICATE -> makeTestLines(testOutLines, prefix, "verifyLinesFlexibleTime", test) - FLEXIBLE_THREAD_PREDICATE -> makeTestLines(testOutLines, prefix, "verifyLinesFlexibleThread", test) - LINES_START_UNORDERED_PREDICATE -> makeTestLines(testOutLines, prefix, "verifyLinesStartUnordered", test) - LINES_START_PREDICATE -> makeTestLines(testOutLines, prefix, "verifyLinesStart", test) - else -> { - testOutLines += prefix + ".also { lines ->" - testOutLines += " check($predicate)" - testOutLines += " }" - } - } - testOutLines += " }" -} - -private fun makeTestLines(testOutLines: MutableList, prefix: String, method: String, test: List) { - testOutLines += "$prefix.$method(" - for ((index, testLine) in test.withIndex()) { - val commaOpt = if (index < test.size - 1) "," else "" - val escapedLine = testLine.replace("\"", "\\\"") - testOutLines += " \"$escapedLine\"$commaOpt" - } - testOutLines += " )" -} - -private fun makeReplacements(line: String, match: MatchResult): String { - var result = line - for ((id, group) in match.groups.withIndex()) { - if (group != null) - result = result.replace("\$\$$id", group.value) - } - return result -} - -private fun flushTestOut(parentDir: File?, testOut: String?, testOutLines: MutableList) { - if (testOut == null) return - val file = File(parentDir, testOut) - testOutLines += "}" - writeLinesIfNeeded(file, testOutLines) - testOutLines.clear() -} - -private fun MarkdownTextReader.readUntil(marker: String): List = - arrayListOf().also { readUntilTo(marker, it) } - -private fun MarkdownTextReader.readUntilTo(marker: String, list: MutableList) { - while (true) { - val line = readLine() ?: break - if (line.startsWith(marker)) break - list += line - } -} - -private inline fun buildList(block: ArrayList.() -> Unit): List { - val result = arrayListOf() - result.block() - return result -} - -private fun requireSingleLine(directive: Directive) { - require(directive.singleLine) { "${directive.name} directive must end on the same line with '$DIRECTIVE_END'" } -} - -fun makeSectionRef(name: String): String = name.replace(' ', '-').replace(".", "").toLowerCase() - -class Include(val regex: Regex, val lines: MutableList = arrayListOf()) - -class Directive( - val name: String, - val param: String, - val singleLine: Boolean -) - -fun directive(line: String): Directive? { - if (!line.startsWith(DIRECTIVE_START)) return null - var s = line.substring(DIRECTIVE_START.length).trim() - val singleLine = s.endsWith(DIRECTIVE_END) - if (singleLine) s = s.substring(0, s.length - DIRECTIVE_END.length) - val i = s.indexOf(' ') - val name = if (i < 0) s else s.substring(0, i) - val param = if (i < 0) "" else s.substring(i).trim() - return Directive(name, param, singleLine) -} - -class ApiRef(val line: Int, val name: String) - -enum class MarkdownPart { PRE_TOC, TOC, POST_TOC } - -class MarkdownTextReader(r: Reader) : LineNumberReader(r) { - val inText = arrayListOf() - val preTocText = arrayListOf() - val postTocText = arrayListOf() - var markdownPart: MarkdownPart = MarkdownPart.PRE_TOC - var skip = false - - val outText: MutableList get() = when (markdownPart) { - MarkdownPart.PRE_TOC -> preTocText - MarkdownPart.POST_TOC -> postTocText - else -> throw IllegalStateException("Wrong state: $markdownPart") - } - - override fun readLine(): String? { - val line = super.readLine() ?: return null - inText += line - if (!skip && markdownPart != MarkdownPart.TOC) - outText += line - return line - } -} - -fun File.withLineNumberReader(factory: (Reader) -> T, block: T.() -> Unit): T { - val reader = factory(reader()) - reader.use { - try { - it.block() - } catch (e: IllegalArgumentException) { - println("ERROR: ${this@withLineNumberReader}: ${it.lineNumber}: ${e.message}") - } - } - return reader -} - -fun File.withMarkdownTextReader(block: MarkdownTextReader.() -> Unit): MarkdownTextReader = - withLineNumberReader(::MarkdownTextReader, block) - -fun writeLinesIfNeeded(file: File, outLines: List) { - val oldLines = try { - file.readLines() - } catch (e: IOException) { - emptyList() - } - if (outLines != oldLines) writeLines(file, outLines) -} - -fun writeLines(file: File, lines: List) { - println(" Writing $file ...") - file.parentFile?.mkdirs() - file.printWriter().use { out -> - lines.forEach { out.println(it) } - } -} - -data class ApiIndexKey( - val docsRoot: String, - val pkg: String -) - -val apiIndexCache: MutableMap> = HashMap() - -val REF_LINE_REGEX = Regex("([a-zA-z.]+)") -val INDEX_HTML = "/index.html" -val INDEX_MD = "/index.md" - -fun loadApiIndex( - docsRoot: String, - path: String, - pkg: String, - namePrefix: String = "" -): Map { - val fileName = docsRoot + "/" + path + INDEX_MD - val visited = mutableSetOf() - val map = HashMap() - File(fileName).withLineNumberReader(::LineNumberReader) { - while (true) { - val line = readLine() ?: break - val result = REF_LINE_REGEX.matchEntire(line) ?: continue - val refLink = result.groups[1]!!.value - if (refLink.startsWith("..")) continue // ignore cross-references - val refName = namePrefix + result.groups[2]!!.value - map.put(refName, path + "/" + refLink) - map.put(pkg + "." + refName, path + "/" + refLink) - if (refLink.endsWith(INDEX_HTML)) { - if (visited.add(refLink)) { - val path2 = path + "/" + refLink.substring(0, refLink.length - INDEX_HTML.length) - map += loadApiIndex(docsRoot, path2, pkg, refName + ".") - } - } - } - } - return map -} - -fun processApiIndex( - siteRoot: String, - docsRoot: String, - pkg: String, - remainingApiRefNames: MutableSet -): List { - val key = ApiIndexKey(docsRoot, pkg) - val map = apiIndexCache.getOrPut(key, { - print("Parsing API docs at $docsRoot/$pkg: ") - val result = loadApiIndex(docsRoot, pkg, pkg) - println("${result.size} definitions") - result - }) - val indexList = arrayListOf() - val it = remainingApiRefNames.iterator() - while (it.hasNext()) { - val refName = it.next() - val refLink = map[refName] ?: continue - indexList += "[$refName]: $siteRoot/$refLink" - it.remove() - } - return indexList -} diff --git a/kotlinx-coroutines-bom/build.gradle.kts b/kotlinx-coroutines-bom/build.gradle.kts new file mode 100644 index 0000000000..b600e9c3c4 --- /dev/null +++ b/kotlinx-coroutines-bom/build.gradle.kts @@ -0,0 +1,55 @@ +import org.gradle.api.publish.maven.internal.publication.DefaultMavenPublication +import java.util.Locale + +plugins { + id("java-platform") +} + +val name = project.name + +dependencies { + constraints { + rootProject.subprojects.forEach { + if (unpublished.contains(it.name)) return@forEach + if (it.name == name) return@forEach + if (!it.plugins.hasPlugin("maven-publish")) return@forEach + evaluationDependsOn(it.path) + it.publishing.publications.all { + this as MavenPublication + if (artifactId.endsWith("-kotlinMultiplatform")) return@all + if (artifactId.endsWith("-metadata")) return@all + // Skip platform artifacts (like *-linuxx64, *-macosx64) + // It leads to inconsistent bom when publishing from different platforms + // (e.g. on linux it will include only linuxx64 artifacts and no macosx64) + // It shouldn't be a problem as usually consumers need to use generic *-native artifact + // Gradle will choose correct variant by using metadata attributes + if (artifacts.any { it.extension == "klib" }) return@all + this@constraints.api(mapOf("group" to groupId, "name" to artifactId, "version" to version)) + } + } + } +} + +publishing { + publications { + val mavenBom by creating(MavenPublication::class) { + from(components["javaPlatform"]) + } + // Disable metadata publication + forEach { pub -> + pub as DefaultMavenPublication + pub.unsetModuleDescriptorGenerator() + tasks.matching { + it.name == "generateMetadataFileFor${ pub.name.replaceFirstChar { it.uppercaseChar() } }Publication" + }.all { + onlyIf { false } + } + } + } +} + +fun DefaultMavenPublication.unsetModuleDescriptorGenerator() { + @Suppress("NULL_FOR_NONNULL_TYPE") + val generator: TaskProvider = null + setModuleDescriptorGenerator(generator) +} diff --git a/kotlinx-coroutines-core/README.md b/kotlinx-coroutines-core/README.md index 2e392843b5..1fbd90d86b 100644 --- a/kotlinx-coroutines-core/README.md +++ b/kotlinx-coroutines-core/README.md @@ -4,129 +4,190 @@ Core primitives to work with coroutines. Coroutine builder functions: -| **Name** | **Result** | **Scope** | **Description** -| ------------- | ------------- | ---------------- | --------------- -| [launch] | [Job] | [CoroutineScope] | Launches coroutine that does not have any result -| [async] | [Deferred] | [CoroutineScope] | Returns a single value with the future result -| [produce][kotlinx.coroutines.experimental.channels.produce] | [ProducerJob][kotlinx.coroutines.experimental.channels.ProducerJob] | [ProducerScope][kotlinx.coroutines.experimental.channels.ProducerScope] | Produces a stream of elements -| [actor][kotlinx.coroutines.experimental.channels.actor] | [ActorJob][kotlinx.coroutines.experimental.channels.ActorJob] | [ActorScope][kotlinx.coroutines.experimental.channels.ActorScope] | Processes a stream of messages -| [runBlocking] | `T` | [CoroutineScope] | Blocks the thread while the coroutine runs +| **Name** | **Result** | **Scope** | **Description** +| ---------------------------------------------- | ------------------------------------------------------------ | ---------------------------------------------------------- | --------------- +| [launch][kotlinx.coroutines.launch] | [Job][kotlinx.coroutines.Job] | [CoroutineScope][kotlinx.coroutines.CoroutineScope] | Launches coroutine that does not have any result +| [async][kotlinx.coroutines.async] | [Deferred][kotlinx.coroutines.Deferred] | [CoroutineScope][kotlinx.coroutines.CoroutineScope] | Returns a single value with the future result +| [produce][kotlinx.coroutines.channels.produce] | [ReceiveChannel][kotlinx.coroutines.channels.ReceiveChannel] | [ProducerScope][kotlinx.coroutines.channels.ProducerScope] | Produces a stream of elements +| [runBlocking][kotlinx.coroutines.runBlocking] | `T` | [CoroutineScope][kotlinx.coroutines.CoroutineScope] | Blocks the thread while the coroutine runs Coroutine dispatchers implementing [CoroutineDispatcher]: -| **Name** | **Description** -| --------------------------- | --------------- -| [CommonPool] | Confines coroutine execution to a shared pool of threads -| [newSingleThreadContext] | Create new single-threaded coroutine context -| [newFixedThreadPoolContext] | Creates new thread pool of a fixed size -| [Executor.asCoroutineDispatcher][java.util.concurrent.Executor.asCoroutineDispatcher] | Extension to convert any executor -| [Unconfined] | Does not confine coroutine execution in any way +| **Name** | **Description** +| --------------------------------------------------------------------------------------------------- | --------------- +| [Dispatchers.Main][kotlinx.coroutines.Dispatchers.Main] | Confines coroutine execution to the UI thread +| [Dispatchers.Default][kotlinx.coroutines.Dispatchers.Default] | Confines coroutine execution to a shared pool of background threads +| [Dispatchers.Unconfined][kotlinx.coroutines.Dispatchers.Unconfined] | Does not confine coroutine execution in any way +| [CoroutineDispatcher.limitedParallelism][kotlinx.coroutines.CoroutineDispatcher.limitedParallelism] | Creates a view of the given dispatcher, limiting the number of tasks executing in parallel + +More context elements: + +| **Name** | **Description** +| ------------------------------------------------------------------------- | --------------- +| [NonCancellable][kotlinx.coroutines.NonCancellable] | A non-cancelable job that is always active +| [CoroutineExceptionHandler][kotlinx.coroutines.CoroutineExceptionHandler] | Handler for uncaught exception Synchronization primitives for coroutines: -| **Name** | **Suspending functions** | **Description** -| ---------- | ----------------------------------------------------------- | --------------- -| [Mutex][kotlinx.coroutines.experimental.sync.Mutex] | [lock][kotlinx.coroutines.experimental.sync.Mutex.lock] | Mutual exclusion -| [Channel][kotlinx.coroutines.experimental.channels.Channel] | [send][kotlinx.coroutines.experimental.channels.SendChannel.send], [receive][kotlinx.coroutines.experimental.channels.ReceiveChannel.receive] | Communication channel (aka queue or exchanger) +| **Name** | **Suspending functions** | **Description** +|------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------| --------------- +| [Mutex][kotlinx.coroutines.sync.Mutex] | [lock][kotlinx.coroutines.sync.Mutex.lock] | Mutual exclusion +| [Semaphore][kotlinx.coroutines.sync.Semaphore] | [acquire][kotlinx.coroutines.sync.Semaphore.acquire] | Limiting the maximum concurrency +| [Channel][kotlinx.coroutines.channels.Channel] | [send][kotlinx.coroutines.channels.SendChannel.send], [receive][kotlinx.coroutines.channels.ReceiveChannel.receive] | Communication channel (aka queue or exchanger) +| [Flow](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/) | [collect][kotlinx.coroutines.flow.Flow.collect] | Asynchronous stream of values + + Top-level suspending functions: -| **Name** | **Description** -| ------------------- | --------------- -| [delay] | Non-blocking sleep -| [yield] | Yields thread in single-threaded dispatchers -| [run] | Switches to a different context -| [withTimeout] | Set execution time-limit with exception on timeout -| [withTimeoutOrNull] | Set execution time-limit will null result on timeout - -[Select][kotlinx.coroutines.experimental.selects.select] expression waits for the result of multiple suspending functions simultaneously: - -| **Receiver** | **Suspending function** | **Select clause** | **Non-suspending version** -| ---------------- | --------------------------------------------- | ------------------------------------------------ | -------------------------- -| [Job] | [join][Job.join] | [onJoin][kotlinx.coroutines.experimental.selects.SelectBuilder.onJoin] | [isCompleted][Job.isCompleted] -| [Deferred] | [await][Deferred.await] | [onAwait][kotlinx.coroutines.experimental.selects.SelectBuilder.onAwait] | [isCompleted][Job.isCompleted] -| [SendChannel][kotlinx.coroutines.experimental.channels.SendChannel] | [send][kotlinx.coroutines.experimental.channels.SendChannel.send] | [onSend][kotlinx.coroutines.experimental.selects.SelectBuilder.onSend] | [offer][kotlinx.coroutines.experimental.channels.SendChannel.offer] -| [ReceiveChannel][kotlinx.coroutines.experimental.channels.ReceiveChannel] | [receive][kotlinx.coroutines.experimental.channels.ReceiveChannel.receive] | [onReceive][kotlinx.coroutines.experimental.selects.SelectBuilder.onReceive] | [poll][kotlinx.coroutines.experimental.channels.ReceiveChannel.poll] -| [ReceiveChannel][kotlinx.coroutines.experimental.channels.ReceiveChannel] | [receiveOrNull][kotlinx.coroutines.experimental.channels.ReceiveChannel.receiveOrNull] | [onReceiveOrNull][kotlinx.coroutines.experimental.selects.SelectBuilder.onReceiveOrNull] | [poll][kotlinx.coroutines.experimental.channels.ReceiveChannel.poll] -| [Mutex][kotlinx.coroutines.experimental.sync.Mutex] | [lock][kotlinx.coroutines.experimental.sync.Mutex.lock] | [onLock][kotlinx.coroutines.experimental.selects.SelectBuilder.onLock] | [tryLock][kotlinx.coroutines.experimental.sync.Mutex.tryLock] -| none | [delay] | [onTimeout][kotlinx.coroutines.experimental.selects.SelectBuilder.onTimeout] | none +| **Name** | **Description** +| --------------------------------------------------------- | --------------- +| [delay][kotlinx.coroutines.delay] | Non-blocking sleep +| [yield][kotlinx.coroutines.yield] | Yields thread in single-threaded dispatchers +| [withContext][kotlinx.coroutines.withContext] | Switches to a different context +| [withTimeout][kotlinx.coroutines.withTimeout] | Set execution time-limit with exception on timeout +| [withTimeoutOrNull][kotlinx.coroutines.withTimeoutOrNull] | Set execution time-limit will null result on timeout +| [awaitAll][kotlinx.coroutines.awaitAll] | Awaits for successful completion of all given jobs or exceptional completion of any +| [joinAll][kotlinx.coroutines.joinAll] | Joins on all given jobs Cancellation support for user-defined suspending functions is available with [suspendCancellableCoroutine] -helper function. [NonCancellable] job object is provided to suppress cancellation with -`run(NonCancellable) {...}` block of code. +helper function. +The [NonCancellable] job object is provided to suppress cancellation inside the +`withContext(NonCancellable) {...}` block of code. + +Ways to construct asynchronous streams of values: + +| **Name** | **Type** | **Description** +| --------------------------------------------------------------------- | -------- | --------------- +| [flow][kotlinx.coroutines.flow.flow] | cold | Runs a generator-style block of code that emits values +| [flowOf][kotlinx.coroutines.flow.flowOf] | cold | Emits the values passed as arguments +| [channelFlow][kotlinx.coroutines.flow.channelFlow] | cold | Runs the given code, providing a channel sending to which means emitting from the flow +| [callbackFlow][kotlinx.coroutines.flow.callbackFlow] | cold | Allows transforming a callback-based API into a flow +| [ReceiveChannel.consumeAsFlow][kotlinx.coroutines.flow.consumeAsFlow] | hot | Transforms a channel into a flow, emitting all of the received values to a single subscriber +| [ReceiveChannel.receiveAsFlow][kotlinx.coroutines.flow.receiveAsFlow] | hot | Transforms a channel into a flow, distributing the received values among its subscribers +| [MutableSharedFlow][kotlinx.coroutines.flow.MutableSharedFlow] | hot | Allows emitting each value to arbitrarily many subscribers at once +| [MutableStateFlow][kotlinx.coroutines.flow.MutableStateFlow] | hot | Represents mutable state as a flow + +A *cold* stream is some process of generating values, and this process is performed separately for each subscriber. +A *hot* stream uses the same source of values independently of whether there are subscribers. + +A [select][kotlinx.coroutines.selects.select] expression waits for the result of multiple suspending functions simultaneously: + +| **Receiver** | **Suspending function** | **Select clause** | **Non-suspending version** +| ------------------------------------------------------------ | --------------------------------------------------------------- | ----------------------------------------------------------------- | -------------------------- +| [Job][kotlinx.coroutines.Job] | [join][kotlinx.coroutines.Job.join] | [onJoin][kotlinx.coroutines.Job.onJoin] | [isCompleted][kotlinx.coroutines.Job.isCompleted] +| [Deferred][kotlinx.coroutines.Deferred] | [await][kotlinx.coroutines.Deferred.await] | [onAwait][kotlinx.coroutines.Deferred.onAwait] | [isCompleted][kotlinx.coroutines.Job.isCompleted] +| [SendChannel][kotlinx.coroutines.channels.SendChannel] | [send][kotlinx.coroutines.channels.SendChannel.send] | [onSend][kotlinx.coroutines.channels.SendChannel.onSend] | [trySend][kotlinx.coroutines.channels.SendChannel.trySend] +| [ReceiveChannel][kotlinx.coroutines.channels.ReceiveChannel] | [receive][kotlinx.coroutines.channels.ReceiveChannel.receive] | [onReceive][kotlinx.coroutines.channels.ReceiveChannel.onReceive] | [tryReceive][kotlinx.coroutines.channels.ReceiveChannel.tryReceive] +| [ReceiveChannel][kotlinx.coroutines.channels.ReceiveChannel] | [receiveCatching][kotlinx.coroutines.channels.receiveCatching] | [onReceiveCatching][kotlinx.coroutines.channels.onReceiveCatching] | [tryReceive][kotlinx.coroutines.channels.ReceiveChannel.tryReceive] +| none | [delay][kotlinx.coroutines.delay] | [onTimeout][kotlinx.coroutines.selects.SelectBuilder.onTimeout] | none + +# Package kotlinx.coroutines + +General-purpose coroutine builders, contexts, and helper functions. -This module provides debugging facilities for coroutines (run JVM with `-ea` or `-Dkotlinx.coroutines.debug` options) -and [newCoroutineContext] function to write user-defined coroutine builders that work with these -debugging facilities. +# Package kotlinx.coroutines.sync -# Package kotlinx.coroutines.experimental +Synchronization primitives (mutex and semaphore). -General-purpose coroutine builders, contexts, and helper functions. +# Package kotlinx.coroutines.channels + +Channels — non-blocking primitives for communicating a stream of elements between coroutines. + +# Package kotlinx.coroutines.flow + +Flow — asynchronous cold and hot streams of elements. + +# Package kotlinx.coroutines.selects + +Select — expressions that perform multiple suspending operations simultaneously until one of them succeeds. + +# Package kotlinx.coroutines.intrinsics + +Low-level primitives for finer-grained control of coroutines. + +# Package kotlinx.coroutines.future + +[JDK 8's `CompletableFuture`](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html) support. + +# Package kotlinx.coroutines.stream + +[JDK 8's `Stream`](https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html) support. + +# Package kotlinx.coroutines.time + +[JDK 8's `Duration`](https://docs.oracle.com/javase/8/docs/api/java/time/Duration.html) support via additional overloads for existing time-based operators. + + + + +[kotlinx.coroutines.launch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html +[kotlinx.coroutines.Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[kotlinx.coroutines.CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[kotlinx.coroutines.async]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html +[kotlinx.coroutines.Deferred]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/index.html +[kotlinx.coroutines.runBlocking]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html +[CoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/index.html +[kotlinx.coroutines.Dispatchers.Main]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-main.html +[kotlinx.coroutines.Dispatchers.Default]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-default.html +[kotlinx.coroutines.Dispatchers.Unconfined]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-unconfined.html +[kotlinx.coroutines.CoroutineDispatcher.limitedParallelism]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/limited-parallelism.html +[kotlinx.coroutines.NonCancellable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-non-cancellable/index.html +[kotlinx.coroutines.CoroutineExceptionHandler]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-exception-handler/index.html +[kotlinx.coroutines.delay]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html +[kotlinx.coroutines.yield]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/yield.html +[kotlinx.coroutines.withContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-context.html +[kotlinx.coroutines.withTimeout]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-timeout.html +[kotlinx.coroutines.withTimeoutOrNull]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-timeout-or-null.html +[kotlinx.coroutines.awaitAll]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/await-all.html +[kotlinx.coroutines.joinAll]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/join-all.html +[suspendCancellableCoroutine]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/suspend-cancellable-coroutine.html +[NonCancellable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-non-cancellable/index.html +[kotlinx.coroutines.Job.join]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/join.html +[kotlinx.coroutines.Job.onJoin]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/on-join.html +[kotlinx.coroutines.Job.isCompleted]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/is-completed.html +[kotlinx.coroutines.Deferred.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/await.html +[kotlinx.coroutines.Deferred.onAwait]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/on-await.html + + + +[kotlinx.coroutines.flow.Flow.collect]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/collect.html +[kotlinx.coroutines.flow.flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flow.html +[kotlinx.coroutines.flow.flowOf]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flow-of.html +[kotlinx.coroutines.flow.channelFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/channel-flow.html +[kotlinx.coroutines.flow.callbackFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/callback-flow.html +[kotlinx.coroutines.flow.consumeAsFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/consume-as-flow.html +[kotlinx.coroutines.flow.receiveAsFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/receive-as-flow.html +[kotlinx.coroutines.flow.MutableSharedFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-mutable-shared-flow/index.html +[kotlinx.coroutines.flow.MutableStateFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-mutable-state-flow/index.html + + + +[kotlinx.coroutines.sync.Mutex]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/index.html +[kotlinx.coroutines.sync.Mutex.lock]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/lock.html +[kotlinx.coroutines.sync.Semaphore]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-semaphore/index.html +[kotlinx.coroutines.sync.Semaphore.acquire]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-semaphore/acquire.html + + + +[kotlinx.coroutines.channels.produce]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/produce.html +[kotlinx.coroutines.channels.ReceiveChannel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/index.html +[kotlinx.coroutines.channels.ProducerScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-producer-scope/index.html +[kotlinx.coroutines.channels.Channel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-channel/index.html +[kotlinx.coroutines.channels.SendChannel.send]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/send.html +[kotlinx.coroutines.channels.ReceiveChannel.receive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/receive.html +[kotlinx.coroutines.channels.SendChannel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/index.html +[kotlinx.coroutines.channels.SendChannel.onSend]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/on-send.html +[kotlinx.coroutines.channels.SendChannel.trySend]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/try-send.html +[kotlinx.coroutines.channels.ReceiveChannel.onReceive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/on-receive.html +[kotlinx.coroutines.channels.ReceiveChannel.tryReceive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/try-receive.html +[kotlinx.coroutines.channels.receiveCatching]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/receive-catching.html +[kotlinx.coroutines.channels.onReceiveCatching]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/on-receive-catching.html + + + +[kotlinx.coroutines.selects.select]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.selects/select.html +[kotlinx.coroutines.selects.SelectBuilder.onTimeout]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.selects/on-timeout.html -# Package kotlinx.coroutines.experimental.sync - -Synchronization primitives (mutex). - -# Package kotlinx.coroutines.experimental.channels - -Channels -- non-blocking primitives for communicating a stream of elements between coroutines. - -# Package kotlinx.coroutines.experimental.selects - -Select expression to perform multiple suspending operations simultaneously until one of them succeeds. - - - - -[launch]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/launch.html -[Job]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/index.html -[CoroutineScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-scope/index.html -[async]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/async.html -[Deferred]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-deferred/index.html -[runBlocking]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/run-blocking.html -[CoroutineDispatcher]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-dispatcher/index.html -[CommonPool]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-common-pool/index.html -[newSingleThreadContext]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/new-single-thread-context.html -[newFixedThreadPoolContext]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/new-fixed-thread-pool-context.html -[java.util.concurrent.Executor.asCoroutineDispatcher]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/java.util.concurrent.-executor/as-coroutine-dispatcher.html -[Unconfined]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-unconfined/index.html -[delay]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/delay.html -[yield]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/yield.html -[run]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/run.html -[withTimeout]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/with-timeout.html -[withTimeoutOrNull]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/with-timeout-or-null.html -[Job.join]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/join.html -[Job.isCompleted]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/is-completed.html -[Deferred.await]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-deferred/await.html -[suspendCancellableCoroutine]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/suspend-cancellable-coroutine.html -[NonCancellable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-non-cancellable/index.html -[newCoroutineContext]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/new-coroutine-context.html - -[kotlinx.coroutines.experimental.sync.Mutex]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.sync/-mutex/index.html -[kotlinx.coroutines.experimental.sync.Mutex.lock]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.sync/-mutex/lock.html -[kotlinx.coroutines.experimental.sync.Mutex.tryLock]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.sync/-mutex/try-lock.html - -[kotlinx.coroutines.experimental.channels.produce]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/produce.html -[kotlinx.coroutines.experimental.channels.ProducerJob]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-producer-job/index.html -[kotlinx.coroutines.experimental.channels.ProducerScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-producer-scope/index.html -[kotlinx.coroutines.experimental.channels.actor]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/actor.html -[kotlinx.coroutines.experimental.channels.ActorJob]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-actor-job/index.html -[kotlinx.coroutines.experimental.channels.ActorScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-actor-scope/index.html -[kotlinx.coroutines.experimental.channels.Channel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel/index.html -[kotlinx.coroutines.experimental.channels.SendChannel.send]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-send-channel/send.html -[kotlinx.coroutines.experimental.channels.ReceiveChannel.receive]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/receive.html -[kotlinx.coroutines.experimental.channels.SendChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-send-channel/index.html -[kotlinx.coroutines.experimental.channels.SendChannel.offer]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-send-channel/offer.html -[kotlinx.coroutines.experimental.channels.ReceiveChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/index.html -[kotlinx.coroutines.experimental.channels.ReceiveChannel.poll]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/poll.html -[kotlinx.coroutines.experimental.channels.ReceiveChannel.receiveOrNull]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/receive-or-null.html - -[kotlinx.coroutines.experimental.selects.select]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/select.html -[kotlinx.coroutines.experimental.selects.SelectBuilder.onJoin]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-join.html -[kotlinx.coroutines.experimental.selects.SelectBuilder.onAwait]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-await.html -[kotlinx.coroutines.experimental.selects.SelectBuilder.onSend]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-send.html -[kotlinx.coroutines.experimental.selects.SelectBuilder.onReceive]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-receive.html -[kotlinx.coroutines.experimental.selects.SelectBuilder.onReceiveOrNull]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-receive-or-null.html -[kotlinx.coroutines.experimental.selects.SelectBuilder.onLock]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-lock.html -[kotlinx.coroutines.experimental.selects.SelectBuilder.onTimeout]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/-select-builder/on-timeout.html + diff --git a/kotlinx-coroutines-core/api/kotlinx-coroutines-core.api b/kotlinx-coroutines-core/api/kotlinx-coroutines-core.api new file mode 100644 index 0000000000..0e35d5fb38 --- /dev/null +++ b/kotlinx-coroutines-core/api/kotlinx-coroutines-core.api @@ -0,0 +1,1413 @@ +public abstract class kotlinx/coroutines/AbstractCoroutine : kotlinx/coroutines/JobSupport, kotlin/coroutines/Continuation, kotlinx/coroutines/CoroutineScope, kotlinx/coroutines/Job { + public fun (Lkotlin/coroutines/CoroutineContext;ZZ)V + protected fun afterResume (Ljava/lang/Object;)V + protected fun cancellationExceptionMessage ()Ljava/lang/String; + public final fun getContext ()Lkotlin/coroutines/CoroutineContext; + public fun getCoroutineContext ()Lkotlin/coroutines/CoroutineContext; + public fun isActive ()Z + protected fun onCancelled (Ljava/lang/Throwable;Z)V + protected fun onCompleted (Ljava/lang/Object;)V + protected final fun onCompletionInternal (Ljava/lang/Object;)V + public final fun resumeWith (Ljava/lang/Object;)V + public final fun start (Lkotlinx/coroutines/CoroutineStart;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V +} + +public final class kotlinx/coroutines/AwaitKt { + public static final fun awaitAll (Ljava/util/Collection;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitAll ([Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun joinAll (Ljava/util/Collection;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun joinAll ([Lkotlinx/coroutines/Job;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/BuildersKt { + public static final fun async (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/Deferred; + public static synthetic fun async$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/Deferred; + public static final fun invoke (Lkotlinx/coroutines/CoroutineDispatcher;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun launch (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/Job; + public static synthetic fun launch$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/Job; + public static final fun runBlocking (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static synthetic fun runBlocking$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Ljava/lang/Object; + public static final fun withContext (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public abstract interface class kotlinx/coroutines/CancellableContinuation : kotlin/coroutines/Continuation { + public abstract fun cancel (Ljava/lang/Throwable;)Z + public abstract fun completeResume (Ljava/lang/Object;)V + public abstract fun initCancellability ()V + public abstract fun invokeOnCancellation (Lkotlin/jvm/functions/Function1;)V + public abstract fun isActive ()Z + public abstract fun isCancelled ()Z + public abstract fun isCompleted ()Z + public abstract fun resume (Ljava/lang/Object;Lkotlin/jvm/functions/Function1;)V + public abstract fun resume (Ljava/lang/Object;Lkotlin/jvm/functions/Function3;)V + public abstract fun resumeUndispatched (Lkotlinx/coroutines/CoroutineDispatcher;Ljava/lang/Object;)V + public abstract fun resumeUndispatchedWithException (Lkotlinx/coroutines/CoroutineDispatcher;Ljava/lang/Throwable;)V + public abstract fun tryResume (Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; + public abstract fun tryResume (Ljava/lang/Object;Ljava/lang/Object;Lkotlin/jvm/functions/Function3;)Ljava/lang/Object; + public abstract fun tryResumeWithException (Ljava/lang/Throwable;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/CancellableContinuation$DefaultImpls { + public static synthetic fun cancel$default (Lkotlinx/coroutines/CancellableContinuation;Ljava/lang/Throwable;ILjava/lang/Object;)Z + public static synthetic fun tryResume$default (Lkotlinx/coroutines/CancellableContinuation;Ljava/lang/Object;Ljava/lang/Object;ILjava/lang/Object;)Ljava/lang/Object; +} + +public class kotlinx/coroutines/CancellableContinuationImpl : kotlin/coroutines/jvm/internal/CoroutineStackFrame, kotlinx/coroutines/CancellableContinuation, kotlinx/coroutines/Waiter { + public fun (Lkotlin/coroutines/Continuation;I)V + public final fun callCancelHandler (Lkotlinx/coroutines/CancelHandler;Ljava/lang/Throwable;)V + public final fun callOnCancellation (Lkotlin/jvm/functions/Function3;Ljava/lang/Throwable;Ljava/lang/Object;)V + public fun cancel (Ljava/lang/Throwable;)Z + public fun completeResume (Ljava/lang/Object;)V + public fun getCallerFrame ()Lkotlin/coroutines/jvm/internal/CoroutineStackFrame; + public fun getContext ()Lkotlin/coroutines/CoroutineContext; + public fun getContinuationCancellationCause (Lkotlinx/coroutines/Job;)Ljava/lang/Throwable; + public final fun getResult ()Ljava/lang/Object; + public fun getStackTraceElement ()Ljava/lang/StackTraceElement; + public fun initCancellability ()V + public fun invokeOnCancellation (Lkotlin/jvm/functions/Function1;)V + public fun invokeOnCancellation (Lkotlinx/coroutines/internal/Segment;I)V + public fun isActive ()Z + public fun isCancelled ()Z + public fun isCompleted ()Z + protected fun nameString ()Ljava/lang/String; + public fun resume (Ljava/lang/Object;Lkotlin/jvm/functions/Function1;)V + public fun resume (Ljava/lang/Object;Lkotlin/jvm/functions/Function3;)V + public fun resumeUndispatched (Lkotlinx/coroutines/CoroutineDispatcher;Ljava/lang/Object;)V + public fun resumeUndispatchedWithException (Lkotlinx/coroutines/CoroutineDispatcher;Ljava/lang/Throwable;)V + public fun resumeWith (Ljava/lang/Object;)V + public fun toString ()Ljava/lang/String; + public fun tryResume (Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; + public fun tryResume (Ljava/lang/Object;Ljava/lang/Object;Lkotlin/jvm/functions/Function3;)Ljava/lang/Object; + public fun tryResumeWithException (Ljava/lang/Throwable;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/CancellableContinuationKt { + public static final fun disposeOnCancellation (Lkotlinx/coroutines/CancellableContinuation;Lkotlinx/coroutines/DisposableHandle;)V + public static final fun suspendCancellableCoroutine (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public abstract interface class kotlinx/coroutines/ChildHandle : kotlinx/coroutines/DisposableHandle { + public abstract fun childCancelled (Ljava/lang/Throwable;)Z + public abstract fun getParent ()Lkotlinx/coroutines/Job; +} + +public abstract interface class kotlinx/coroutines/ChildJob : kotlinx/coroutines/Job { + public abstract fun parentCancelled (Lkotlinx/coroutines/ParentJob;)V +} + +public final class kotlinx/coroutines/ChildJob$DefaultImpls { + public static synthetic fun cancel (Lkotlinx/coroutines/ChildJob;)V + public static fun fold (Lkotlinx/coroutines/ChildJob;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static fun get (Lkotlinx/coroutines/ChildJob;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public static fun minusKey (Lkotlinx/coroutines/ChildJob;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/ChildJob;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/ChildJob;Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; +} + +public abstract interface class kotlinx/coroutines/CompletableDeferred : kotlinx/coroutines/Deferred { + public abstract fun complete (Ljava/lang/Object;)Z + public abstract fun completeExceptionally (Ljava/lang/Throwable;)Z +} + +public final class kotlinx/coroutines/CompletableDeferred$DefaultImpls { + public static synthetic fun cancel (Lkotlinx/coroutines/CompletableDeferred;)V + public static fun fold (Lkotlinx/coroutines/CompletableDeferred;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static fun get (Lkotlinx/coroutines/CompletableDeferred;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public static fun minusKey (Lkotlinx/coroutines/CompletableDeferred;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/CompletableDeferred;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/CompletableDeferred;Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; +} + +public final class kotlinx/coroutines/CompletableDeferredKt { + public static final fun CompletableDeferred (Ljava/lang/Object;)Lkotlinx/coroutines/CompletableDeferred; + public static final fun CompletableDeferred (Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/CompletableDeferred; + public static synthetic fun CompletableDeferred$default (Lkotlinx/coroutines/Job;ILjava/lang/Object;)Lkotlinx/coroutines/CompletableDeferred; + public static final fun completeWith (Lkotlinx/coroutines/CompletableDeferred;Ljava/lang/Object;)Z +} + +public abstract interface class kotlinx/coroutines/CompletableJob : kotlinx/coroutines/Job { + public abstract fun complete ()Z + public abstract fun completeExceptionally (Ljava/lang/Throwable;)Z +} + +public final class kotlinx/coroutines/CompletableJob$DefaultImpls { + public static synthetic fun cancel (Lkotlinx/coroutines/CompletableJob;)V + public static fun fold (Lkotlinx/coroutines/CompletableJob;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static fun get (Lkotlinx/coroutines/CompletableJob;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public static fun minusKey (Lkotlinx/coroutines/CompletableJob;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/CompletableJob;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/CompletableJob;Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; +} + +public final class kotlinx/coroutines/CompletionHandlerException : java/lang/RuntimeException { + public fun (Ljava/lang/String;Ljava/lang/Throwable;)V +} + +public abstract interface class kotlinx/coroutines/CopyableThreadContextElement : kotlinx/coroutines/ThreadContextElement { + public abstract fun copyForChild ()Lkotlinx/coroutines/CopyableThreadContextElement; + public abstract fun mergeForChild (Lkotlin/coroutines/CoroutineContext$Element;)Lkotlin/coroutines/CoroutineContext; +} + +public final class kotlinx/coroutines/CopyableThreadContextElement$DefaultImpls { + public static fun fold (Lkotlinx/coroutines/CopyableThreadContextElement;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static fun get (Lkotlinx/coroutines/CopyableThreadContextElement;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public static fun minusKey (Lkotlinx/coroutines/CopyableThreadContextElement;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/CopyableThreadContextElement;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; +} + +public abstract interface class kotlinx/coroutines/CopyableThrowable { + public abstract fun createCopy ()Ljava/lang/Throwable; +} + +public final class kotlinx/coroutines/CoroutineContextKt { + public static final fun newCoroutineContext (Lkotlin/coroutines/CoroutineContext;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; + public static final fun newCoroutineContext (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; +} + +public abstract class kotlinx/coroutines/CoroutineDispatcher : kotlin/coroutines/AbstractCoroutineContextElement, kotlin/coroutines/ContinuationInterceptor { + public static final field Key Lkotlinx/coroutines/CoroutineDispatcher$Key; + public fun ()V + public abstract fun dispatch (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Runnable;)V + public fun dispatchYield (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Runnable;)V + public fun get (Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public final fun interceptContinuation (Lkotlin/coroutines/Continuation;)Lkotlin/coroutines/Continuation; + public fun isDispatchNeeded (Lkotlin/coroutines/CoroutineContext;)Z + public synthetic fun limitedParallelism (I)Lkotlinx/coroutines/CoroutineDispatcher; + public fun limitedParallelism (ILjava/lang/String;)Lkotlinx/coroutines/CoroutineDispatcher; + public static synthetic fun limitedParallelism$default (Lkotlinx/coroutines/CoroutineDispatcher;ILjava/lang/String;ILjava/lang/Object;)Lkotlinx/coroutines/CoroutineDispatcher; + public fun minusKey (Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public final fun plus (Lkotlinx/coroutines/CoroutineDispatcher;)Lkotlinx/coroutines/CoroutineDispatcher; + public final fun releaseInterceptedContinuation (Lkotlin/coroutines/Continuation;)V + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/CoroutineDispatcher$Key : kotlin/coroutines/AbstractCoroutineContextKey { +} + +public abstract interface class kotlinx/coroutines/CoroutineExceptionHandler : kotlin/coroutines/CoroutineContext$Element { + public static final field Key Lkotlinx/coroutines/CoroutineExceptionHandler$Key; + public abstract fun handleException (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Throwable;)V +} + +public final class kotlinx/coroutines/CoroutineExceptionHandler$DefaultImpls { + public static fun fold (Lkotlinx/coroutines/CoroutineExceptionHandler;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static fun get (Lkotlinx/coroutines/CoroutineExceptionHandler;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public static fun minusKey (Lkotlinx/coroutines/CoroutineExceptionHandler;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/CoroutineExceptionHandler;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; +} + +public final class kotlinx/coroutines/CoroutineExceptionHandler$Key : kotlin/coroutines/CoroutineContext$Key { +} + +public final class kotlinx/coroutines/CoroutineExceptionHandlerKt { + public static final fun CoroutineExceptionHandler (Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/CoroutineExceptionHandler; + public static final fun handleCoroutineException (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Throwable;)V +} + +public final class kotlinx/coroutines/CoroutineId : kotlin/coroutines/AbstractCoroutineContextElement, kotlinx/coroutines/ThreadContextElement { + public static final field Key Lkotlinx/coroutines/CoroutineId$Key; + public fun (J)V + public final fun component1 ()J + public final fun copy (J)Lkotlinx/coroutines/CoroutineId; + public static synthetic fun copy$default (Lkotlinx/coroutines/CoroutineId;JILjava/lang/Object;)Lkotlinx/coroutines/CoroutineId; + public fun equals (Ljava/lang/Object;)Z + public final fun getId ()J + public fun hashCode ()I + public synthetic fun restoreThreadContext (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Object;)V + public fun restoreThreadContext (Lkotlin/coroutines/CoroutineContext;Ljava/lang/String;)V + public fun toString ()Ljava/lang/String; + public synthetic fun updateThreadContext (Lkotlin/coroutines/CoroutineContext;)Ljava/lang/Object; + public fun updateThreadContext (Lkotlin/coroutines/CoroutineContext;)Ljava/lang/String; +} + +public final class kotlinx/coroutines/CoroutineId$Key : kotlin/coroutines/CoroutineContext$Key { +} + +public final class kotlinx/coroutines/CoroutineName : kotlin/coroutines/AbstractCoroutineContextElement { + public static final field Key Lkotlinx/coroutines/CoroutineName$Key; + public fun (Ljava/lang/String;)V + public final fun component1 ()Ljava/lang/String; + public final fun copy (Ljava/lang/String;)Lkotlinx/coroutines/CoroutineName; + public static synthetic fun copy$default (Lkotlinx/coroutines/CoroutineName;Ljava/lang/String;ILjava/lang/Object;)Lkotlinx/coroutines/CoroutineName; + public fun equals (Ljava/lang/Object;)Z + public final fun getName ()Ljava/lang/String; + public fun hashCode ()I + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/CoroutineName$Key : kotlin/coroutines/CoroutineContext$Key { +} + +public abstract interface class kotlinx/coroutines/CoroutineScope { + public abstract fun getCoroutineContext ()Lkotlin/coroutines/CoroutineContext; +} + +public final class kotlinx/coroutines/CoroutineScopeKt { + public static final fun CoroutineScope (Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/CoroutineScope; + public static final fun MainScope ()Lkotlinx/coroutines/CoroutineScope; + public static final fun cancel (Lkotlinx/coroutines/CoroutineScope;Ljava/lang/String;Ljava/lang/Throwable;)V + public static final fun cancel (Lkotlinx/coroutines/CoroutineScope;Ljava/util/concurrent/CancellationException;)V + public static synthetic fun cancel$default (Lkotlinx/coroutines/CoroutineScope;Ljava/lang/String;Ljava/lang/Throwable;ILjava/lang/Object;)V + public static synthetic fun cancel$default (Lkotlinx/coroutines/CoroutineScope;Ljava/util/concurrent/CancellationException;ILjava/lang/Object;)V + public static final fun coroutineScope (Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun currentCoroutineContext (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun ensureActive (Lkotlinx/coroutines/CoroutineScope;)V + public static final fun isActive (Lkotlinx/coroutines/CoroutineScope;)Z + public static final fun plus (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/CoroutineScope; +} + +public final class kotlinx/coroutines/CoroutineStart : java/lang/Enum { + public static final field ATOMIC Lkotlinx/coroutines/CoroutineStart; + public static final field DEFAULT Lkotlinx/coroutines/CoroutineStart; + public static final field LAZY Lkotlinx/coroutines/CoroutineStart; + public static final field UNDISPATCHED Lkotlinx/coroutines/CoroutineStart; + public static fun getEntries ()Lkotlin/enums/EnumEntries; + public final fun invoke (Lkotlin/jvm/functions/Function2;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)V + public final fun isLazy ()Z + public static fun valueOf (Ljava/lang/String;)Lkotlinx/coroutines/CoroutineStart; + public static fun values ()[Lkotlinx/coroutines/CoroutineStart; +} + +public final class kotlinx/coroutines/DebugKt { + public static final field DEBUG_PROPERTY_NAME Ljava/lang/String; + public static final field DEBUG_PROPERTY_VALUE_AUTO Ljava/lang/String; + public static final field DEBUG_PROPERTY_VALUE_OFF Ljava/lang/String; + public static final field DEBUG_PROPERTY_VALUE_ON Ljava/lang/String; + public static final fun getRECOVER_STACK_TRACES ()Z +} + +public final class kotlinx/coroutines/DefaultExecutorKt { + public static final fun getDefaultDelay ()Lkotlinx/coroutines/Delay; +} + +public abstract interface class kotlinx/coroutines/Deferred : kotlinx/coroutines/Job { + public abstract fun await (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun getCompleted ()Ljava/lang/Object; + public abstract fun getCompletionExceptionOrNull ()Ljava/lang/Throwable; + public abstract fun getOnAwait ()Lkotlinx/coroutines/selects/SelectClause1; +} + +public final class kotlinx/coroutines/Deferred$DefaultImpls { + public static synthetic fun cancel (Lkotlinx/coroutines/Deferred;)V + public static fun fold (Lkotlinx/coroutines/Deferred;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static fun get (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public static fun minusKey (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/Deferred;Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; +} + +public abstract interface class kotlinx/coroutines/Delay { + public abstract fun delay (JLkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun invokeOnTimeout (JLjava/lang/Runnable;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/DisposableHandle; + public abstract fun scheduleResumeAfterDelay (JLkotlinx/coroutines/CancellableContinuation;)V +} + +public final class kotlinx/coroutines/Delay$DefaultImpls { + public static fun delay (Lkotlinx/coroutines/Delay;JLkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static fun invokeOnTimeout (Lkotlinx/coroutines/Delay;JLjava/lang/Runnable;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/DisposableHandle; +} + +public final class kotlinx/coroutines/DelayKt { + public static final fun awaitCancellation (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun delay (JLkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun delay-VtjQ1oo (JLkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public abstract interface annotation class kotlinx/coroutines/DelicateCoroutinesApi : java/lang/annotation/Annotation { +} + +public final class kotlinx/coroutines/DispatchedTaskKt { + public static final field MODE_CANCELLABLE I +} + +public final class kotlinx/coroutines/Dispatchers { + public static final field INSTANCE Lkotlinx/coroutines/Dispatchers; + public static final fun getDefault ()Lkotlinx/coroutines/CoroutineDispatcher; + public static final fun getIO ()Lkotlinx/coroutines/CoroutineDispatcher; + public static final fun getMain ()Lkotlinx/coroutines/MainCoroutineDispatcher; + public static final fun getUnconfined ()Lkotlinx/coroutines/CoroutineDispatcher; + public final fun shutdown ()V +} + +public final class kotlinx/coroutines/DispatchersKt { + public static final field IO_PARALLELISM_PROPERTY_NAME Ljava/lang/String; + public static final synthetic fun getIO (Lkotlinx/coroutines/Dispatchers;)Lkotlinx/coroutines/CoroutineDispatcher; +} + +public abstract interface class kotlinx/coroutines/DisposableHandle { + public abstract fun dispose ()V +} + +public final class kotlinx/coroutines/EventLoopKt { + public static final fun isIoDispatcherThread (Ljava/lang/Thread;)Z + public static final fun processNextEventInCurrentThread ()J + public static final fun runSingleTaskFromCurrentSystemDispatcher ()J +} + +public final class kotlinx/coroutines/ExceptionsKt { + public static final fun CancellationException (Ljava/lang/String;Ljava/lang/Throwable;)Ljava/util/concurrent/CancellationException; +} + +public abstract class kotlinx/coroutines/ExecutorCoroutineDispatcher : kotlinx/coroutines/CoroutineDispatcher, java/io/Closeable, java/lang/AutoCloseable { + public static final field Key Lkotlinx/coroutines/ExecutorCoroutineDispatcher$Key; + public fun ()V + public abstract fun close ()V + public abstract fun getExecutor ()Ljava/util/concurrent/Executor; +} + +public final class kotlinx/coroutines/ExecutorCoroutineDispatcher$Key : kotlin/coroutines/AbstractCoroutineContextKey { +} + +public final class kotlinx/coroutines/ExecutorsKt { + public static final fun asExecutor (Lkotlinx/coroutines/CoroutineDispatcher;)Ljava/util/concurrent/Executor; + public static final fun from (Ljava/util/concurrent/Executor;)Lkotlinx/coroutines/CoroutineDispatcher; + public static final fun from (Ljava/util/concurrent/ExecutorService;)Lkotlinx/coroutines/ExecutorCoroutineDispatcher; +} + +public abstract interface annotation class kotlinx/coroutines/ExperimentalCoroutinesApi : java/lang/annotation/Annotation { +} + +public abstract interface annotation class kotlinx/coroutines/ExperimentalForInheritanceCoroutinesApi : java/lang/annotation/Annotation { +} + +public abstract interface annotation class kotlinx/coroutines/FlowPreview : java/lang/annotation/Annotation { +} + +public final class kotlinx/coroutines/GlobalScope : kotlinx/coroutines/CoroutineScope { + public static final field INSTANCE Lkotlinx/coroutines/GlobalScope; + public fun getCoroutineContext ()Lkotlin/coroutines/CoroutineContext; +} + +public final class kotlinx/coroutines/GuidanceKt { + public static final fun async (Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/Deferred; + public static synthetic fun async$default (Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/Deferred; + public static final fun launch (Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/Job; + public static synthetic fun launch$default (Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/Job; +} + +public abstract interface annotation class kotlinx/coroutines/InternalCoroutinesApi : java/lang/annotation/Annotation { +} + +public abstract interface annotation class kotlinx/coroutines/InternalForInheritanceCoroutinesApi : java/lang/annotation/Annotation { +} + +public final class kotlinx/coroutines/InterruptibleKt { + public static final fun runInterruptible (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static synthetic fun runInterruptible$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;ILjava/lang/Object;)Ljava/lang/Object; +} + +public abstract interface class kotlinx/coroutines/Job : kotlin/coroutines/CoroutineContext$Element { + public static final field Key Lkotlinx/coroutines/Job$Key; + public abstract fun attachChild (Lkotlinx/coroutines/ChildJob;)Lkotlinx/coroutines/ChildHandle; + public abstract synthetic fun cancel ()V + public abstract synthetic fun cancel (Ljava/lang/Throwable;)Z + public abstract fun cancel (Ljava/util/concurrent/CancellationException;)V + public abstract fun getCancellationException ()Ljava/util/concurrent/CancellationException; + public abstract fun getChildren ()Lkotlin/sequences/Sequence; + public abstract fun getOnJoin ()Lkotlinx/coroutines/selects/SelectClause0; + public abstract fun getParent ()Lkotlinx/coroutines/Job; + public abstract fun invokeOnCompletion (Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; + public abstract fun invokeOnCompletion (ZZLkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; + public abstract fun isActive ()Z + public abstract fun isCancelled ()Z + public abstract fun isCompleted ()Z + public abstract fun join (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun plus (Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; + public abstract fun start ()Z +} + +public final class kotlinx/coroutines/Job$DefaultImpls { + public static synthetic fun cancel (Lkotlinx/coroutines/Job;)V + public static synthetic fun cancel$default (Lkotlinx/coroutines/Job;Ljava/lang/Throwable;ILjava/lang/Object;)Z + public static synthetic fun cancel$default (Lkotlinx/coroutines/Job;Ljava/util/concurrent/CancellationException;ILjava/lang/Object;)V + public static fun fold (Lkotlinx/coroutines/Job;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static fun get (Lkotlinx/coroutines/Job;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public static synthetic fun invokeOnCompletion$default (Lkotlinx/coroutines/Job;ZZLkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lkotlinx/coroutines/DisposableHandle; + public static fun minusKey (Lkotlinx/coroutines/Job;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/Job;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/Job;Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; +} + +public final class kotlinx/coroutines/Job$Key : kotlin/coroutines/CoroutineContext$Key { +} + +public class kotlinx/coroutines/JobImpl : kotlinx/coroutines/JobSupport, kotlinx/coroutines/CompletableJob { + public fun (Lkotlinx/coroutines/Job;)V + public fun complete ()Z + public fun completeExceptionally (Ljava/lang/Throwable;)Z +} + +public final class kotlinx/coroutines/JobKt { + public static final fun Job (Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/CompletableJob; + public static final synthetic fun Job (Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; + public static synthetic fun Job$default (Lkotlinx/coroutines/Job;ILjava/lang/Object;)Lkotlinx/coroutines/CompletableJob; + public static synthetic fun Job$default (Lkotlinx/coroutines/Job;ILjava/lang/Object;)Lkotlinx/coroutines/Job; + public static final synthetic fun cancel (Lkotlin/coroutines/CoroutineContext;)V + public static final synthetic fun cancel (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Throwable;)Z + public static final fun cancel (Lkotlin/coroutines/CoroutineContext;Ljava/util/concurrent/CancellationException;)V + public static final fun cancel (Lkotlinx/coroutines/Job;Ljava/lang/String;Ljava/lang/Throwable;)V + public static synthetic fun cancel$default (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Throwable;ILjava/lang/Object;)Z + public static synthetic fun cancel$default (Lkotlin/coroutines/CoroutineContext;Ljava/util/concurrent/CancellationException;ILjava/lang/Object;)V + public static synthetic fun cancel$default (Lkotlinx/coroutines/Job;Ljava/lang/String;Ljava/lang/Throwable;ILjava/lang/Object;)V + public static final fun cancelAndJoin (Lkotlinx/coroutines/Job;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun cancelChildren (Lkotlin/coroutines/CoroutineContext;)V + public static final synthetic fun cancelChildren (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Throwable;)V + public static final fun cancelChildren (Lkotlin/coroutines/CoroutineContext;Ljava/util/concurrent/CancellationException;)V + public static final synthetic fun cancelChildren (Lkotlinx/coroutines/Job;)V + public static final synthetic fun cancelChildren (Lkotlinx/coroutines/Job;Ljava/lang/Throwable;)V + public static final fun cancelChildren (Lkotlinx/coroutines/Job;Ljava/util/concurrent/CancellationException;)V + public static synthetic fun cancelChildren$default (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Throwable;ILjava/lang/Object;)V + public static synthetic fun cancelChildren$default (Lkotlin/coroutines/CoroutineContext;Ljava/util/concurrent/CancellationException;ILjava/lang/Object;)V + public static synthetic fun cancelChildren$default (Lkotlinx/coroutines/Job;Ljava/lang/Throwable;ILjava/lang/Object;)V + public static synthetic fun cancelChildren$default (Lkotlinx/coroutines/Job;Ljava/util/concurrent/CancellationException;ILjava/lang/Object;)V + public static final fun cancelFutureOnCancellation (Lkotlinx/coroutines/CancellableContinuation;Ljava/util/concurrent/Future;)V + public static final fun ensureActive (Lkotlin/coroutines/CoroutineContext;)V + public static final fun ensureActive (Lkotlinx/coroutines/Job;)V + public static final fun getJob (Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/Job; + public static final fun isActive (Lkotlin/coroutines/CoroutineContext;)Z +} + +public class kotlinx/coroutines/JobSupport : kotlinx/coroutines/ChildJob, kotlinx/coroutines/Job, kotlinx/coroutines/ParentJob { + public fun (Z)V + protected fun afterCompletion (Ljava/lang/Object;)V + public final fun attachChild (Lkotlinx/coroutines/ChildJob;)Lkotlinx/coroutines/ChildHandle; + protected final fun awaitInternal (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public synthetic fun cancel ()V + public synthetic fun cancel (Ljava/lang/Throwable;)Z + public fun cancel (Ljava/util/concurrent/CancellationException;)V + public final fun cancelCoroutine (Ljava/lang/Throwable;)Z + public fun cancelInternal (Ljava/lang/Throwable;)V + protected fun cancellationExceptionMessage ()Ljava/lang/String; + public fun childCancelled (Ljava/lang/Throwable;)Z + public fun fold (Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public fun get (Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public final fun getCancellationException ()Ljava/util/concurrent/CancellationException; + public fun getChildJobCancellationCause ()Ljava/util/concurrent/CancellationException; + public final fun getChildren ()Lkotlin/sequences/Sequence; + protected final fun getCompletionCause ()Ljava/lang/Throwable; + protected final fun getCompletionCauseHandled ()Z + public final fun getCompletionExceptionOrNull ()Ljava/lang/Throwable; + public final fun getKey ()Lkotlin/coroutines/CoroutineContext$Key; + protected final fun getOnAwaitInternal ()Lkotlinx/coroutines/selects/SelectClause1; + public final fun getOnJoin ()Lkotlinx/coroutines/selects/SelectClause0; + public fun getParent ()Lkotlinx/coroutines/Job; + protected fun handleJobException (Ljava/lang/Throwable;)Z + protected final fun initParentJob (Lkotlinx/coroutines/Job;)V + public final fun invokeOnCompletion (Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; + public final fun invokeOnCompletion (ZZLkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; + public fun isActive ()Z + public final fun isCancelled ()Z + public final fun isCompleted ()Z + public final fun isCompletedExceptionally ()Z + protected fun isScopedCoroutine ()Z + public final fun join (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun minusKey (Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + protected fun onCancelling (Ljava/lang/Throwable;)V + protected fun onCompletionInternal (Ljava/lang/Object;)V + protected fun onStart ()V + public final fun parentCancelled (Lkotlinx/coroutines/ParentJob;)V + public fun plus (Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; + public fun plus (Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; + public final fun start ()Z + protected final fun toCancellationException (Ljava/lang/Throwable;Ljava/lang/String;)Ljava/util/concurrent/CancellationException; + public static synthetic fun toCancellationException$default (Lkotlinx/coroutines/JobSupport;Ljava/lang/Throwable;Ljava/lang/String;ILjava/lang/Object;)Ljava/util/concurrent/CancellationException; + public final fun toDebugString ()Ljava/lang/String; + public fun toString ()Ljava/lang/String; +} + +public abstract class kotlinx/coroutines/MainCoroutineDispatcher : kotlinx/coroutines/CoroutineDispatcher { + public fun ()V + public abstract fun getImmediate ()Lkotlinx/coroutines/MainCoroutineDispatcher; + public fun limitedParallelism (ILjava/lang/String;)Lkotlinx/coroutines/CoroutineDispatcher; + public fun toString ()Ljava/lang/String; + protected final fun toStringInternalImpl ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/NonCancellable : kotlin/coroutines/AbstractCoroutineContextElement, kotlinx/coroutines/Job { + public static final field INSTANCE Lkotlinx/coroutines/NonCancellable; + public fun attachChild (Lkotlinx/coroutines/ChildJob;)Lkotlinx/coroutines/ChildHandle; + public synthetic fun cancel ()V + public synthetic fun cancel (Ljava/lang/Throwable;)Z + public fun cancel (Ljava/util/concurrent/CancellationException;)V + public fun getCancellationException ()Ljava/util/concurrent/CancellationException; + public fun getChildren ()Lkotlin/sequences/Sequence; + public fun getOnJoin ()Lkotlinx/coroutines/selects/SelectClause0; + public fun getParent ()Lkotlinx/coroutines/Job; + public fun invokeOnCompletion (Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; + public fun invokeOnCompletion (ZZLkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/DisposableHandle; + public fun isActive ()Z + public fun isCancelled ()Z + public fun isCompleted ()Z + public fun join (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun plus (Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; + public fun start ()Z + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/NonDisposableHandle : kotlinx/coroutines/ChildHandle, kotlinx/coroutines/DisposableHandle { + public static final field INSTANCE Lkotlinx/coroutines/NonDisposableHandle; + public fun childCancelled (Ljava/lang/Throwable;)Z + public fun dispose ()V + public fun getParent ()Lkotlinx/coroutines/Job; + public fun toString ()Ljava/lang/String; +} + +public abstract interface annotation class kotlinx/coroutines/ObsoleteCoroutinesApi : java/lang/annotation/Annotation { +} + +public abstract interface class kotlinx/coroutines/ParentJob : kotlinx/coroutines/Job { + public abstract fun getChildJobCancellationCause ()Ljava/util/concurrent/CancellationException; +} + +public final class kotlinx/coroutines/ParentJob$DefaultImpls { + public static synthetic fun cancel (Lkotlinx/coroutines/ParentJob;)V + public static fun fold (Lkotlinx/coroutines/ParentJob;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static fun get (Lkotlinx/coroutines/ParentJob;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public static fun minusKey (Lkotlinx/coroutines/ParentJob;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/ParentJob;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/ParentJob;Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; +} + +public final class kotlinx/coroutines/SupervisorKt { + public static final fun SupervisorJob (Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/CompletableJob; + public static final synthetic fun SupervisorJob (Lkotlinx/coroutines/Job;)Lkotlinx/coroutines/Job; + public static synthetic fun SupervisorJob$default (Lkotlinx/coroutines/Job;ILjava/lang/Object;)Lkotlinx/coroutines/CompletableJob; + public static synthetic fun SupervisorJob$default (Lkotlinx/coroutines/Job;ILjava/lang/Object;)Lkotlinx/coroutines/Job; + public static final fun supervisorScope (Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public abstract interface class kotlinx/coroutines/ThreadContextElement : kotlin/coroutines/CoroutineContext$Element { + public abstract fun restoreThreadContext (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Object;)V + public abstract fun updateThreadContext (Lkotlin/coroutines/CoroutineContext;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/ThreadContextElement$DefaultImpls { + public static fun fold (Lkotlinx/coroutines/ThreadContextElement;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)Ljava/lang/Object; + public static fun get (Lkotlinx/coroutines/ThreadContextElement;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext$Element; + public static fun minusKey (Lkotlinx/coroutines/ThreadContextElement;Lkotlin/coroutines/CoroutineContext$Key;)Lkotlin/coroutines/CoroutineContext; + public static fun plus (Lkotlinx/coroutines/ThreadContextElement;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext; +} + +public final class kotlinx/coroutines/ThreadContextElementKt { + public static final fun asContextElement (Ljava/lang/ThreadLocal;Ljava/lang/Object;)Lkotlinx/coroutines/ThreadContextElement; + public static synthetic fun asContextElement$default (Ljava/lang/ThreadLocal;Ljava/lang/Object;ILjava/lang/Object;)Lkotlinx/coroutines/ThreadContextElement; + public static final fun ensurePresent (Ljava/lang/ThreadLocal;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun isPresent (Ljava/lang/ThreadLocal;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/ThreadPoolDispatcherKt { + public static final fun newFixedThreadPoolContext (ILjava/lang/String;)Lkotlinx/coroutines/ExecutorCoroutineDispatcher; + public static final fun newSingleThreadContext (Ljava/lang/String;)Lkotlinx/coroutines/ExecutorCoroutineDispatcher; +} + +public final class kotlinx/coroutines/TimeoutCancellationException : java/util/concurrent/CancellationException, kotlinx/coroutines/CopyableThrowable { + public synthetic fun createCopy ()Ljava/lang/Throwable; + public fun createCopy ()Lkotlinx/coroutines/TimeoutCancellationException; +} + +public final class kotlinx/coroutines/TimeoutKt { + public static final fun withTimeout (JLkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun withTimeout-KLykuaI (JLkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun withTimeoutOrNull (JLkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun withTimeoutOrNull-KLykuaI (JLkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/YieldContext : kotlin/coroutines/AbstractCoroutineContextElement { + public static final field Key Lkotlinx/coroutines/YieldContext$Key; + public field dispatcherWasUnconfined Z + public fun ()V +} + +public final class kotlinx/coroutines/YieldContext$Key : kotlin/coroutines/CoroutineContext$Key { +} + +public final class kotlinx/coroutines/YieldKt { + public static final fun yield (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/ActorKt { + public static final fun actor (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/SendChannel; + public static synthetic fun actor$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/SendChannel; +} + +public abstract interface class kotlinx/coroutines/channels/ActorScope : kotlinx/coroutines/CoroutineScope, kotlinx/coroutines/channels/ReceiveChannel { + public abstract fun getChannel ()Lkotlinx/coroutines/channels/Channel; +} + +public final class kotlinx/coroutines/channels/ActorScope$DefaultImpls { + public static synthetic fun cancel (Lkotlinx/coroutines/channels/ActorScope;)V + public static fun getOnReceiveOrNull (Lkotlinx/coroutines/channels/ActorScope;)Lkotlinx/coroutines/selects/SelectClause1; + public static fun poll (Lkotlinx/coroutines/channels/ActorScope;)Ljava/lang/Object; + public static fun receiveOrNull (Lkotlinx/coroutines/channels/ActorScope;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public abstract interface class kotlinx/coroutines/channels/BroadcastChannel : kotlinx/coroutines/channels/SendChannel { + public abstract synthetic fun cancel (Ljava/lang/Throwable;)Z + public abstract fun cancel (Ljava/util/concurrent/CancellationException;)V + public abstract fun openSubscription ()Lkotlinx/coroutines/channels/ReceiveChannel; +} + +public final class kotlinx/coroutines/channels/BroadcastChannel$DefaultImpls { + public static synthetic fun cancel$default (Lkotlinx/coroutines/channels/BroadcastChannel;Ljava/lang/Throwable;ILjava/lang/Object;)Z + public static synthetic fun cancel$default (Lkotlinx/coroutines/channels/BroadcastChannel;Ljava/util/concurrent/CancellationException;ILjava/lang/Object;)V + public static fun offer (Lkotlinx/coroutines/channels/BroadcastChannel;Ljava/lang/Object;)Z +} + +public final class kotlinx/coroutines/channels/BroadcastChannelKt { + public static final fun BroadcastChannel (I)Lkotlinx/coroutines/channels/BroadcastChannel; +} + +public final class kotlinx/coroutines/channels/BroadcastKt { + public static final fun broadcast (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/BroadcastChannel; + public static final fun broadcast (Lkotlinx/coroutines/channels/ReceiveChannel;ILkotlinx/coroutines/CoroutineStart;)Lkotlinx/coroutines/channels/BroadcastChannel; + public static synthetic fun broadcast$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/BroadcastChannel; + public static synthetic fun broadcast$default (Lkotlinx/coroutines/channels/ReceiveChannel;ILkotlinx/coroutines/CoroutineStart;ILjava/lang/Object;)Lkotlinx/coroutines/channels/BroadcastChannel; +} + +public final class kotlinx/coroutines/channels/BufferOverflow : java/lang/Enum { + public static final field DROP_LATEST Lkotlinx/coroutines/channels/BufferOverflow; + public static final field DROP_OLDEST Lkotlinx/coroutines/channels/BufferOverflow; + public static final field SUSPEND Lkotlinx/coroutines/channels/BufferOverflow; + public static fun getEntries ()Lkotlin/enums/EnumEntries; + public static fun valueOf (Ljava/lang/String;)Lkotlinx/coroutines/channels/BufferOverflow; + public static fun values ()[Lkotlinx/coroutines/channels/BufferOverflow; +} + +public abstract interface class kotlinx/coroutines/channels/Channel : kotlinx/coroutines/channels/ReceiveChannel, kotlinx/coroutines/channels/SendChannel { + public static final field BUFFERED I + public static final field CONFLATED I + public static final field DEFAULT_BUFFER_PROPERTY_NAME Ljava/lang/String; + public static final field Factory Lkotlinx/coroutines/channels/Channel$Factory; + public static final field RENDEZVOUS I + public static final field UNLIMITED I +} + +public final class kotlinx/coroutines/channels/Channel$DefaultImpls { + public static synthetic fun cancel (Lkotlinx/coroutines/channels/Channel;)V + public static fun getOnReceiveOrNull (Lkotlinx/coroutines/channels/Channel;)Lkotlinx/coroutines/selects/SelectClause1; + public static fun offer (Lkotlinx/coroutines/channels/Channel;Ljava/lang/Object;)Z + public static fun poll (Lkotlinx/coroutines/channels/Channel;)Ljava/lang/Object; + public static fun receiveOrNull (Lkotlinx/coroutines/channels/Channel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/Channel$Factory { + public static final field BUFFERED I + public static final field CONFLATED I + public static final field DEFAULT_BUFFER_PROPERTY_NAME Ljava/lang/String; + public static final field RENDEZVOUS I + public static final field UNLIMITED I +} + +public abstract interface class kotlinx/coroutines/channels/ChannelIterator { + public abstract fun hasNext (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun next ()Ljava/lang/Object; + public abstract synthetic fun next (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/ChannelIterator$DefaultImpls { + public static synthetic fun next (Lkotlinx/coroutines/channels/ChannelIterator;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/ChannelKt { + public static final synthetic fun Channel (I)Lkotlinx/coroutines/channels/Channel; + public static final fun Channel (ILkotlinx/coroutines/channels/BufferOverflow;Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/channels/Channel; + public static synthetic fun Channel$default (IILjava/lang/Object;)Lkotlinx/coroutines/channels/Channel; + public static synthetic fun Channel$default (ILkotlinx/coroutines/channels/BufferOverflow;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lkotlinx/coroutines/channels/Channel; + public static final fun getOrElse-WpGqRn0 (Ljava/lang/Object;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object; + public static final fun onClosed-WpGqRn0 (Ljava/lang/Object;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object; + public static final fun onFailure-WpGqRn0 (Ljava/lang/Object;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object; + public static final fun onSuccess-WpGqRn0 (Ljava/lang/Object;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/ChannelResult { + public static final field Companion Lkotlinx/coroutines/channels/ChannelResult$Companion; + public static final synthetic fun box-impl (Ljava/lang/Object;)Lkotlinx/coroutines/channels/ChannelResult; + public static fun constructor-impl (Ljava/lang/Object;)Ljava/lang/Object; + public fun equals (Ljava/lang/Object;)Z + public static fun equals-impl (Ljava/lang/Object;Ljava/lang/Object;)Z + public static final fun equals-impl0 (Ljava/lang/Object;Ljava/lang/Object;)Z + public static final fun exceptionOrNull-impl (Ljava/lang/Object;)Ljava/lang/Throwable; + public static final fun getOrNull-impl (Ljava/lang/Object;)Ljava/lang/Object; + public static final fun getOrThrow-impl (Ljava/lang/Object;)Ljava/lang/Object; + public fun hashCode ()I + public static fun hashCode-impl (Ljava/lang/Object;)I + public static final fun isClosed-impl (Ljava/lang/Object;)Z + public static final fun isFailure-impl (Ljava/lang/Object;)Z + public static final fun isSuccess-impl (Ljava/lang/Object;)Z + public fun toString ()Ljava/lang/String; + public static fun toString-impl (Ljava/lang/Object;)Ljava/lang/String; + public final synthetic fun unbox-impl ()Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/ChannelResult$Companion { + public final fun closed-JP2dKIU (Ljava/lang/Throwable;)Ljava/lang/Object; + public final fun failure-PtdJZtk ()Ljava/lang/Object; + public final fun success-JP2dKIU (Ljava/lang/Object;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/ChannelsKt { + public static final synthetic fun any (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun cancelConsumed (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/lang/Throwable;)V + public static final fun consume (Lkotlinx/coroutines/channels/BroadcastChannel;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object; + public static final fun consume (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object; + public static final fun consumeEach (Lkotlinx/coroutines/channels/BroadcastChannel;Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun consumeEach (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun consumes (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlin/jvm/functions/Function1; + public static final fun consumesAll ([Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlin/jvm/functions/Function1; + public static final synthetic fun count (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun distinct (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun distinctBy (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun distinctBy$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun drop (Lkotlinx/coroutines/channels/ReceiveChannel;ILkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun drop$default (Lkotlinx/coroutines/channels/ReceiveChannel;ILkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun dropWhile (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun dropWhile$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun elementAt (Lkotlinx/coroutines/channels/ReceiveChannel;ILkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun elementAtOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;ILkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun filter (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun filter$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun filterIndexed (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun filterIndexed$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function3;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun filterNot (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun filterNot$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun filterNotNull (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun filterNotNullTo (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/util/Collection;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun filterNotNullTo (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlinx/coroutines/channels/SendChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun first (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun firstOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun flatMap (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun flatMap$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun indexOf (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun last (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun lastIndexOf (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun lastOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun map (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun map$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun mapIndexed (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun mapIndexed$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function3;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun mapIndexedNotNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun mapIndexedNotNull$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function3;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun mapNotNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun mapNotNull$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun maxWith (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/util/Comparator;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun minWith (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/util/Comparator;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun none (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun onReceiveOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/selects/SelectClause1; + public static final synthetic fun receiveOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun requireNoNulls (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun sendBlocking (Lkotlinx/coroutines/channels/SendChannel;Ljava/lang/Object;)V + public static final synthetic fun single (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun singleOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun take (Lkotlinx/coroutines/channels/ReceiveChannel;ILkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun take$default (Lkotlinx/coroutines/channels/ReceiveChannel;ILkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun takeWhile (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun takeWhile$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun toChannel (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlinx/coroutines/channels/SendChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun toCollection (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/util/Collection;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun toList (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun toMap (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/util/Map;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun toMap (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun toMutableList (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun toMutableSet (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun toSet (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun trySendBlocking (Lkotlinx/coroutines/channels/SendChannel;Ljava/lang/Object;)Ljava/lang/Object; + public static final synthetic fun withIndex (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun withIndex$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun zip (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun zip (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun zip$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; +} + +public final class kotlinx/coroutines/channels/ClosedReceiveChannelException : java/util/NoSuchElementException { + public fun (Ljava/lang/String;)V +} + +public final class kotlinx/coroutines/channels/ClosedSendChannelException : java/lang/IllegalStateException { + public fun (Ljava/lang/String;)V +} + +public final class kotlinx/coroutines/channels/ConflatedBroadcastChannel : kotlinx/coroutines/channels/BroadcastChannel { + public fun ()V + public fun (Ljava/lang/Object;)V + public synthetic fun cancel (Ljava/lang/Throwable;)Z + public fun cancel (Ljava/util/concurrent/CancellationException;)V + public fun close (Ljava/lang/Throwable;)Z + public fun getOnSend ()Lkotlinx/coroutines/selects/SelectClause2; + public final fun getValue ()Ljava/lang/Object; + public final fun getValueOrNull ()Ljava/lang/Object; + public fun invokeOnClose (Lkotlin/jvm/functions/Function1;)V + public fun isClosedForSend ()Z + public fun offer (Ljava/lang/Object;)Z + public fun openSubscription ()Lkotlinx/coroutines/channels/ReceiveChannel; + public fun send (Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun trySend-JP2dKIU (Ljava/lang/Object;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/ProduceKt { + public static final fun awaitClose (Lkotlinx/coroutines/channels/ProducerScope;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static synthetic fun awaitClose$default (Lkotlinx/coroutines/channels/ProducerScope;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;ILjava/lang/Object;)Ljava/lang/Object; + public static final fun produce (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;ILkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun produce (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun produce$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;ILkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun produce$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; +} + +public abstract interface class kotlinx/coroutines/channels/ProducerScope : kotlinx/coroutines/CoroutineScope, kotlinx/coroutines/channels/SendChannel { + public abstract fun getChannel ()Lkotlinx/coroutines/channels/SendChannel; +} + +public final class kotlinx/coroutines/channels/ProducerScope$DefaultImpls { + public static fun offer (Lkotlinx/coroutines/channels/ProducerScope;Ljava/lang/Object;)Z +} + +public abstract interface class kotlinx/coroutines/channels/ReceiveChannel { + public abstract synthetic fun cancel ()V + public abstract synthetic fun cancel (Ljava/lang/Throwable;)Z + public abstract fun cancel (Ljava/util/concurrent/CancellationException;)V + public abstract fun getOnReceive ()Lkotlinx/coroutines/selects/SelectClause1; + public abstract fun getOnReceiveCatching ()Lkotlinx/coroutines/selects/SelectClause1; + public abstract fun getOnReceiveOrNull ()Lkotlinx/coroutines/selects/SelectClause1; + public abstract fun isClosedForReceive ()Z + public abstract fun isEmpty ()Z + public abstract fun iterator ()Lkotlinx/coroutines/channels/ChannelIterator; + public abstract fun poll ()Ljava/lang/Object; + public abstract fun receive (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun receiveCatching-JP2dKIU (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun receiveOrNull (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun tryReceive-PtdJZtk ()Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/ReceiveChannel$DefaultImpls { + public static synthetic fun cancel (Lkotlinx/coroutines/channels/ReceiveChannel;)V + public static synthetic fun cancel$default (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/lang/Throwable;ILjava/lang/Object;)Z + public static synthetic fun cancel$default (Lkotlinx/coroutines/channels/ReceiveChannel;Ljava/util/concurrent/CancellationException;ILjava/lang/Object;)V + public static fun getOnReceiveOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/selects/SelectClause1; + public static fun poll (Lkotlinx/coroutines/channels/ReceiveChannel;)Ljava/lang/Object; + public static fun receiveOrNull (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public abstract interface class kotlinx/coroutines/channels/SendChannel { + public abstract fun close (Ljava/lang/Throwable;)Z + public abstract fun getOnSend ()Lkotlinx/coroutines/selects/SelectClause2; + public abstract fun invokeOnClose (Lkotlin/jvm/functions/Function1;)V + public abstract fun isClosedForSend ()Z + public abstract fun offer (Ljava/lang/Object;)Z + public abstract fun send (Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun trySend-JP2dKIU (Ljava/lang/Object;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/channels/SendChannel$DefaultImpls { + public static synthetic fun close$default (Lkotlinx/coroutines/channels/SendChannel;Ljava/lang/Throwable;ILjava/lang/Object;)Z + public static fun offer (Lkotlinx/coroutines/channels/SendChannel;Ljava/lang/Object;)Z +} + +public final class kotlinx/coroutines/channels/TickerChannelsKt { + public static final fun ticker (JJLkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/channels/TickerMode;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun ticker$default (JJLkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/channels/TickerMode;ILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; +} + +public final class kotlinx/coroutines/channels/TickerMode : java/lang/Enum { + public static final field FIXED_DELAY Lkotlinx/coroutines/channels/TickerMode; + public static final field FIXED_PERIOD Lkotlinx/coroutines/channels/TickerMode; + public static fun getEntries ()Lkotlin/enums/EnumEntries; + public static fun valueOf (Ljava/lang/String;)Lkotlinx/coroutines/channels/TickerMode; + public static fun values ()[Lkotlinx/coroutines/channels/TickerMode; +} + +public final class kotlinx/coroutines/debug/internal/AgentInstallationType { + public static final field INSTANCE Lkotlinx/coroutines/debug/internal/AgentInstallationType; +} + +public final class kotlinx/coroutines/debug/internal/DebugCoroutineInfo { + public final fun getContext ()Lkotlin/coroutines/CoroutineContext; + public final fun getCreationStackTrace ()Ljava/util/List; + public final fun getLastObservedFrame ()Lkotlin/coroutines/jvm/internal/CoroutineStackFrame; + public final fun getLastObservedThread ()Ljava/lang/Thread; + public final fun getSequenceNumber ()J + public final fun getState ()Ljava/lang/String; + public final fun lastObservedStackTrace ()Ljava/util/List; +} + +public final class kotlinx/coroutines/debug/internal/DebugCoroutineInfoImpl { + public field _lastObservedFrame Ljava/lang/ref/WeakReference; + public field _state Ljava/lang/String; + public field lastObservedThread Ljava/lang/Thread; + public final field sequenceNumber J + public final fun getContext ()Lkotlin/coroutines/CoroutineContext; + public final fun getCreationStackTrace ()Ljava/util/List; + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/debug/internal/DebugProbesImpl { + public static final field INSTANCE Lkotlinx/coroutines/debug/internal/DebugProbesImpl; + public final fun dumpCoroutinesInfo ()Ljava/util/List; + public final fun dumpCoroutinesInfoAsJsonAndReferences ()[Ljava/lang/Object; + public final fun dumpDebuggerInfo ()Ljava/util/List; + public final fun enhanceStackTraceWithThreadDump (Lkotlinx/coroutines/debug/internal/DebugCoroutineInfo;Ljava/util/List;)Ljava/util/List; + public final fun enhanceStackTraceWithThreadDumpAsJson (Lkotlinx/coroutines/debug/internal/DebugCoroutineInfo;)Ljava/lang/String; + public final fun getIgnoreCoroutinesWithEmptyContext ()Z + public final fun isInstalled$kotlinx_coroutines_debug ()Z + public final fun setIgnoreCoroutinesWithEmptyContext (Z)V +} + +public final class kotlinx/coroutines/debug/internal/DebugProbesImpl$CoroutineOwner : kotlin/coroutines/Continuation, kotlin/coroutines/jvm/internal/CoroutineStackFrame { + public final field info Lkotlinx/coroutines/debug/internal/DebugCoroutineInfoImpl; + public fun getCallerFrame ()Lkotlin/coroutines/jvm/internal/CoroutineStackFrame; + public fun getContext ()Lkotlin/coroutines/CoroutineContext; + public fun getStackTraceElement ()Ljava/lang/StackTraceElement; + public fun resumeWith (Ljava/lang/Object;)V + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/debug/internal/DebuggerInfo : java/io/Serializable { + public fun (Lkotlinx/coroutines/debug/internal/DebugCoroutineInfoImpl;Lkotlin/coroutines/CoroutineContext;)V + public final fun getCoroutineId ()Ljava/lang/Long; + public final fun getDispatcher ()Ljava/lang/String; + public final fun getLastObservedStackTrace ()Ljava/util/List; + public final fun getLastObservedThreadName ()Ljava/lang/String; + public final fun getLastObservedThreadState ()Ljava/lang/String; + public final fun getName ()Ljava/lang/String; + public final fun getSequenceNumber ()J + public final fun getState ()Ljava/lang/String; +} + +public abstract class kotlinx/coroutines/flow/AbstractFlow : kotlinx/coroutines/flow/CancellableFlow, kotlinx/coroutines/flow/Flow { + public fun ()V + public final fun collect (Lkotlinx/coroutines/flow/FlowCollector;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun collectSafely (Lkotlinx/coroutines/flow/FlowCollector;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public abstract interface class kotlinx/coroutines/flow/Flow { + public abstract fun collect (Lkotlinx/coroutines/flow/FlowCollector;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public abstract interface class kotlinx/coroutines/flow/FlowCollector { + public abstract fun emit (Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/flow/FlowKt { + public static final field DEFAULT_CONCURRENCY_PROPERTY_NAME Ljava/lang/String; + public static final fun all (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun any (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun asFlow (Ljava/lang/Iterable;)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlow (Ljava/util/Iterator;)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlow (Lkotlin/jvm/functions/Function0;)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlow (Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlow (Lkotlin/ranges/IntRange;)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlow (Lkotlin/ranges/LongRange;)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlow (Lkotlin/sequences/Sequence;)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlow ([I)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlow ([J)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlow ([Ljava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun asSharedFlow (Lkotlinx/coroutines/flow/MutableSharedFlow;)Lkotlinx/coroutines/flow/SharedFlow; + public static final fun asStateFlow (Lkotlinx/coroutines/flow/MutableStateFlow;)Lkotlinx/coroutines/flow/StateFlow; + public static final synthetic fun buffer (Lkotlinx/coroutines/flow/Flow;I)Lkotlinx/coroutines/flow/Flow; + public static final fun buffer (Lkotlinx/coroutines/flow/Flow;ILkotlinx/coroutines/channels/BufferOverflow;)Lkotlinx/coroutines/flow/Flow; + public static synthetic fun buffer$default (Lkotlinx/coroutines/flow/Flow;IILjava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static synthetic fun buffer$default (Lkotlinx/coroutines/flow/Flow;ILkotlinx/coroutines/channels/BufferOverflow;ILjava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun cache (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun callbackFlow (Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun cancellable (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun catch (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun channelFlow (Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun chunked (Lkotlinx/coroutines/flow/Flow;I)Lkotlinx/coroutines/flow/Flow; + public static final fun collect (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun collect (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun collectIndexed (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun collectLatest (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun combine (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun combine (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function4;)Lkotlinx/coroutines/flow/Flow; + public static final fun combine (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function5;)Lkotlinx/coroutines/flow/Flow; + public static final fun combine (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function6;)Lkotlinx/coroutines/flow/Flow; + public static final fun combineLatest (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun combineLatest (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function4;)Lkotlinx/coroutines/flow/Flow; + public static final fun combineLatest (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function5;)Lkotlinx/coroutines/flow/Flow; + public static final fun combineLatest (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function6;)Lkotlinx/coroutines/flow/Flow; + public static final fun combineTransform (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function4;)Lkotlinx/coroutines/flow/Flow; + public static final fun combineTransform (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function5;)Lkotlinx/coroutines/flow/Flow; + public static final fun combineTransform (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function6;)Lkotlinx/coroutines/flow/Flow; + public static final fun combineTransform (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function7;)Lkotlinx/coroutines/flow/Flow; + public static final fun compose (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/flow/Flow; + public static final fun concatMap (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/flow/Flow; + public static final fun concatWith (Lkotlinx/coroutines/flow/Flow;Ljava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun concatWith (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun conflate (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun consumeAsFlow (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/flow/Flow; + public static final fun count (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun count (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun debounce (Lkotlinx/coroutines/flow/Flow;J)Lkotlinx/coroutines/flow/Flow; + public static final fun debounce (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/flow/Flow; + public static final fun debounce-HG0u8IE (Lkotlinx/coroutines/flow/Flow;J)Lkotlinx/coroutines/flow/Flow; + public static final fun debounceDuration (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/flow/Flow; + public static final fun delayEach (Lkotlinx/coroutines/flow/Flow;J)Lkotlinx/coroutines/flow/Flow; + public static final fun delayFlow (Lkotlinx/coroutines/flow/Flow;J)Lkotlinx/coroutines/flow/Flow; + public static final fun distinctUntilChanged (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun distinctUntilChanged (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun distinctUntilChangedBy (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/flow/Flow; + public static final fun drop (Lkotlinx/coroutines/flow/Flow;I)Lkotlinx/coroutines/flow/Flow; + public static final fun dropWhile (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun emitAll (Lkotlinx/coroutines/flow/FlowCollector;Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun emitAll (Lkotlinx/coroutines/flow/FlowCollector;Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun emptyFlow ()Lkotlinx/coroutines/flow/Flow; + public static final fun filter (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun filterIsInstance (Lkotlinx/coroutines/flow/Flow;Lkotlin/reflect/KClass;)Lkotlinx/coroutines/flow/Flow; + public static final fun filterNot (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun filterNotNull (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun first (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun first (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun firstOrNull (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun firstOrNull (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun flatMap (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun flatMapConcat (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun flatMapLatest (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun flatMapMerge (Lkotlinx/coroutines/flow/Flow;ILkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static synthetic fun flatMapMerge$default (Lkotlinx/coroutines/flow/Flow;ILkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun flatten (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun flattenConcat (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun flattenMerge (Lkotlinx/coroutines/flow/Flow;I)Lkotlinx/coroutines/flow/Flow; + public static synthetic fun flattenMerge$default (Lkotlinx/coroutines/flow/Flow;IILjava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun flow (Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun flowCombine (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun flowCombineTransform (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function4;)Lkotlinx/coroutines/flow/Flow; + public static final fun flowOf (Ljava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun flowOf ([Ljava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun flowOn (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/flow/Flow; + public static final fun fold (Lkotlinx/coroutines/flow/Flow;Ljava/lang/Object;Lkotlin/jvm/functions/Function3;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun forEach (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)V + public static final fun getDEFAULT_CONCURRENCY ()I + public static final fun last (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun lastOrNull (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun launchIn (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/CoroutineScope;)Lkotlinx/coroutines/Job; + public static final fun map (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun mapLatest (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun mapNotNull (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun merge (Ljava/lang/Iterable;)Lkotlinx/coroutines/flow/Flow; + public static final fun merge (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun merge ([Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun none (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun observeOn (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/flow/Flow; + public static final fun onCompletion (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun onEach (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun onEmpty (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun onErrorResume (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun onErrorResumeNext (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun onErrorReturn (Lkotlinx/coroutines/flow/Flow;Ljava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun onErrorReturn (Lkotlinx/coroutines/flow/Flow;Ljava/lang/Object;Lkotlin/jvm/functions/Function1;)Lkotlinx/coroutines/flow/Flow; + public static synthetic fun onErrorReturn$default (Lkotlinx/coroutines/flow/Flow;Ljava/lang/Object;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun onStart (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun onSubscription (Lkotlinx/coroutines/flow/SharedFlow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/SharedFlow; + public static final fun produceIn (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/CoroutineScope;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun publish (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun publish (Lkotlinx/coroutines/flow/Flow;I)Lkotlinx/coroutines/flow/Flow; + public static final fun publishOn (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/flow/Flow; + public static final fun receiveAsFlow (Lkotlinx/coroutines/channels/ReceiveChannel;)Lkotlinx/coroutines/flow/Flow; + public static final fun reduce (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun replay (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun replay (Lkotlinx/coroutines/flow/Flow;I)Lkotlinx/coroutines/flow/Flow; + public static final fun retry (Lkotlinx/coroutines/flow/Flow;JLkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static synthetic fun retry$default (Lkotlinx/coroutines/flow/Flow;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun retryWhen (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function4;)Lkotlinx/coroutines/flow/Flow; + public static final fun runningFold (Lkotlinx/coroutines/flow/Flow;Ljava/lang/Object;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun runningReduce (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun sample (Lkotlinx/coroutines/flow/Flow;J)Lkotlinx/coroutines/flow/Flow; + public static final fun sample-HG0u8IE (Lkotlinx/coroutines/flow/Flow;J)Lkotlinx/coroutines/flow/Flow; + public static final fun scan (Lkotlinx/coroutines/flow/Flow;Ljava/lang/Object;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun scanFold (Lkotlinx/coroutines/flow/Flow;Ljava/lang/Object;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun scanReduce (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun shareIn (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/CoroutineScope;Lkotlinx/coroutines/flow/SharingStarted;I)Lkotlinx/coroutines/flow/SharedFlow; + public static synthetic fun shareIn$default (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/CoroutineScope;Lkotlinx/coroutines/flow/SharingStarted;IILjava/lang/Object;)Lkotlinx/coroutines/flow/SharedFlow; + public static final fun single (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun singleOrNull (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun skip (Lkotlinx/coroutines/flow/Flow;I)Lkotlinx/coroutines/flow/Flow; + public static final fun startWith (Lkotlinx/coroutines/flow/Flow;Ljava/lang/Object;)Lkotlinx/coroutines/flow/Flow; + public static final fun startWith (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun stateIn (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun stateIn (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/CoroutineScope;Lkotlinx/coroutines/flow/SharingStarted;Ljava/lang/Object;)Lkotlinx/coroutines/flow/StateFlow; + public static final fun subscribe (Lkotlinx/coroutines/flow/Flow;)V + public static final fun subscribe (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)V + public static final fun subscribe (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/jvm/functions/Function2;)V + public static final fun subscribeOn (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/flow/Flow; + public static final fun switchMap (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun take (Lkotlinx/coroutines/flow/Flow;I)Lkotlinx/coroutines/flow/Flow; + public static final fun takeWhile (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; + public static final fun timeout-HG0u8IE (Lkotlinx/coroutines/flow/Flow;J)Lkotlinx/coroutines/flow/Flow; + public static final fun toCollection (Lkotlinx/coroutines/flow/Flow;Ljava/util/Collection;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun toList (Lkotlinx/coroutines/flow/Flow;Ljava/util/List;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static synthetic fun toList$default (Lkotlinx/coroutines/flow/Flow;Ljava/util/List;Lkotlin/coroutines/Continuation;ILjava/lang/Object;)Ljava/lang/Object; + public static final fun toSet (Lkotlinx/coroutines/flow/Flow;Ljava/util/Set;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static synthetic fun toSet$default (Lkotlinx/coroutines/flow/Flow;Ljava/util/Set;Lkotlin/coroutines/Continuation;ILjava/lang/Object;)Ljava/lang/Object; + public static final fun transform (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun transformLatest (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun transformWhile (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun unsafeTransform (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; + public static final fun withIndex (Lkotlinx/coroutines/flow/Flow;)Lkotlinx/coroutines/flow/Flow; + public static final fun zip (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow; +} + +public final class kotlinx/coroutines/flow/LintKt { + public static final fun cancel (Lkotlinx/coroutines/flow/FlowCollector;Ljava/util/concurrent/CancellationException;)V + public static synthetic fun cancel$default (Lkotlinx/coroutines/flow/FlowCollector;Ljava/util/concurrent/CancellationException;ILjava/lang/Object;)V + public static final fun cancellable (Lkotlinx/coroutines/flow/SharedFlow;)Lkotlinx/coroutines/flow/Flow; + public static final fun conflate (Lkotlinx/coroutines/flow/StateFlow;)Lkotlinx/coroutines/flow/Flow; + public static final fun distinctUntilChanged (Lkotlinx/coroutines/flow/StateFlow;)Lkotlinx/coroutines/flow/Flow; + public static final fun flowOn (Lkotlinx/coroutines/flow/SharedFlow;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/flow/Flow; + public static final fun getCoroutineContext (Lkotlinx/coroutines/flow/FlowCollector;)Lkotlin/coroutines/CoroutineContext; + public static final fun isActive (Lkotlinx/coroutines/flow/FlowCollector;)Z +} + +public abstract interface class kotlinx/coroutines/flow/MutableSharedFlow : kotlinx/coroutines/flow/FlowCollector, kotlinx/coroutines/flow/SharedFlow { + public abstract fun emit (Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun getSubscriptionCount ()Lkotlinx/coroutines/flow/StateFlow; + public abstract fun resetReplayCache ()V + public abstract fun tryEmit (Ljava/lang/Object;)Z +} + +public abstract interface class kotlinx/coroutines/flow/MutableStateFlow : kotlinx/coroutines/flow/MutableSharedFlow, kotlinx/coroutines/flow/StateFlow { + public abstract fun compareAndSet (Ljava/lang/Object;Ljava/lang/Object;)Z + public abstract fun getValue ()Ljava/lang/Object; + public abstract fun setValue (Ljava/lang/Object;)V +} + +public abstract interface class kotlinx/coroutines/flow/SharedFlow : kotlinx/coroutines/flow/Flow { + public abstract fun collect (Lkotlinx/coroutines/flow/FlowCollector;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun getReplayCache ()Ljava/util/List; +} + +public final class kotlinx/coroutines/flow/SharedFlowKt { + public static final fun MutableSharedFlow (IILkotlinx/coroutines/channels/BufferOverflow;)Lkotlinx/coroutines/flow/MutableSharedFlow; + public static synthetic fun MutableSharedFlow$default (IILkotlinx/coroutines/channels/BufferOverflow;ILjava/lang/Object;)Lkotlinx/coroutines/flow/MutableSharedFlow; +} + +public final class kotlinx/coroutines/flow/SharingCommand : java/lang/Enum { + public static final field START Lkotlinx/coroutines/flow/SharingCommand; + public static final field STOP Lkotlinx/coroutines/flow/SharingCommand; + public static final field STOP_AND_RESET_REPLAY_CACHE Lkotlinx/coroutines/flow/SharingCommand; + public static fun getEntries ()Lkotlin/enums/EnumEntries; + public static fun valueOf (Ljava/lang/String;)Lkotlinx/coroutines/flow/SharingCommand; + public static fun values ()[Lkotlinx/coroutines/flow/SharingCommand; +} + +public abstract interface class kotlinx/coroutines/flow/SharingStarted { + public static final field Companion Lkotlinx/coroutines/flow/SharingStarted$Companion; + public abstract fun command (Lkotlinx/coroutines/flow/StateFlow;)Lkotlinx/coroutines/flow/Flow; +} + +public final class kotlinx/coroutines/flow/SharingStarted$Companion { + public final fun WhileSubscribed (JJ)Lkotlinx/coroutines/flow/SharingStarted; + public static synthetic fun WhileSubscribed$default (Lkotlinx/coroutines/flow/SharingStarted$Companion;JJILjava/lang/Object;)Lkotlinx/coroutines/flow/SharingStarted; + public final fun getEagerly ()Lkotlinx/coroutines/flow/SharingStarted; + public final fun getLazily ()Lkotlinx/coroutines/flow/SharingStarted; +} + +public final class kotlinx/coroutines/flow/SharingStartedKt { + public static final fun WhileSubscribed-5qebJ5I (Lkotlinx/coroutines/flow/SharingStarted$Companion;JJ)Lkotlinx/coroutines/flow/SharingStarted; + public static synthetic fun WhileSubscribed-5qebJ5I$default (Lkotlinx/coroutines/flow/SharingStarted$Companion;JJILjava/lang/Object;)Lkotlinx/coroutines/flow/SharingStarted; +} + +public abstract interface class kotlinx/coroutines/flow/StateFlow : kotlinx/coroutines/flow/SharedFlow { + public abstract fun getValue ()Ljava/lang/Object; +} + +public final class kotlinx/coroutines/flow/StateFlowKt { + public static final fun MutableStateFlow (Ljava/lang/Object;)Lkotlinx/coroutines/flow/MutableStateFlow; + public static final fun getAndUpdate (Lkotlinx/coroutines/flow/MutableStateFlow;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object; + public static final fun update (Lkotlinx/coroutines/flow/MutableStateFlow;Lkotlin/jvm/functions/Function1;)V + public static final fun updateAndGet (Lkotlinx/coroutines/flow/MutableStateFlow;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object; +} + +public abstract class kotlinx/coroutines/flow/internal/ChannelFlow : kotlinx/coroutines/flow/internal/FusibleFlow { + public final field capacity I + public final field context Lkotlin/coroutines/CoroutineContext; + public final field onBufferOverflow Lkotlinx/coroutines/channels/BufferOverflow; + public fun (Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/channels/BufferOverflow;)V + protected fun additionalToStringProps ()Ljava/lang/String; + public fun collect (Lkotlinx/coroutines/flow/FlowCollector;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + protected abstract fun collectTo (Lkotlinx/coroutines/channels/ProducerScope;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + protected abstract fun create (Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/channels/BufferOverflow;)Lkotlinx/coroutines/flow/internal/ChannelFlow; + public fun dropChannelOperators ()Lkotlinx/coroutines/flow/Flow; + public fun fuse (Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/channels/BufferOverflow;)Lkotlinx/coroutines/flow/Flow; + public fun produceImpl (Lkotlinx/coroutines/CoroutineScope;)Lkotlinx/coroutines/channels/ReceiveChannel; + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/flow/internal/CombineKt { + public static final fun combineInternal (Lkotlinx/coroutines/flow/FlowCollector;[Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function0;Lkotlin/jvm/functions/Function3;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/flow/internal/FlowExceptions_commonKt { + public static final fun checkIndexOverflow (I)I +} + +public abstract interface class kotlinx/coroutines/flow/internal/FusibleFlow : kotlinx/coroutines/flow/Flow { + public abstract fun fuse (Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/channels/BufferOverflow;)Lkotlinx/coroutines/flow/Flow; +} + +public final class kotlinx/coroutines/flow/internal/FusibleFlow$DefaultImpls { + public static synthetic fun fuse$default (Lkotlinx/coroutines/flow/internal/FusibleFlow;Lkotlin/coroutines/CoroutineContext;ILkotlinx/coroutines/channels/BufferOverflow;ILjava/lang/Object;)Lkotlinx/coroutines/flow/Flow; +} + +public final class kotlinx/coroutines/flow/internal/SafeCollector_commonKt { + public static final fun unsafeFlow (Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow; +} + +public final class kotlinx/coroutines/flow/internal/SendingCollector : kotlinx/coroutines/flow/FlowCollector { + public fun (Lkotlinx/coroutines/channels/SendChannel;)V + public fun emit (Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/future/FutureKt { + public static final fun asCompletableFuture (Lkotlinx/coroutines/Deferred;)Ljava/util/concurrent/CompletableFuture; + public static final fun asCompletableFuture (Lkotlinx/coroutines/Job;)Ljava/util/concurrent/CompletableFuture; + public static final fun asDeferred (Ljava/util/concurrent/CompletionStage;)Lkotlinx/coroutines/Deferred; + public static final fun await (Ljava/util/concurrent/CompletionStage;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun future (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;)Ljava/util/concurrent/CompletableFuture; + public static synthetic fun future$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlinx/coroutines/CoroutineStart;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Ljava/util/concurrent/CompletableFuture; +} + +public final class kotlinx/coroutines/intrinsics/CancellableKt { + public static final fun startCoroutineCancellable (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)V +} + +public final class kotlinx/coroutines/selects/OnTimeoutKt { + public static final fun onTimeout (Lkotlinx/coroutines/selects/SelectBuilder;JLkotlin/jvm/functions/Function1;)V + public static final fun onTimeout-8Mi8wO0 (Lkotlinx/coroutines/selects/SelectBuilder;JLkotlin/jvm/functions/Function1;)V +} + +public abstract interface class kotlinx/coroutines/selects/SelectBuilder { + public abstract fun invoke (Lkotlinx/coroutines/selects/SelectClause0;Lkotlin/jvm/functions/Function1;)V + public abstract fun invoke (Lkotlinx/coroutines/selects/SelectClause1;Lkotlin/jvm/functions/Function2;)V + public abstract fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V + public abstract fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Lkotlin/jvm/functions/Function2;)V + public abstract fun onTimeout (JLkotlin/jvm/functions/Function1;)V +} + +public final class kotlinx/coroutines/selects/SelectBuilder$DefaultImpls { + public static fun invoke (Lkotlinx/coroutines/selects/SelectBuilder;Lkotlinx/coroutines/selects/SelectClause2;Lkotlin/jvm/functions/Function2;)V + public static fun onTimeout (Lkotlinx/coroutines/selects/SelectBuilder;JLkotlin/jvm/functions/Function1;)V +} + +public final class kotlinx/coroutines/selects/SelectBuilderImpl : kotlinx/coroutines/selects/SelectImplementation { + public fun (Lkotlin/coroutines/Continuation;)V + public final fun getResult ()Ljava/lang/Object; + public final fun handleBuilderException (Ljava/lang/Throwable;)V +} + +public abstract interface class kotlinx/coroutines/selects/SelectClause { + public abstract fun getClauseObject ()Ljava/lang/Object; + public abstract fun getOnCancellationConstructor ()Lkotlin/jvm/functions/Function3; + public abstract fun getProcessResFunc ()Lkotlin/jvm/functions/Function3; + public abstract fun getRegFunc ()Lkotlin/jvm/functions/Function3; +} + +public abstract interface class kotlinx/coroutines/selects/SelectClause0 : kotlinx/coroutines/selects/SelectClause { +} + +public abstract interface class kotlinx/coroutines/selects/SelectClause1 : kotlinx/coroutines/selects/SelectClause { +} + +public abstract interface class kotlinx/coroutines/selects/SelectClause2 : kotlinx/coroutines/selects/SelectClause { +} + +public class kotlinx/coroutines/selects/SelectImplementation : kotlinx/coroutines/CancelHandler, kotlinx/coroutines/selects/SelectBuilder, kotlinx/coroutines/selects/SelectInstanceInternal { + public fun (Lkotlin/coroutines/CoroutineContext;)V + public fun disposeOnCompletion (Lkotlinx/coroutines/DisposableHandle;)V + public fun doSelect (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun getContext ()Lkotlin/coroutines/CoroutineContext; + public fun invoke (Ljava/lang/Throwable;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause0;Lkotlin/jvm/functions/Function1;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause1;Lkotlin/jvm/functions/Function2;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Lkotlin/jvm/functions/Function2;)V + public fun invokeOnCancellation (Lkotlinx/coroutines/internal/Segment;I)V + public fun onTimeout (JLkotlin/jvm/functions/Function1;)V + public fun selectInRegistrationPhase (Ljava/lang/Object;)V + public fun trySelect (Ljava/lang/Object;Ljava/lang/Object;)Z + public final fun trySelectDetailed (Ljava/lang/Object;Ljava/lang/Object;)Lkotlinx/coroutines/selects/TrySelectDetailedResult; +} + +public abstract interface class kotlinx/coroutines/selects/SelectInstance { + public abstract fun disposeOnCompletion (Lkotlinx/coroutines/DisposableHandle;)V + public abstract fun getContext ()Lkotlin/coroutines/CoroutineContext; + public abstract fun selectInRegistrationPhase (Ljava/lang/Object;)V + public abstract fun trySelect (Ljava/lang/Object;Ljava/lang/Object;)Z +} + +public final class kotlinx/coroutines/selects/SelectKt { + public static final fun select (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/selects/SelectOldKt { + public static final fun selectOld (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun selectUnbiasedOld (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/selects/SelectUnbiasedKt { + public static final fun selectUnbiased (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/selects/UnbiasedSelectBuilderImpl : kotlinx/coroutines/selects/UnbiasedSelectImplementation { + public fun (Lkotlin/coroutines/Continuation;)V + public final fun handleBuilderException (Ljava/lang/Throwable;)V + public final fun initSelectResult ()Ljava/lang/Object; +} + +public class kotlinx/coroutines/selects/UnbiasedSelectImplementation : kotlinx/coroutines/selects/SelectImplementation { + public fun (Lkotlin/coroutines/CoroutineContext;)V + public fun doSelect (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun invoke (Lkotlinx/coroutines/selects/SelectClause0;Lkotlin/jvm/functions/Function1;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause1;Lkotlin/jvm/functions/Function2;)V + public fun invoke (Lkotlinx/coroutines/selects/SelectClause2;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V +} + +public final class kotlinx/coroutines/selects/WhileSelectKt { + public static final fun whileSelect (Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/stream/StreamKt { + public static final fun consumeAsFlow (Ljava/util/stream/Stream;)Lkotlinx/coroutines/flow/Flow; +} + +public abstract interface class kotlinx/coroutines/sync/Mutex { + public abstract fun getOnLock ()Lkotlinx/coroutines/selects/SelectClause2; + public abstract fun holdsLock (Ljava/lang/Object;)Z + public abstract fun isLocked ()Z + public abstract fun lock (Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun tryLock (Ljava/lang/Object;)Z + public abstract fun unlock (Ljava/lang/Object;)V +} + +public final class kotlinx/coroutines/sync/Mutex$DefaultImpls { + public static synthetic fun lock$default (Lkotlinx/coroutines/sync/Mutex;Ljava/lang/Object;Lkotlin/coroutines/Continuation;ILjava/lang/Object;)Ljava/lang/Object; + public static synthetic fun tryLock$default (Lkotlinx/coroutines/sync/Mutex;Ljava/lang/Object;ILjava/lang/Object;)Z + public static synthetic fun unlock$default (Lkotlinx/coroutines/sync/Mutex;Ljava/lang/Object;ILjava/lang/Object;)V +} + +public final class kotlinx/coroutines/sync/MutexKt { + public static final fun Mutex (Z)Lkotlinx/coroutines/sync/Mutex; + public static synthetic fun Mutex$default (ZILjava/lang/Object;)Lkotlinx/coroutines/sync/Mutex; + public static final fun withLock (Lkotlinx/coroutines/sync/Mutex;Ljava/lang/Object;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static synthetic fun withLock$default (Lkotlinx/coroutines/sync/Mutex;Ljava/lang/Object;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;ILjava/lang/Object;)Ljava/lang/Object; +} + +public abstract interface class kotlinx/coroutines/sync/Semaphore { + public abstract fun acquire (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun getAvailablePermits ()I + public abstract fun release ()V + public abstract fun tryAcquire ()Z +} + +public final class kotlinx/coroutines/sync/SemaphoreKt { + public static final fun Semaphore (II)Lkotlinx/coroutines/sync/Semaphore; + public static synthetic fun Semaphore$default (IIILjava/lang/Object;)Lkotlinx/coroutines/sync/Semaphore; + public static final fun withPermit (Lkotlinx/coroutines/sync/Semaphore;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/time/TimeKt { + public static final fun debounce (Lkotlinx/coroutines/flow/Flow;Ljava/time/Duration;)Lkotlinx/coroutines/flow/Flow; + public static final fun delay (Ljava/time/Duration;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun onTimeout (Lkotlinx/coroutines/selects/SelectBuilder;Ljava/time/Duration;Lkotlin/jvm/functions/Function1;)V + public static final fun sample (Lkotlinx/coroutines/flow/Flow;Ljava/time/Duration;)Lkotlinx/coroutines/flow/Flow; + public static final fun withTimeout (Ljava/time/Duration;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun withTimeoutOrNull (Ljava/time/Duration;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + diff --git a/kotlinx-coroutines-core/api/kotlinx-coroutines-core.klib.api b/kotlinx-coroutines-core/api/kotlinx-coroutines-core.klib.api new file mode 100644 index 0000000000..373a1eee52 --- /dev/null +++ b/kotlinx-coroutines-core/api/kotlinx-coroutines-core.klib.api @@ -0,0 +1,1103 @@ +// Klib ABI Dump +// Targets: [androidNativeArm32, androidNativeArm64, androidNativeX64, androidNativeX86, iosArm64, iosSimulatorArm64, iosX64, js, linuxArm64, linuxX64, macosArm64, macosX64, mingwX64, tvosArm64, tvosSimulatorArm64, tvosX64, wasmJs, wasmWasi, watchosArm32, watchosArm64, watchosDeviceArm64, watchosSimulatorArm64, watchosX64] +// Alias: native => [androidNativeArm32, androidNativeArm64, androidNativeX64, androidNativeX86, iosArm64, iosSimulatorArm64, iosX64, linuxArm64, linuxX64, macosArm64, macosX64, mingwX64, tvosArm64, tvosSimulatorArm64, tvosX64, watchosArm32, watchosArm64, watchosDeviceArm64, watchosSimulatorArm64, watchosX64] +// Rendering settings: +// - Signature version: 2 +// - Show manifest properties: true +// - Show declarations: true + +// Library unique name: +open annotation class kotlinx.coroutines/DelicateCoroutinesApi : kotlin/Annotation { // kotlinx.coroutines/DelicateCoroutinesApi|null[0] + constructor () // kotlinx.coroutines/DelicateCoroutinesApi.|(){}[0] +} + +open annotation class kotlinx.coroutines/ExperimentalCoroutinesApi : kotlin/Annotation { // kotlinx.coroutines/ExperimentalCoroutinesApi|null[0] + constructor () // kotlinx.coroutines/ExperimentalCoroutinesApi.|(){}[0] +} + +open annotation class kotlinx.coroutines/ExperimentalForInheritanceCoroutinesApi : kotlin/Annotation { // kotlinx.coroutines/ExperimentalForInheritanceCoroutinesApi|null[0] + constructor () // kotlinx.coroutines/ExperimentalForInheritanceCoroutinesApi.|(){}[0] +} + +open annotation class kotlinx.coroutines/FlowPreview : kotlin/Annotation { // kotlinx.coroutines/FlowPreview|null[0] + constructor () // kotlinx.coroutines/FlowPreview.|(){}[0] +} + +open annotation class kotlinx.coroutines/InternalCoroutinesApi : kotlin/Annotation { // kotlinx.coroutines/InternalCoroutinesApi|null[0] + constructor () // kotlinx.coroutines/InternalCoroutinesApi.|(){}[0] +} + +open annotation class kotlinx.coroutines/InternalForInheritanceCoroutinesApi : kotlin/Annotation { // kotlinx.coroutines/InternalForInheritanceCoroutinesApi|null[0] + constructor () // kotlinx.coroutines/InternalForInheritanceCoroutinesApi.|(){}[0] +} + +open annotation class kotlinx.coroutines/ObsoleteCoroutinesApi : kotlin/Annotation { // kotlinx.coroutines/ObsoleteCoroutinesApi|null[0] + constructor () // kotlinx.coroutines/ObsoleteCoroutinesApi.|(){}[0] +} + +final enum class kotlinx.coroutines.channels/BufferOverflow : kotlin/Enum { // kotlinx.coroutines.channels/BufferOverflow|null[0] + enum entry DROP_LATEST // kotlinx.coroutines.channels/BufferOverflow.DROP_LATEST|null[0] + enum entry DROP_OLDEST // kotlinx.coroutines.channels/BufferOverflow.DROP_OLDEST|null[0] + enum entry SUSPEND // kotlinx.coroutines.channels/BufferOverflow.SUSPEND|null[0] + + final val entries // kotlinx.coroutines.channels/BufferOverflow.entries|#static{}entries[0] + final fun (): kotlin.enums/EnumEntries // kotlinx.coroutines.channels/BufferOverflow.entries.|#static(){}[0] + + final fun valueOf(kotlin/String): kotlinx.coroutines.channels/BufferOverflow // kotlinx.coroutines.channels/BufferOverflow.valueOf|valueOf#static(kotlin.String){}[0] + final fun values(): kotlin/Array // kotlinx.coroutines.channels/BufferOverflow.values|values#static(){}[0] +} + +final enum class kotlinx.coroutines.flow/SharingCommand : kotlin/Enum { // kotlinx.coroutines.flow/SharingCommand|null[0] + enum entry START // kotlinx.coroutines.flow/SharingCommand.START|null[0] + enum entry STOP // kotlinx.coroutines.flow/SharingCommand.STOP|null[0] + enum entry STOP_AND_RESET_REPLAY_CACHE // kotlinx.coroutines.flow/SharingCommand.STOP_AND_RESET_REPLAY_CACHE|null[0] + + final val entries // kotlinx.coroutines.flow/SharingCommand.entries|#static{}entries[0] + final fun (): kotlin.enums/EnumEntries // kotlinx.coroutines.flow/SharingCommand.entries.|#static(){}[0] + + final fun valueOf(kotlin/String): kotlinx.coroutines.flow/SharingCommand // kotlinx.coroutines.flow/SharingCommand.valueOf|valueOf#static(kotlin.String){}[0] + final fun values(): kotlin/Array // kotlinx.coroutines.flow/SharingCommand.values|values#static(){}[0] +} + +final enum class kotlinx.coroutines/CoroutineStart : kotlin/Enum { // kotlinx.coroutines/CoroutineStart|null[0] + enum entry ATOMIC // kotlinx.coroutines/CoroutineStart.ATOMIC|null[0] + enum entry DEFAULT // kotlinx.coroutines/CoroutineStart.DEFAULT|null[0] + enum entry LAZY // kotlinx.coroutines/CoroutineStart.LAZY|null[0] + enum entry UNDISPATCHED // kotlinx.coroutines/CoroutineStart.UNDISPATCHED|null[0] + + final val entries // kotlinx.coroutines/CoroutineStart.entries|#static{}entries[0] + final fun (): kotlin.enums/EnumEntries // kotlinx.coroutines/CoroutineStart.entries.|#static(){}[0] + final val isLazy // kotlinx.coroutines/CoroutineStart.isLazy|{}isLazy[0] + final fun (): kotlin/Boolean // kotlinx.coroutines/CoroutineStart.isLazy.|(){}[0] + + final fun <#A1: kotlin/Any?, #B1: kotlin/Any?> invoke(kotlin.coroutines/SuspendFunction1<#A1, #B1>, #A1, kotlin.coroutines/Continuation<#B1>) // kotlinx.coroutines/CoroutineStart.invoke|invoke(kotlin.coroutines.SuspendFunction1<0:0,0:1>;0:0;kotlin.coroutines.Continuation<0:1>){0§;1§}[0] + final fun valueOf(kotlin/String): kotlinx.coroutines/CoroutineStart // kotlinx.coroutines/CoroutineStart.valueOf|valueOf#static(kotlin.String){}[0] + final fun values(): kotlin/Array // kotlinx.coroutines/CoroutineStart.values|values#static(){}[0] +} + +abstract fun interface <#A: in kotlin/Any?> kotlinx.coroutines.flow/FlowCollector { // kotlinx.coroutines.flow/FlowCollector|null[0] + abstract suspend fun emit(#A) // kotlinx.coroutines.flow/FlowCollector.emit|emit(1:0){}[0] +} + +abstract fun interface kotlinx.coroutines.flow/SharingStarted { // kotlinx.coroutines.flow/SharingStarted|null[0] + abstract fun command(kotlinx.coroutines.flow/StateFlow): kotlinx.coroutines.flow/Flow // kotlinx.coroutines.flow/SharingStarted.command|command(kotlinx.coroutines.flow.StateFlow){}[0] + + final object Companion { // kotlinx.coroutines.flow/SharingStarted.Companion|null[0] + final val Eagerly // kotlinx.coroutines.flow/SharingStarted.Companion.Eagerly|{}Eagerly[0] + final fun (): kotlinx.coroutines.flow/SharingStarted // kotlinx.coroutines.flow/SharingStarted.Companion.Eagerly.|(){}[0] + final val Lazily // kotlinx.coroutines.flow/SharingStarted.Companion.Lazily|{}Lazily[0] + final fun (): kotlinx.coroutines.flow/SharingStarted // kotlinx.coroutines.flow/SharingStarted.Companion.Lazily.|(){}[0] + + final fun WhileSubscribed(kotlin/Long = ..., kotlin/Long = ...): kotlinx.coroutines.flow/SharingStarted // kotlinx.coroutines.flow/SharingStarted.Companion.WhileSubscribed|WhileSubscribed(kotlin.Long;kotlin.Long){}[0] + } +} + +abstract fun interface kotlinx.coroutines/DisposableHandle { // kotlinx.coroutines/DisposableHandle|null[0] + abstract fun dispose() // kotlinx.coroutines/DisposableHandle.dispose|dispose(){}[0] +} + +abstract fun interface kotlinx.coroutines/Runnable { // kotlinx.coroutines/Runnable|null[0] + abstract fun run() // kotlinx.coroutines/Runnable.run|run(){}[0] +} + +abstract interface <#A: in kotlin/Any?> kotlinx.coroutines.channels/ProducerScope : kotlinx.coroutines.channels/SendChannel<#A>, kotlinx.coroutines/CoroutineScope { // kotlinx.coroutines.channels/ProducerScope|null[0] + abstract val channel // kotlinx.coroutines.channels/ProducerScope.channel|{}channel[0] + abstract fun (): kotlinx.coroutines.channels/SendChannel<#A> // kotlinx.coroutines.channels/ProducerScope.channel.|(){}[0] +} + +abstract interface <#A: in kotlin/Any?> kotlinx.coroutines.channels/SendChannel { // kotlinx.coroutines.channels/SendChannel|null[0] + abstract val isClosedForSend // kotlinx.coroutines.channels/SendChannel.isClosedForSend|{}isClosedForSend[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines.channels/SendChannel.isClosedForSend.|(){}[0] + abstract val onSend // kotlinx.coroutines.channels/SendChannel.onSend|{}onSend[0] + abstract fun (): kotlinx.coroutines.selects/SelectClause2<#A, kotlinx.coroutines.channels/SendChannel<#A>> // kotlinx.coroutines.channels/SendChannel.onSend.|(){}[0] + + abstract fun close(kotlin/Throwable? = ...): kotlin/Boolean // kotlinx.coroutines.channels/SendChannel.close|close(kotlin.Throwable?){}[0] + abstract fun invokeOnClose(kotlin/Function1) // kotlinx.coroutines.channels/SendChannel.invokeOnClose|invokeOnClose(kotlin.Function1){}[0] + abstract fun trySend(#A): kotlinx.coroutines.channels/ChannelResult // kotlinx.coroutines.channels/SendChannel.trySend|trySend(1:0){}[0] + abstract suspend fun send(#A) // kotlinx.coroutines.channels/SendChannel.send|send(1:0){}[0] + open fun offer(#A): kotlin/Boolean // kotlinx.coroutines.channels/SendChannel.offer|offer(1:0){}[0] +} + +abstract interface <#A: in kotlin/Any?> kotlinx.coroutines/CancellableContinuation : kotlin.coroutines/Continuation<#A> { // kotlinx.coroutines/CancellableContinuation|null[0] + abstract val isActive // kotlinx.coroutines/CancellableContinuation.isActive|{}isActive[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines/CancellableContinuation.isActive.|(){}[0] + abstract val isCancelled // kotlinx.coroutines/CancellableContinuation.isCancelled|{}isCancelled[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines/CancellableContinuation.isCancelled.|(){}[0] + abstract val isCompleted // kotlinx.coroutines/CancellableContinuation.isCompleted|{}isCompleted[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines/CancellableContinuation.isCompleted.|(){}[0] + + abstract fun (kotlinx.coroutines/CoroutineDispatcher).resumeUndispatched(#A) // kotlinx.coroutines/CancellableContinuation.resumeUndispatched|resumeUndispatched@kotlinx.coroutines.CoroutineDispatcher(1:0){}[0] + abstract fun (kotlinx.coroutines/CoroutineDispatcher).resumeUndispatchedWithException(kotlin/Throwable) // kotlinx.coroutines/CancellableContinuation.resumeUndispatchedWithException|resumeUndispatchedWithException@kotlinx.coroutines.CoroutineDispatcher(kotlin.Throwable){}[0] + abstract fun <#A1: #A> resume(#A1, kotlin/Function3?) // kotlinx.coroutines/CancellableContinuation.resume|resume(0:0;kotlin.Function3?){0§<1:0>}[0] + abstract fun <#A1: #A> tryResume(#A1, kotlin/Any?, kotlin/Function3?): kotlin/Any? // kotlinx.coroutines/CancellableContinuation.tryResume|tryResume(0:0;kotlin.Any?;kotlin.Function3?){0§<1:0>}[0] + abstract fun cancel(kotlin/Throwable? = ...): kotlin/Boolean // kotlinx.coroutines/CancellableContinuation.cancel|cancel(kotlin.Throwable?){}[0] + abstract fun completeResume(kotlin/Any) // kotlinx.coroutines/CancellableContinuation.completeResume|completeResume(kotlin.Any){}[0] + abstract fun initCancellability() // kotlinx.coroutines/CancellableContinuation.initCancellability|initCancellability(){}[0] + abstract fun invokeOnCancellation(kotlin/Function1) // kotlinx.coroutines/CancellableContinuation.invokeOnCancellation|invokeOnCancellation(kotlin.Function1){}[0] + abstract fun resume(#A, kotlin/Function1?) // kotlinx.coroutines/CancellableContinuation.resume|resume(1:0;kotlin.Function1?){}[0] + abstract fun tryResume(#A, kotlin/Any? = ...): kotlin/Any? // kotlinx.coroutines/CancellableContinuation.tryResume|tryResume(1:0;kotlin.Any?){}[0] + abstract fun tryResumeWithException(kotlin/Throwable): kotlin/Any? // kotlinx.coroutines/CancellableContinuation.tryResumeWithException|tryResumeWithException(kotlin.Throwable){}[0] +} + +abstract interface <#A: kotlin/Any?> kotlinx.coroutines.channels/BroadcastChannel : kotlinx.coroutines.channels/SendChannel<#A> { // kotlinx.coroutines.channels/BroadcastChannel|null[0] + abstract fun cancel(kotlin.coroutines.cancellation/CancellationException? = ...) // kotlinx.coroutines.channels/BroadcastChannel.cancel|cancel(kotlin.coroutines.cancellation.CancellationException?){}[0] + abstract fun cancel(kotlin/Throwable? = ...): kotlin/Boolean // kotlinx.coroutines.channels/BroadcastChannel.cancel|cancel(kotlin.Throwable?){}[0] + abstract fun openSubscription(): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/BroadcastChannel.openSubscription|openSubscription(){}[0] +} + +abstract interface <#A: kotlin/Any?> kotlinx.coroutines.channels/Channel : kotlinx.coroutines.channels/ReceiveChannel<#A>, kotlinx.coroutines.channels/SendChannel<#A> { // kotlinx.coroutines.channels/Channel|null[0] + final object Factory { // kotlinx.coroutines.channels/Channel.Factory|null[0] + final const val BUFFERED // kotlinx.coroutines.channels/Channel.Factory.BUFFERED|{}BUFFERED[0] + final fun (): kotlin/Int // kotlinx.coroutines.channels/Channel.Factory.BUFFERED.|(){}[0] + final const val CONFLATED // kotlinx.coroutines.channels/Channel.Factory.CONFLATED|{}CONFLATED[0] + final fun (): kotlin/Int // kotlinx.coroutines.channels/Channel.Factory.CONFLATED.|(){}[0] + final const val DEFAULT_BUFFER_PROPERTY_NAME // kotlinx.coroutines.channels/Channel.Factory.DEFAULT_BUFFER_PROPERTY_NAME|{}DEFAULT_BUFFER_PROPERTY_NAME[0] + final fun (): kotlin/String // kotlinx.coroutines.channels/Channel.Factory.DEFAULT_BUFFER_PROPERTY_NAME.|(){}[0] + final const val RENDEZVOUS // kotlinx.coroutines.channels/Channel.Factory.RENDEZVOUS|{}RENDEZVOUS[0] + final fun (): kotlin/Int // kotlinx.coroutines.channels/Channel.Factory.RENDEZVOUS.|(){}[0] + final const val UNLIMITED // kotlinx.coroutines.channels/Channel.Factory.UNLIMITED|{}UNLIMITED[0] + final fun (): kotlin/Int // kotlinx.coroutines.channels/Channel.Factory.UNLIMITED.|(){}[0] + } +} + +abstract interface <#A: kotlin/Any?> kotlinx.coroutines.flow.internal/FusibleFlow : kotlinx.coroutines.flow/Flow<#A> { // kotlinx.coroutines.flow.internal/FusibleFlow|null[0] + abstract fun fuse(kotlin.coroutines/CoroutineContext = ..., kotlin/Int = ..., kotlinx.coroutines.channels/BufferOverflow = ...): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow.internal/FusibleFlow.fuse|fuse(kotlin.coroutines.CoroutineContext;kotlin.Int;kotlinx.coroutines.channels.BufferOverflow){}[0] +} + +abstract interface <#A: kotlin/Any?> kotlinx.coroutines.flow/MutableSharedFlow : kotlinx.coroutines.flow/FlowCollector<#A>, kotlinx.coroutines.flow/SharedFlow<#A> { // kotlinx.coroutines.flow/MutableSharedFlow|null[0] + abstract val subscriptionCount // kotlinx.coroutines.flow/MutableSharedFlow.subscriptionCount|{}subscriptionCount[0] + abstract fun (): kotlinx.coroutines.flow/StateFlow // kotlinx.coroutines.flow/MutableSharedFlow.subscriptionCount.|(){}[0] + + abstract fun resetReplayCache() // kotlinx.coroutines.flow/MutableSharedFlow.resetReplayCache|resetReplayCache(){}[0] + abstract fun tryEmit(#A): kotlin/Boolean // kotlinx.coroutines.flow/MutableSharedFlow.tryEmit|tryEmit(1:0){}[0] + abstract suspend fun emit(#A) // kotlinx.coroutines.flow/MutableSharedFlow.emit|emit(1:0){}[0] +} + +abstract interface <#A: kotlin/Any?> kotlinx.coroutines.flow/MutableStateFlow : kotlinx.coroutines.flow/MutableSharedFlow<#A>, kotlinx.coroutines.flow/StateFlow<#A> { // kotlinx.coroutines.flow/MutableStateFlow|null[0] + abstract var value // kotlinx.coroutines.flow/MutableStateFlow.value|{}value[0] + abstract fun (): #A // kotlinx.coroutines.flow/MutableStateFlow.value.|(){}[0] + abstract fun (#A) // kotlinx.coroutines.flow/MutableStateFlow.value.|(1:0){}[0] + + abstract fun compareAndSet(#A, #A): kotlin/Boolean // kotlinx.coroutines.flow/MutableStateFlow.compareAndSet|compareAndSet(1:0;1:0){}[0] +} + +abstract interface <#A: kotlin/Any?> kotlinx.coroutines/CompletableDeferred : kotlinx.coroutines/Deferred<#A> { // kotlinx.coroutines/CompletableDeferred|null[0] + abstract fun complete(#A): kotlin/Boolean // kotlinx.coroutines/CompletableDeferred.complete|complete(1:0){}[0] + abstract fun completeExceptionally(kotlin/Throwable): kotlin/Boolean // kotlinx.coroutines/CompletableDeferred.completeExceptionally|completeExceptionally(kotlin.Throwable){}[0] +} + +abstract interface <#A: kotlin/Throwable & kotlinx.coroutines/CopyableThrowable<#A>> kotlinx.coroutines/CopyableThrowable { // kotlinx.coroutines/CopyableThrowable|null[0] + abstract fun createCopy(): #A? // kotlinx.coroutines/CopyableThrowable.createCopy|createCopy(){}[0] +} + +abstract interface <#A: out kotlin/Any?> kotlinx.coroutines.channels/ChannelIterator { // kotlinx.coroutines.channels/ChannelIterator|null[0] + abstract fun next(): #A // kotlinx.coroutines.channels/ChannelIterator.next|next(){}[0] + abstract suspend fun hasNext(): kotlin/Boolean // kotlinx.coroutines.channels/ChannelIterator.hasNext|hasNext(){}[0] + open suspend fun next0(): #A // kotlinx.coroutines.channels/ChannelIterator.next0|next0(){}[0] +} + +abstract interface <#A: out kotlin/Any?> kotlinx.coroutines.channels/ReceiveChannel { // kotlinx.coroutines.channels/ReceiveChannel|null[0] + abstract val isClosedForReceive // kotlinx.coroutines.channels/ReceiveChannel.isClosedForReceive|{}isClosedForReceive[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines.channels/ReceiveChannel.isClosedForReceive.|(){}[0] + abstract val isEmpty // kotlinx.coroutines.channels/ReceiveChannel.isEmpty|{}isEmpty[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines.channels/ReceiveChannel.isEmpty.|(){}[0] + abstract val onReceive // kotlinx.coroutines.channels/ReceiveChannel.onReceive|{}onReceive[0] + abstract fun (): kotlinx.coroutines.selects/SelectClause1<#A> // kotlinx.coroutines.channels/ReceiveChannel.onReceive.|(){}[0] + abstract val onReceiveCatching // kotlinx.coroutines.channels/ReceiveChannel.onReceiveCatching|{}onReceiveCatching[0] + abstract fun (): kotlinx.coroutines.selects/SelectClause1> // kotlinx.coroutines.channels/ReceiveChannel.onReceiveCatching.|(){}[0] + open val onReceiveOrNull // kotlinx.coroutines.channels/ReceiveChannel.onReceiveOrNull|{}onReceiveOrNull[0] + open fun (): kotlinx.coroutines.selects/SelectClause1<#A?> // kotlinx.coroutines.channels/ReceiveChannel.onReceiveOrNull.|(){}[0] + + abstract fun cancel(kotlin.coroutines.cancellation/CancellationException? = ...) // kotlinx.coroutines.channels/ReceiveChannel.cancel|cancel(kotlin.coroutines.cancellation.CancellationException?){}[0] + abstract fun cancel(kotlin/Throwable? = ...): kotlin/Boolean // kotlinx.coroutines.channels/ReceiveChannel.cancel|cancel(kotlin.Throwable?){}[0] + abstract fun iterator(): kotlinx.coroutines.channels/ChannelIterator<#A> // kotlinx.coroutines.channels/ReceiveChannel.iterator|iterator(){}[0] + abstract fun tryReceive(): kotlinx.coroutines.channels/ChannelResult<#A> // kotlinx.coroutines.channels/ReceiveChannel.tryReceive|tryReceive(){}[0] + abstract suspend fun receive(): #A // kotlinx.coroutines.channels/ReceiveChannel.receive|receive(){}[0] + abstract suspend fun receiveCatching(): kotlinx.coroutines.channels/ChannelResult<#A> // kotlinx.coroutines.channels/ReceiveChannel.receiveCatching|receiveCatching(){}[0] + open fun cancel() // kotlinx.coroutines.channels/ReceiveChannel.cancel|cancel(){}[0] + open fun poll(): #A? // kotlinx.coroutines.channels/ReceiveChannel.poll|poll(){}[0] + open suspend fun receiveOrNull(): #A? // kotlinx.coroutines.channels/ReceiveChannel.receiveOrNull|receiveOrNull(){}[0] +} + +abstract interface <#A: out kotlin/Any?> kotlinx.coroutines.flow/Flow { // kotlinx.coroutines.flow/Flow|null[0] + abstract suspend fun collect(kotlinx.coroutines.flow/FlowCollector<#A>) // kotlinx.coroutines.flow/Flow.collect|collect(kotlinx.coroutines.flow.FlowCollector<1:0>){}[0] +} + +abstract interface <#A: out kotlin/Any?> kotlinx.coroutines.flow/SharedFlow : kotlinx.coroutines.flow/Flow<#A> { // kotlinx.coroutines.flow/SharedFlow|null[0] + abstract val replayCache // kotlinx.coroutines.flow/SharedFlow.replayCache|{}replayCache[0] + abstract fun (): kotlin.collections/List<#A> // kotlinx.coroutines.flow/SharedFlow.replayCache.|(){}[0] + + abstract suspend fun collect(kotlinx.coroutines.flow/FlowCollector<#A>): kotlin/Nothing // kotlinx.coroutines.flow/SharedFlow.collect|collect(kotlinx.coroutines.flow.FlowCollector<1:0>){}[0] +} + +abstract interface <#A: out kotlin/Any?> kotlinx.coroutines.flow/StateFlow : kotlinx.coroutines.flow/SharedFlow<#A> { // kotlinx.coroutines.flow/StateFlow|null[0] + abstract val value // kotlinx.coroutines.flow/StateFlow.value|{}value[0] + abstract fun (): #A // kotlinx.coroutines.flow/StateFlow.value.|(){}[0] +} + +abstract interface <#A: out kotlin/Any?> kotlinx.coroutines/Deferred : kotlinx.coroutines/Job { // kotlinx.coroutines/Deferred|null[0] + abstract val onAwait // kotlinx.coroutines/Deferred.onAwait|{}onAwait[0] + abstract fun (): kotlinx.coroutines.selects/SelectClause1<#A> // kotlinx.coroutines/Deferred.onAwait.|(){}[0] + + abstract fun getCompleted(): #A // kotlinx.coroutines/Deferred.getCompleted|getCompleted(){}[0] + abstract fun getCompletionExceptionOrNull(): kotlin/Throwable? // kotlinx.coroutines/Deferred.getCompletionExceptionOrNull|getCompletionExceptionOrNull(){}[0] + abstract suspend fun await(): #A // kotlinx.coroutines/Deferred.await|await(){}[0] +} + +abstract interface kotlinx.coroutines.sync/Mutex { // kotlinx.coroutines.sync/Mutex|null[0] + abstract val isLocked // kotlinx.coroutines.sync/Mutex.isLocked|{}isLocked[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines.sync/Mutex.isLocked.|(){}[0] + abstract val onLock // kotlinx.coroutines.sync/Mutex.onLock|{}onLock[0] + abstract fun (): kotlinx.coroutines.selects/SelectClause2 // kotlinx.coroutines.sync/Mutex.onLock.|(){}[0] + + abstract fun holdsLock(kotlin/Any): kotlin/Boolean // kotlinx.coroutines.sync/Mutex.holdsLock|holdsLock(kotlin.Any){}[0] + abstract fun tryLock(kotlin/Any? = ...): kotlin/Boolean // kotlinx.coroutines.sync/Mutex.tryLock|tryLock(kotlin.Any?){}[0] + abstract fun unlock(kotlin/Any? = ...) // kotlinx.coroutines.sync/Mutex.unlock|unlock(kotlin.Any?){}[0] + abstract suspend fun lock(kotlin/Any? = ...) // kotlinx.coroutines.sync/Mutex.lock|lock(kotlin.Any?){}[0] +} + +abstract interface kotlinx.coroutines.sync/Semaphore { // kotlinx.coroutines.sync/Semaphore|null[0] + abstract val availablePermits // kotlinx.coroutines.sync/Semaphore.availablePermits|{}availablePermits[0] + abstract fun (): kotlin/Int // kotlinx.coroutines.sync/Semaphore.availablePermits.|(){}[0] + + abstract fun release() // kotlinx.coroutines.sync/Semaphore.release|release(){}[0] + abstract fun tryAcquire(): kotlin/Boolean // kotlinx.coroutines.sync/Semaphore.tryAcquire|tryAcquire(){}[0] + abstract suspend fun acquire() // kotlinx.coroutines.sync/Semaphore.acquire|acquire(){}[0] +} + +abstract interface kotlinx.coroutines/ChildHandle : kotlinx.coroutines/DisposableHandle { // kotlinx.coroutines/ChildHandle|null[0] + abstract val parent // kotlinx.coroutines/ChildHandle.parent|{}parent[0] + abstract fun (): kotlinx.coroutines/Job? // kotlinx.coroutines/ChildHandle.parent.|(){}[0] + + abstract fun childCancelled(kotlin/Throwable): kotlin/Boolean // kotlinx.coroutines/ChildHandle.childCancelled|childCancelled(kotlin.Throwable){}[0] +} + +abstract interface kotlinx.coroutines/ChildJob : kotlinx.coroutines/Job { // kotlinx.coroutines/ChildJob|null[0] + abstract fun parentCancelled(kotlinx.coroutines/ParentJob) // kotlinx.coroutines/ChildJob.parentCancelled|parentCancelled(kotlinx.coroutines.ParentJob){}[0] +} + +abstract interface kotlinx.coroutines/CompletableJob : kotlinx.coroutines/Job { // kotlinx.coroutines/CompletableJob|null[0] + abstract fun complete(): kotlin/Boolean // kotlinx.coroutines/CompletableJob.complete|complete(){}[0] + abstract fun completeExceptionally(kotlin/Throwable): kotlin/Boolean // kotlinx.coroutines/CompletableJob.completeExceptionally|completeExceptionally(kotlin.Throwable){}[0] +} + +abstract interface kotlinx.coroutines/CoroutineExceptionHandler : kotlin.coroutines/CoroutineContext.Element { // kotlinx.coroutines/CoroutineExceptionHandler|null[0] + abstract fun handleException(kotlin.coroutines/CoroutineContext, kotlin/Throwable) // kotlinx.coroutines/CoroutineExceptionHandler.handleException|handleException(kotlin.coroutines.CoroutineContext;kotlin.Throwable){}[0] + + final object Key : kotlin.coroutines/CoroutineContext.Key // kotlinx.coroutines/CoroutineExceptionHandler.Key|null[0] +} + +abstract interface kotlinx.coroutines/CoroutineScope { // kotlinx.coroutines/CoroutineScope|null[0] + abstract val coroutineContext // kotlinx.coroutines/CoroutineScope.coroutineContext|{}coroutineContext[0] + abstract fun (): kotlin.coroutines/CoroutineContext // kotlinx.coroutines/CoroutineScope.coroutineContext.|(){}[0] +} + +abstract interface kotlinx.coroutines/Delay { // kotlinx.coroutines/Delay|null[0] + abstract fun scheduleResumeAfterDelay(kotlin/Long, kotlinx.coroutines/CancellableContinuation) // kotlinx.coroutines/Delay.scheduleResumeAfterDelay|scheduleResumeAfterDelay(kotlin.Long;kotlinx.coroutines.CancellableContinuation){}[0] + open fun invokeOnTimeout(kotlin/Long, kotlinx.coroutines/Runnable, kotlin.coroutines/CoroutineContext): kotlinx.coroutines/DisposableHandle // kotlinx.coroutines/Delay.invokeOnTimeout|invokeOnTimeout(kotlin.Long;kotlinx.coroutines.Runnable;kotlin.coroutines.CoroutineContext){}[0] + open suspend fun delay(kotlin/Long) // kotlinx.coroutines/Delay.delay|delay(kotlin.Long){}[0] +} + +abstract interface kotlinx.coroutines/Job : kotlin.coroutines/CoroutineContext.Element { // kotlinx.coroutines/Job|null[0] + abstract val children // kotlinx.coroutines/Job.children|{}children[0] + abstract fun (): kotlin.sequences/Sequence // kotlinx.coroutines/Job.children.|(){}[0] + abstract val isActive // kotlinx.coroutines/Job.isActive|{}isActive[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines/Job.isActive.|(){}[0] + abstract val isCancelled // kotlinx.coroutines/Job.isCancelled|{}isCancelled[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines/Job.isCancelled.|(){}[0] + abstract val isCompleted // kotlinx.coroutines/Job.isCompleted|{}isCompleted[0] + abstract fun (): kotlin/Boolean // kotlinx.coroutines/Job.isCompleted.|(){}[0] + abstract val onJoin // kotlinx.coroutines/Job.onJoin|{}onJoin[0] + abstract fun (): kotlinx.coroutines.selects/SelectClause0 // kotlinx.coroutines/Job.onJoin.|(){}[0] + abstract val parent // kotlinx.coroutines/Job.parent|{}parent[0] + abstract fun (): kotlinx.coroutines/Job? // kotlinx.coroutines/Job.parent.|(){}[0] + + abstract fun attachChild(kotlinx.coroutines/ChildJob): kotlinx.coroutines/ChildHandle // kotlinx.coroutines/Job.attachChild|attachChild(kotlinx.coroutines.ChildJob){}[0] + abstract fun cancel(kotlin.coroutines.cancellation/CancellationException? = ...) // kotlinx.coroutines/Job.cancel|cancel(kotlin.coroutines.cancellation.CancellationException?){}[0] + abstract fun cancel(kotlin/Throwable? = ...): kotlin/Boolean // kotlinx.coroutines/Job.cancel|cancel(kotlin.Throwable?){}[0] + abstract fun getCancellationException(): kotlin.coroutines.cancellation/CancellationException // kotlinx.coroutines/Job.getCancellationException|getCancellationException(){}[0] + abstract fun invokeOnCompletion(kotlin/Boolean = ..., kotlin/Boolean = ..., kotlin/Function1): kotlinx.coroutines/DisposableHandle // kotlinx.coroutines/Job.invokeOnCompletion|invokeOnCompletion(kotlin.Boolean;kotlin.Boolean;kotlin.Function1){}[0] + abstract fun invokeOnCompletion(kotlin/Function1): kotlinx.coroutines/DisposableHandle // kotlinx.coroutines/Job.invokeOnCompletion|invokeOnCompletion(kotlin.Function1){}[0] + abstract fun start(): kotlin/Boolean // kotlinx.coroutines/Job.start|start(){}[0] + abstract suspend fun join() // kotlinx.coroutines/Job.join|join(){}[0] + open fun cancel() // kotlinx.coroutines/Job.cancel|cancel(){}[0] + open fun plus(kotlinx.coroutines/Job): kotlinx.coroutines/Job // kotlinx.coroutines/Job.plus|plus(kotlinx.coroutines.Job){}[0] + + final object Key : kotlin.coroutines/CoroutineContext.Key // kotlinx.coroutines/Job.Key|null[0] +} + +abstract interface kotlinx.coroutines/ParentJob : kotlinx.coroutines/Job { // kotlinx.coroutines/ParentJob|null[0] + abstract fun getChildJobCancellationCause(): kotlin.coroutines.cancellation/CancellationException // kotlinx.coroutines/ParentJob.getChildJobCancellationCause|getChildJobCancellationCause(){}[0] +} + +sealed interface <#A: in kotlin/Any?, #B: out kotlin/Any?> kotlinx.coroutines.selects/SelectClause2 : kotlinx.coroutines.selects/SelectClause // kotlinx.coroutines.selects/SelectClause2|null[0] + +sealed interface <#A: in kotlin/Any?> kotlinx.coroutines.selects/SelectBuilder { // kotlinx.coroutines.selects/SelectBuilder|null[0] + abstract fun (kotlinx.coroutines.selects/SelectClause0).invoke(kotlin.coroutines/SuspendFunction0<#A>) // kotlinx.coroutines.selects/SelectBuilder.invoke|invoke@kotlinx.coroutines.selects.SelectClause0(kotlin.coroutines.SuspendFunction0<1:0>){}[0] + abstract fun <#A1: kotlin/Any?, #B1: kotlin/Any?> (kotlinx.coroutines.selects/SelectClause2<#A1, #B1>).invoke(#A1, kotlin.coroutines/SuspendFunction1<#B1, #A>) // kotlinx.coroutines.selects/SelectBuilder.invoke|invoke@kotlinx.coroutines.selects.SelectClause2<0:0,0:1>(0:0;kotlin.coroutines.SuspendFunction1<0:1,1:0>){0§;1§}[0] + abstract fun <#A1: kotlin/Any?> (kotlinx.coroutines.selects/SelectClause1<#A1>).invoke(kotlin.coroutines/SuspendFunction1<#A1, #A>) // kotlinx.coroutines.selects/SelectBuilder.invoke|invoke@kotlinx.coroutines.selects.SelectClause1<0:0>(kotlin.coroutines.SuspendFunction1<0:0,1:0>){0§}[0] + open fun <#A1: kotlin/Any?, #B1: kotlin/Any?> (kotlinx.coroutines.selects/SelectClause2<#A1?, #B1>).invoke(kotlin.coroutines/SuspendFunction1<#B1, #A>) // kotlinx.coroutines.selects/SelectBuilder.invoke|invoke@kotlinx.coroutines.selects.SelectClause2<0:0?,0:1>(kotlin.coroutines.SuspendFunction1<0:1,1:0>){0§;1§}[0] + open fun onTimeout(kotlin/Long, kotlin.coroutines/SuspendFunction0<#A>) // kotlinx.coroutines.selects/SelectBuilder.onTimeout|onTimeout(kotlin.Long;kotlin.coroutines.SuspendFunction0<1:0>){}[0] +} + +sealed interface <#A: in kotlin/Any?> kotlinx.coroutines.selects/SelectInstance { // kotlinx.coroutines.selects/SelectInstance|null[0] + abstract val context // kotlinx.coroutines.selects/SelectInstance.context|{}context[0] + abstract fun (): kotlin.coroutines/CoroutineContext // kotlinx.coroutines.selects/SelectInstance.context.|(){}[0] + + abstract fun disposeOnCompletion(kotlinx.coroutines/DisposableHandle) // kotlinx.coroutines.selects/SelectInstance.disposeOnCompletion|disposeOnCompletion(kotlinx.coroutines.DisposableHandle){}[0] + abstract fun selectInRegistrationPhase(kotlin/Any?) // kotlinx.coroutines.selects/SelectInstance.selectInRegistrationPhase|selectInRegistrationPhase(kotlin.Any?){}[0] + abstract fun trySelect(kotlin/Any, kotlin/Any?): kotlin/Boolean // kotlinx.coroutines.selects/SelectInstance.trySelect|trySelect(kotlin.Any;kotlin.Any?){}[0] +} + +sealed interface <#A: out kotlin/Any?> kotlinx.coroutines.selects/SelectClause1 : kotlinx.coroutines.selects/SelectClause // kotlinx.coroutines.selects/SelectClause1|null[0] + +sealed interface kotlinx.coroutines.selects/SelectClause { // kotlinx.coroutines.selects/SelectClause|null[0] + abstract val clauseObject // kotlinx.coroutines.selects/SelectClause.clauseObject|{}clauseObject[0] + abstract fun (): kotlin/Any // kotlinx.coroutines.selects/SelectClause.clauseObject.|(){}[0] + abstract val onCancellationConstructor // kotlinx.coroutines.selects/SelectClause.onCancellationConstructor|{}onCancellationConstructor[0] + abstract fun (): kotlin/Function3, kotlin/Any?, kotlin/Any?, kotlin/Function3>? // kotlinx.coroutines.selects/SelectClause.onCancellationConstructor.|(){}[0] + abstract val processResFunc // kotlinx.coroutines.selects/SelectClause.processResFunc|{}processResFunc[0] + abstract fun (): kotlin/Function3 // kotlinx.coroutines.selects/SelectClause.processResFunc.|(){}[0] + abstract val regFunc // kotlinx.coroutines.selects/SelectClause.regFunc|{}regFunc[0] + abstract fun (): kotlin/Function3, kotlin/Any?, kotlin/Unit> // kotlinx.coroutines.selects/SelectClause.regFunc.|(){}[0] +} + +sealed interface kotlinx.coroutines.selects/SelectClause0 : kotlinx.coroutines.selects/SelectClause // kotlinx.coroutines.selects/SelectClause0|null[0] + +abstract class <#A: in kotlin/Any?> kotlinx.coroutines/AbstractCoroutine : kotlin.coroutines/Continuation<#A>, kotlinx.coroutines/CoroutineScope, kotlinx.coroutines/Job, kotlinx.coroutines/JobSupport { // kotlinx.coroutines/AbstractCoroutine|null[0] + constructor (kotlin.coroutines/CoroutineContext, kotlin/Boolean, kotlin/Boolean) // kotlinx.coroutines/AbstractCoroutine.|(kotlin.coroutines.CoroutineContext;kotlin.Boolean;kotlin.Boolean){}[0] + + final val context // kotlinx.coroutines/AbstractCoroutine.context|{}context[0] + final fun (): kotlin.coroutines/CoroutineContext // kotlinx.coroutines/AbstractCoroutine.context.|(){}[0] + open val coroutineContext // kotlinx.coroutines/AbstractCoroutine.coroutineContext|{}coroutineContext[0] + open fun (): kotlin.coroutines/CoroutineContext // kotlinx.coroutines/AbstractCoroutine.coroutineContext.|(){}[0] + open val isActive // kotlinx.coroutines/AbstractCoroutine.isActive|{}isActive[0] + open fun (): kotlin/Boolean // kotlinx.coroutines/AbstractCoroutine.isActive.|(){}[0] + + final fun <#A1: kotlin/Any?> start(kotlinx.coroutines/CoroutineStart, #A1, kotlin.coroutines/SuspendFunction1<#A1, #A>) // kotlinx.coroutines/AbstractCoroutine.start|start(kotlinx.coroutines.CoroutineStart;0:0;kotlin.coroutines.SuspendFunction1<0:0,1:0>){0§}[0] + final fun onCompletionInternal(kotlin/Any?) // kotlinx.coroutines/AbstractCoroutine.onCompletionInternal|onCompletionInternal(kotlin.Any?){}[0] + final fun resumeWith(kotlin/Result<#A>) // kotlinx.coroutines/AbstractCoroutine.resumeWith|resumeWith(kotlin.Result<1:0>){}[0] + open fun afterResume(kotlin/Any?) // kotlinx.coroutines/AbstractCoroutine.afterResume|afterResume(kotlin.Any?){}[0] + open fun cancellationExceptionMessage(): kotlin/String // kotlinx.coroutines/AbstractCoroutine.cancellationExceptionMessage|cancellationExceptionMessage(){}[0] + open fun onCancelled(kotlin/Throwable, kotlin/Boolean) // kotlinx.coroutines/AbstractCoroutine.onCancelled|onCancelled(kotlin.Throwable;kotlin.Boolean){}[0] + open fun onCompleted(#A) // kotlinx.coroutines/AbstractCoroutine.onCompleted|onCompleted(1:0){}[0] +} + +abstract class <#A: kotlin/Any?> kotlinx.coroutines.flow.internal/ChannelFlow : kotlinx.coroutines.flow.internal/FusibleFlow<#A> { // kotlinx.coroutines.flow.internal/ChannelFlow|null[0] + constructor (kotlin.coroutines/CoroutineContext, kotlin/Int, kotlinx.coroutines.channels/BufferOverflow) // kotlinx.coroutines.flow.internal/ChannelFlow.|(kotlin.coroutines.CoroutineContext;kotlin.Int;kotlinx.coroutines.channels.BufferOverflow){}[0] + + final val capacity // kotlinx.coroutines.flow.internal/ChannelFlow.capacity|{}capacity[0] + final fun (): kotlin/Int // kotlinx.coroutines.flow.internal/ChannelFlow.capacity.|(){}[0] + final val context // kotlinx.coroutines.flow.internal/ChannelFlow.context|{}context[0] + final fun (): kotlin.coroutines/CoroutineContext // kotlinx.coroutines.flow.internal/ChannelFlow.context.|(){}[0] + final val onBufferOverflow // kotlinx.coroutines.flow.internal/ChannelFlow.onBufferOverflow|{}onBufferOverflow[0] + final fun (): kotlinx.coroutines.channels/BufferOverflow // kotlinx.coroutines.flow.internal/ChannelFlow.onBufferOverflow.|(){}[0] + + abstract fun create(kotlin.coroutines/CoroutineContext, kotlin/Int, kotlinx.coroutines.channels/BufferOverflow): kotlinx.coroutines.flow.internal/ChannelFlow<#A> // kotlinx.coroutines.flow.internal/ChannelFlow.create|create(kotlin.coroutines.CoroutineContext;kotlin.Int;kotlinx.coroutines.channels.BufferOverflow){}[0] + abstract suspend fun collectTo(kotlinx.coroutines.channels/ProducerScope<#A>) // kotlinx.coroutines.flow.internal/ChannelFlow.collectTo|collectTo(kotlinx.coroutines.channels.ProducerScope<1:0>){}[0] + open fun additionalToStringProps(): kotlin/String? // kotlinx.coroutines.flow.internal/ChannelFlow.additionalToStringProps|additionalToStringProps(){}[0] + open fun dropChannelOperators(): kotlinx.coroutines.flow/Flow<#A>? // kotlinx.coroutines.flow.internal/ChannelFlow.dropChannelOperators|dropChannelOperators(){}[0] + open fun fuse(kotlin.coroutines/CoroutineContext, kotlin/Int, kotlinx.coroutines.channels/BufferOverflow): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow.internal/ChannelFlow.fuse|fuse(kotlin.coroutines.CoroutineContext;kotlin.Int;kotlinx.coroutines.channels.BufferOverflow){}[0] + open fun produceImpl(kotlinx.coroutines/CoroutineScope): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.flow.internal/ChannelFlow.produceImpl|produceImpl(kotlinx.coroutines.CoroutineScope){}[0] + open fun toString(): kotlin/String // kotlinx.coroutines.flow.internal/ChannelFlow.toString|toString(){}[0] + open suspend fun collect(kotlinx.coroutines.flow/FlowCollector<#A>) // kotlinx.coroutines.flow.internal/ChannelFlow.collect|collect(kotlinx.coroutines.flow.FlowCollector<1:0>){}[0] +} + +abstract class <#A: kotlin/Any?> kotlinx.coroutines.flow/AbstractFlow : kotlinx.coroutines.flow/CancellableFlow<#A>, kotlinx.coroutines.flow/Flow<#A> { // kotlinx.coroutines.flow/AbstractFlow|null[0] + constructor () // kotlinx.coroutines.flow/AbstractFlow.|(){}[0] + + abstract suspend fun collectSafely(kotlinx.coroutines.flow/FlowCollector<#A>) // kotlinx.coroutines.flow/AbstractFlow.collectSafely|collectSafely(kotlinx.coroutines.flow.FlowCollector<1:0>){}[0] + final suspend fun collect(kotlinx.coroutines.flow/FlowCollector<#A>) // kotlinx.coroutines.flow/AbstractFlow.collect|collect(kotlinx.coroutines.flow.FlowCollector<1:0>){}[0] +} + +abstract class kotlinx.coroutines/CloseableCoroutineDispatcher : kotlin/AutoCloseable, kotlinx.coroutines/CoroutineDispatcher { // kotlinx.coroutines/CloseableCoroutineDispatcher|null[0] + constructor () // kotlinx.coroutines/CloseableCoroutineDispatcher.|(){}[0] + + abstract fun close() // kotlinx.coroutines/CloseableCoroutineDispatcher.close|close(){}[0] +} + +abstract class kotlinx.coroutines/CoroutineDispatcher : kotlin.coroutines/AbstractCoroutineContextElement, kotlin.coroutines/ContinuationInterceptor { // kotlinx.coroutines/CoroutineDispatcher|null[0] + constructor () // kotlinx.coroutines/CoroutineDispatcher.|(){}[0] + + abstract fun dispatch(kotlin.coroutines/CoroutineContext, kotlinx.coroutines/Runnable) // kotlinx.coroutines/CoroutineDispatcher.dispatch|dispatch(kotlin.coroutines.CoroutineContext;kotlinx.coroutines.Runnable){}[0] + final fun <#A1: kotlin/Any?> interceptContinuation(kotlin.coroutines/Continuation<#A1>): kotlin.coroutines/Continuation<#A1> // kotlinx.coroutines/CoroutineDispatcher.interceptContinuation|interceptContinuation(kotlin.coroutines.Continuation<0:0>){0§}[0] + final fun plus(kotlinx.coroutines/CoroutineDispatcher): kotlinx.coroutines/CoroutineDispatcher // kotlinx.coroutines/CoroutineDispatcher.plus|plus(kotlinx.coroutines.CoroutineDispatcher){}[0] + final fun releaseInterceptedContinuation(kotlin.coroutines/Continuation<*>) // kotlinx.coroutines/CoroutineDispatcher.releaseInterceptedContinuation|releaseInterceptedContinuation(kotlin.coroutines.Continuation<*>){}[0] + open fun dispatchYield(kotlin.coroutines/CoroutineContext, kotlinx.coroutines/Runnable) // kotlinx.coroutines/CoroutineDispatcher.dispatchYield|dispatchYield(kotlin.coroutines.CoroutineContext;kotlinx.coroutines.Runnable){}[0] + open fun isDispatchNeeded(kotlin.coroutines/CoroutineContext): kotlin/Boolean // kotlinx.coroutines/CoroutineDispatcher.isDispatchNeeded|isDispatchNeeded(kotlin.coroutines.CoroutineContext){}[0] + open fun limitedParallelism(kotlin/Int): kotlinx.coroutines/CoroutineDispatcher // kotlinx.coroutines/CoroutineDispatcher.limitedParallelism|limitedParallelism(kotlin.Int){}[0] + open fun limitedParallelism(kotlin/Int, kotlin/String? = ...): kotlinx.coroutines/CoroutineDispatcher // kotlinx.coroutines/CoroutineDispatcher.limitedParallelism|limitedParallelism(kotlin.Int;kotlin.String?){}[0] + open fun toString(): kotlin/String // kotlinx.coroutines/CoroutineDispatcher.toString|toString(){}[0] + + final object Key : kotlin.coroutines/AbstractCoroutineContextKey // kotlinx.coroutines/CoroutineDispatcher.Key|null[0] +} + +abstract class kotlinx.coroutines/MainCoroutineDispatcher : kotlinx.coroutines/CoroutineDispatcher { // kotlinx.coroutines/MainCoroutineDispatcher|null[0] + constructor () // kotlinx.coroutines/MainCoroutineDispatcher.|(){}[0] + + abstract val immediate // kotlinx.coroutines/MainCoroutineDispatcher.immediate|{}immediate[0] + abstract fun (): kotlinx.coroutines/MainCoroutineDispatcher // kotlinx.coroutines/MainCoroutineDispatcher.immediate.|(){}[0] + + final fun toStringInternalImpl(): kotlin/String? // kotlinx.coroutines/MainCoroutineDispatcher.toStringInternalImpl|toStringInternalImpl(){}[0] + open fun limitedParallelism(kotlin/Int, kotlin/String?): kotlinx.coroutines/CoroutineDispatcher // kotlinx.coroutines/MainCoroutineDispatcher.limitedParallelism|limitedParallelism(kotlin.Int;kotlin.String?){}[0] + open fun toString(): kotlin/String // kotlinx.coroutines/MainCoroutineDispatcher.toString|toString(){}[0] +} + +final class <#A: kotlin/Any?> kotlinx.coroutines.channels/ConflatedBroadcastChannel : kotlinx.coroutines.channels/BroadcastChannel<#A> { // kotlinx.coroutines.channels/ConflatedBroadcastChannel|null[0] + constructor (#A) // kotlinx.coroutines.channels/ConflatedBroadcastChannel.|(1:0){}[0] + constructor () // kotlinx.coroutines.channels/ConflatedBroadcastChannel.|(){}[0] + + final val isClosedForSend // kotlinx.coroutines.channels/ConflatedBroadcastChannel.isClosedForSend|{}isClosedForSend[0] + final fun (): kotlin/Boolean // kotlinx.coroutines.channels/ConflatedBroadcastChannel.isClosedForSend.|(){}[0] + final val onSend // kotlinx.coroutines.channels/ConflatedBroadcastChannel.onSend|{}onSend[0] + final fun (): kotlinx.coroutines.selects/SelectClause2<#A, kotlinx.coroutines.channels/SendChannel<#A>> // kotlinx.coroutines.channels/ConflatedBroadcastChannel.onSend.|(){}[0] + final val value // kotlinx.coroutines.channels/ConflatedBroadcastChannel.value|{}value[0] + final fun (): #A // kotlinx.coroutines.channels/ConflatedBroadcastChannel.value.|(){}[0] + final val valueOrNull // kotlinx.coroutines.channels/ConflatedBroadcastChannel.valueOrNull|{}valueOrNull[0] + final fun (): #A? // kotlinx.coroutines.channels/ConflatedBroadcastChannel.valueOrNull.|(){}[0] + + final fun cancel(kotlin.coroutines.cancellation/CancellationException?) // kotlinx.coroutines.channels/ConflatedBroadcastChannel.cancel|cancel(kotlin.coroutines.cancellation.CancellationException?){}[0] + final fun cancel(kotlin/Throwable?): kotlin/Boolean // kotlinx.coroutines.channels/ConflatedBroadcastChannel.cancel|cancel(kotlin.Throwable?){}[0] + final fun close(kotlin/Throwable?): kotlin/Boolean // kotlinx.coroutines.channels/ConflatedBroadcastChannel.close|close(kotlin.Throwable?){}[0] + final fun invokeOnClose(kotlin/Function1) // kotlinx.coroutines.channels/ConflatedBroadcastChannel.invokeOnClose|invokeOnClose(kotlin.Function1){}[0] + final fun offer(#A): kotlin/Boolean // kotlinx.coroutines.channels/ConflatedBroadcastChannel.offer|offer(1:0){}[0] + final fun openSubscription(): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/ConflatedBroadcastChannel.openSubscription|openSubscription(){}[0] + final fun trySend(#A): kotlinx.coroutines.channels/ChannelResult // kotlinx.coroutines.channels/ConflatedBroadcastChannel.trySend|trySend(1:0){}[0] + final suspend fun send(#A) // kotlinx.coroutines.channels/ConflatedBroadcastChannel.send|send(1:0){}[0] +} + +final class <#A: kotlin/Any?> kotlinx.coroutines.flow.internal/SendingCollector : kotlinx.coroutines.flow/FlowCollector<#A> { // kotlinx.coroutines.flow.internal/SendingCollector|null[0] + constructor (kotlinx.coroutines.channels/SendChannel<#A>) // kotlinx.coroutines.flow.internal/SendingCollector.|(kotlinx.coroutines.channels.SendChannel<1:0>){}[0] + + final suspend fun emit(#A) // kotlinx.coroutines.flow.internal/SendingCollector.emit|emit(1:0){}[0] +} + +final class <#A: kotlin/Any?> kotlinx.coroutines.selects/SelectBuilderImpl : kotlinx.coroutines.selects/SelectImplementation<#A> { // kotlinx.coroutines.selects/SelectBuilderImpl|null[0] + constructor (kotlin.coroutines/Continuation<#A>) // kotlinx.coroutines.selects/SelectBuilderImpl.|(kotlin.coroutines.Continuation<1:0>){}[0] + + final fun getResult(): kotlin/Any? // kotlinx.coroutines.selects/SelectBuilderImpl.getResult|getResult(){}[0] + final fun handleBuilderException(kotlin/Throwable) // kotlinx.coroutines.selects/SelectBuilderImpl.handleBuilderException|handleBuilderException(kotlin.Throwable){}[0] +} + +final class <#A: kotlin/Any?> kotlinx.coroutines.selects/UnbiasedSelectBuilderImpl : kotlinx.coroutines.selects/UnbiasedSelectImplementation<#A> { // kotlinx.coroutines.selects/UnbiasedSelectBuilderImpl|null[0] + constructor (kotlin.coroutines/Continuation<#A>) // kotlinx.coroutines.selects/UnbiasedSelectBuilderImpl.|(kotlin.coroutines.Continuation<1:0>){}[0] + + final fun handleBuilderException(kotlin/Throwable) // kotlinx.coroutines.selects/UnbiasedSelectBuilderImpl.handleBuilderException|handleBuilderException(kotlin.Throwable){}[0] + final fun initSelectResult(): kotlin/Any? // kotlinx.coroutines.selects/UnbiasedSelectBuilderImpl.initSelectResult|initSelectResult(){}[0] +} + +final class kotlinx.coroutines.channels/ClosedReceiveChannelException : kotlin/NoSuchElementException { // kotlinx.coroutines.channels/ClosedReceiveChannelException|null[0] + constructor (kotlin/String?) // kotlinx.coroutines.channels/ClosedReceiveChannelException.|(kotlin.String?){}[0] +} + +final class kotlinx.coroutines.channels/ClosedSendChannelException : kotlin/IllegalStateException { // kotlinx.coroutines.channels/ClosedSendChannelException|null[0] + constructor (kotlin/String?) // kotlinx.coroutines.channels/ClosedSendChannelException.|(kotlin.String?){}[0] +} + +final class kotlinx.coroutines/CompletionHandlerException : kotlin/RuntimeException { // kotlinx.coroutines/CompletionHandlerException|null[0] + constructor (kotlin/String, kotlin/Throwable) // kotlinx.coroutines/CompletionHandlerException.|(kotlin.String;kotlin.Throwable){}[0] +} + +final class kotlinx.coroutines/CoroutineName : kotlin.coroutines/AbstractCoroutineContextElement { // kotlinx.coroutines/CoroutineName|null[0] + constructor (kotlin/String) // kotlinx.coroutines/CoroutineName.|(kotlin.String){}[0] + + final val name // kotlinx.coroutines/CoroutineName.name|{}name[0] + final fun (): kotlin/String // kotlinx.coroutines/CoroutineName.name.|(){}[0] + + final fun component1(): kotlin/String // kotlinx.coroutines/CoroutineName.component1|component1(){}[0] + final fun copy(kotlin/String = ...): kotlinx.coroutines/CoroutineName // kotlinx.coroutines/CoroutineName.copy|copy(kotlin.String){}[0] + final fun equals(kotlin/Any?): kotlin/Boolean // kotlinx.coroutines/CoroutineName.equals|equals(kotlin.Any?){}[0] + final fun hashCode(): kotlin/Int // kotlinx.coroutines/CoroutineName.hashCode|hashCode(){}[0] + final fun toString(): kotlin/String // kotlinx.coroutines/CoroutineName.toString|toString(){}[0] + + final object Key : kotlin.coroutines/CoroutineContext.Key // kotlinx.coroutines/CoroutineName.Key|null[0] +} + +final class kotlinx.coroutines/TimeoutCancellationException : kotlin.coroutines.cancellation/CancellationException, kotlinx.coroutines/CopyableThrowable { // kotlinx.coroutines/TimeoutCancellationException|null[0] + final fun createCopy(): kotlinx.coroutines/TimeoutCancellationException // kotlinx.coroutines/TimeoutCancellationException.createCopy|createCopy(){}[0] +} + +final class kotlinx.coroutines/YieldContext : kotlin.coroutines/AbstractCoroutineContextElement { // kotlinx.coroutines/YieldContext|null[0] + constructor () // kotlinx.coroutines/YieldContext.|(){}[0] + + final var dispatcherWasUnconfined // kotlinx.coroutines/YieldContext.dispatcherWasUnconfined|{}dispatcherWasUnconfined[0] + final fun (): kotlin/Boolean // kotlinx.coroutines/YieldContext.dispatcherWasUnconfined.|(){}[0] + final fun (kotlin/Boolean) // kotlinx.coroutines/YieldContext.dispatcherWasUnconfined.|(kotlin.Boolean){}[0] + + final object Key : kotlin.coroutines/CoroutineContext.Key // kotlinx.coroutines/YieldContext.Key|null[0] +} + +final value class <#A: out kotlin/Any?> kotlinx.coroutines.channels/ChannelResult { // kotlinx.coroutines.channels/ChannelResult|null[0] + constructor (kotlin/Any?) // kotlinx.coroutines.channels/ChannelResult.|(kotlin.Any?){}[0] + + final val holder // kotlinx.coroutines.channels/ChannelResult.holder|{}holder[0] + final fun (): kotlin/Any? // kotlinx.coroutines.channels/ChannelResult.holder.|(){}[0] + final val isClosed // kotlinx.coroutines.channels/ChannelResult.isClosed|{}isClosed[0] + final fun (): kotlin/Boolean // kotlinx.coroutines.channels/ChannelResult.isClosed.|(){}[0] + final val isFailure // kotlinx.coroutines.channels/ChannelResult.isFailure|{}isFailure[0] + final fun (): kotlin/Boolean // kotlinx.coroutines.channels/ChannelResult.isFailure.|(){}[0] + final val isSuccess // kotlinx.coroutines.channels/ChannelResult.isSuccess|{}isSuccess[0] + final fun (): kotlin/Boolean // kotlinx.coroutines.channels/ChannelResult.isSuccess.|(){}[0] + + final fun equals(kotlin/Any?): kotlin/Boolean // kotlinx.coroutines.channels/ChannelResult.equals|equals(kotlin.Any?){}[0] + final fun exceptionOrNull(): kotlin/Throwable? // kotlinx.coroutines.channels/ChannelResult.exceptionOrNull|exceptionOrNull(){}[0] + final fun getOrNull(): #A? // kotlinx.coroutines.channels/ChannelResult.getOrNull|getOrNull(){}[0] + final fun getOrThrow(): #A // kotlinx.coroutines.channels/ChannelResult.getOrThrow|getOrThrow(){}[0] + final fun hashCode(): kotlin/Int // kotlinx.coroutines.channels/ChannelResult.hashCode|hashCode(){}[0] + final fun toString(): kotlin/String // kotlinx.coroutines.channels/ChannelResult.toString|toString(){}[0] + + final object Companion { // kotlinx.coroutines.channels/ChannelResult.Companion|null[0] + final fun <#A2: kotlin/Any?> closed(kotlin/Throwable?): kotlinx.coroutines.channels/ChannelResult<#A2> // kotlinx.coroutines.channels/ChannelResult.Companion.closed|closed(kotlin.Throwable?){0§}[0] + final fun <#A2: kotlin/Any?> failure(): kotlinx.coroutines.channels/ChannelResult<#A2> // kotlinx.coroutines.channels/ChannelResult.Companion.failure|failure(){0§}[0] + final fun <#A2: kotlin/Any?> success(#A2): kotlinx.coroutines.channels/ChannelResult<#A2> // kotlinx.coroutines.channels/ChannelResult.Companion.success|success(0:0){0§}[0] + } +} + +open class <#A: in kotlin/Any?> kotlinx.coroutines/CancellableContinuationImpl : kotlinx.coroutines.internal/CoroutineStackFrame, kotlinx.coroutines/CancellableContinuation<#A>, kotlinx.coroutines/DispatchedTask<#A>, kotlinx.coroutines/Waiter { // kotlinx.coroutines/CancellableContinuationImpl|null[0] + constructor (kotlin.coroutines/Continuation<#A>, kotlin/Int) // kotlinx.coroutines/CancellableContinuationImpl.|(kotlin.coroutines.Continuation<1:0>;kotlin.Int){}[0] + + open val callerFrame // kotlinx.coroutines/CancellableContinuationImpl.callerFrame|{}callerFrame[0] + open fun (): kotlinx.coroutines.internal/CoroutineStackFrame? // kotlinx.coroutines/CancellableContinuationImpl.callerFrame.|(){}[0] + open val context // kotlinx.coroutines/CancellableContinuationImpl.context|{}context[0] + open fun (): kotlin.coroutines/CoroutineContext // kotlinx.coroutines/CancellableContinuationImpl.context.|(){}[0] + open val isActive // kotlinx.coroutines/CancellableContinuationImpl.isActive|{}isActive[0] + open fun (): kotlin/Boolean // kotlinx.coroutines/CancellableContinuationImpl.isActive.|(){}[0] + open val isCancelled // kotlinx.coroutines/CancellableContinuationImpl.isCancelled|{}isCancelled[0] + open fun (): kotlin/Boolean // kotlinx.coroutines/CancellableContinuationImpl.isCancelled.|(){}[0] + open val isCompleted // kotlinx.coroutines/CancellableContinuationImpl.isCompleted|{}isCompleted[0] + open fun (): kotlin/Boolean // kotlinx.coroutines/CancellableContinuationImpl.isCompleted.|(){}[0] + + final fun <#A1: kotlin/Any?> callOnCancellation(kotlin/Function3, kotlin/Throwable, #A1) // kotlinx.coroutines/CancellableContinuationImpl.callOnCancellation|callOnCancellation(kotlin.Function3;kotlin.Throwable;0:0){0§}[0] + final fun callCancelHandler(kotlinx.coroutines/CancelHandler, kotlin/Throwable?) // kotlinx.coroutines/CancellableContinuationImpl.callCancelHandler|callCancelHandler(kotlinx.coroutines.CancelHandler;kotlin.Throwable?){}[0] + final fun getResult(): kotlin/Any? // kotlinx.coroutines/CancellableContinuationImpl.getResult|getResult(){}[0] + open fun (kotlinx.coroutines/CoroutineDispatcher).resumeUndispatched(#A) // kotlinx.coroutines/CancellableContinuationImpl.resumeUndispatched|resumeUndispatched@kotlinx.coroutines.CoroutineDispatcher(1:0){}[0] + open fun (kotlinx.coroutines/CoroutineDispatcher).resumeUndispatchedWithException(kotlin/Throwable) // kotlinx.coroutines/CancellableContinuationImpl.resumeUndispatchedWithException|resumeUndispatchedWithException@kotlinx.coroutines.CoroutineDispatcher(kotlin.Throwable){}[0] + open fun <#A1: #A> resume(#A1, kotlin/Function3?) // kotlinx.coroutines/CancellableContinuationImpl.resume|resume(0:0;kotlin.Function3?){0§<1:0>}[0] + open fun <#A1: #A> tryResume(#A1, kotlin/Any?, kotlin/Function3?): kotlin/Any? // kotlinx.coroutines/CancellableContinuationImpl.tryResume|tryResume(0:0;kotlin.Any?;kotlin.Function3?){0§<1:0>}[0] + open fun cancel(kotlin/Throwable?): kotlin/Boolean // kotlinx.coroutines/CancellableContinuationImpl.cancel|cancel(kotlin.Throwable?){}[0] + open fun completeResume(kotlin/Any) // kotlinx.coroutines/CancellableContinuationImpl.completeResume|completeResume(kotlin.Any){}[0] + open fun getContinuationCancellationCause(kotlinx.coroutines/Job): kotlin/Throwable // kotlinx.coroutines/CancellableContinuationImpl.getContinuationCancellationCause|getContinuationCancellationCause(kotlinx.coroutines.Job){}[0] + open fun getStackTraceElement(): kotlin/Any? // kotlinx.coroutines/CancellableContinuationImpl.getStackTraceElement|getStackTraceElement(){}[0] + open fun initCancellability() // kotlinx.coroutines/CancellableContinuationImpl.initCancellability|initCancellability(){}[0] + open fun invokeOnCancellation(kotlin/Function1) // kotlinx.coroutines/CancellableContinuationImpl.invokeOnCancellation|invokeOnCancellation(kotlin.Function1){}[0] + open fun invokeOnCancellation(kotlinx.coroutines.internal/Segment<*>, kotlin/Int) // kotlinx.coroutines/CancellableContinuationImpl.invokeOnCancellation|invokeOnCancellation(kotlinx.coroutines.internal.Segment<*>;kotlin.Int){}[0] + open fun nameString(): kotlin/String // kotlinx.coroutines/CancellableContinuationImpl.nameString|nameString(){}[0] + open fun resume(#A, kotlin/Function1?) // kotlinx.coroutines/CancellableContinuationImpl.resume|resume(1:0;kotlin.Function1?){}[0] + open fun resumeWith(kotlin/Result<#A>) // kotlinx.coroutines/CancellableContinuationImpl.resumeWith|resumeWith(kotlin.Result<1:0>){}[0] + open fun toString(): kotlin/String // kotlinx.coroutines/CancellableContinuationImpl.toString|toString(){}[0] + open fun tryResume(#A, kotlin/Any?): kotlin/Any? // kotlinx.coroutines/CancellableContinuationImpl.tryResume|tryResume(1:0;kotlin.Any?){}[0] + open fun tryResumeWithException(kotlin/Throwable): kotlin/Any? // kotlinx.coroutines/CancellableContinuationImpl.tryResumeWithException|tryResumeWithException(kotlin.Throwable){}[0] +} + +open class <#A: kotlin/Any?> kotlinx.coroutines.selects/SelectImplementation : kotlinx.coroutines.selects/SelectBuilder<#A>, kotlinx.coroutines.selects/SelectInstanceInternal<#A>, kotlinx.coroutines/CancelHandler { // kotlinx.coroutines.selects/SelectImplementation|null[0] + constructor (kotlin.coroutines/CoroutineContext) // kotlinx.coroutines.selects/SelectImplementation.|(kotlin.coroutines.CoroutineContext){}[0] + + open val context // kotlinx.coroutines.selects/SelectImplementation.context|{}context[0] + open fun (): kotlin.coroutines/CoroutineContext // kotlinx.coroutines.selects/SelectImplementation.context.|(){}[0] + + final fun trySelectDetailed(kotlin/Any, kotlin/Any?): kotlinx.coroutines.selects/TrySelectDetailedResult // kotlinx.coroutines.selects/SelectImplementation.trySelectDetailed|trySelectDetailed(kotlin.Any;kotlin.Any?){}[0] + open fun (kotlinx.coroutines.selects/SelectClause0).invoke(kotlin.coroutines/SuspendFunction0<#A>) // kotlinx.coroutines.selects/SelectImplementation.invoke|invoke@kotlinx.coroutines.selects.SelectClause0(kotlin.coroutines.SuspendFunction0<1:0>){}[0] + open fun <#A1: kotlin/Any?, #B1: kotlin/Any?> (kotlinx.coroutines.selects/SelectClause2<#A1, #B1>).invoke(#A1, kotlin.coroutines/SuspendFunction1<#B1, #A>) // kotlinx.coroutines.selects/SelectImplementation.invoke|invoke@kotlinx.coroutines.selects.SelectClause2<0:0,0:1>(0:0;kotlin.coroutines.SuspendFunction1<0:1,1:0>){0§;1§}[0] + open fun <#A1: kotlin/Any?> (kotlinx.coroutines.selects/SelectClause1<#A1>).invoke(kotlin.coroutines/SuspendFunction1<#A1, #A>) // kotlinx.coroutines.selects/SelectImplementation.invoke|invoke@kotlinx.coroutines.selects.SelectClause1<0:0>(kotlin.coroutines.SuspendFunction1<0:0,1:0>){0§}[0] + open fun disposeOnCompletion(kotlinx.coroutines/DisposableHandle) // kotlinx.coroutines.selects/SelectImplementation.disposeOnCompletion|disposeOnCompletion(kotlinx.coroutines.DisposableHandle){}[0] + open fun invoke(kotlin/Throwable?) // kotlinx.coroutines.selects/SelectImplementation.invoke|invoke(kotlin.Throwable?){}[0] + open fun invokeOnCancellation(kotlinx.coroutines.internal/Segment<*>, kotlin/Int) // kotlinx.coroutines.selects/SelectImplementation.invokeOnCancellation|invokeOnCancellation(kotlinx.coroutines.internal.Segment<*>;kotlin.Int){}[0] + open fun selectInRegistrationPhase(kotlin/Any?) // kotlinx.coroutines.selects/SelectImplementation.selectInRegistrationPhase|selectInRegistrationPhase(kotlin.Any?){}[0] + open fun trySelect(kotlin/Any, kotlin/Any?): kotlin/Boolean // kotlinx.coroutines.selects/SelectImplementation.trySelect|trySelect(kotlin.Any;kotlin.Any?){}[0] + open suspend fun doSelect(): #A // kotlinx.coroutines.selects/SelectImplementation.doSelect|doSelect(){}[0] +} + +open class <#A: kotlin/Any?> kotlinx.coroutines.selects/UnbiasedSelectImplementation : kotlinx.coroutines.selects/SelectImplementation<#A> { // kotlinx.coroutines.selects/UnbiasedSelectImplementation|null[0] + constructor (kotlin.coroutines/CoroutineContext) // kotlinx.coroutines.selects/UnbiasedSelectImplementation.|(kotlin.coroutines.CoroutineContext){}[0] + + open fun (kotlinx.coroutines.selects/SelectClause0).invoke(kotlin.coroutines/SuspendFunction0<#A>) // kotlinx.coroutines.selects/UnbiasedSelectImplementation.invoke|invoke@kotlinx.coroutines.selects.SelectClause0(kotlin.coroutines.SuspendFunction0<1:0>){}[0] + open fun <#A1: kotlin/Any?, #B1: kotlin/Any?> (kotlinx.coroutines.selects/SelectClause2<#A1, #B1>).invoke(#A1, kotlin.coroutines/SuspendFunction1<#B1, #A>) // kotlinx.coroutines.selects/UnbiasedSelectImplementation.invoke|invoke@kotlinx.coroutines.selects.SelectClause2<0:0,0:1>(0:0;kotlin.coroutines.SuspendFunction1<0:1,1:0>){0§;1§}[0] + open fun <#A1: kotlin/Any?> (kotlinx.coroutines.selects/SelectClause1<#A1>).invoke(kotlin.coroutines/SuspendFunction1<#A1, #A>) // kotlinx.coroutines.selects/UnbiasedSelectImplementation.invoke|invoke@kotlinx.coroutines.selects.SelectClause1<0:0>(kotlin.coroutines.SuspendFunction1<0:0,1:0>){0§}[0] + open suspend fun doSelect(): #A // kotlinx.coroutines.selects/UnbiasedSelectImplementation.doSelect|doSelect(){}[0] +} + +open class kotlinx.coroutines/JobImpl : kotlinx.coroutines/CompletableJob, kotlinx.coroutines/JobSupport { // kotlinx.coroutines/JobImpl|null[0] + constructor (kotlinx.coroutines/Job?) // kotlinx.coroutines/JobImpl.|(kotlinx.coroutines.Job?){}[0] + + open fun complete(): kotlin/Boolean // kotlinx.coroutines/JobImpl.complete|complete(){}[0] + open fun completeExceptionally(kotlin/Throwable): kotlin/Boolean // kotlinx.coroutines/JobImpl.completeExceptionally|completeExceptionally(kotlin.Throwable){}[0] +} + +open class kotlinx.coroutines/JobSupport : kotlinx.coroutines/ChildJob, kotlinx.coroutines/Job, kotlinx.coroutines/ParentJob { // kotlinx.coroutines/JobSupport|null[0] + constructor (kotlin/Boolean) // kotlinx.coroutines/JobSupport.|(kotlin.Boolean){}[0] + + final val children // kotlinx.coroutines/JobSupport.children|{}children[0] + final fun (): kotlin.sequences/Sequence // kotlinx.coroutines/JobSupport.children.|(){}[0] + final val completionCause // kotlinx.coroutines/JobSupport.completionCause|{}completionCause[0] + final fun (): kotlin/Throwable? // kotlinx.coroutines/JobSupport.completionCause.|(){}[0] + final val completionCauseHandled // kotlinx.coroutines/JobSupport.completionCauseHandled|{}completionCauseHandled[0] + final fun (): kotlin/Boolean // kotlinx.coroutines/JobSupport.completionCauseHandled.|(){}[0] + final val isCancelled // kotlinx.coroutines/JobSupport.isCancelled|{}isCancelled[0] + final fun (): kotlin/Boolean // kotlinx.coroutines/JobSupport.isCancelled.|(){}[0] + final val isCompleted // kotlinx.coroutines/JobSupport.isCompleted|{}isCompleted[0] + final fun (): kotlin/Boolean // kotlinx.coroutines/JobSupport.isCompleted.|(){}[0] + final val isCompletedExceptionally // kotlinx.coroutines/JobSupport.isCompletedExceptionally|{}isCompletedExceptionally[0] + final fun (): kotlin/Boolean // kotlinx.coroutines/JobSupport.isCompletedExceptionally.|(){}[0] + final val key // kotlinx.coroutines/JobSupport.key|{}key[0] + final fun (): kotlin.coroutines/CoroutineContext.Key<*> // kotlinx.coroutines/JobSupport.key.|(){}[0] + final val onAwaitInternal // kotlinx.coroutines/JobSupport.onAwaitInternal|{}onAwaitInternal[0] + final fun (): kotlinx.coroutines.selects/SelectClause1<*> // kotlinx.coroutines/JobSupport.onAwaitInternal.|(){}[0] + final val onJoin // kotlinx.coroutines/JobSupport.onJoin|{}onJoin[0] + final fun (): kotlinx.coroutines.selects/SelectClause0 // kotlinx.coroutines/JobSupport.onJoin.|(){}[0] + open val isActive // kotlinx.coroutines/JobSupport.isActive|{}isActive[0] + open fun (): kotlin/Boolean // kotlinx.coroutines/JobSupport.isActive.|(){}[0] + open val isScopedCoroutine // kotlinx.coroutines/JobSupport.isScopedCoroutine|{}isScopedCoroutine[0] + open fun (): kotlin/Boolean // kotlinx.coroutines/JobSupport.isScopedCoroutine.|(){}[0] + open val parent // kotlinx.coroutines/JobSupport.parent|{}parent[0] + open fun (): kotlinx.coroutines/Job? // kotlinx.coroutines/JobSupport.parent.|(){}[0] + + final fun (kotlin/Throwable).toCancellationException(kotlin/String? = ...): kotlin.coroutines.cancellation/CancellationException // kotlinx.coroutines/JobSupport.toCancellationException|toCancellationException@kotlin.Throwable(kotlin.String?){}[0] + final fun attachChild(kotlinx.coroutines/ChildJob): kotlinx.coroutines/ChildHandle // kotlinx.coroutines/JobSupport.attachChild|attachChild(kotlinx.coroutines.ChildJob){}[0] + final fun cancelCoroutine(kotlin/Throwable?): kotlin/Boolean // kotlinx.coroutines/JobSupport.cancelCoroutine|cancelCoroutine(kotlin.Throwable?){}[0] + final fun getCancellationException(): kotlin.coroutines.cancellation/CancellationException // kotlinx.coroutines/JobSupport.getCancellationException|getCancellationException(){}[0] + final fun getCompletionExceptionOrNull(): kotlin/Throwable? // kotlinx.coroutines/JobSupport.getCompletionExceptionOrNull|getCompletionExceptionOrNull(){}[0] + final fun initParentJob(kotlinx.coroutines/Job?) // kotlinx.coroutines/JobSupport.initParentJob|initParentJob(kotlinx.coroutines.Job?){}[0] + final fun invokeOnCompletion(kotlin/Boolean, kotlin/Boolean, kotlin/Function1): kotlinx.coroutines/DisposableHandle // kotlinx.coroutines/JobSupport.invokeOnCompletion|invokeOnCompletion(kotlin.Boolean;kotlin.Boolean;kotlin.Function1){}[0] + final fun invokeOnCompletion(kotlin/Function1): kotlinx.coroutines/DisposableHandle // kotlinx.coroutines/JobSupport.invokeOnCompletion|invokeOnCompletion(kotlin.Function1){}[0] + final fun parentCancelled(kotlinx.coroutines/ParentJob) // kotlinx.coroutines/JobSupport.parentCancelled|parentCancelled(kotlinx.coroutines.ParentJob){}[0] + final fun start(): kotlin/Boolean // kotlinx.coroutines/JobSupport.start|start(){}[0] + final fun toDebugString(): kotlin/String // kotlinx.coroutines/JobSupport.toDebugString|toDebugString(){}[0] + final suspend fun awaitInternal(): kotlin/Any? // kotlinx.coroutines/JobSupport.awaitInternal|awaitInternal(){}[0] + final suspend fun join() // kotlinx.coroutines/JobSupport.join|join(){}[0] + open fun afterCompletion(kotlin/Any?) // kotlinx.coroutines/JobSupport.afterCompletion|afterCompletion(kotlin.Any?){}[0] + open fun cancel(kotlin.coroutines.cancellation/CancellationException?) // kotlinx.coroutines/JobSupport.cancel|cancel(kotlin.coroutines.cancellation.CancellationException?){}[0] + open fun cancel(kotlin/Throwable?): kotlin/Boolean // kotlinx.coroutines/JobSupport.cancel|cancel(kotlin.Throwable?){}[0] + open fun cancelInternal(kotlin/Throwable) // kotlinx.coroutines/JobSupport.cancelInternal|cancelInternal(kotlin.Throwable){}[0] + open fun cancellationExceptionMessage(): kotlin/String // kotlinx.coroutines/JobSupport.cancellationExceptionMessage|cancellationExceptionMessage(){}[0] + open fun childCancelled(kotlin/Throwable): kotlin/Boolean // kotlinx.coroutines/JobSupport.childCancelled|childCancelled(kotlin.Throwable){}[0] + open fun getChildJobCancellationCause(): kotlin.coroutines.cancellation/CancellationException // kotlinx.coroutines/JobSupport.getChildJobCancellationCause|getChildJobCancellationCause(){}[0] + open fun handleJobException(kotlin/Throwable): kotlin/Boolean // kotlinx.coroutines/JobSupport.handleJobException|handleJobException(kotlin.Throwable){}[0] + open fun onCancelling(kotlin/Throwable?) // kotlinx.coroutines/JobSupport.onCancelling|onCancelling(kotlin.Throwable?){}[0] + open fun onCompletionInternal(kotlin/Any?) // kotlinx.coroutines/JobSupport.onCompletionInternal|onCompletionInternal(kotlin.Any?){}[0] + open fun onStart() // kotlinx.coroutines/JobSupport.onStart|onStart(){}[0] + open fun toString(): kotlin/String // kotlinx.coroutines/JobSupport.toString|toString(){}[0] +} + +final object kotlinx.coroutines/Dispatchers { // kotlinx.coroutines/Dispatchers|null[0] + final val Default // kotlinx.coroutines/Dispatchers.Default|{}Default[0] + final fun (): kotlinx.coroutines/CoroutineDispatcher // kotlinx.coroutines/Dispatchers.Default.|(){}[0] + final val Main // kotlinx.coroutines/Dispatchers.Main|{}Main[0] + final fun (): kotlinx.coroutines/MainCoroutineDispatcher // kotlinx.coroutines/Dispatchers.Main.|(){}[0] + final val Unconfined // kotlinx.coroutines/Dispatchers.Unconfined|{}Unconfined[0] + final fun (): kotlinx.coroutines/CoroutineDispatcher // kotlinx.coroutines/Dispatchers.Unconfined.|(){}[0] + + final fun injectMain(kotlinx.coroutines/MainCoroutineDispatcher) // kotlinx.coroutines/Dispatchers.injectMain|injectMain(kotlinx.coroutines.MainCoroutineDispatcher){}[0] +} + +final object kotlinx.coroutines/GlobalScope : kotlinx.coroutines/CoroutineScope { // kotlinx.coroutines/GlobalScope|null[0] + final val coroutineContext // kotlinx.coroutines/GlobalScope.coroutineContext|{}coroutineContext[0] + final fun (): kotlin.coroutines/CoroutineContext // kotlinx.coroutines/GlobalScope.coroutineContext.|(){}[0] +} + +final object kotlinx.coroutines/NonCancellable : kotlin.coroutines/AbstractCoroutineContextElement, kotlinx.coroutines/Job { // kotlinx.coroutines/NonCancellable|null[0] + final val children // kotlinx.coroutines/NonCancellable.children|{}children[0] + final fun (): kotlin.sequences/Sequence // kotlinx.coroutines/NonCancellable.children.|(){}[0] + final val isActive // kotlinx.coroutines/NonCancellable.isActive|{}isActive[0] + final fun (): kotlin/Boolean // kotlinx.coroutines/NonCancellable.isActive.|(){}[0] + final val isCancelled // kotlinx.coroutines/NonCancellable.isCancelled|{}isCancelled[0] + final fun (): kotlin/Boolean // kotlinx.coroutines/NonCancellable.isCancelled.|(){}[0] + final val isCompleted // kotlinx.coroutines/NonCancellable.isCompleted|{}isCompleted[0] + final fun (): kotlin/Boolean // kotlinx.coroutines/NonCancellable.isCompleted.|(){}[0] + final val onJoin // kotlinx.coroutines/NonCancellable.onJoin|{}onJoin[0] + final fun (): kotlinx.coroutines.selects/SelectClause0 // kotlinx.coroutines/NonCancellable.onJoin.|(){}[0] + final val parent // kotlinx.coroutines/NonCancellable.parent|{}parent[0] + final fun (): kotlinx.coroutines/Job? // kotlinx.coroutines/NonCancellable.parent.|(){}[0] + + final fun attachChild(kotlinx.coroutines/ChildJob): kotlinx.coroutines/ChildHandle // kotlinx.coroutines/NonCancellable.attachChild|attachChild(kotlinx.coroutines.ChildJob){}[0] + final fun cancel(kotlin.coroutines.cancellation/CancellationException?) // kotlinx.coroutines/NonCancellable.cancel|cancel(kotlin.coroutines.cancellation.CancellationException?){}[0] + final fun cancel(kotlin/Throwable?): kotlin/Boolean // kotlinx.coroutines/NonCancellable.cancel|cancel(kotlin.Throwable?){}[0] + final fun getCancellationException(): kotlin.coroutines.cancellation/CancellationException // kotlinx.coroutines/NonCancellable.getCancellationException|getCancellationException(){}[0] + final fun invokeOnCompletion(kotlin/Boolean, kotlin/Boolean, kotlin/Function1): kotlinx.coroutines/DisposableHandle // kotlinx.coroutines/NonCancellable.invokeOnCompletion|invokeOnCompletion(kotlin.Boolean;kotlin.Boolean;kotlin.Function1){}[0] + final fun invokeOnCompletion(kotlin/Function1): kotlinx.coroutines/DisposableHandle // kotlinx.coroutines/NonCancellable.invokeOnCompletion|invokeOnCompletion(kotlin.Function1){}[0] + final fun start(): kotlin/Boolean // kotlinx.coroutines/NonCancellable.start|start(){}[0] + final fun toString(): kotlin/String // kotlinx.coroutines/NonCancellable.toString|toString(){}[0] + final suspend fun join() // kotlinx.coroutines/NonCancellable.join|join(){}[0] +} + +final object kotlinx.coroutines/NonDisposableHandle : kotlinx.coroutines/ChildHandle, kotlinx.coroutines/DisposableHandle { // kotlinx.coroutines/NonDisposableHandle|null[0] + final val parent // kotlinx.coroutines/NonDisposableHandle.parent|{}parent[0] + final fun (): kotlinx.coroutines/Job? // kotlinx.coroutines/NonDisposableHandle.parent.|(){}[0] + + final fun childCancelled(kotlin/Throwable): kotlin/Boolean // kotlinx.coroutines/NonDisposableHandle.childCancelled|childCancelled(kotlin.Throwable){}[0] + final fun dispose() // kotlinx.coroutines/NonDisposableHandle.dispose|dispose(){}[0] + final fun toString(): kotlin/String // kotlinx.coroutines/NonDisposableHandle.toString|toString(){}[0] +} + +final const val kotlinx.coroutines.flow/DEFAULT_CONCURRENCY_PROPERTY_NAME // kotlinx.coroutines.flow/DEFAULT_CONCURRENCY_PROPERTY_NAME|{}DEFAULT_CONCURRENCY_PROPERTY_NAME[0] + final fun (): kotlin/String // kotlinx.coroutines.flow/DEFAULT_CONCURRENCY_PROPERTY_NAME.|(){}[0] +final const val kotlinx.coroutines/MODE_CANCELLABLE // kotlinx.coroutines/MODE_CANCELLABLE|{}MODE_CANCELLABLE[0] + final fun (): kotlin/Int // kotlinx.coroutines/MODE_CANCELLABLE.|(){}[0] + +final val kotlinx.coroutines.flow/DEFAULT_CONCURRENCY // kotlinx.coroutines.flow/DEFAULT_CONCURRENCY|{}DEFAULT_CONCURRENCY[0] + final fun (): kotlin/Int // kotlinx.coroutines.flow/DEFAULT_CONCURRENCY.|(){}[0] +final val kotlinx.coroutines.flow/coroutineContext // kotlinx.coroutines.flow/coroutineContext|@kotlinx.coroutines.flow.FlowCollector<*>{}coroutineContext[0] + final fun (kotlinx.coroutines.flow/FlowCollector<*>).(): kotlin.coroutines/CoroutineContext // kotlinx.coroutines.flow/coroutineContext.|@kotlinx.coroutines.flow.FlowCollector<*>(){}[0] +final val kotlinx.coroutines.flow/isActive // kotlinx.coroutines.flow/isActive|@kotlinx.coroutines.flow.FlowCollector<*>{}isActive[0] + final fun (kotlinx.coroutines.flow/FlowCollector<*>).(): kotlin/Boolean // kotlinx.coroutines.flow/isActive.|@kotlinx.coroutines.flow.FlowCollector<*>(){}[0] +final val kotlinx.coroutines/DefaultDelay // kotlinx.coroutines/DefaultDelay|{}DefaultDelay[0] + final fun (): kotlinx.coroutines/Delay // kotlinx.coroutines/DefaultDelay.|(){}[0] +final val kotlinx.coroutines/isActive // kotlinx.coroutines/isActive|@kotlin.coroutines.CoroutineContext{}isActive[0] + final fun (kotlin.coroutines/CoroutineContext).(): kotlin/Boolean // kotlinx.coroutines/isActive.|@kotlin.coroutines.CoroutineContext(){}[0] +final val kotlinx.coroutines/isActive // kotlinx.coroutines/isActive|@kotlinx.coroutines.CoroutineScope{}isActive[0] + final fun (kotlinx.coroutines/CoroutineScope).(): kotlin/Boolean // kotlinx.coroutines/isActive.|@kotlinx.coroutines.CoroutineScope(){}[0] +final val kotlinx.coroutines/job // kotlinx.coroutines/job|@kotlin.coroutines.CoroutineContext{}job[0] + final fun (kotlin.coroutines/CoroutineContext).(): kotlinx.coroutines/Job // kotlinx.coroutines/job.|@kotlin.coroutines.CoroutineContext(){}[0] + +final fun (kotlin.coroutines/CoroutineContext).kotlinx.coroutines/cancel() // kotlinx.coroutines/cancel|cancel@kotlin.coroutines.CoroutineContext(){}[0] +final fun (kotlin.coroutines/CoroutineContext).kotlinx.coroutines/cancel(kotlin.coroutines.cancellation/CancellationException? = ...) // kotlinx.coroutines/cancel|cancel@kotlin.coroutines.CoroutineContext(kotlin.coroutines.cancellation.CancellationException?){}[0] +final fun (kotlin.coroutines/CoroutineContext).kotlinx.coroutines/cancel(kotlin/Throwable? = ...): kotlin/Boolean // kotlinx.coroutines/cancel|cancel@kotlin.coroutines.CoroutineContext(kotlin.Throwable?){}[0] +final fun (kotlin.coroutines/CoroutineContext).kotlinx.coroutines/cancelChildren() // kotlinx.coroutines/cancelChildren|cancelChildren@kotlin.coroutines.CoroutineContext(){}[0] +final fun (kotlin.coroutines/CoroutineContext).kotlinx.coroutines/cancelChildren(kotlin.coroutines.cancellation/CancellationException? = ...) // kotlinx.coroutines/cancelChildren|cancelChildren@kotlin.coroutines.CoroutineContext(kotlin.coroutines.cancellation.CancellationException?){}[0] +final fun (kotlin.coroutines/CoroutineContext).kotlinx.coroutines/cancelChildren(kotlin/Throwable? = ...) // kotlinx.coroutines/cancelChildren|cancelChildren@kotlin.coroutines.CoroutineContext(kotlin.Throwable?){}[0] +final fun (kotlin.coroutines/CoroutineContext).kotlinx.coroutines/ensureActive() // kotlinx.coroutines/ensureActive|ensureActive@kotlin.coroutines.CoroutineContext(){}[0] +final fun (kotlin.coroutines/CoroutineContext).kotlinx.coroutines/newCoroutineContext(kotlin.coroutines/CoroutineContext): kotlin.coroutines/CoroutineContext // kotlinx.coroutines/newCoroutineContext|newCoroutineContext@kotlin.coroutines.CoroutineContext(kotlin.coroutines.CoroutineContext){}[0] +final fun (kotlin.ranges/IntRange).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.ranges.IntRange(){}[0] +final fun (kotlin.ranges/LongRange).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.ranges.LongRange(){}[0] +final fun (kotlin/IntArray).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.IntArray(){}[0] +final fun (kotlin/LongArray).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.LongArray(){}[0] +final fun (kotlinx.coroutines.channels/ReceiveChannel<*>).kotlinx.coroutines.channels/cancelConsumed(kotlin/Throwable?) // kotlinx.coroutines.channels/cancelConsumed|cancelConsumed@kotlinx.coroutines.channels.ReceiveChannel<*>(kotlin.Throwable?){}[0] +final fun (kotlinx.coroutines.channels/ReceiveChannel<*>).kotlinx.coroutines.channels/consumes(): kotlin/Function1 // kotlinx.coroutines.channels/consumes|consumes@kotlinx.coroutines.channels.ReceiveChannel<*>(){}[0] +final fun (kotlinx.coroutines.flow/FlowCollector<*>).kotlinx.coroutines.flow/cancel(kotlin.coroutines.cancellation/CancellationException? = ...) // kotlinx.coroutines.flow/cancel|cancel@kotlinx.coroutines.flow.FlowCollector<*>(kotlin.coroutines.cancellation.CancellationException?){}[0] +final fun (kotlinx.coroutines.flow/SharingStarted.Companion).kotlinx.coroutines.flow/WhileSubscribed(kotlin.time/Duration = ..., kotlin.time/Duration = ...): kotlinx.coroutines.flow/SharingStarted // kotlinx.coroutines.flow/WhileSubscribed|WhileSubscribed@kotlinx.coroutines.flow.SharingStarted.Companion(kotlin.time.Duration;kotlin.time.Duration){}[0] +final fun (kotlinx.coroutines/CancellableContinuation<*>).kotlinx.coroutines/disposeOnCancellation(kotlinx.coroutines/DisposableHandle) // kotlinx.coroutines/disposeOnCancellation|disposeOnCancellation@kotlinx.coroutines.CancellableContinuation<*>(kotlinx.coroutines.DisposableHandle){}[0] +final fun (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines/cancel(kotlin.coroutines.cancellation/CancellationException? = ...) // kotlinx.coroutines/cancel|cancel@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.cancellation.CancellationException?){}[0] +final fun (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines/cancel(kotlin/String, kotlin/Throwable? = ...) // kotlinx.coroutines/cancel|cancel@kotlinx.coroutines.CoroutineScope(kotlin.String;kotlin.Throwable?){}[0] +final fun (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines/ensureActive() // kotlinx.coroutines/ensureActive|ensureActive@kotlinx.coroutines.CoroutineScope(){}[0] +final fun (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines/launch(kotlin.coroutines/CoroutineContext = ..., kotlinx.coroutines/CoroutineStart = ..., kotlin.coroutines/SuspendFunction1): kotlinx.coroutines/Job // kotlinx.coroutines/launch|launch@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.CoroutineContext;kotlinx.coroutines.CoroutineStart;kotlin.coroutines.SuspendFunction1){}[0] +final fun (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines/newCoroutineContext(kotlin.coroutines/CoroutineContext): kotlin.coroutines/CoroutineContext // kotlinx.coroutines/newCoroutineContext|newCoroutineContext@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.CoroutineContext){}[0] +final fun (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines/plus(kotlin.coroutines/CoroutineContext): kotlinx.coroutines/CoroutineScope // kotlinx.coroutines/plus|plus@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.CoroutineContext){}[0] +final fun (kotlinx.coroutines/Job).kotlinx.coroutines/cancel(kotlin/String, kotlin/Throwable? = ...) // kotlinx.coroutines/cancel|cancel@kotlinx.coroutines.Job(kotlin.String;kotlin.Throwable?){}[0] +final fun (kotlinx.coroutines/Job).kotlinx.coroutines/cancelChildren() // kotlinx.coroutines/cancelChildren|cancelChildren@kotlinx.coroutines.Job(){}[0] +final fun (kotlinx.coroutines/Job).kotlinx.coroutines/cancelChildren(kotlin.coroutines.cancellation/CancellationException? = ...) // kotlinx.coroutines/cancelChildren|cancelChildren@kotlinx.coroutines.Job(kotlin.coroutines.cancellation.CancellationException?){}[0] +final fun (kotlinx.coroutines/Job).kotlinx.coroutines/cancelChildren(kotlin/Throwable? = ...) // kotlinx.coroutines/cancelChildren|cancelChildren@kotlinx.coroutines.Job(kotlin.Throwable?){}[0] +final fun (kotlinx.coroutines/Job).kotlinx.coroutines/ensureActive() // kotlinx.coroutines/ensureActive|ensureActive@kotlinx.coroutines.Job(){}[0] +final fun <#A: kotlin/Any> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/onReceiveOrNull(): kotlinx.coroutines.selects/SelectClause1<#A?> // kotlinx.coroutines.channels/onReceiveOrNull|onReceiveOrNull@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final fun <#A: kotlin/Any> (kotlinx.coroutines.channels/ReceiveChannel<#A?>).kotlinx.coroutines.channels/filterNotNull(): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/filterNotNull|filterNotNull@kotlinx.coroutines.channels.ReceiveChannel<0:0?>(){0§}[0] +final fun <#A: kotlin/Any> (kotlinx.coroutines.channels/ReceiveChannel<#A?>).kotlinx.coroutines.channels/requireNoNulls(): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/requireNoNulls|requireNoNulls@kotlinx.coroutines.channels.ReceiveChannel<0:0?>(){0§}[0] +final fun <#A: kotlin/Any> (kotlinx.coroutines.flow/Flow<#A?>).kotlinx.coroutines.flow/filterNotNull(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/filterNotNull|filterNotNull@kotlinx.coroutines.flow.Flow<0:0?>(){0§}[0] +final fun <#A: kotlin/Any> (kotlinx.coroutines.flow/Flow<*>).kotlinx.coroutines.flow/filterIsInstance(kotlin.reflect/KClass<#A>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/filterIsInstance|filterIsInstance@kotlinx.coroutines.flow.Flow<*>(kotlin.reflect.KClass<0:0>){0§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/mapIndexedNotNull(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction2): kotlinx.coroutines.channels/ReceiveChannel<#B> // kotlinx.coroutines.channels/mapIndexedNotNull|mapIndexedNotNull@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction2){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/mapNotNull(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction1<#A, #B?>): kotlinx.coroutines.channels/ReceiveChannel<#B> // kotlinx.coroutines.channels/mapNotNull|mapNotNull@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<0:0,0:1?>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?, #D: kotlin/Any?, #E: kotlin/Any?, #F: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/combineLatest(kotlinx.coroutines.flow/Flow<#B>, kotlinx.coroutines.flow/Flow<#C>, kotlinx.coroutines.flow/Flow<#D>, kotlinx.coroutines.flow/Flow<#E>, kotlin.coroutines/SuspendFunction5<#A, #B, #C, #D, #E, #F>): kotlinx.coroutines.flow/Flow<#F> // kotlinx.coroutines.flow/combineLatest|combineLatest@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:1>;kotlinx.coroutines.flow.Flow<0:2>;kotlinx.coroutines.flow.Flow<0:3>;kotlinx.coroutines.flow.Flow<0:4>;kotlin.coroutines.SuspendFunction5<0:0,0:1,0:2,0:3,0:4,0:5>){0§;1§;2§;3§;4§;5§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?, #D: kotlin/Any?, #E: kotlin/Any?, #F: kotlin/Any?> kotlinx.coroutines.flow/combine(kotlinx.coroutines.flow/Flow<#A>, kotlinx.coroutines.flow/Flow<#B>, kotlinx.coroutines.flow/Flow<#C>, kotlinx.coroutines.flow/Flow<#D>, kotlinx.coroutines.flow/Flow<#E>, kotlin.coroutines/SuspendFunction5<#A, #B, #C, #D, #E, #F>): kotlinx.coroutines.flow/Flow<#F> // kotlinx.coroutines.flow/combine|combine(kotlinx.coroutines.flow.Flow<0:0>;kotlinx.coroutines.flow.Flow<0:1>;kotlinx.coroutines.flow.Flow<0:2>;kotlinx.coroutines.flow.Flow<0:3>;kotlinx.coroutines.flow.Flow<0:4>;kotlin.coroutines.SuspendFunction5<0:0,0:1,0:2,0:3,0:4,0:5>){0§;1§;2§;3§;4§;5§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?, #D: kotlin/Any?, #E: kotlin/Any?, #F: kotlin/Any?> kotlinx.coroutines.flow/combineTransform(kotlinx.coroutines.flow/Flow<#A>, kotlinx.coroutines.flow/Flow<#B>, kotlinx.coroutines.flow/Flow<#C>, kotlinx.coroutines.flow/Flow<#D>, kotlinx.coroutines.flow/Flow<#E>, kotlin.coroutines/SuspendFunction6, #A, #B, #C, #D, #E, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#F> // kotlinx.coroutines.flow/combineTransform|combineTransform(kotlinx.coroutines.flow.Flow<0:0>;kotlinx.coroutines.flow.Flow<0:1>;kotlinx.coroutines.flow.Flow<0:2>;kotlinx.coroutines.flow.Flow<0:3>;kotlinx.coroutines.flow.Flow<0:4>;kotlin.coroutines.SuspendFunction6,0:0,0:1,0:2,0:3,0:4,kotlin.Unit>){0§;1§;2§;3§;4§;5§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?, #D: kotlin/Any?, #E: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/combineLatest(kotlinx.coroutines.flow/Flow<#B>, kotlinx.coroutines.flow/Flow<#C>, kotlinx.coroutines.flow/Flow<#D>, kotlin.coroutines/SuspendFunction4<#A, #B, #C, #D, #E>): kotlinx.coroutines.flow/Flow<#E> // kotlinx.coroutines.flow/combineLatest|combineLatest@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:1>;kotlinx.coroutines.flow.Flow<0:2>;kotlinx.coroutines.flow.Flow<0:3>;kotlin.coroutines.SuspendFunction4<0:0,0:1,0:2,0:3,0:4>){0§;1§;2§;3§;4§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?, #D: kotlin/Any?, #E: kotlin/Any?> kotlinx.coroutines.flow/combine(kotlinx.coroutines.flow/Flow<#A>, kotlinx.coroutines.flow/Flow<#B>, kotlinx.coroutines.flow/Flow<#C>, kotlinx.coroutines.flow/Flow<#D>, kotlin.coroutines/SuspendFunction4<#A, #B, #C, #D, #E>): kotlinx.coroutines.flow/Flow<#E> // kotlinx.coroutines.flow/combine|combine(kotlinx.coroutines.flow.Flow<0:0>;kotlinx.coroutines.flow.Flow<0:1>;kotlinx.coroutines.flow.Flow<0:2>;kotlinx.coroutines.flow.Flow<0:3>;kotlin.coroutines.SuspendFunction4<0:0,0:1,0:2,0:3,0:4>){0§;1§;2§;3§;4§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?, #D: kotlin/Any?, #E: kotlin/Any?> kotlinx.coroutines.flow/combineTransform(kotlinx.coroutines.flow/Flow<#A>, kotlinx.coroutines.flow/Flow<#B>, kotlinx.coroutines.flow/Flow<#C>, kotlinx.coroutines.flow/Flow<#D>, kotlin.coroutines/SuspendFunction5, #A, #B, #C, #D, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#E> // kotlinx.coroutines.flow/combineTransform|combineTransform(kotlinx.coroutines.flow.Flow<0:0>;kotlinx.coroutines.flow.Flow<0:1>;kotlinx.coroutines.flow.Flow<0:2>;kotlinx.coroutines.flow.Flow<0:3>;kotlin.coroutines.SuspendFunction5,0:0,0:1,0:2,0:3,kotlin.Unit>){0§;1§;2§;3§;4§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?, #D: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/combineLatest(kotlinx.coroutines.flow/Flow<#B>, kotlinx.coroutines.flow/Flow<#C>, kotlin.coroutines/SuspendFunction3<#A, #B, #C, #D>): kotlinx.coroutines.flow/Flow<#D> // kotlinx.coroutines.flow/combineLatest|combineLatest@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:1>;kotlinx.coroutines.flow.Flow<0:2>;kotlin.coroutines.SuspendFunction3<0:0,0:1,0:2,0:3>){0§;1§;2§;3§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?, #D: kotlin/Any?> kotlinx.coroutines.flow/combine(kotlinx.coroutines.flow/Flow<#A>, kotlinx.coroutines.flow/Flow<#B>, kotlinx.coroutines.flow/Flow<#C>, kotlin.coroutines/SuspendFunction3<#A, #B, #C, #D>): kotlinx.coroutines.flow/Flow<#D> // kotlinx.coroutines.flow/combine|combine(kotlinx.coroutines.flow.Flow<0:0>;kotlinx.coroutines.flow.Flow<0:1>;kotlinx.coroutines.flow.Flow<0:2>;kotlin.coroutines.SuspendFunction3<0:0,0:1,0:2,0:3>){0§;1§;2§;3§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?, #D: kotlin/Any?> kotlinx.coroutines.flow/combineTransform(kotlinx.coroutines.flow/Flow<#A>, kotlinx.coroutines.flow/Flow<#B>, kotlinx.coroutines.flow/Flow<#C>, kotlin.coroutines/SuspendFunction4, #A, #B, #C, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#D> // kotlinx.coroutines.flow/combineTransform|combineTransform(kotlinx.coroutines.flow.Flow<0:0>;kotlinx.coroutines.flow.Flow<0:1>;kotlinx.coroutines.flow.Flow<0:2>;kotlin.coroutines.SuspendFunction4,0:0,0:1,0:2,kotlin.Unit>){0§;1§;2§;3§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/zip(kotlinx.coroutines.channels/ReceiveChannel<#B>, kotlin.coroutines/CoroutineContext = ..., kotlin/Function2<#A, #B, #C>): kotlinx.coroutines.channels/ReceiveChannel<#C> // kotlinx.coroutines.channels/zip|zip@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlinx.coroutines.channels.ReceiveChannel<0:1>;kotlin.coroutines.CoroutineContext;kotlin.Function2<0:0,0:1,0:2>){0§;1§;2§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/combine(kotlinx.coroutines.flow/Flow<#B>, kotlin.coroutines/SuspendFunction2<#A, #B, #C>): kotlinx.coroutines.flow/Flow<#C> // kotlinx.coroutines.flow/combine|combine@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:1>;kotlin.coroutines.SuspendFunction2<0:0,0:1,0:2>){0§;1§;2§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/combineLatest(kotlinx.coroutines.flow/Flow<#B>, kotlin.coroutines/SuspendFunction2<#A, #B, #C>): kotlinx.coroutines.flow/Flow<#C> // kotlinx.coroutines.flow/combineLatest|combineLatest@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:1>;kotlin.coroutines.SuspendFunction2<0:0,0:1,0:2>){0§;1§;2§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/combineTransform(kotlinx.coroutines.flow/Flow<#B>, kotlin.coroutines/SuspendFunction3, #A, #B, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#C> // kotlinx.coroutines.flow/combineTransform|combineTransform@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:1>;kotlin.coroutines.SuspendFunction3,0:0,0:1,kotlin.Unit>){0§;1§;2§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/zip(kotlinx.coroutines.flow/Flow<#B>, kotlin.coroutines/SuspendFunction2<#A, #B, #C>): kotlinx.coroutines.flow/Flow<#C> // kotlinx.coroutines.flow/zip|zip@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:1>;kotlin.coroutines.SuspendFunction2<0:0,0:1,0:2>){0§;1§;2§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?> kotlinx.coroutines.flow/combine(kotlinx.coroutines.flow/Flow<#A>, kotlinx.coroutines.flow/Flow<#B>, kotlin.coroutines/SuspendFunction2<#A, #B, #C>): kotlinx.coroutines.flow/Flow<#C> // kotlinx.coroutines.flow/combine|combine(kotlinx.coroutines.flow.Flow<0:0>;kotlinx.coroutines.flow.Flow<0:1>;kotlin.coroutines.SuspendFunction2<0:0,0:1,0:2>){0§;1§;2§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin/Any?> kotlinx.coroutines.flow/combineTransform(kotlinx.coroutines.flow/Flow<#A>, kotlinx.coroutines.flow/Flow<#B>, kotlin.coroutines/SuspendFunction3, #A, #B, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#C> // kotlinx.coroutines.flow/combineTransform|combineTransform(kotlinx.coroutines.flow.Flow<0:0>;kotlinx.coroutines.flow.Flow<0:1>;kotlin.coroutines.SuspendFunction3,0:0,0:1,kotlin.Unit>){0§;1§;2§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/distinctBy(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction1<#A, #B>): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/distinctBy|distinctBy@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<0:0,0:1>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/flatMap(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction1<#A, kotlinx.coroutines.channels/ReceiveChannel<#B>>): kotlinx.coroutines.channels/ReceiveChannel<#B> // kotlinx.coroutines.channels/flatMap|flatMap@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<0:0,kotlinx.coroutines.channels.ReceiveChannel<0:1>>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/map(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction1<#A, #B>): kotlinx.coroutines.channels/ReceiveChannel<#B> // kotlinx.coroutines.channels/map|map@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<0:0,0:1>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/mapIndexed(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction2): kotlinx.coroutines.channels/ReceiveChannel<#B> // kotlinx.coroutines.channels/mapIndexed|mapIndexed@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction2){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/zip(kotlinx.coroutines.channels/ReceiveChannel<#B>): kotlinx.coroutines.channels/ReceiveChannel> // kotlinx.coroutines.channels/zip|zip@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlinx.coroutines.channels.ReceiveChannel<0:1>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/compose(kotlin/Function1, kotlinx.coroutines.flow/Flow<#B>>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/compose|compose@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Function1,kotlinx.coroutines.flow.Flow<0:1>>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/concatMap(kotlin/Function1<#A, kotlinx.coroutines.flow/Flow<#B>>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/concatMap|concatMap@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Function1<0:0,kotlinx.coroutines.flow.Flow<0:1>>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/distinctUntilChangedBy(kotlin/Function1<#A, #B>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/distinctUntilChangedBy|distinctUntilChangedBy@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Function1<0:0,0:1>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/flatMap(kotlin.coroutines/SuspendFunction1<#A, kotlinx.coroutines.flow/Flow<#B>>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/flatMap|flatMap@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlinx.coroutines.flow.Flow<0:1>>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/flatMapConcat(kotlin.coroutines/SuspendFunction1<#A, kotlinx.coroutines.flow/Flow<#B>>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/flatMapConcat|flatMapConcat@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlinx.coroutines.flow.Flow<0:1>>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/flatMapMerge(kotlin/Int = ..., kotlin.coroutines/SuspendFunction1<#A, kotlinx.coroutines.flow/Flow<#B>>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/flatMapMerge|flatMapMerge@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Int;kotlin.coroutines.SuspendFunction1<0:0,kotlinx.coroutines.flow.Flow<0:1>>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/mapLatest(kotlin.coroutines/SuspendFunction1<#A, #B>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/mapLatest|mapLatest@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,0:1>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/runningFold(#B, kotlin.coroutines/SuspendFunction2<#B, #A, #B>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/runningFold|runningFold@kotlinx.coroutines.flow.Flow<0:0>(0:1;kotlin.coroutines.SuspendFunction2<0:1,0:0,0:1>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/scan(#B, kotlin.coroutines/SuspendFunction2<#B, #A, #B>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/scan|scan@kotlinx.coroutines.flow.Flow<0:0>(0:1;kotlin.coroutines.SuspendFunction2<0:1,0:0,0:1>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/scanFold(#B, kotlin.coroutines/SuspendFunction2<#B, #A, #B>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/scanFold|scanFold@kotlinx.coroutines.flow.Flow<0:0>(0:1;kotlin.coroutines.SuspendFunction2<0:1,0:0,0:1>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/switchMap(kotlin.coroutines/SuspendFunction1<#A, kotlinx.coroutines.flow/Flow<#B>>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/switchMap|switchMap@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlinx.coroutines.flow.Flow<0:1>>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/transformLatest(kotlin.coroutines/SuspendFunction2, #A, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/transformLatest|transformLatest@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction2,0:0,kotlin.Unit>){0§;1§}[0] +final fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/transformWhile(kotlin.coroutines/SuspendFunction2, #A, kotlin/Boolean>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/transformWhile|transformWhile@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction2,0:0,kotlin.Boolean>){0§;1§}[0] +final fun <#A: kotlin/Any?> (kotlin.collections/Iterable<#A>).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.collections.Iterable<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlin.collections/Iterable>).kotlinx.coroutines.flow/merge(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/merge|merge@kotlin.collections.Iterable>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlin.collections/Iterator<#A>).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.collections.Iterator<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlin.coroutines/SuspendFunction0<#A>).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.coroutines.SuspendFunction0<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlin.coroutines/SuspendFunction0<#A>).kotlinx.coroutines.intrinsics/startCoroutineCancellable(kotlin.coroutines/Continuation<#A>) // kotlinx.coroutines.intrinsics/startCoroutineCancellable|startCoroutineCancellable@kotlin.coroutines.SuspendFunction0<0:0>(kotlin.coroutines.Continuation<0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlin.sequences/Sequence<#A>).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.sequences.Sequence<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlin/Array<#A>).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.Array<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlin/Function0<#A>).kotlinx.coroutines.flow/asFlow(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/asFlow|asFlow@kotlin.Function0<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/broadcast(kotlin/Int = ..., kotlinx.coroutines/CoroutineStart = ...): kotlinx.coroutines.channels/BroadcastChannel<#A> // kotlinx.coroutines.channels/broadcast|broadcast@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.Int;kotlinx.coroutines.CoroutineStart){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/distinct(): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/distinct|distinct@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/drop(kotlin/Int, kotlin.coroutines/CoroutineContext = ...): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/drop|drop@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.Int;kotlin.coroutines.CoroutineContext){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/dropWhile(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/dropWhile|dropWhile@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/filter(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/filter|filter@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/filterIndexed(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction2): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/filterIndexed|filterIndexed@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction2){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/filterNot(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/filterNot|filterNot@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/take(kotlin/Int, kotlin.coroutines/CoroutineContext = ...): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/take|take@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.Int;kotlin.coroutines.CoroutineContext){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/takeWhile(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/takeWhile|takeWhile@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/withIndex(kotlin.coroutines/CoroutineContext = ...): kotlinx.coroutines.channels/ReceiveChannel> // kotlinx.coroutines.channels/withIndex|withIndex@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.coroutines.CoroutineContext){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.flow/consumeAsFlow(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/consumeAsFlow|consumeAsFlow@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.flow/receiveAsFlow(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/receiveAsFlow|receiveAsFlow@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/buffer(kotlin/Int = ...): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/buffer|buffer@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/buffer(kotlin/Int = ..., kotlinx.coroutines.channels/BufferOverflow = ...): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/buffer|buffer@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Int;kotlinx.coroutines.channels.BufferOverflow){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/cache(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/cache|cache@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/cancellable(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/cancellable|cancellable@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/catch(kotlin.coroutines/SuspendFunction2, kotlin/Throwable, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/catch|catch@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction2,kotlin.Throwable,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/chunked(kotlin/Int): kotlinx.coroutines.flow/Flow> // kotlinx.coroutines.flow/chunked|chunked@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/concatWith(#A): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/concatWith|concatWith@kotlinx.coroutines.flow.Flow<0:0>(0:0){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/concatWith(kotlinx.coroutines.flow/Flow<#A>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/concatWith|concatWith@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/conflate(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/conflate|conflate@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/debounce(kotlin.time/Duration): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/debounce|debounce@kotlinx.coroutines.flow.Flow<0:0>(kotlin.time.Duration){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/debounce(kotlin/Function1<#A, kotlin.time/Duration>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/debounce|debounce@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Function1<0:0,kotlin.time.Duration>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/debounce(kotlin/Function1<#A, kotlin/Long>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/debounce|debounce@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Function1<0:0,kotlin.Long>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/debounce(kotlin/Long): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/debounce|debounce@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Long){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/delayEach(kotlin/Long): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/delayEach|delayEach@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Long){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/delayFlow(kotlin/Long): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/delayFlow|delayFlow@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Long){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/distinctUntilChanged(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/distinctUntilChanged|distinctUntilChanged@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/distinctUntilChanged(kotlin/Function2<#A, #A, kotlin/Boolean>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/distinctUntilChanged|distinctUntilChanged@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Function2<0:0,0:0,kotlin.Boolean>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/drop(kotlin/Int): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/drop|drop@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/dropWhile(kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/dropWhile|dropWhile@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/flowOn(kotlin.coroutines/CoroutineContext): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/flowOn|flowOn@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.CoroutineContext){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/forEach(kotlin.coroutines/SuspendFunction1<#A, kotlin/Unit>) // kotlinx.coroutines.flow/forEach|forEach@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/launchIn(kotlinx.coroutines/CoroutineScope): kotlinx.coroutines/Job // kotlinx.coroutines.flow/launchIn|launchIn@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.CoroutineScope){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/observeOn(kotlin.coroutines/CoroutineContext): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/observeOn|observeOn@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.CoroutineContext){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/onCompletion(kotlin.coroutines/SuspendFunction2, kotlin/Throwable?, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/onCompletion|onCompletion@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction2,kotlin.Throwable?,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/onEach(kotlin.coroutines/SuspendFunction1<#A, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/onEach|onEach@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/onEmpty(kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/onEmpty|onEmpty@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/onErrorResume(kotlinx.coroutines.flow/Flow<#A>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/onErrorResume|onErrorResume@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/onErrorResumeNext(kotlinx.coroutines.flow/Flow<#A>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/onErrorResumeNext|onErrorResumeNext@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/onErrorReturn(#A): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/onErrorReturn|onErrorReturn@kotlinx.coroutines.flow.Flow<0:0>(0:0){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/onErrorReturn(#A, kotlin/Function1 = ...): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/onErrorReturn|onErrorReturn@kotlinx.coroutines.flow.Flow<0:0>(0:0;kotlin.Function1){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/onStart(kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/onStart|onStart@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/produceIn(kotlinx.coroutines/CoroutineScope): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.flow/produceIn|produceIn@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.CoroutineScope){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/publish(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/publish|publish@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/publish(kotlin/Int): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/publish|publish@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/publishOn(kotlin.coroutines/CoroutineContext): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/publishOn|publishOn@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.CoroutineContext){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/replay(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/replay|replay@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/replay(kotlin/Int): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/replay|replay@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/retry(kotlin/Long = ..., kotlin.coroutines/SuspendFunction1 = ...): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/retry|retry@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Long;kotlin.coroutines.SuspendFunction1){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/retryWhen(kotlin.coroutines/SuspendFunction3, kotlin/Throwable, kotlin/Long, kotlin/Boolean>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/retryWhen|retryWhen@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction3,kotlin.Throwable,kotlin.Long,kotlin.Boolean>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/runningReduce(kotlin.coroutines/SuspendFunction2<#A, #A, #A>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/runningReduce|runningReduce@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction2<0:0,0:0,0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/sample(kotlin.time/Duration): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/sample|sample@kotlinx.coroutines.flow.Flow<0:0>(kotlin.time.Duration){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/sample(kotlin/Long): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/sample|sample@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Long){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/scanReduce(kotlin.coroutines/SuspendFunction2<#A, #A, #A>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/scanReduce|scanReduce@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction2<0:0,0:0,0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/shareIn(kotlinx.coroutines/CoroutineScope, kotlinx.coroutines.flow/SharingStarted, kotlin/Int = ...): kotlinx.coroutines.flow/SharedFlow<#A> // kotlinx.coroutines.flow/shareIn|shareIn@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.CoroutineScope;kotlinx.coroutines.flow.SharingStarted;kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/skip(kotlin/Int): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/skip|skip@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/startWith(#A): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/startWith|startWith@kotlinx.coroutines.flow.Flow<0:0>(0:0){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/startWith(kotlinx.coroutines.flow/Flow<#A>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/startWith|startWith@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.flow.Flow<0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/stateIn(kotlinx.coroutines/CoroutineScope, kotlinx.coroutines.flow/SharingStarted, #A): kotlinx.coroutines.flow/StateFlow<#A> // kotlinx.coroutines.flow/stateIn|stateIn@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.CoroutineScope;kotlinx.coroutines.flow.SharingStarted;0:0){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/subscribe() // kotlinx.coroutines.flow/subscribe|subscribe@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/subscribe(kotlin.coroutines/SuspendFunction1<#A, kotlin/Unit>) // kotlinx.coroutines.flow/subscribe|subscribe@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/subscribe(kotlin.coroutines/SuspendFunction1<#A, kotlin/Unit>, kotlin.coroutines/SuspendFunction1) // kotlinx.coroutines.flow/subscribe|subscribe@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Unit>;kotlin.coroutines.SuspendFunction1){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/subscribeOn(kotlin.coroutines/CoroutineContext): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/subscribeOn|subscribeOn@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.CoroutineContext){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/take(kotlin/Int): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/take|take@kotlinx.coroutines.flow.Flow<0:0>(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/takeWhile(kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/takeWhile|takeWhile@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/timeout(kotlin.time/Duration): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/timeout|timeout@kotlinx.coroutines.flow.Flow<0:0>(kotlin.time.Duration){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/withIndex(): kotlinx.coroutines.flow/Flow> // kotlinx.coroutines.flow/withIndex|withIndex@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow>).kotlinx.coroutines.flow/flatten(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/flatten|flatten@kotlinx.coroutines.flow.Flow>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow>).kotlinx.coroutines.flow/flattenConcat(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/flattenConcat|flattenConcat@kotlinx.coroutines.flow.Flow>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow>).kotlinx.coroutines.flow/flattenMerge(kotlin/Int = ...): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/flattenMerge|flattenMerge@kotlinx.coroutines.flow.Flow>(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow>).kotlinx.coroutines.flow/merge(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/merge|merge@kotlinx.coroutines.flow.Flow>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/MutableSharedFlow<#A>).kotlinx.coroutines.flow/asSharedFlow(): kotlinx.coroutines.flow/SharedFlow<#A> // kotlinx.coroutines.flow/asSharedFlow|asSharedFlow@kotlinx.coroutines.flow.MutableSharedFlow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/MutableStateFlow<#A>).kotlinx.coroutines.flow/asStateFlow(): kotlinx.coroutines.flow/StateFlow<#A> // kotlinx.coroutines.flow/asStateFlow|asStateFlow@kotlinx.coroutines.flow.MutableStateFlow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/cancellable(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/cancellable|cancellable@kotlinx.coroutines.flow.SharedFlow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/flowOn(kotlin.coroutines/CoroutineContext): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/flowOn|flowOn@kotlinx.coroutines.flow.SharedFlow<0:0>(kotlin.coroutines.CoroutineContext){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/onSubscription(kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.flow/SharedFlow<#A> // kotlinx.coroutines.flow/onSubscription|onSubscription@kotlinx.coroutines.flow.SharedFlow<0:0>(kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/StateFlow<#A>).kotlinx.coroutines.flow/conflate(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/conflate|conflate@kotlinx.coroutines.flow.StateFlow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/StateFlow<#A>).kotlinx.coroutines.flow/distinctUntilChanged(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/distinctUntilChanged|distinctUntilChanged@kotlinx.coroutines.flow.StateFlow<0:0>(){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.selects/SelectBuilder<#A>).kotlinx.coroutines.selects/onTimeout(kotlin.time/Duration, kotlin.coroutines/SuspendFunction0<#A>) // kotlinx.coroutines.selects/onTimeout|onTimeout@kotlinx.coroutines.selects.SelectBuilder<0:0>(kotlin.time.Duration;kotlin.coroutines.SuspendFunction0<0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.selects/SelectBuilder<#A>).kotlinx.coroutines.selects/onTimeout(kotlin/Long, kotlin.coroutines/SuspendFunction0<#A>) // kotlinx.coroutines.selects/onTimeout|onTimeout@kotlinx.coroutines.selects.SelectBuilder<0:0>(kotlin.Long;kotlin.coroutines.SuspendFunction0<0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines/CompletableDeferred<#A>).kotlinx.coroutines/completeWith(kotlin/Result<#A>): kotlin/Boolean // kotlinx.coroutines/completeWith|completeWith@kotlinx.coroutines.CompletableDeferred<0:0>(kotlin.Result<0:0>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines.channels/broadcast(kotlin.coroutines/CoroutineContext = ..., kotlin/Int = ..., kotlinx.coroutines/CoroutineStart = ..., kotlin/Function1? = ..., kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.channels/BroadcastChannel<#A> // kotlinx.coroutines.channels/broadcast|broadcast@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.CoroutineContext;kotlin.Int;kotlinx.coroutines.CoroutineStart;kotlin.Function1?;kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines.channels/produce(kotlin.coroutines/CoroutineContext = ..., kotlin/Int = ..., kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/produce|produce@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.CoroutineContext;kotlin.Int;kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines.channels/produce(kotlin.coroutines/CoroutineContext = ..., kotlin/Int = ..., kotlinx.coroutines/CoroutineStart = ..., kotlin/Function1? = ..., kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.channels/ReceiveChannel<#A> // kotlinx.coroutines.channels/produce|produce@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.CoroutineContext;kotlin.Int;kotlinx.coroutines.CoroutineStart;kotlin.Function1?;kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines/async(kotlin.coroutines/CoroutineContext = ..., kotlinx.coroutines/CoroutineStart = ..., kotlin.coroutines/SuspendFunction1): kotlinx.coroutines/Deferred<#A> // kotlinx.coroutines/async|async@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.CoroutineContext;kotlinx.coroutines.CoroutineStart;kotlin.coroutines.SuspendFunction1){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.channels/BroadcastChannel(kotlin/Int): kotlinx.coroutines.channels/BroadcastChannel<#A> // kotlinx.coroutines.channels/BroadcastChannel|BroadcastChannel(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.channels/Channel(kotlin/Int = ...): kotlinx.coroutines.channels/Channel<#A> // kotlinx.coroutines.channels/Channel|Channel(kotlin.Int){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.channels/Channel(kotlin/Int = ..., kotlinx.coroutines.channels/BufferOverflow = ..., kotlin/Function1<#A, kotlin/Unit>? = ...): kotlinx.coroutines.channels/Channel<#A> // kotlinx.coroutines.channels/Channel|Channel(kotlin.Int;kotlinx.coroutines.channels.BufferOverflow;kotlin.Function1<0:0,kotlin.Unit>?){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.flow/MutableSharedFlow(kotlin/Int = ..., kotlin/Int = ..., kotlinx.coroutines.channels/BufferOverflow = ...): kotlinx.coroutines.flow/MutableSharedFlow<#A> // kotlinx.coroutines.flow/MutableSharedFlow|MutableSharedFlow(kotlin.Int;kotlin.Int;kotlinx.coroutines.channels.BufferOverflow){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.flow/MutableStateFlow(#A): kotlinx.coroutines.flow/MutableStateFlow<#A> // kotlinx.coroutines.flow/MutableStateFlow|MutableStateFlow(0:0){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.flow/callbackFlow(kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/callbackFlow|callbackFlow(kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.flow/channelFlow(kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/channelFlow|channelFlow(kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.flow/emptyFlow(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/emptyFlow|emptyFlow(){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.flow/flow(kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/flow|flow(kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.flow/flowOf(#A): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/flowOf|flowOf(0:0){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.flow/flowOf(kotlin/Array...): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/flowOf|flowOf(kotlin.Array...){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines.flow/merge(kotlin/Array>...): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/merge|merge(kotlin.Array>...){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines/CompletableDeferred(#A): kotlinx.coroutines/CompletableDeferred<#A> // kotlinx.coroutines/CompletableDeferred|CompletableDeferred(0:0){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines/CompletableDeferred(kotlinx.coroutines/Job? = ...): kotlinx.coroutines/CompletableDeferred<#A> // kotlinx.coroutines/CompletableDeferred|CompletableDeferred(kotlinx.coroutines.Job?){0§}[0] +final fun <#A: kotlin/Any?> kotlinx.coroutines/async(kotlin.coroutines/CoroutineContext = ..., kotlinx.coroutines/CoroutineStart = ..., kotlin.coroutines/SuspendFunction1): kotlinx.coroutines/Deferred<#A> // kotlinx.coroutines/async|async(kotlin.coroutines.CoroutineContext;kotlinx.coroutines.CoroutineStart;kotlin.coroutines.SuspendFunction1){0§}[0] +final fun kotlinx.coroutines.channels/consumesAll(kotlin/Array>...): kotlin/Function1 // kotlinx.coroutines.channels/consumesAll|consumesAll(kotlin.Array>...){}[0] +final fun kotlinx.coroutines.sync/Mutex(kotlin/Boolean = ...): kotlinx.coroutines.sync/Mutex // kotlinx.coroutines.sync/Mutex|Mutex(kotlin.Boolean){}[0] +final fun kotlinx.coroutines.sync/Semaphore(kotlin/Int, kotlin/Int = ...): kotlinx.coroutines.sync/Semaphore // kotlinx.coroutines.sync/Semaphore|Semaphore(kotlin.Int;kotlin.Int){}[0] +final fun kotlinx.coroutines/CancellationException(kotlin/String?, kotlin/Throwable?): kotlin.coroutines.cancellation/CancellationException // kotlinx.coroutines/CancellationException|CancellationException(kotlin.String?;kotlin.Throwable?){}[0] +final fun kotlinx.coroutines/CoroutineScope(kotlin.coroutines/CoroutineContext): kotlinx.coroutines/CoroutineScope // kotlinx.coroutines/CoroutineScope|CoroutineScope(kotlin.coroutines.CoroutineContext){}[0] +final fun kotlinx.coroutines/Job(kotlinx.coroutines/Job? = ...): kotlinx.coroutines/CompletableJob // kotlinx.coroutines/Job|Job(kotlinx.coroutines.Job?){}[0] +final fun kotlinx.coroutines/Job0(kotlinx.coroutines/Job? = ...): kotlinx.coroutines/Job // kotlinx.coroutines/Job0|Job0(kotlinx.coroutines.Job?){}[0] +final fun kotlinx.coroutines/MainScope(): kotlinx.coroutines/CoroutineScope // kotlinx.coroutines/MainScope|MainScope(){}[0] +final fun kotlinx.coroutines/SupervisorJob(kotlinx.coroutines/Job? = ...): kotlinx.coroutines/CompletableJob // kotlinx.coroutines/SupervisorJob|SupervisorJob(kotlinx.coroutines.Job?){}[0] +final fun kotlinx.coroutines/SupervisorJob0(kotlinx.coroutines/Job? = ...): kotlinx.coroutines/Job // kotlinx.coroutines/SupervisorJob0|SupervisorJob0(kotlinx.coroutines.Job?){}[0] +final fun kotlinx.coroutines/handleCoroutineException(kotlin.coroutines/CoroutineContext, kotlin/Throwable) // kotlinx.coroutines/handleCoroutineException|handleCoroutineException(kotlin.coroutines.CoroutineContext;kotlin.Throwable){}[0] +final fun kotlinx.coroutines/launch(kotlin.coroutines/CoroutineContext = ..., kotlinx.coroutines/CoroutineStart = ..., kotlin.coroutines/SuspendFunction1): kotlinx.coroutines/Job // kotlinx.coroutines/launch|launch(kotlin.coroutines.CoroutineContext;kotlinx.coroutines.CoroutineStart;kotlin.coroutines.SuspendFunction1){}[0] +final inline fun <#A: kotlin/Any?, #B: kotlin/Any> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/mapNotNull(crossinline kotlin.coroutines/SuspendFunction1<#A, #B?>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/mapNotNull|mapNotNull@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,0:1?>){0§;1§}[0] +final inline fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.channels/BroadcastChannel<#A>).kotlinx.coroutines.channels/consume(kotlin/Function1, #B>): #B // kotlinx.coroutines.channels/consume|consume@kotlinx.coroutines.channels.BroadcastChannel<0:0>(kotlin.Function1,0:1>){0§;1§}[0] +final inline fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/consume(kotlin/Function1, #B>): #B // kotlinx.coroutines.channels/consume|consume@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.Function1,0:1>){0§;1§}[0] +final inline fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/flatMapLatest(crossinline kotlin.coroutines/SuspendFunction1<#A, kotlinx.coroutines.flow/Flow<#B>>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/flatMapLatest|flatMapLatest@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlinx.coroutines.flow.Flow<0:1>>){0§;1§}[0] +final inline fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/map(crossinline kotlin.coroutines/SuspendFunction1<#A, #B>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/map|map@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,0:1>){0§;1§}[0] +final inline fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/transform(crossinline kotlin.coroutines/SuspendFunction2, #A, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/transform|transform@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction2,0:0,kotlin.Unit>){0§;1§}[0] +final inline fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/unsafeTransform(crossinline kotlin.coroutines/SuspendFunction2, #A, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/unsafeTransform|unsafeTransform@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction2,0:0,kotlin.Unit>){0§;1§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ChannelResult<#A>).kotlinx.coroutines.channels/getOrElse(kotlin/Function1): #A // kotlinx.coroutines.channels/getOrElse|getOrElse@kotlinx.coroutines.channels.ChannelResult<0:0>(kotlin.Function1){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ChannelResult<#A>).kotlinx.coroutines.channels/onClosed(kotlin/Function1): kotlinx.coroutines.channels/ChannelResult<#A> // kotlinx.coroutines.channels/onClosed|onClosed@kotlinx.coroutines.channels.ChannelResult<0:0>(kotlin.Function1){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ChannelResult<#A>).kotlinx.coroutines.channels/onFailure(kotlin/Function1): kotlinx.coroutines.channels/ChannelResult<#A> // kotlinx.coroutines.channels/onFailure|onFailure@kotlinx.coroutines.channels.ChannelResult<0:0>(kotlin.Function1){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ChannelResult<#A>).kotlinx.coroutines.channels/onSuccess(kotlin/Function1<#A, kotlin/Unit>): kotlinx.coroutines.channels/ChannelResult<#A> // kotlinx.coroutines.channels/onSuccess|onSuccess@kotlinx.coroutines.channels.ChannelResult<0:0>(kotlin.Function1<0:0,kotlin.Unit>){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/filter(crossinline kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/filter|filter@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/filterNot(crossinline kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/filterNot|filterNot@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/MutableStateFlow<#A>).kotlinx.coroutines.flow/getAndUpdate(kotlin/Function1<#A, #A>): #A // kotlinx.coroutines.flow/getAndUpdate|getAndUpdate@kotlinx.coroutines.flow.MutableStateFlow<0:0>(kotlin.Function1<0:0,0:0>){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/MutableStateFlow<#A>).kotlinx.coroutines.flow/update(kotlin/Function1<#A, #A>) // kotlinx.coroutines.flow/update|update@kotlinx.coroutines.flow.MutableStateFlow<0:0>(kotlin.Function1<0:0,0:0>){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/MutableStateFlow<#A>).kotlinx.coroutines.flow/updateAndGet(kotlin/Function1<#A, #A>): #A // kotlinx.coroutines.flow/updateAndGet|updateAndGet@kotlinx.coroutines.flow.MutableStateFlow<0:0>(kotlin.Function1<0:0,0:0>){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/catch(noinline kotlin.coroutines/SuspendFunction2, kotlin/Throwable, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/catch|catch@kotlinx.coroutines.flow.SharedFlow<0:0>(kotlin.coroutines.SuspendFunction2,kotlin.Throwable,kotlin.Unit>){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/retry(kotlin/Long = ..., noinline kotlin.coroutines/SuspendFunction1 = ...): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/retry|retry@kotlinx.coroutines.flow.SharedFlow<0:0>(kotlin.Long;kotlin.coroutines.SuspendFunction1){0§}[0] +final inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/retryWhen(noinline kotlin.coroutines/SuspendFunction3, kotlin/Throwable, kotlin/Long, kotlin/Boolean>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/retryWhen|retryWhen@kotlinx.coroutines.flow.SharedFlow<0:0>(kotlin.coroutines.SuspendFunction3,kotlin.Throwable,kotlin.Long,kotlin.Boolean>){0§}[0] +final inline fun <#A: kotlin/Any?> kotlinx.coroutines.flow.internal/unsafeFlow(crossinline kotlin.coroutines/SuspendFunction1, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow.internal/unsafeFlow|unsafeFlow(kotlin.coroutines.SuspendFunction1,kotlin.Unit>){0§}[0] +final inline fun <#A: reified kotlin/Any?, #B: kotlin/Any?> kotlinx.coroutines.flow/combine(kotlin.collections/Iterable>, crossinline kotlin.coroutines/SuspendFunction1, #B>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/combine|combine(kotlin.collections.Iterable>;kotlin.coroutines.SuspendFunction1,0:1>){0§;1§}[0] +final inline fun <#A: reified kotlin/Any?, #B: kotlin/Any?> kotlinx.coroutines.flow/combine(kotlin/Array>..., crossinline kotlin.coroutines/SuspendFunction1, #B>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/combine|combine(kotlin.Array>...;kotlin.coroutines.SuspendFunction1,0:1>){0§;1§}[0] +final inline fun <#A: reified kotlin/Any?, #B: kotlin/Any?> kotlinx.coroutines.flow/combineTransform(kotlin.collections/Iterable>, crossinline kotlin.coroutines/SuspendFunction2, kotlin/Array<#A>, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/combineTransform|combineTransform(kotlin.collections.Iterable>;kotlin.coroutines.SuspendFunction2,kotlin.Array<0:0>,kotlin.Unit>){0§;1§}[0] +final inline fun <#A: reified kotlin/Any?, #B: kotlin/Any?> kotlinx.coroutines.flow/combineTransform(kotlin/Array>..., crossinline kotlin.coroutines/SuspendFunction2, kotlin/Array<#A>, kotlin/Unit>): kotlinx.coroutines.flow/Flow<#B> // kotlinx.coroutines.flow/combineTransform|combineTransform(kotlin.Array>...;kotlin.coroutines.SuspendFunction2,kotlin.Array<0:0>,kotlin.Unit>){0§;1§}[0] +final inline fun <#A: reified kotlin/Any?> (kotlinx.coroutines.flow/Flow<*>).kotlinx.coroutines.flow/filterIsInstance(): kotlinx.coroutines.flow/Flow<#A> // kotlinx.coroutines.flow/filterIsInstance|filterIsInstance@kotlinx.coroutines.flow.Flow<*>(){0§}[0] +final inline fun kotlinx.coroutines.flow.internal/checkIndexOverflow(kotlin/Int): kotlin/Int // kotlinx.coroutines.flow.internal/checkIndexOverflow|checkIndexOverflow(kotlin.Int){}[0] +final inline fun kotlinx.coroutines/CoroutineExceptionHandler(crossinline kotlin/Function2): kotlinx.coroutines/CoroutineExceptionHandler // kotlinx.coroutines/CoroutineExceptionHandler|CoroutineExceptionHandler(kotlin.Function2){}[0] +final inline fun kotlinx.coroutines/Runnable(crossinline kotlin/Function0): kotlinx.coroutines/Runnable // kotlinx.coroutines/Runnable|Runnable(kotlin.Function0){}[0] +final suspend fun (kotlin.collections/Collection).kotlinx.coroutines/joinAll() // kotlinx.coroutines/joinAll|joinAll@kotlin.collections.Collection(){}[0] +final suspend fun (kotlinx.coroutines.channels/ProducerScope<*>).kotlinx.coroutines.channels/awaitClose(kotlin/Function0 = ...) // kotlinx.coroutines.channels/awaitClose|awaitClose@kotlinx.coroutines.channels.ProducerScope<*>(kotlin.Function0){}[0] +final suspend fun (kotlinx.coroutines.flow/Flow<*>).kotlinx.coroutines.flow/collect() // kotlinx.coroutines.flow/collect|collect@kotlinx.coroutines.flow.Flow<*>(){}[0] +final suspend fun (kotlinx.coroutines/Job).kotlinx.coroutines/cancelAndJoin() // kotlinx.coroutines/cancelAndJoin|cancelAndJoin@kotlinx.coroutines.Job(){}[0] +final suspend fun <#A: kotlin/Any, #B: kotlin.collections/MutableCollection> (kotlinx.coroutines.channels/ReceiveChannel<#A?>).kotlinx.coroutines.channels/filterNotNullTo(#B): #B // kotlinx.coroutines.channels/filterNotNullTo|filterNotNullTo@kotlinx.coroutines.channels.ReceiveChannel<0:0?>(0:1){0§;1§>}[0] +final suspend fun <#A: kotlin/Any, #B: kotlinx.coroutines.channels/SendChannel<#A>> (kotlinx.coroutines.channels/ReceiveChannel<#A?>).kotlinx.coroutines.channels/filterNotNullTo(#B): #B // kotlinx.coroutines.channels/filterNotNullTo|filterNotNullTo@kotlinx.coroutines.channels.ReceiveChannel<0:0?>(0:1){0§;1§>}[0] +final suspend fun <#A: kotlin/Any> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/receiveOrNull(): #A? // kotlinx.coroutines.channels/receiveOrNull|receiveOrNull@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?, #B: #A> (kotlinx.coroutines.flow/Flow<#B>).kotlinx.coroutines.flow/reduce(kotlin.coroutines/SuspendFunction2<#A, #B, #A>): #A // kotlinx.coroutines.flow/reduce|reduce@kotlinx.coroutines.flow.Flow<0:1>(kotlin.coroutines.SuspendFunction2<0:0,0:1,0:0>){0§;1§<0:0>}[0] +final suspend fun <#A: kotlin/Any?, #B: kotlin.collections/MutableCollection> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/toCollection(#B): #B // kotlinx.coroutines.channels/toCollection|toCollection@kotlinx.coroutines.channels.ReceiveChannel<0:0>(0:1){0§;1§>}[0] +final suspend fun <#A: kotlin/Any?, #B: kotlin.collections/MutableCollection> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/toCollection(#B): #B // kotlinx.coroutines.flow/toCollection|toCollection@kotlinx.coroutines.flow.Flow<0:0>(0:1){0§;1§>}[0] +final suspend fun <#A: kotlin/Any?, #B: kotlin/Any?, #C: kotlin.collections/MutableMap> (kotlinx.coroutines.channels/ReceiveChannel>).kotlinx.coroutines.channels/toMap(#C): #C // kotlinx.coroutines.channels/toMap|toMap@kotlinx.coroutines.channels.ReceiveChannel>(0:2){0§;1§;2§>}[0] +final suspend fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel>).kotlinx.coroutines.channels/toMap(): kotlin.collections/Map<#A, #B> // kotlinx.coroutines.channels/toMap|toMap@kotlinx.coroutines.channels.ReceiveChannel>(){0§;1§}[0] +final suspend fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/FlowCollector<#A>).kotlinx.coroutines.flow.internal/combineInternal(kotlin/Array>, kotlin/Function0?>, kotlin.coroutines/SuspendFunction2, kotlin/Array<#B>, kotlin/Unit>) // kotlinx.coroutines.flow.internal/combineInternal|combineInternal@kotlinx.coroutines.flow.FlowCollector<0:0>(kotlin.Array>;kotlin.Function0?>;kotlin.coroutines.SuspendFunction2,kotlin.Array<0:1>,kotlin.Unit>){0§;1§}[0] +final suspend fun <#A: kotlin/Any?, #B: kotlinx.coroutines.channels/SendChannel<#A>> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/toChannel(#B): #B // kotlinx.coroutines.channels/toChannel|toChannel@kotlinx.coroutines.channels.ReceiveChannel<0:0>(0:1){0§;1§>}[0] +final suspend fun <#A: kotlin/Any?> (kotlin.collections/Collection>).kotlinx.coroutines/awaitAll(): kotlin.collections/List<#A> // kotlinx.coroutines/awaitAll|awaitAll@kotlin.collections.Collection>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/any(): kotlin/Boolean // kotlinx.coroutines.channels/any|any@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/count(): kotlin/Int // kotlinx.coroutines.channels/count|count@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/elementAt(kotlin/Int): #A // kotlinx.coroutines.channels/elementAt|elementAt@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.Int){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/elementAtOrNull(kotlin/Int): #A? // kotlinx.coroutines.channels/elementAtOrNull|elementAtOrNull@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.Int){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/first(): #A // kotlinx.coroutines.channels/first|first@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/firstOrNull(): #A? // kotlinx.coroutines.channels/firstOrNull|firstOrNull@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/indexOf(#A): kotlin/Int // kotlinx.coroutines.channels/indexOf|indexOf@kotlinx.coroutines.channels.ReceiveChannel<0:0>(0:0){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/last(): #A // kotlinx.coroutines.channels/last|last@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/lastIndexOf(#A): kotlin/Int // kotlinx.coroutines.channels/lastIndexOf|lastIndexOf@kotlinx.coroutines.channels.ReceiveChannel<0:0>(0:0){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/lastOrNull(): #A? // kotlinx.coroutines.channels/lastOrNull|lastOrNull@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/maxWith(kotlin/Comparator): #A? // kotlinx.coroutines.channels/maxWith|maxWith@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.Comparator){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/minWith(kotlin/Comparator): #A? // kotlinx.coroutines.channels/minWith|minWith@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.Comparator){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/none(): kotlin/Boolean // kotlinx.coroutines.channels/none|none@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/single(): #A // kotlinx.coroutines.channels/single|single@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/singleOrNull(): #A? // kotlinx.coroutines.channels/singleOrNull|singleOrNull@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/toList(): kotlin.collections/List<#A> // kotlinx.coroutines.channels/toList|toList@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/toMutableList(): kotlin.collections/MutableList<#A> // kotlinx.coroutines.channels/toMutableList|toMutableList@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/toMutableSet(): kotlin.collections/MutableSet<#A> // kotlinx.coroutines.channels/toMutableSet|toMutableSet@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/toSet(): kotlin.collections/Set<#A> // kotlinx.coroutines.channels/toSet|toSet@kotlinx.coroutines.channels.ReceiveChannel<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/all(kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlin/Boolean // kotlinx.coroutines.flow/all|all@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/any(kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlin/Boolean // kotlinx.coroutines.flow/any|any@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/collectLatest(kotlin.coroutines/SuspendFunction1<#A, kotlin/Unit>) // kotlinx.coroutines.flow/collectLatest|collectLatest@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Unit>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/count(): kotlin/Int // kotlinx.coroutines.flow/count|count@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/count(kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlin/Int // kotlinx.coroutines.flow/count|count@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/first(): #A // kotlinx.coroutines.flow/first|first@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/first(kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): #A // kotlinx.coroutines.flow/first|first@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/firstOrNull(): #A? // kotlinx.coroutines.flow/firstOrNull|firstOrNull@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/firstOrNull(kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): #A? // kotlinx.coroutines.flow/firstOrNull|firstOrNull@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/last(): #A // kotlinx.coroutines.flow/last|last@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/lastOrNull(): #A? // kotlinx.coroutines.flow/lastOrNull|lastOrNull@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/none(kotlin.coroutines/SuspendFunction1<#A, kotlin/Boolean>): kotlin/Boolean // kotlinx.coroutines.flow/none|none@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Boolean>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/single(): #A // kotlinx.coroutines.flow/single|single@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/singleOrNull(): #A? // kotlinx.coroutines.flow/singleOrNull|singleOrNull@kotlinx.coroutines.flow.Flow<0:0>(){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/stateIn(kotlinx.coroutines/CoroutineScope): kotlinx.coroutines.flow/StateFlow<#A> // kotlinx.coroutines.flow/stateIn|stateIn@kotlinx.coroutines.flow.Flow<0:0>(kotlinx.coroutines.CoroutineScope){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/toList(kotlin.collections/MutableList<#A> = ...): kotlin.collections/List<#A> // kotlinx.coroutines.flow/toList|toList@kotlinx.coroutines.flow.Flow<0:0>(kotlin.collections.MutableList<0:0>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/toSet(kotlin.collections/MutableSet<#A> = ...): kotlin.collections/Set<#A> // kotlinx.coroutines.flow/toSet|toSet@kotlinx.coroutines.flow.Flow<0:0>(kotlin.collections.MutableSet<0:0>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/FlowCollector<#A>).kotlinx.coroutines.flow/emitAll(kotlinx.coroutines.channels/ReceiveChannel<#A>) // kotlinx.coroutines.flow/emitAll|emitAll@kotlinx.coroutines.flow.FlowCollector<0:0>(kotlinx.coroutines.channels.ReceiveChannel<0:0>){0§}[0] +final suspend fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/FlowCollector<#A>).kotlinx.coroutines.flow/emitAll(kotlinx.coroutines.flow/Flow<#A>) // kotlinx.coroutines.flow/emitAll|emitAll@kotlinx.coroutines.flow.FlowCollector<0:0>(kotlinx.coroutines.flow.Flow<0:0>){0§}[0] +final suspend fun <#A: kotlin/Any?> kotlinx.coroutines/awaitAll(kotlin/Array>...): kotlin.collections/List<#A> // kotlinx.coroutines/awaitAll|awaitAll(kotlin.Array>...){0§}[0] +final suspend fun <#A: kotlin/Any?> kotlinx.coroutines/coroutineScope(kotlin.coroutines/SuspendFunction1): #A // kotlinx.coroutines/coroutineScope|coroutineScope(kotlin.coroutines.SuspendFunction1){0§}[0] +final suspend fun <#A: kotlin/Any?> kotlinx.coroutines/supervisorScope(kotlin.coroutines/SuspendFunction1): #A // kotlinx.coroutines/supervisorScope|supervisorScope(kotlin.coroutines.SuspendFunction1){0§}[0] +final suspend fun <#A: kotlin/Any?> kotlinx.coroutines/withContext(kotlin.coroutines/CoroutineContext, kotlin.coroutines/SuspendFunction1): #A // kotlinx.coroutines/withContext|withContext(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1){0§}[0] +final suspend fun <#A: kotlin/Any?> kotlinx.coroutines/withTimeout(kotlin.time/Duration, kotlin.coroutines/SuspendFunction1): #A // kotlinx.coroutines/withTimeout|withTimeout(kotlin.time.Duration;kotlin.coroutines.SuspendFunction1){0§}[0] +final suspend fun <#A: kotlin/Any?> kotlinx.coroutines/withTimeout(kotlin/Long, kotlin.coroutines/SuspendFunction1): #A // kotlinx.coroutines/withTimeout|withTimeout(kotlin.Long;kotlin.coroutines.SuspendFunction1){0§}[0] +final suspend fun <#A: kotlin/Any?> kotlinx.coroutines/withTimeoutOrNull(kotlin.time/Duration, kotlin.coroutines/SuspendFunction1): #A? // kotlinx.coroutines/withTimeoutOrNull|withTimeoutOrNull(kotlin.time.Duration;kotlin.coroutines.SuspendFunction1){0§}[0] +final suspend fun <#A: kotlin/Any?> kotlinx.coroutines/withTimeoutOrNull(kotlin/Long, kotlin.coroutines/SuspendFunction1): #A? // kotlinx.coroutines/withTimeoutOrNull|withTimeoutOrNull(kotlin.Long;kotlin.coroutines.SuspendFunction1){0§}[0] +final suspend fun kotlinx.coroutines/awaitCancellation(): kotlin/Nothing // kotlinx.coroutines/awaitCancellation|awaitCancellation(){}[0] +final suspend fun kotlinx.coroutines/delay(kotlin.time/Duration) // kotlinx.coroutines/delay|delay(kotlin.time.Duration){}[0] +final suspend fun kotlinx.coroutines/delay(kotlin/Long) // kotlinx.coroutines/delay|delay(kotlin.Long){}[0] +final suspend fun kotlinx.coroutines/joinAll(kotlin/Array...) // kotlinx.coroutines/joinAll|joinAll(kotlin.Array...){}[0] +final suspend fun kotlinx.coroutines/yield() // kotlinx.coroutines/yield|yield(){}[0] +final suspend inline fun <#A: kotlin/Any?, #B: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/fold(#B, crossinline kotlin.coroutines/SuspendFunction2<#B, #A, #B>): #B // kotlinx.coroutines.flow/fold|fold@kotlinx.coroutines.flow.Flow<0:0>(0:1;kotlin.coroutines.SuspendFunction2<0:1,0:0,0:1>){0§;1§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/BroadcastChannel<#A>).kotlinx.coroutines.channels/consumeEach(kotlin/Function1<#A, kotlin/Unit>) // kotlinx.coroutines.channels/consumeEach|consumeEach@kotlinx.coroutines.channels.BroadcastChannel<0:0>(kotlin.Function1<0:0,kotlin.Unit>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/ReceiveChannel<#A>).kotlinx.coroutines.channels/consumeEach(kotlin/Function1<#A, kotlin/Unit>) // kotlinx.coroutines.channels/consumeEach|consumeEach@kotlinx.coroutines.channels.ReceiveChannel<0:0>(kotlin.Function1<0:0,kotlin.Unit>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/collect(crossinline kotlin.coroutines/SuspendFunction1<#A, kotlin/Unit>) // kotlinx.coroutines.flow/collect|collect@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction1<0:0,kotlin.Unit>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/Flow<#A>).kotlinx.coroutines.flow/collectIndexed(crossinline kotlin.coroutines/SuspendFunction2) // kotlinx.coroutines.flow/collectIndexed|collectIndexed@kotlinx.coroutines.flow.Flow<0:0>(kotlin.coroutines.SuspendFunction2){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/count(): kotlin/Int // kotlinx.coroutines.flow/count|count@kotlinx.coroutines.flow.SharedFlow<0:0>(){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/toList(): kotlin.collections/List<#A> // kotlinx.coroutines.flow/toList|toList@kotlinx.coroutines.flow.SharedFlow<0:0>(){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/toList(kotlin.collections/MutableList<#A>): kotlin/Nothing // kotlinx.coroutines.flow/toList|toList@kotlinx.coroutines.flow.SharedFlow<0:0>(kotlin.collections.MutableList<0:0>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/toSet(): kotlin.collections/Set<#A> // kotlinx.coroutines.flow/toSet|toSet@kotlinx.coroutines.flow.SharedFlow<0:0>(){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.flow/SharedFlow<#A>).kotlinx.coroutines.flow/toSet(kotlin.collections/MutableSet<#A>): kotlin/Nothing // kotlinx.coroutines.flow/toSet|toSet@kotlinx.coroutines.flow.SharedFlow<0:0>(kotlin.collections.MutableSet<0:0>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.sync/Mutex).kotlinx.coroutines.sync/withLock(kotlin/Any? = ..., kotlin/Function0<#A>): #A // kotlinx.coroutines.sync/withLock|withLock@kotlinx.coroutines.sync.Mutex(kotlin.Any?;kotlin.Function0<0:0>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines.sync/Semaphore).kotlinx.coroutines.sync/withPermit(kotlin/Function0<#A>): #A // kotlinx.coroutines.sync/withPermit|withPermit@kotlinx.coroutines.sync.Semaphore(kotlin.Function0<0:0>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> (kotlinx.coroutines/CoroutineDispatcher).kotlinx.coroutines/invoke(noinline kotlin.coroutines/SuspendFunction1): #A // kotlinx.coroutines/invoke|invoke@kotlinx.coroutines.CoroutineDispatcher(kotlin.coroutines.SuspendFunction1){0§}[0] +final suspend inline fun <#A: kotlin/Any?> kotlinx.coroutines.selects/select(crossinline kotlin/Function1, kotlin/Unit>): #A // kotlinx.coroutines.selects/select|select(kotlin.Function1,kotlin.Unit>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> kotlinx.coroutines.selects/selectOld(crossinline kotlin/Function1, kotlin/Unit>): #A // kotlinx.coroutines.selects/selectOld|selectOld(kotlin.Function1,kotlin.Unit>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> kotlinx.coroutines.selects/selectUnbiased(crossinline kotlin/Function1, kotlin/Unit>): #A // kotlinx.coroutines.selects/selectUnbiased|selectUnbiased(kotlin.Function1,kotlin.Unit>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> kotlinx.coroutines.selects/selectUnbiasedOld(crossinline kotlin/Function1, kotlin/Unit>): #A // kotlinx.coroutines.selects/selectUnbiasedOld|selectUnbiasedOld(kotlin.Function1,kotlin.Unit>){0§}[0] +final suspend inline fun <#A: kotlin/Any?> kotlinx.coroutines/suspendCancellableCoroutine(crossinline kotlin/Function1, kotlin/Unit>): #A // kotlinx.coroutines/suspendCancellableCoroutine|suspendCancellableCoroutine(kotlin.Function1,kotlin.Unit>){0§}[0] +final suspend inline fun kotlinx.coroutines.selects/whileSelect(crossinline kotlin/Function1, kotlin/Unit>) // kotlinx.coroutines.selects/whileSelect|whileSelect(kotlin.Function1,kotlin.Unit>){}[0] +final suspend inline fun kotlinx.coroutines/currentCoroutineContext(): kotlin.coroutines/CoroutineContext // kotlinx.coroutines/currentCoroutineContext|currentCoroutineContext(){}[0] + +// Targets: [native] +final val kotlinx.coroutines/IO // kotlinx.coroutines/IO|@kotlinx.coroutines.Dispatchers{}IO[0] + final fun (kotlinx.coroutines/Dispatchers).(): kotlinx.coroutines/CoroutineDispatcher // kotlinx.coroutines/IO.|@kotlinx.coroutines.Dispatchers(){}[0] + +// Targets: [native] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/SendChannel<#A>).kotlinx.coroutines.channels/sendBlocking(#A) // kotlinx.coroutines.channels/sendBlocking|sendBlocking@kotlinx.coroutines.channels.SendChannel<0:0>(0:0){0§}[0] + +// Targets: [native] +final fun <#A: kotlin/Any?> (kotlinx.coroutines.channels/SendChannel<#A>).kotlinx.coroutines.channels/trySendBlocking(#A): kotlinx.coroutines.channels/ChannelResult // kotlinx.coroutines.channels/trySendBlocking|trySendBlocking@kotlinx.coroutines.channels.SendChannel<0:0>(0:0){0§}[0] + +// Targets: [native] +final fun <#A: kotlin/Any?> kotlinx.coroutines/runBlocking(kotlin.coroutines/CoroutineContext = ..., kotlin.coroutines/SuspendFunction1): #A // kotlinx.coroutines/runBlocking|runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1){0§}[0] + +// Targets: [native] +final fun kotlinx.coroutines/newFixedThreadPoolContext(kotlin/Int, kotlin/String): kotlinx.coroutines/CloseableCoroutineDispatcher // kotlinx.coroutines/newFixedThreadPoolContext|newFixedThreadPoolContext(kotlin.Int;kotlin.String){}[0] + +// Targets: [native] +final fun kotlinx.coroutines/newSingleThreadContext(kotlin/String): kotlinx.coroutines/CloseableCoroutineDispatcher // kotlinx.coroutines/newSingleThreadContext|newSingleThreadContext(kotlin.String){}[0] + +// Targets: [js] +final fun (org.w3c.dom/Window).kotlinx.coroutines/asCoroutineDispatcher(): kotlinx.coroutines/CoroutineDispatcher // kotlinx.coroutines/asCoroutineDispatcher|asCoroutineDispatcher@org.w3c.dom.Window(){}[0] + +// Targets: [js] +final fun <#A: kotlin/Any?> (kotlin.js/Promise<#A>).kotlinx.coroutines/asDeferred(): kotlinx.coroutines/Deferred<#A> // kotlinx.coroutines/asDeferred|asDeferred@kotlin.js.Promise<0:0>(){0§}[0] + +// Targets: [js] +final fun <#A: kotlin/Any?> (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines/promise(kotlin.coroutines/CoroutineContext = ..., kotlinx.coroutines/CoroutineStart = ..., kotlin.coroutines/SuspendFunction1): kotlin.js/Promise<#A> // kotlinx.coroutines/promise|promise@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.CoroutineContext;kotlinx.coroutines.CoroutineStart;kotlin.coroutines.SuspendFunction1){0§}[0] + +// Targets: [js] +final fun <#A: kotlin/Any?> (kotlinx.coroutines/Deferred<#A>).kotlinx.coroutines/asPromise(): kotlin.js/Promise<#A> // kotlinx.coroutines/asPromise|asPromise@kotlinx.coroutines.Deferred<0:0>(){0§}[0] + +// Targets: [js] +final suspend fun (org.w3c.dom/Window).kotlinx.coroutines/awaitAnimationFrame(): kotlin/Double // kotlinx.coroutines/awaitAnimationFrame|awaitAnimationFrame@org.w3c.dom.Window(){}[0] + +// Targets: [js] +final suspend fun <#A: kotlin/Any?> (kotlin.js/Promise<#A>).kotlinx.coroutines/await(): #A // kotlinx.coroutines/await|await@kotlin.js.Promise<0:0>(){0§}[0] + +// Targets: [wasmJs] +final fun <#A: kotlin/Any?> (kotlin.js/Promise).kotlinx.coroutines/asDeferred(): kotlinx.coroutines/Deferred<#A> // kotlinx.coroutines/asDeferred|asDeferred@kotlin.js.Promise(){0§}[0] + +// Targets: [wasmJs] +final fun <#A: kotlin/Any?> (kotlinx.coroutines/CoroutineScope).kotlinx.coroutines/promise(kotlin.coroutines/CoroutineContext = ..., kotlinx.coroutines/CoroutineStart = ..., kotlin.coroutines/SuspendFunction1): kotlin.js/Promise // kotlinx.coroutines/promise|promise@kotlinx.coroutines.CoroutineScope(kotlin.coroutines.CoroutineContext;kotlinx.coroutines.CoroutineStart;kotlin.coroutines.SuspendFunction1){0§}[0] + +// Targets: [wasmJs] +final fun <#A: kotlin/Any?> (kotlinx.coroutines/Deferred<#A>).kotlinx.coroutines/asPromise(): kotlin.js/Promise // kotlinx.coroutines/asPromise|asPromise@kotlinx.coroutines.Deferred<0:0>(){0§}[0] + +// Targets: [wasmJs] +final suspend fun <#A: kotlin/Any?> (kotlin.js/Promise).kotlinx.coroutines/await(): #A // kotlinx.coroutines/await|await@kotlin.js.Promise(){0§}[0] diff --git a/kotlinx-coroutines-core/benchmarks/README.md b/kotlinx-coroutines-core/benchmarks/README.md new file mode 100644 index 0000000000..5ea19735b8 --- /dev/null +++ b/kotlinx-coroutines-core/benchmarks/README.md @@ -0,0 +1,21 @@ +## kotlinx-coroutines-core benchmarks + +Multiplatform benchmarks for kotlinx-coroutines-core. + +This source-set contains benchmarks that leverage `internal` API (e.g. `suspendCancellableCoroutineReusable`) or +that are multiplatform (-> only supported with `kotlinx-benchmarks` which is less convenient than `jmh` plugin). +For JVM-only non-internal benchmarks, consider using `benchmarks` top-level project. + +### Usage + +``` +// JVM only +./gradlew :kotlinx-coroutines-core:jvmBenchmarkBenchmarkJar +java -jar kotlinx-coroutines-core/build/benchmarks/jvmBenchmark/jars/kotlinx-coroutines-core-jvmBenchmark-jmh-*-JMH.jar + +// Native, OS X +./gradlew :kotlinx-coroutines-core:macosArm64BenchmarkBenchmark + +// Figure out what to use +./gradlew :kotlinx-coroutines-core:tasks | grep -i bench +``` diff --git a/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/BenchmarkUtils.kt b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/BenchmarkUtils.kt new file mode 100644 index 0000000000..f0fdd82464 --- /dev/null +++ b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/BenchmarkUtils.kt @@ -0,0 +1,13 @@ +package kotlinx.coroutines + +import java.util.concurrent.* + +public fun doGeomDistrWork(work: Int) { + // We use geometric distribution here. We also checked on macbook pro 13" (2017) that the resulting work times + // are distributed geometrically, see https://github.com/Kotlin/kotlinx.coroutines/pull/1464#discussion_r355705325 + val p = 1.0 / work + val r = ThreadLocalRandom.current() + while (true) { + if (r.nextDouble() < p) break + } +} diff --git a/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/SemaphoreBenchmark.kt b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/SemaphoreBenchmark.kt new file mode 100644 index 0000000000..3135874578 --- /dev/null +++ b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/SemaphoreBenchmark.kt @@ -0,0 +1,84 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.scheduling.* +import kotlinx.coroutines.sync.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +@Warmup(iterations = 3, time = 500, timeUnit = TimeUnit.MICROSECONDS) +@Measurement(iterations = 10, time = 500, timeUnit = TimeUnit.MICROSECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class SemaphoreBenchmark { + @Param + private var _1_dispatcher: SemaphoreBenchDispatcherCreator = SemaphoreBenchDispatcherCreator.DEFAULT + + @Param("0", "1000") + private var _2_coroutines: Int = 0 + + @Param("1", "2", "4", "8", "32", "128", "100000") + private var _3_maxPermits: Int = 0 + + @Param("1", "2", "4", "8", "16") // local machine +// @Param("1", "2", "4", "8", "16", "32", "64", "128") // Server + private var _4_parallelism: Int = 0 + + private lateinit var dispatcher: CoroutineDispatcher + private var coroutines = 0 + + @InternalCoroutinesApi + @Setup + fun setup() { + dispatcher = _1_dispatcher.create(_4_parallelism) + coroutines = if (_2_coroutines == 0) _4_parallelism else _2_coroutines + } + + @Benchmark + fun semaphore() = runBlocking { + val n = BATCH_SIZE / coroutines + val semaphore = Semaphore(_3_maxPermits) + val jobs = ArrayList(coroutines) + repeat(coroutines) { + jobs += GlobalScope.launch { + repeat(n) { + semaphore.withPermit { + doGeomDistrWork(WORK_INSIDE) + } + doGeomDistrWork(WORK_OUTSIDE) + } + } + } + jobs.forEach { it.join() } + } + + @Benchmark + fun channelAsSemaphore() = runBlocking { + val n = BATCH_SIZE / coroutines + val semaphore = Channel(_3_maxPermits) + val jobs = ArrayList(coroutines) + repeat(coroutines) { + jobs += GlobalScope.launch { + repeat(n) { + semaphore.send(Unit) // acquire + doGeomDistrWork(WORK_INSIDE) + semaphore.receive() // release + doGeomDistrWork(WORK_OUTSIDE) + } + } + } + jobs.forEach { it.join() } + } +} + +enum class SemaphoreBenchDispatcherCreator(val create: (parallelism: Int) -> CoroutineDispatcher) { + FORK_JOIN({ parallelism -> ForkJoinPool(parallelism).asCoroutineDispatcher() }), + DEFAULT({ parallelism -> CoroutineScheduler(corePoolSize = parallelism, maxPoolSize = parallelism).asCoroutineDispatcher() }) +} + +private const val WORK_INSIDE = 50 +private const val WORK_OUTSIDE = 50 +private const val BATCH_SIZE = 100000 diff --git a/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/ChannelProducerConsumerBenchmark.kt b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/ChannelProducerConsumerBenchmark.kt new file mode 100644 index 0000000000..1e41e0bf63 --- /dev/null +++ b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/ChannelProducerConsumerBenchmark.kt @@ -0,0 +1,150 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.Channel +import kotlinx.coroutines.scheduling.* +import kotlinx.coroutines.selects.select +import org.openjdk.jmh.annotations.* +import java.lang.Integer.max +import java.util.concurrent.ForkJoinPool +import java.util.concurrent.Phaser +import java.util.concurrent.TimeUnit + + +/** + * Benchmark to measure channel algorithm performance in terms of average time per `send-receive` pair; + * actually, it measures the time for a batch of such operations separated into the specified number of consumers/producers. + * It uses different channels (rendezvous, buffered, unlimited; see [ChannelCreator]) and different dispatchers + * (see [DispatcherCreator]). If the [_3_withSelect] property is set, it invokes `send` and + * `receive` via [select], waiting on a local dummy channel simultaneously, simulating a "cancellation" channel. + * + * Please, be patient, this benchmark takes quite a lot of time to complete. + */ +@Warmup(iterations = 3, time = 500, timeUnit = TimeUnit.MICROSECONDS) +@Measurement(iterations = 20, time = 500, timeUnit = TimeUnit.MICROSECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.Throughput) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +open class ChannelProducerConsumerBenchmark { + @Param + private var _0_dispatcher: DispatcherCreator = DispatcherCreator.DEFAULT + + @Param + private var _1_channel: ChannelCreator = ChannelCreator.RENDEZVOUS + + @Param("0", "1000") + private var _2_coroutines: Int = 0 + + @Param("false", "true") + private var _3_withSelect: Boolean = false + + @Param("1", "2", "4", "8", "16") // local machine +// @Param("1", "2", "4", "8", "16", "32", "64", "128") // Server + private var _4_parallelism: Int = 0 + + @Param("50") + private var _5_workSize: Int = 0 + + private lateinit var dispatcher: CoroutineDispatcher + private lateinit var channel: Channel + + @InternalCoroutinesApi + @Setup + fun setup() { + dispatcher = _0_dispatcher.create(_4_parallelism) + channel = _1_channel.create() + } + + @Benchmark + fun mcsp() { + if (_2_coroutines != 0) return + val producers = max(1, _4_parallelism - 1) + val consumers = 1 + run(producers, consumers) + } + + @Benchmark + fun spmc() { + if (_2_coroutines != 0) return + val producers = 1 + val consumers = max(1, _4_parallelism - 1) + run(producers, consumers) + } + + @Benchmark + fun mpmc() { + val producers = if (_2_coroutines == 0) (_4_parallelism + 1) / 2 else _2_coroutines / 2 + val consumers = producers + run(producers, consumers) + } + + private fun run(producers: Int, consumers: Int) { + val n = (APPROX_BATCH_SIZE / producers * producers) / consumers * consumers + val phaser = Phaser(producers + consumers + 1) + // Run producers + repeat(producers) { + GlobalScope.launch(dispatcher) { + val dummy = if (_3_withSelect) _1_channel.create() else null + repeat(n / producers) { + produce(it, dummy) + } + phaser.arrive() + } + } + // Run consumers + repeat(consumers) { + GlobalScope.launch(dispatcher) { + val dummy = if (_3_withSelect) _1_channel.create() else null + repeat(n / consumers) { + consume(dummy) + } + phaser.arrive() + } + } + // Wait until work is done + phaser.arriveAndAwaitAdvance() + } + + private suspend fun produce(element: Int, dummy: Channel?) { + if (_3_withSelect) { + select { + channel.onSend(element) {} + dummy!!.onReceive {} + } + } else { + channel.send(element) + } + doWork(_5_workSize) + } + + private suspend fun consume(dummy: Channel?) { + if (_3_withSelect) { + select { + channel.onReceive {} + dummy!!.onReceive {} + } + } else { + channel.receive() + } + doWork(_5_workSize) + } +} + +enum class DispatcherCreator(val create: (parallelism: Int) -> CoroutineDispatcher) { + FORK_JOIN({ parallelism -> ForkJoinPool(parallelism).asCoroutineDispatcher() }), + DEFAULT({ parallelism -> CoroutineScheduler(corePoolSize = parallelism, maxPoolSize = parallelism).asCoroutineDispatcher() }) +} + +enum class ChannelCreator(private val capacity: Int) { + RENDEZVOUS(Channel.RENDEZVOUS), + BUFFERED_16(16), + BUFFERED_64(64), + BUFFERED_UNLIMITED(Channel.UNLIMITED); + + fun create(): Channel = Channel(capacity) +} + +private fun doWork(workSize: Int): Unit = doGeomDistrWork(workSize) + +private const val APPROX_BATCH_SIZE = 100_000 diff --git a/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/SelectBenchmark.kt b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/SelectBenchmark.kt new file mode 100644 index 0000000000..0df0c5971c --- /dev/null +++ b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/SelectBenchmark.kt @@ -0,0 +1,38 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +@Warmup(iterations = 8, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 8, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class SelectBenchmark { + // 450 + private val iterations = 1000 + + @Benchmark + fun stressSelect() = runBlocking { + val pingPong = Channel() + launch { + repeat(iterations) { + select { + pingPong.onSend(Unit) {} + } + } + } + + launch { + repeat(iterations) { + select { + pingPong.onReceive() {} + } + } + } + } +} diff --git a/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/SimpleChannel.kt b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/SimpleChannel.kt new file mode 100644 index 0000000000..e198ea6934 --- /dev/null +++ b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/SimpleChannel.kt @@ -0,0 +1,91 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +public abstract class SimpleChannel { + companion object { + const val NULL_SURROGATE: Int = -1 + } + + @JvmField + protected var producer: Continuation? = null + @JvmField + protected var enqueuedValue: Int = NULL_SURROGATE + @JvmField + protected var consumer: Continuation? = null + + suspend fun send(element: Int) { + require(element != NULL_SURROGATE) + if (offer(element)) { + return + } + + return suspendSend(element) + } + + private fun offer(element: Int): Boolean { + if (consumer == null) { + return false + } + + consumer!!.resume(element) + consumer = null + return true + } + + suspend fun receive(): Int { + // Cached value + if (enqueuedValue != NULL_SURROGATE) { + val result = enqueuedValue + enqueuedValue = NULL_SURROGATE + producer!!.resume(Unit) + return result + } + + return suspendReceive() + } + + abstract suspend fun suspendReceive(): Int + abstract suspend fun suspendSend(element: Int) +} + +class NonCancellableChannel : SimpleChannel() { + override suspend fun suspendReceive(): Int = suspendCoroutineUninterceptedOrReturn { + consumer = it.intercepted() + COROUTINE_SUSPENDED + } + + override suspend fun suspendSend(element: Int) = suspendCoroutineUninterceptedOrReturn { + enqueuedValue = element + producer = it.intercepted() + COROUTINE_SUSPENDED + } +} + +class CancellableChannel : SimpleChannel() { + override suspend fun suspendReceive(): Int = suspendCancellableCoroutine { + consumer = it.intercepted() + COROUTINE_SUSPENDED + } + + override suspend fun suspendSend(element: Int) = suspendCancellableCoroutine { + enqueuedValue = element + producer = it.intercepted() + COROUTINE_SUSPENDED + } +} + +class CancellableReusableChannel : SimpleChannel() { + override suspend fun suspendReceive(): Int = suspendCancellableCoroutineReusable { + consumer = it.intercepted() + COROUTINE_SUSPENDED + } + + override suspend fun suspendSend(element: Int) = suspendCancellableCoroutineReusable { + enqueuedValue = element + producer = it.intercepted() + COROUTINE_SUSPENDED + } +} diff --git a/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/SimpleChannelBenchmark.kt b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/SimpleChannelBenchmark.kt new file mode 100644 index 0000000000..3228687c0d --- /dev/null +++ b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/channels/SimpleChannelBenchmark.kt @@ -0,0 +1,57 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class SimpleChannelBenchmark { + + private val iterations = 10_000 + + @Volatile + private var sink: Int = 0 + + @Benchmark + fun cancellable() = runBlocking { + val ch = CancellableChannel() + launch { + repeat(iterations) { ch.send(it) } + } + + launch { + repeat(iterations) { sink = ch.receive() } + } + } + + @Benchmark + fun cancellableReusable() = runBlocking { + val ch = CancellableReusableChannel() + launch { + repeat(iterations) { ch.send(it) } + } + + launch { + repeat(iterations) { sink = ch.receive() } + } + } + + @Benchmark + fun nonCancellable() = runBlocking { + val ch = NonCancellableChannel() + launch { + repeat(iterations) { ch.send(it) } + } + + launch { + repeat(iterations) { + sink = ch.receive() + } + } + } +} diff --git a/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/flow/TakeWhileBenchmark.kt b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/flow/TakeWhileBenchmark.kt new file mode 100644 index 0000000000..bdaabc3afa --- /dev/null +++ b/kotlinx-coroutines-core/benchmarks/jvm/kotlin/kotlinx/coroutines/flow/TakeWhileBenchmark.kt @@ -0,0 +1,63 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.internal.* +import kotlinx.coroutines.flow.internal.AbortFlowException +import kotlinx.coroutines.flow.internal.unsafeFlow +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* + +@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS) +@Fork(value = 1) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +@State(Scope.Benchmark) +open class TakeWhileBenchmark { + @Param("1", "10", "100", "1000") + private var size: Int = 0 + + private suspend inline fun Flow.consume() = + filter { it % 2L != 0L } + .map { it * it }.count() + + @Benchmark + fun baseline() = runBlocking { + (0L until size).asFlow().consume() + } + + @Benchmark + fun takeWhileDirect() = runBlocking { + (0L..Long.MAX_VALUE).asFlow().takeWhileDirect { it < size }.consume() + } + + @Benchmark + fun takeWhileViaCollectWhile() = runBlocking { + (0L..Long.MAX_VALUE).asFlow().takeWhileViaCollectWhile { it < size }.consume() + } + + // Direct implementation by checking predicate and throwing AbortFlowException + private fun Flow.takeWhileDirect(predicate: suspend (T) -> Boolean): Flow = unsafeFlow { + try { + collect { value -> + if (predicate(value)) emit(value) + else throw AbortFlowException(this) + } + } catch (e: AbortFlowException) { + e.checkOwnership(owner = this) + } + } + + // Essentially the same code, but reusing the logic via collectWhile function + private fun Flow.takeWhileViaCollectWhile(predicate: suspend (T) -> Boolean): Flow = unsafeFlow { + // This return is needed to work around a bug in JS BE: KT-39227 + return@unsafeFlow collectWhile { value -> + if (predicate(value)) { + emit(value) + true + } else { + false + } + } + } +} diff --git a/kotlinx-coroutines-core/benchmarks/main/kotlin/SharedFlowBaseline.kt b/kotlinx-coroutines-core/benchmarks/main/kotlin/SharedFlowBaseline.kt new file mode 100644 index 0000000000..67ba1bf0c6 --- /dev/null +++ b/kotlinx-coroutines-core/benchmarks/main/kotlin/SharedFlowBaseline.kt @@ -0,0 +1,24 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.benchmark.* + +// Stresses out 'syncrhonozed' codepath in MutableSharedFlow +@State(Scope.Benchmark) +@Measurement(iterations = 3, time = 1, timeUnit = BenchmarkTimeUnit.SECONDS) +@OutputTimeUnit(BenchmarkTimeUnit.MICROSECONDS) +@BenchmarkMode(Mode.AverageTime) +open class SharedFlowBaseline { + private var size: Int = 10_000 + + @Benchmark + fun baseline() = runBlocking { + val flow = MutableSharedFlow() + launch { + repeat(size) { flow.emit(Unit) } + } + + flow.take(size).collect { } + } +} diff --git a/kotlinx-coroutines-core/build.gradle.kts b/kotlinx-coroutines-core/build.gradle.kts new file mode 100644 index 0000000000..8ddea4f5d3 --- /dev/null +++ b/kotlinx-coroutines-core/build.gradle.kts @@ -0,0 +1,308 @@ +import org.gradle.api.tasks.testing.* +import org.gradle.kotlin.dsl.* +import org.gradle.kotlin.dsl.withType +import org.jetbrains.kotlin.gradle.dsl.KotlinMultiplatformExtension +import org.jetbrains.kotlin.gradle.plugin.KotlinSourceSet +import org.jetbrains.kotlin.gradle.plugin.mpp.* +import org.jetbrains.kotlin.gradle.targets.native.tasks.* +import org.jetbrains.kotlin.gradle.tasks.* +import org.jetbrains.kotlin.gradle.testing.* +import ru.vyarus.gradle.plugin.animalsniffer.AnimalSniffer + +plugins { + kotlin("multiplatform") + id("org.jetbrains.kotlinx.benchmark") + id("org.jetbrains.dokka") + id("org.jetbrains.kotlinx.kover") +} + +apply(plugin = "pub-conventions") + +/* ========================================================================== + Configure source sets structure for kotlinx-coroutines-core: + + TARGETS SOURCE SETS + ------------------------------------------------------------ + wasmJs \------> jsAndWasmJsShared ----+ + js / | + V + wasmWasi --------------------> jsAndWasmShared ----------+ + | + V + jvm ----------------------------> concurrent -------> common + ^ + ios \ | + macos | ---> nativeDarwin ---> native ---+ + tvos | ^ + watchos / | + | + linux \ ---> nativeOther -------+ + mingw / + ========================================================================== */ + +kotlin { + sourceSets { + // using the source set names from + groupSourceSets("concurrent", listOf("jvm", "native"), listOf("common")) + if (project.nativeTargetsAreEnabled) { + // TODO: 'nativeDarwin' behaves exactly like 'apple', we can remove it + groupSourceSets("nativeDarwin", listOf("apple"), listOf("native")) + groupSourceSets("nativeOther", listOf("linux", "mingw", "androidNative"), listOf("native")) + } + jvmMain { + dependencies { + compileOnly("com.google.android:annotations:4.1.1.4") + } + } + jvmTest { + dependencies { + api("org.jetbrains.kotlinx:lincheck:${version("lincheck")}") + api("org.jetbrains.kotlinx:kotlinx-knit-test:${version("knit")}") + implementation(project(":android-unit-tests")) + implementation("org.openjdk.jol:jol-core:0.16") + } + } + } + setupBenchmarkSourceSets(sourceSets) + + /* + * Configure two test runs for Native: + * 1) Main thread + * 2) BG thread (required for Dispatchers.Main tests on Darwin) + * + * All new MM targets are build with optimize = true to have stress tests properly run. + */ + targets.withType(KotlinNativeTargetWithTests::class).configureEach { + binaries.test("workerTest", listOf(DEBUG)) { + val thisTest = this + freeCompilerArgs = freeCompilerArgs + listOf("-e", "kotlinx.coroutines.mainBackground") + testRuns.create("workerTest") { + this as KotlinTaskTestRun<*, *> + setExecutionSourceFrom(thisTest) + executionTask.configure { + this as KotlinNativeTest + targetName = "$targetName worker with new MM" + } + } + } + } +} + +private fun KotlinMultiplatformExtension.setupBenchmarkSourceSets(ss: NamedDomainObjectContainer) { + // Forgive me, Father, for I have sinned. + // Really, that is needed to have benchmark sourcesets be the part of the project, not a separate project + val benchmarkMain by ss.creating { + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:${version("benchmarks")}") + } + // For each source set we have to manually set path to the sources, otherwise lookup will fail + kotlin.srcDir("benchmarks/main/kotlin") + } + + @Suppress("UnusedVariable") + val jvmBenchmark by ss.creating { + // For each source set we have to manually set path to the sources, otherwise lookup will fail + kotlin.srcDir("benchmarks/jvm/kotlin") + } + + targets.matching { + it.name != "metadata" + // Doesn't work, don't want to figure it out for now + && !it.name.contains("wasm") + && !it.name.contains("js") + }.all { + compilations.create("benchmark") { + associateWith(this@all.compilations.getByName("main")) + defaultSourceSet { + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:${version("benchmarks")}") + } + dependsOn(benchmarkMain) + } + } + } + + targets.matching { it.name != "metadata" }.all { + benchmark.targets.register("${name}Benchmark") + } +} + +// Update module name for metadata artifact to avoid conflicts +// see https://github.com/Kotlin/kotlinx.coroutines/issues/1797 +val compileKotlinMetadata by tasks.getting(KotlinCompilationTask::class) { + compilerOptions { + freeCompilerArgs.addAll("-module-name", "kotlinx-coroutines-core-common") + } +} + +val jvmTest by tasks.getting(Test::class) { + minHeapSize = "1g" + maxHeapSize = "1g" + enableAssertions = true + // 'stress' is required to be able to run all subpackage tests like ":jvmTests --tests "*channels*" -Pstress=true" + if (!Idea.active && rootProject.properties["stress"] == null) { + exclude("**/*LincheckTest*") + exclude("**/*StressTest.*") + } + if (Idea.active) { + // Configure the IDEA runner for Lincheck + configureJvmForLincheck() + } +} + +// Setup manifest for kotlinx-coroutines-core-jvm.jar +val jvmJar by tasks.getting(Jar::class) { setupManifest(this) } + +/* + * Setup manifest for kotlinx-coroutines-core.jar + * This is convenient for users that pass -javaagent arg manually and also is a workaround #2619 and KTIJ-5659. + * This manifest contains reference to AgentPremain that belongs to + * kotlinx-coroutines-core-jvm, but our resolving machinery guarantees that + * any JVM project that depends on -core artifact also depends on -core-jvm one. + */ +val allMetadataJar by tasks.getting(Jar::class) { setupManifest(this) } + +fun setupManifest(jar: Jar) { + jar.manifest { + attributes( + mapOf( + "Premain-Class" to "kotlinx.coroutines.debug.internal.AgentPremain", + "Can-Retransform-Classes" to "true", + ) + ) + } +} + +val compileTestKotlinJvm by tasks.getting(KotlinJvmCompile::class) +val jvmTestClasses by tasks.getting + +val jvmStressTest by tasks.registering(Test::class) { + dependsOn(compileTestKotlinJvm) + classpath = jvmTest.classpath + testClassesDirs = jvmTest.testClassesDirs + minHeapSize = "1g" + maxHeapSize = "1g" + include("**/*StressTest.*") + enableAssertions = true + testLogging.showStandardStreams = true + systemProperty("kotlinx.coroutines.scheduler.keep.alive.sec", 100000) // any unpark problem hangs test + // Adjust internal algorithmic parameters to increase the testing quality instead of performance. + systemProperty("kotlinx.coroutines.semaphore.segmentSize", 1) + systemProperty("kotlinx.coroutines.semaphore.maxSpinCycles", 10) + systemProperty("kotlinx.coroutines.bufferedChannel.segmentSize", 2) + systemProperty("kotlinx.coroutines.bufferedChannel.expandBufferCompletionWaitIterations", 1) +} + +val jvmLincheckTest by tasks.registering(Test::class) { + dependsOn(compileTestKotlinJvm) + classpath = jvmTest.classpath + testClassesDirs = jvmTest.testClassesDirs + include("**/*LincheckTest*") + enableAssertions = true + testLogging.showStandardStreams = true + configureJvmForLincheck() +} + +// Additional Lincheck tests with `segmentSize = 2`. +// Some bugs cannot be revealed when storing one request per segment, +// and some are hard to detect when storing multiple requests. +val jvmLincheckTestAdditional by tasks.registering(Test::class) { + dependsOn(compileTestKotlinJvm) + classpath = jvmTest.classpath + testClassesDirs = jvmTest.testClassesDirs + include("**/RendezvousChannelLincheckTest*") + include("**/Buffered1ChannelLincheckTest*") + include("**/Semaphore*LincheckTest*") + enableAssertions = true + testLogging.showStandardStreams = true + configureJvmForLincheck(segmentSize = 2) +} + +fun Test.configureJvmForLincheck(segmentSize: Int = 1) { + minHeapSize = "1g" + maxHeapSize = "4g" // we may need more space for building an interleaving tree in the model checking mode + // https://github.com/JetBrains/lincheck#java-9 + jvmArgs = listOf( + "--add-opens", "java.base/jdk.internal.misc=ALL-UNNAMED", // required for transformation + "--add-exports", "java.base/sun.security.action=ALL-UNNAMED", + "--add-exports", "java.base/jdk.internal.util=ALL-UNNAMED" + ) // in the model checking mode + // Adjust internal algorithmic parameters to increase the testing quality instead of performance. + systemProperty("kotlinx.coroutines.semaphore.segmentSize", segmentSize) + systemProperty("kotlinx.coroutines.semaphore.maxSpinCycles", 1) // better for the model checking mode + systemProperty("kotlinx.coroutines.bufferedChannel.segmentSize", segmentSize) + systemProperty("kotlinx.coroutines.bufferedChannel.expandBufferCompletionWaitIterations", 1) +} + +// Always check additional test sets +val moreTest by tasks.registering { + dependsOn(listOf(jvmStressTest, jvmLincheckTest, jvmLincheckTestAdditional)) +} + +val check by tasks.getting { + dependsOn(moreTest) +} + +kover { + currentProject { + instrumentation { + // Always disabled, lincheck doesn't really support coverage + disabledForTestTasks.addAll("jvmLincheckTest") + + // lincheck has NPE error on `ManagedStrategyStateHolder` class + excludedClasses.addAll("org.jetbrains.kotlinx.lincheck.*") + } + sources { + excludedSourceSets.addAll("benchmark") + } + } + + reports { + filters { + excludes { + classes( + "kotlinx.coroutines.debug.*", // Tested by debug module + "kotlinx.coroutines.channels.ChannelsKt__DeprecatedKt*", // Deprecated + "kotlinx.coroutines.scheduling.LimitingDispatcher", // Deprecated + "kotlinx.coroutines.scheduling.ExperimentalCoroutineDispatcher", // Deprecated + "kotlinx.coroutines.flow.FlowKt__MigrationKt*", // Migrations + "kotlinx.coroutines.flow.LintKt*", // Migrations + "kotlinx.coroutines.internal.WeakMapCtorCache", // Fallback implementation that we never test + "_COROUTINE._CREATION", // For IDE navigation + "_COROUTINE._BOUNDARY", // For IDE navigation + ) + } + } + } +} + +val testsJar by tasks.registering(Jar::class) { + dependsOn(jvmTestClasses) + archiveClassifier = "tests" + from(compileTestKotlinJvm.destinationDirectory) +} + +artifacts { + archives(testsJar) +} + +// Workaround for https://github.com/Kotlin/dokka/issues/1833: make implicit dependency explicit +tasks.named("dokkaHtmlPartial") { + dependsOn(jvmJar) +} + +// Specific files so nothing from core is accidentally skipped +tasks.withType { + exclude("**/future/FutureKt*") + exclude("**/future/ContinuationHandler*") + exclude("**/future/CompletableFutureCoroutine*") + + exclude("**/stream/StreamKt*") + exclude("**/stream/StreamFlow*") + + exclude("**/time/TimeKt*") +} + +animalsniffer { + defaultTargets = setOf("jvmMain") +} diff --git a/kotlinx-coroutines-core/common/README.md b/kotlinx-coroutines-core/common/README.md new file mode 100644 index 0000000000..041cd3ec64 --- /dev/null +++ b/kotlinx-coroutines-core/common/README.md @@ -0,0 +1,153 @@ +# Module kotlinx-coroutines-core + +Core primitives to work with coroutines available on all platforms. + +Coroutine builder functions: + +| **Name** | **Result** | **Scope** | **Description** +| ------------- | ------------- | ---------------- | --------------- +| [launch] | [Job] | [CoroutineScope] | Launches coroutine that does not have any result +| [async] | [Deferred] | [CoroutineScope] | Returns a single value with the future result +| [produce][kotlinx.coroutines.channels.produce] | [ReceiveChannel][kotlinx.coroutines.channels.ReceiveChannel] | [ProducerScope][kotlinx.coroutines.channels.ProducerScope] | Produces a stream of elements +| [actor][kotlinx.coroutines.channels.actor] | [SendChannel][kotlinx.coroutines.channels.SendChannel] | [ActorScope][kotlinx.coroutines.channels.ActorScope] | Processes a stream of messages + +Coroutine dispatchers implementing [CoroutineDispatcher]: + +| **Name** | **Description** +| --------------------------- | --------------- +| [Dispatchers.Default] | Confines coroutine execution to a shared pool of background threads +| [Dispatchers.Unconfined] | Does not confine coroutine execution in any way +| [newSingleThreadContext] | Creates a single-threaded coroutine context +| [newFixedThreadPoolContext] | Creates a thread pool of a fixed size +| [Executor.asCoroutineDispatcher][asCoroutineDispatcher] | Extension to convert any executor + +More context elements: + +| **Name** | **Description** +| --------------------------- | --------------- +| [NonCancellable] | A non-cancelable job that is always active +| [CoroutineExceptionHandler] | Handler for uncaught exception + +Synchronization primitives for coroutines: + +| **Name** | **Suspending functions** | **Description** +| ---------- | ----------------------------------------------------------- | --------------- +| [Mutex][kotlinx.coroutines.sync.Mutex] | [lock][kotlinx.coroutines.sync.Mutex.lock] | Mutual exclusion +| [Channel][kotlinx.coroutines.channels.Channel] | [send][kotlinx.coroutines.channels.SendChannel.send], [receive][kotlinx.coroutines.channels.ReceiveChannel.receive] | Communication channel (aka queue or exchanger) + +Top-level suspending functions: + +| **Name** | **Description** +| ------------------- | --------------- +| [delay] | Non-blocking sleep +| [yield] | Yields thread in single-threaded dispatchers +| [withContext] | Switches to a different context +| [withTimeout] | Set execution time-limit with exception on timeout +| [withTimeoutOrNull] | Set execution time-limit will null result on timeout +| [awaitAll] | Awaits for successful completion of all given jobs or exceptional completion of any +| [joinAll] | Joins on all given jobs + +Cancellation support for user-defined suspending functions is available with [suspendCancellableCoroutine] +helper function. [NonCancellable] job object is provided to suppress cancellation with +`withContext(NonCancellable) {...}` block of code. + +[Select][kotlinx.coroutines.selects.select] expression waits for the result of multiple suspending functions simultaneously: + +| **Receiver** | **Suspending function** | **Select clause** | **Non-suspending version** +| ---------------- | --------------------------------------------- | ------------------------------------------------ | -------------------------- +| [Job] | [join][Job.join] | [onJoin][Job.onJoin] | [isCompleted][Job.isCompleted] +| [Deferred] | [await][Deferred.await] | [onAwait][Deferred.onAwait] | [isCompleted][Job.isCompleted] +| [SendChannel][kotlinx.coroutines.channels.SendChannel] | [send][kotlinx.coroutines.channels.SendChannel.send] | [onSend][kotlinx.coroutines.channels.SendChannel.onSend] | [trySend][kotlinx.coroutines.channels.SendChannel.trySend] +| [ReceiveChannel][kotlinx.coroutines.channels.ReceiveChannel] | [receive][kotlinx.coroutines.channels.ReceiveChannel.receive] | [onReceive][kotlinx.coroutines.channels.ReceiveChannel.onReceive] | [tryReceive][kotlinx.coroutines.channels.ReceiveChannel.tryReceive] +| [ReceiveChannel][kotlinx.coroutines.channels.ReceiveChannel] | [receiveCatching][kotlinx.coroutines.channels.ReceiveChannel.receiveCatching] | [onReceiveCatching][kotlinx.coroutines.channels.ReceiveChannel.onReceiveCatching] | [tryReceive][kotlinx.coroutines.channels.ReceiveChannel.tryReceive] +| none | [delay] | [onTimeout][kotlinx.coroutines.selects.SelectBuilder.onTimeout] | none + +This module provides debugging facilities for coroutines (run JVM with `-ea` or `-Dkotlinx.coroutines.debug` options) +and [newCoroutineContext] function to write user-defined coroutine builders that work with these +debugging facilities. See [DEBUG_PROPERTY_NAME] for more details. + +# Package kotlinx.coroutines + +General-purpose coroutine builders, contexts, and helper functions. + +# Package kotlinx.coroutines.flow + +Flow -- primitive to work with asynchronous and event-based streams of data. + +# Package kotlinx.coroutines.sync + +Synchronization primitives (mutex). + +# Package kotlinx.coroutines.channels + +Channels -- non-blocking primitives for communicating a stream of elements between coroutines. + +# Package kotlinx.coroutines.selects + +Select expression to perform multiple suspending operations simultaneously until one of them succeeds. + +# Package kotlinx.coroutines.intrinsics + +Low-level primitives for finer-grained control of coroutines. + + + + +[launch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[async]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html +[Deferred]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/index.html +[CoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/index.html +[Dispatchers.Default]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-default.html +[Dispatchers.Unconfined]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-unconfined.html +[newSingleThreadContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/new-single-thread-context.html +[newFixedThreadPoolContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/new-fixed-thread-pool-context.html +[asCoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/as-coroutine-dispatcher.html +[NonCancellable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-non-cancellable/index.html +[CoroutineExceptionHandler]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-exception-handler/index.html +[delay]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html +[yield]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/yield.html +[withContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-context.html +[withTimeout]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-timeout.html +[withTimeoutOrNull]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-timeout-or-null.html +[awaitAll]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/await-all.html +[joinAll]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/join-all.html +[suspendCancellableCoroutine]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/suspend-cancellable-coroutine.html +[Job.join]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/join.html +[Job.onJoin]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/on-join.html +[Job.isCompleted]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/is-completed.html +[Deferred.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/await.html +[Deferred.onAwait]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-deferred/on-await.html +[newCoroutineContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/new-coroutine-context.html +[DEBUG_PROPERTY_NAME]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-d-e-b-u-g_-p-r-o-p-e-r-t-y_-n-a-m-e.html + + + +[kotlinx.coroutines.sync.Mutex]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/index.html +[kotlinx.coroutines.sync.Mutex.lock]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/lock.html + + + +[kotlinx.coroutines.channels.produce]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/produce.html +[kotlinx.coroutines.channels.ReceiveChannel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/index.html +[kotlinx.coroutines.channels.ProducerScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-producer-scope/index.html +[kotlinx.coroutines.channels.actor]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/actor.html +[kotlinx.coroutines.channels.SendChannel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/index.html +[kotlinx.coroutines.channels.ActorScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-actor-scope/index.html +[kotlinx.coroutines.channels.Channel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-channel/index.html +[kotlinx.coroutines.channels.SendChannel.send]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/send.html +[kotlinx.coroutines.channels.ReceiveChannel.receive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/receive.html +[kotlinx.coroutines.channels.SendChannel.onSend]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/on-send.html +[kotlinx.coroutines.channels.SendChannel.trySend]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/try-send.html +[kotlinx.coroutines.channels.ReceiveChannel.onReceive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/on-receive.html +[kotlinx.coroutines.channels.ReceiveChannel.tryReceive]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/try-receive.html +[kotlinx.coroutines.channels.ReceiveChannel.receiveCatching]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/receive-catching.html +[kotlinx.coroutines.channels.ReceiveChannel.onReceiveCatching]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-receive-channel/on-receive-catching.html + + + +[kotlinx.coroutines.selects.select]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.selects/select.html +[kotlinx.coroutines.selects.SelectBuilder.onTimeout]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.selects/on-timeout.html + + diff --git a/kotlinx-coroutines-core/common/src/AbstractCoroutine.kt b/kotlinx-coroutines-core/common/src/AbstractCoroutine.kt new file mode 100644 index 0000000000..2d6273acc9 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/AbstractCoroutine.kt @@ -0,0 +1,136 @@ +@file:Suppress("DEPRECATION_ERROR") + +package kotlinx.coroutines + +import kotlinx.coroutines.CoroutineStart.* +import kotlinx.coroutines.intrinsics.* +import kotlin.coroutines.* +import kotlinx.coroutines.internal.ScopeCoroutine + +/** + * Abstract base class for implementation of coroutines in coroutine builders. + * + * This class implements completion [Continuation], [Job], and [CoroutineScope] interfaces. + * It stores the result of continuation in the state of the job. + * This coroutine waits for children coroutines to finish before completing and + * fails through an intermediate _failing_ state. + * + * The following methods are available for override: + * + * - [onStart] is invoked when the coroutine was created in non-active state and is being [started][Job.start]. + * - [onCancelling] is invoked as soon as the coroutine starts being cancelled for any reason (or completes). + * - [onCompleted] is invoked when the coroutine completes with a value. + * - [onCancelled] in invoked when the coroutine completes with an exception (cancelled). + * + * @param parentContext the context of the parent coroutine. + * @param initParentJob specifies whether the parent-child relationship should be instantiated directly + * in `AbstractCoroutine` constructor. If set to `false`, it's the responsibility of the child class + * to invoke [initParentJob] manually. + * @param active when `true` (by default), the coroutine is created in the _active_ state, otherwise it is created in the _new_ state. + * See [Job] for details. + * + * @suppress **This an internal API and should not be used from general code.** + */ +@OptIn(InternalForInheritanceCoroutinesApi::class) +@InternalCoroutinesApi +public abstract class AbstractCoroutine( + parentContext: CoroutineContext, + initParentJob: Boolean, + active: Boolean +) : JobSupport(active), Job, Continuation, CoroutineScope { + + init { + /* + * Setup parent-child relationship between the parent in the context and the current coroutine. + * It may cause this coroutine to become _cancelling_ if the parent is already cancelled. + * It is dangerous to install parent-child relationship here if the coroutine class + * operates its state from within onCancelled or onCancelling + * (with exceptions for rx integrations that can't have any parent) + */ + if (initParentJob) initParentJob(parentContext[Job]) + } + + /** + * The context of this coroutine that includes this coroutine as a [Job]. + */ + @Suppress("LeakingThis") + public final override val context: CoroutineContext = parentContext + this + + /** + * The context of this scope which is the same as the [context] of this coroutine. + */ + public override val coroutineContext: CoroutineContext get() = context + + override val isActive: Boolean get() = super.isActive + + /** + * This function is invoked once when the job was completed normally with the specified [value], + * right before all the waiters for the coroutine's completion are notified. + */ + protected open fun onCompleted(value: T) {} + + /** + * This function is invoked once when the job was cancelled with the specified [cause], + * right before all the waiters for coroutine's completion are notified. + * + * **Note:** the state of the coroutine might not be final yet in this function and should not be queried. + * You can use [completionCause] and [completionCauseHandled] to recover parameters that we passed + * to this `onCancelled` invocation only when [isCompleted] returns `true`. + * + * @param cause The cancellation (failure) cause + * @param handled `true` if the exception was handled by parent (always `true` when it is a [CancellationException]) + */ + protected open fun onCancelled(cause: Throwable, handled: Boolean) {} + + override fun cancellationExceptionMessage(): String = "$classSimpleName was cancelled" + + @Suppress("UNCHECKED_CAST") + protected final override fun onCompletionInternal(state: Any?) { + if (state is CompletedExceptionally) + onCancelled(state.cause, state.handled) + else + onCompleted(state as T) + } + + /** + * Completes execution of this with coroutine with the specified result. + */ + public final override fun resumeWith(result: Result) { + val state = makeCompletingOnce(result.toState()) + if (state === COMPLETING_WAITING_CHILDREN) return + afterResume(state) + } + + /** + * Invoked when the corresponding `AbstractCoroutine` was **conceptually** resumed, but not mechanically. + * Currently, this function only invokes `resume` on the underlying continuation for [ScopeCoroutine] + * or does nothing otherwise. + * + * Examples of resumes: + * - `afterCompletion` calls when the corresponding `Job` changed its state (i.e. got cancelled) + * - [AbstractCoroutine.resumeWith] was invoked + */ + protected open fun afterResume(state: Any?): Unit = afterCompletion(state) + + internal final override fun handleOnCompletionException(exception: Throwable) { + handleCoroutineException(context, exception) + } + + internal override fun nameString(): String { + val coroutineName = context.coroutineName ?: return super.nameString() + return "\"$coroutineName\":${super.nameString()}" + } + + /** + * Starts this coroutine with the given code [block] and [start] strategy. + * This function shall be invoked at most once on this coroutine. + * + * - [DEFAULT] uses [startCoroutineCancellable]. + * - [ATOMIC] uses [startCoroutine]. + * - [UNDISPATCHED] uses [startCoroutineUndispatched]. + * - [LAZY] does nothing. + */ + public fun start(start: CoroutineStart, receiver: R, block: suspend R.() -> T) { + start(block, receiver, this) + } +} diff --git a/kotlinx-coroutines-core/common/src/Annotations.kt b/kotlinx-coroutines-core/common/src/Annotations.kt new file mode 100644 index 0000000000..5fcca4639d --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Annotations.kt @@ -0,0 +1,118 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.flow.* + +/** + * Marks declarations in the coroutines that are **delicate** — + * they have limited use-case and shall be used with care in general code. + * Any use of a delicate declaration has to be carefully reviewed to make sure it is + * properly used and does not create problems like memory and resource leaks. + * Carefully read documentation of any declaration marked as `DelicateCoroutinesApi`. + */ +@MustBeDocumented +@Retention(value = AnnotationRetention.BINARY) +@RequiresOptIn( + level = RequiresOptIn.Level.WARNING, + message = "This is a delicate API and its use requires care." + + " Make sure you fully read and understand documentation of the declaration that is marked as a delicate API." +) +public annotation class DelicateCoroutinesApi + +/** + * Marks declarations that are still **experimental** in coroutines API, which means that the design of the + * corresponding declarations has open issues which may (or may not) lead to their changes in the future. + * Roughly speaking, there is a chance that those declarations will be deprecated in the near future or + * the semantics of their behavior may change in some way that may break some code. + */ +@MustBeDocumented +@Retention(value = AnnotationRetention.BINARY) +@Target( + AnnotationTarget.CLASS, + AnnotationTarget.ANNOTATION_CLASS, + AnnotationTarget.PROPERTY, + AnnotationTarget.FIELD, + AnnotationTarget.LOCAL_VARIABLE, + AnnotationTarget.VALUE_PARAMETER, + AnnotationTarget.CONSTRUCTOR, + AnnotationTarget.FUNCTION, + AnnotationTarget.PROPERTY_GETTER, + AnnotationTarget.PROPERTY_SETTER, + AnnotationTarget.TYPEALIAS +) +@RequiresOptIn(level = RequiresOptIn.Level.WARNING) +public annotation class ExperimentalCoroutinesApi + +/** + * Marks [Flow]-related API as a feature preview. + * + * Flow preview has **no** backward compatibility guarantees, including both binary and source compatibility. + * Its API and semantics can and will be changed in next releases. + * + * Feature preview can be used to evaluate its real-world strengths and weaknesses, gather and provide feedback. + * According to the feedback, [Flow] will be refined on its road to stabilization and promotion to a stable API. + * + * The best way to speed up preview feature promotion is providing the feedback on the feature. + */ +@MustBeDocumented +@Retention(value = AnnotationRetention.BINARY) +@RequiresOptIn( + level = RequiresOptIn.Level.WARNING, + message = "This declaration is in a preview state and can be changed in a backwards-incompatible manner with a best-effort migration. " + + "Its usage should be marked with '@kotlinx.coroutines.FlowPreview' or '@OptIn(kotlinx.coroutines.FlowPreview::class)' " + + "if you accept the drawback of relying on preview API" +) +@Target(AnnotationTarget.CLASS, AnnotationTarget.FUNCTION, AnnotationTarget.TYPEALIAS, AnnotationTarget.PROPERTY) +public annotation class FlowPreview + +/** + * Marks declarations that are **obsolete** in coroutines API, which means that the design of the corresponding + * declarations has serious known flaws and they will be redesigned in the future. + * Roughly speaking, these declarations will be deprecated in the future but there is no replacement for them yet, + * so they cannot be deprecated right away. + */ +@MustBeDocumented +@Retention(value = AnnotationRetention.BINARY) +@RequiresOptIn(level = RequiresOptIn.Level.WARNING) +public annotation class ObsoleteCoroutinesApi + +/** + * Marks declarations that are **internal** in coroutines API, which means that should not be used outside of + * `kotlinx.coroutines`, because their signatures and semantics will change between future releases without any + * warnings and without providing any migration aids. + */ +@MustBeDocumented +@Retention(value = AnnotationRetention.BINARY) +@Target(AnnotationTarget.CLASS, AnnotationTarget.FUNCTION, AnnotationTarget.TYPEALIAS, AnnotationTarget.PROPERTY) +@RequiresOptIn( + level = RequiresOptIn.Level.ERROR, message = "This is an internal kotlinx.coroutines API that " + + "should not be used from outside of kotlinx.coroutines. No compatibility guarantees are provided. " + + "It is recommended to report your use-case of internal API to kotlinx.coroutines issue tracker, " + + "so stable API could be provided instead" +) +public annotation class InternalCoroutinesApi + +/** + * Marks declarations that cannot be safely inherited from. + */ +@Target(AnnotationTarget.CLASS) +@RequiresOptIn( + level = RequiresOptIn.Level.WARNING, message = + "Inheriting from this kotlinx.coroutines API is unstable. " + + "Either new methods may be added in the future, which would break the inheritance, " + + "or correctly inheriting from it requires fulfilling contracts that may change in the future." +) +public annotation class ExperimentalForInheritanceCoroutinesApi + +/** + * Marks declarations that cannot be safely inherited from. + */ +@Target(AnnotationTarget.CLASS) +@RequiresOptIn( + level = RequiresOptIn.Level.WARNING, message = + "This is a kotlinx.coroutines API that is not intended to be inherited from, " + + "as the library may handle predefined instances of this in a special manner. " + + "This will be an error in a future release. " + + "If you need to inherit from this, please describe your use case in " + + "https://github.com/Kotlin/kotlinx.coroutines/issues,%20so%20that%20we%20can%20provide%20a%20stable%20API%20for%20inheritance." +) +public annotation class InternalForInheritanceCoroutinesApi diff --git a/kotlinx-coroutines-core/common/src/Await.kt b/kotlinx-coroutines-core/common/src/Await.kt new file mode 100644 index 0000000000..845f01e506 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Await.kt @@ -0,0 +1,120 @@ +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlin.coroutines.* + +/** + * Awaits for completion of given deferred values without blocking a thread and resumes normally with the list of values + * when all deferred computations are complete or resumes with the first thrown exception if any of computations + * complete exceptionally including cancellation. + * + * This function is **not** equivalent to `deferreds.map { it.await() }` which fails only when it sequentially + * gets to wait for the failing deferred, while this `awaitAll` fails immediately as soon as any of the deferreds fail. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + */ +public suspend fun awaitAll(vararg deferreds: Deferred): List = + if (deferreds.isEmpty()) emptyList() else AwaitAll(deferreds).await() + +/** + * Awaits for completion of given deferred values without blocking a thread and resumes normally with the list of values + * when all deferred computations are complete or resumes with the first thrown exception if any of computations + * complete exceptionally including cancellation. + * + * This function is **not** equivalent to `this.map { it.await() }` which fails only when it sequentially + * gets to wait for the failing deferred, while this `awaitAll` fails immediately as soon as any of the deferreds fail. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + */ +public suspend fun Collection>.awaitAll(): List = + if (isEmpty()) emptyList() else AwaitAll(toTypedArray()).await() + +/** + * Suspends current coroutine until all given jobs are complete. + * This method is semantically equivalent to joining all given jobs one by one with `jobs.forEach { it.join() }`. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + */ +public suspend fun joinAll(vararg jobs: Job): Unit = jobs.forEach { it.join() } + +/** + * Suspends current coroutine until all given jobs are complete. + * This method is semantically equivalent to joining all given jobs one by one with `forEach { it.join() }`. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + */ +public suspend fun Collection.joinAll(): Unit = forEach { it.join() } + +private class AwaitAll(private val deferreds: Array>) { + private val notCompletedCount = atomic(deferreds.size) + + suspend fun await(): List = suspendCancellableCoroutine { cont -> + // Intricate dance here + // Step 1: Create nodes and install them as completion handlers, they may fire! + val nodes = Array(deferreds.size) { i -> + val deferred = deferreds[i] + deferred.start() // To properly await lazily started deferreds + AwaitAllNode(cont).apply { + handle = deferred.invokeOnCompletion(handler = this) + } + } + val disposer = DisposeHandlersOnCancel(nodes) + // Step 2: Set disposer to each node + nodes.forEach { it.disposer = disposer } + // Here we know that if any code the nodes complete, it will dispose the rest + // Step 3: Now we can check if continuation is complete + if (cont.isCompleted) { + // it is already complete while handlers were being installed -- dispose them all + disposer.disposeAll() + } else { + cont.invokeOnCancellation(handler = disposer) + } + } + + private inner class DisposeHandlersOnCancel(private val nodes: Array) : CancelHandler { + fun disposeAll() { + nodes.forEach { it.handle.dispose() } + } + + override fun invoke(cause: Throwable?) { disposeAll() } + override fun toString(): String = "DisposeHandlersOnCancel[$nodes]" + } + + private inner class AwaitAllNode(private val continuation: CancellableContinuation>) : JobNode() { + lateinit var handle: DisposableHandle + + private val _disposer = atomic(null) + var disposer: DisposeHandlersOnCancel? + get() = _disposer.value + set(value) { _disposer.value = value } + + override val onCancelling get() = false + + override fun invoke(cause: Throwable?) { + if (cause != null) { + val token = continuation.tryResumeWithException(cause) + if (token != null) { + continuation.completeResume(token) + // volatile read of disposer AFTER continuation is complete + // and if disposer was already set (all handlers where already installed, then dispose them all) + disposer?.disposeAll() + } + } else if (notCompletedCount.decrementAndGet() == 0) { + continuation.resume(deferreds.map { it.getCompleted() }) + // Note that all deferreds are complete here, so we don't need to dispose their nodes + } + } + } +} diff --git a/kotlinx-coroutines-core/common/src/Builders.common.kt b/kotlinx-coroutines-core/common/src/Builders.common.kt new file mode 100644 index 0000000000..23ef7665b5 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Builders.common.kt @@ -0,0 +1,267 @@ +@file:JvmMultifileClass +@file:JvmName("BuildersKt") +@file:OptIn(ExperimentalContracts::class) +@file:Suppress("LEAKED_IN_PLACE_LAMBDA", "WRONG_INVOCATION_KIND") + +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.intrinsics.* +import kotlinx.coroutines.selects.* +import kotlin.contracts.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlin.jvm.* + +// --------------- launch --------------- + +/** + * Launches a new coroutine without blocking the current thread and returns a reference to the coroutine as a [Job]. + * The coroutine is cancelled when the resulting job is [cancelled][Job.cancel]. + * + * The coroutine context is inherited from a [CoroutineScope]. Additional context elements can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * The parent job is inherited from a [CoroutineScope] as well, but it can also be overridden + * with a corresponding [context] element. + * + * By default, the coroutine is immediately scheduled for execution. + * Other start options can be specified via `start` parameter. See [CoroutineStart] for details. + * An optional [start] parameter can be set to [CoroutineStart.LAZY] to start coroutine _lazily_. In this case, + * the coroutine [Job] is created in _new_ state. It can be explicitly started with [start][Job.start] function + * and will be started implicitly on the first invocation of [join][Job.join]. + * + * Uncaught exceptions in this coroutine cancel the parent job in the context by default + * (unless [CoroutineExceptionHandler] is explicitly specified), which means that when `launch` is used with + * the context of another coroutine, then any uncaught exception leads to the cancellation of the parent coroutine. + * + * See [newCoroutineContext] for a description of debugging facilities that are available for a newly created coroutine. + * + * @param context additional to [CoroutineScope.coroutineContext] context of the coroutine. + * @param start coroutine start option. The default value is [CoroutineStart.DEFAULT]. + * @param block the coroutine code which will be invoked in the context of the provided scope. + **/ +public fun CoroutineScope.launch( + context: CoroutineContext = EmptyCoroutineContext, + start: CoroutineStart = CoroutineStart.DEFAULT, + block: suspend CoroutineScope.() -> Unit +): Job { + val newContext = newCoroutineContext(context) + val coroutine = if (start.isLazy) + LazyStandaloneCoroutine(newContext, block) else + StandaloneCoroutine(newContext, active = true) + coroutine.start(start, coroutine, block) + return coroutine +} + +// --------------- async --------------- + +/** + * Creates a coroutine and returns its future result as an implementation of [Deferred]. + * The running coroutine is cancelled when the resulting deferred is [cancelled][Job.cancel]. + * The resulting coroutine has a key difference compared with similar primitives in other languages + * and frameworks: it cancels the parent job (or outer scope) on failure to enforce *structured concurrency* paradigm. + * To change that behaviour, supervising parent ([SupervisorJob] or [supervisorScope]) can be used. + * + * Coroutine context is inherited from a [CoroutineScope], additional context elements can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * The parent job is inherited from a [CoroutineScope] as well, but it can also be overridden + * with corresponding [context] element. + * + * By default, the coroutine is immediately scheduled for execution. + * Other options can be specified via `start` parameter. See [CoroutineStart] for details. + * An optional [start] parameter can be set to [CoroutineStart.LAZY] to start coroutine _lazily_. In this case, + * the resulting [Deferred] is created in _new_ state. It can be explicitly started with [start][Job.start] + * function and will be started implicitly on the first invocation of [join][Job.join], [await][Deferred.await] or [awaitAll]. + * + * @param block the coroutine code. + */ +public fun CoroutineScope.async( + context: CoroutineContext = EmptyCoroutineContext, + start: CoroutineStart = CoroutineStart.DEFAULT, + block: suspend CoroutineScope.() -> T +): Deferred { + val newContext = newCoroutineContext(context) + val coroutine = if (start.isLazy) + LazyDeferredCoroutine(newContext, block) else + DeferredCoroutine(newContext, active = true) + coroutine.start(start, coroutine, block) + return coroutine +} + +@OptIn(InternalForInheritanceCoroutinesApi::class) +@Suppress("UNCHECKED_CAST") +private open class DeferredCoroutine( + parentContext: CoroutineContext, + active: Boolean +) : AbstractCoroutine(parentContext, true, active = active), Deferred { + override fun getCompleted(): T = getCompletedInternal() as T + override suspend fun await(): T = awaitInternal() as T + override val onAwait: SelectClause1 get() = onAwaitInternal as SelectClause1 +} + +private class LazyDeferredCoroutine( + parentContext: CoroutineContext, + block: suspend CoroutineScope.() -> T +) : DeferredCoroutine(parentContext, active = false) { + private val continuation = block.createCoroutineUnintercepted(this, this) + + override fun onStart() { + continuation.startCoroutineCancellable(this) + } +} + +// --------------- withContext --------------- + +/** + * Calls the specified suspending block with a given coroutine context, suspends until it completes, and returns + * the result. + * + * The resulting context for the [block] is derived by merging the current [coroutineContext] with the + * specified [context] using `coroutineContext + context` (see [CoroutineContext.plus]). + * This suspending function is cancellable. It immediately checks for cancellation of + * the resulting context and throws [CancellationException] if it is not [active][CoroutineContext.isActive]. + * + * Calls to [withContext] whose [context] argument provides a [CoroutineDispatcher] that is + * different from the current one, by necessity, perform additional dispatches: the [block] + * can not be executed immediately and needs to be dispatched for execution on + * the passed [CoroutineDispatcher], and then when the [block] completes, the execution + * has to shift back to the original dispatcher. + * + * Note that the result of `withContext` invocation is dispatched into the original context in a cancellable way + * with a **prompt cancellation guarantee**, which means that if the original [coroutineContext] + * in which `withContext` was invoked is cancelled by the time its dispatcher starts to execute the code, + * it discards the result of `withContext` and throws [CancellationException]. + * + * The cancellation behaviour described above is enabled if and only if the dispatcher is being changed. + * For example, when using `withContext(NonCancellable) { ... }` there is no change in dispatcher and + * this call will not be cancelled neither on entry to the block inside `withContext` nor on exit from it. + */ +public suspend fun withContext( + context: CoroutineContext, + block: suspend CoroutineScope.() -> T +): T { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + return suspendCoroutineUninterceptedOrReturn sc@ { uCont -> + // compute new context + val oldContext = uCont.context + // Copy CopyableThreadContextElement if necessary + val newContext = oldContext.newCoroutineContext(context) + // always check for cancellation of new context + newContext.ensureActive() + // FAST PATH #1 -- new context is the same as the old one + if (newContext === oldContext) { + val coroutine = ScopeCoroutine(newContext, uCont) + return@sc coroutine.startUndispatchedOrReturn(coroutine, block) + } + // FAST PATH #2 -- the new dispatcher is the same as the old one (something else changed) + // `equals` is used by design (see equals implementation is wrapper context like ExecutorCoroutineDispatcher) + if (newContext[ContinuationInterceptor] == oldContext[ContinuationInterceptor]) { + val coroutine = UndispatchedCoroutine(newContext, uCont) + // There are changes in the context, so this thread needs to be updated + withCoroutineContext(coroutine.context, null) { + return@sc coroutine.startUndispatchedOrReturn(coroutine, block) + } + } + // SLOW PATH -- use new dispatcher + val coroutine = DispatchedCoroutine(newContext, uCont) + block.startCoroutineCancellable(coroutine, coroutine) + coroutine.getResult() + } +} + +/** + * Calls the specified suspending block with the given [CoroutineDispatcher], suspends until it + * completes, and returns the result. + * + * This inline function calls [withContext]. + */ +public suspend inline operator fun CoroutineDispatcher.invoke( + noinline block: suspend CoroutineScope.() -> T +): T = withContext(this, block) + +// --------------- implementation --------------- + +private open class StandaloneCoroutine( + parentContext: CoroutineContext, + active: Boolean +) : AbstractCoroutine(parentContext, initParentJob = true, active = active) { + override fun handleJobException(exception: Throwable): Boolean { + handleCoroutineException(context, exception) + return true + } +} + +private class LazyStandaloneCoroutine( + parentContext: CoroutineContext, + block: suspend CoroutineScope.() -> Unit +) : StandaloneCoroutine(parentContext, active = false) { + private val continuation = block.createCoroutineUnintercepted(this, this) + + override fun onStart() { + continuation.startCoroutineCancellable(this) + } +} + +// Used by withContext when context changes, but dispatcher stays the same +internal expect class UndispatchedCoroutine( + context: CoroutineContext, + uCont: Continuation +) : ScopeCoroutine + +private const val UNDECIDED = 0 +private const val SUSPENDED = 1 +private const val RESUMED = 2 + +// Used by withContext when context dispatcher changes +internal class DispatchedCoroutine( + context: CoroutineContext, + uCont: Continuation +) : ScopeCoroutine(context, uCont) { + // this is copy-and-paste of a decision state machine inside AbstractionContinuation + // todo: we may some-how abstract it via inline class + private val _decision = atomic(UNDECIDED) + + private fun trySuspend(): Boolean { + _decision.loop { decision -> + when (decision) { + UNDECIDED -> if (this._decision.compareAndSet(UNDECIDED, SUSPENDED)) return true + RESUMED -> return false + else -> error("Already suspended") + } + } + } + + private fun tryResume(): Boolean { + _decision.loop { decision -> + when (decision) { + UNDECIDED -> if (this._decision.compareAndSet(UNDECIDED, RESUMED)) return true + SUSPENDED -> return false + else -> error("Already resumed") + } + } + } + + override fun afterCompletion(state: Any?) { + // Call afterResume from afterCompletion and not vice-versa, because stack-size is more + // important for afterResume implementation + afterResume(state) + } + + override fun afterResume(state: Any?) { + if (tryResume()) return // completed before getResult invocation -- bail out + // Resume in a cancellable way because we have to switch back to the original dispatcher + uCont.intercepted().resumeCancellableWith(recoverResult(state, uCont)) + } + + internal fun getResult(): Any? { + if (trySuspend()) return COROUTINE_SUSPENDED + // otherwise, onCompletionInternal was already invoked & invoked tryResume, and the result is in the state + val state = this.state.unboxState() + if (state is CompletedExceptionally) throw state.cause + @Suppress("UNCHECKED_CAST") + return state as T + } +} diff --git a/kotlinx-coroutines-core/common/src/CancellableContinuation.kt b/kotlinx-coroutines-core/common/src/CancellableContinuation.kt new file mode 100644 index 0000000000..44d61fe861 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CancellableContinuation.kt @@ -0,0 +1,497 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +/** + * Cancellable [continuation][Continuation] is a thread-safe continuation primitive with the support of + * an asynchronous cancellation. + * + * Cancellable continuation can be [resumed][Continuation.resumeWith], but unlike regular [Continuation], + * it also might be [cancelled][CancellableContinuation.cancel] explicitly or [implicitly][Job.cancel] via a parent [job][Job]. + * + * If the continuation is cancelled successfully, it resumes with a [CancellationException] or + * the specified cancel cause. + * + * ### Usage + * + * An instance of `CancellableContinuation` can only be obtained by the [suspendCancellableCoroutine] function. + * The interface itself is public for use and private for implementation. + * + * A typical usages of this function is to suspend a coroutine while waiting for a result + * from a callback or an external source of values that optionally supports cancellation: + * + * ``` + * suspend fun CompletableFuture.await(): T = suspendCancellableCoroutine { c -> + * val future = this + * future.whenComplete { result, throwable -> + * if (throwable != null) { + * // Resume continuation with an exception if an external source failed + * c.resumeWithException(throwable) + * } else { + * // Resume continuation with a value if it was computed + * c.resume(result) + * } + * } + * // Cancel the computation if the continuation itself was cancelled because a caller of 'await' is cancelled + * c.invokeOnCancellation { future.cancel(true) } + * } + * ``` + * + * ### Thread-safety + * + * Instances of [CancellableContinuation] are thread-safe and can be safely shared across multiple threads. + * [CancellableContinuation] allows concurrent invocations of the [cancel] and [resume] pair, guaranteeing + * that only one of these operations will succeed. + * Concurrent invocations of [resume] methods lead to a [IllegalStateException] and are considered a programmatic error. + * Concurrent invocations of [cancel] methods is permitted, and at most one of them succeeds. + * + * ### Prompt cancellation guarantee + * + * A cancellable continuation provides a **prompt cancellation guarantee**. + * + * If the [Job] of the coroutine that obtained a cancellable continuation was cancelled while this continuation was suspended it will not resume + * successfully, even if [CancellableContinuation.resume] was already invoked but not yet executed. + * + * The cancellation of the coroutine's job is generally asynchronous with respect to the suspended coroutine. + * The suspended coroutine is resumed with a call to its [Continuation.resumeWith] member function or to the + * [resume][Continuation.resume] extension function. + * However, when the coroutine is resumed, it does not immediately start executing but is passed to its + * [CoroutineDispatcher] to schedule its execution when the dispatcher's resources become available for execution. + * The job's cancellation can happen before, after, and concurrently with the call to `resume`. In any + * case, prompt cancellation guarantees that the coroutine will not resume its code successfully. + * + * If the coroutine was resumed with an exception (for example, using the [Continuation.resumeWithException] extension + * function) and cancelled, then the exception thrown by the `suspendCancellableCoroutine` function is determined + * by what happened first: exceptional resume or cancellation. + * + * ### Resuming with a closeable resource + * + * [CancellableContinuation] provides the capability to work with values that represent a resource that should be + * closed. For that, it provides `resume(value: R, onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)` + * function that guarantees that either the given `value` will be successfully returned from the corresponding + * `suspend` function or that `onCancellation` will be invoked with the supplied value: + * + * ``` + * continuation.resume(resourceToResumeWith) { _, resourceToClose, _ + * // Will be invoked if the continuation is cancelled while being dispatched + * resourceToClose.close() + * } + * ``` + * + * #### Continuation states + * + * A cancellable continuation has three observable states: + * + * | **State** | [isActive] | [isCompleted] | [isCancelled] | + * | ----------------------------------- | ---------- | ------------- | ------------- | + * | _Active_ (initial state) | `true` | `false` | `false` | + * | _Resumed_ (final _completed_ state) | `false` | `true` | `false` | + * | _Canceled_ (final _completed_ state)| `false` | `true` | `true` | + * + * For a detailed description of each state, see the corresponding properties' documentation. + * + * A successful invocation of [cancel] transitions the continuation from an _active_ to a _cancelled_ state, while + * an invocation of [Continuation.resume] or [Continuation.resumeWithException] transitions it from + * an _active_ to _resumed_ state. + * + * Possible state transitions diagram: + * ``` + * +-----------+ resume +---------+ + * | Active | ----------> | Resumed | + * +-----------+ +---------+ + * | + * | cancel + * V + * +-----------+ + * | Cancelled | + * +-----------+ + * ``` + */ +@OptIn(ExperimentalSubclassOptIn::class) +@SubclassOptInRequired(InternalForInheritanceCoroutinesApi::class) +public interface CancellableContinuation : Continuation { + /** + * Returns `true` when this continuation is active -- it was created, + * but not yet [resumed][Continuation.resumeWith] or [cancelled][CancellableContinuation.cancel]. + * + * This state implies that [isCompleted] and [isCancelled] are `false`, + * but this can change immediately after the invocation because of parallel calls to [cancel] and [resume]. + */ + public val isActive: Boolean + + /** + * Returns `true` when this continuation was completed -- [resumed][Continuation.resumeWith] or + * [cancelled][CancellableContinuation.cancel]. + * + * This state implies that [isActive] is `false`. + */ + public val isCompleted: Boolean + + /** + * Returns `true` if this continuation was [cancelled][CancellableContinuation.cancel]. + * + * It implies that [isActive] is `false` and [isCompleted] is `true`. + */ + public val isCancelled: Boolean + + /** + * Tries to resume this continuation with the specified [value] and returns a non-null object token if successful, + * or `null` otherwise (it was already resumed or cancelled). When a non-null object is returned, + * [completeResume] must be invoked with it. + * + * When [idempotent] is not `null`, this function performs an _idempotent_ operation, so that + * further invocations with the same non-null reference produce the same result. + * + * @suppress **This is unstable API and it is subject to change.** + */ + @InternalCoroutinesApi + public fun tryResume(value: T, idempotent: Any? = null): Any? + + /** + * Same as [tryResume] but with an [onCancellation] handler that is called if and only if the value is not + * delivered to the caller because of the dispatch in the process. + * + * The purpose of this function is to enable atomic delivery guarantees: either resumption succeeded, passing + * the responsibility for [value] to the continuation, or the [onCancellation] block will be invoked, + * allowing one to free the resources in [value]. + * + * Implementation note: current implementation always returns RESUME_TOKEN or `null` + * + * @suppress **This is unstable API and it is subject to change.** + */ + @InternalCoroutinesApi + public fun tryResume( + value: R, idempotent: Any?, onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)? + ): Any? + + /** + * Tries to resume this continuation with the specified [exception] and returns a non-null object token if successful, + * or `null` otherwise (it was already resumed or cancelled). When a non-null object is returned, + * [completeResume] must be invoked with it. + * + * @suppress **This is unstable API and it is subject to change.** + */ + @InternalCoroutinesApi + public fun tryResumeWithException(exception: Throwable): Any? + + /** + * Completes the execution of [tryResume] or [tryResumeWithException] on its non-null result. + * + * @suppress **This is unstable API and it is subject to change.** + */ + @InternalCoroutinesApi + public fun completeResume(token: Any) + + /** + * Internal function that setups cancellation behavior in [suspendCancellableCoroutine]. + * It's illegal to call this function in any non-`kotlinx.coroutines` code and + * such calls lead to undefined behaviour. + * Exposed in our ABI since 1.0.0 withing `suspendCancellableCoroutine` body. + * + * @suppress **This is unstable API and it is subject to change.** + */ + @InternalCoroutinesApi + public fun initCancellability() + + /** + * Cancels this continuation with an optional cancellation `cause`. The result is `true` if this continuation was + * cancelled as a result of this invocation, and `false` otherwise. + * [cancel] might return `false` when the continuation was either [resumed][resume] or already [cancelled][cancel]. + */ + public fun cancel(cause: Throwable? = null): Boolean + + /** + * Registers a [handler] to be **synchronously** invoked on [cancellation][cancel] (regular or exceptional) of this continuation. + * When the continuation is already cancelled, the handler is immediately invoked with the cancellation exception. + * Otherwise, the handler will be invoked as soon as this continuation is cancelled. + * + * The installed [handler] should not throw any exceptions. + * If it does, they will get caught, wrapped into a `CompletionHandlerException` and + * processed as an uncaught exception in the context of the current coroutine + * (see [CoroutineExceptionHandler]). + * + * At most one [handler] can be installed on a continuation. + * Attempting to call `invokeOnCancellation` a second time produces an [IllegalStateException]. + * + * This handler is also called when this continuation [resumes][Continuation.resume] normally (with a value) and then + * is cancelled while waiting to be dispatched. More generally speaking, this handler is called whenever + * the caller of [suspendCancellableCoroutine] is getting a [CancellationException]. + * + * A typical example of `invokeOnCancellation` usage is given in + * the documentation for the [suspendCancellableCoroutine] function. + * + * **Note**: Implementations of [CompletionHandler] must be fast, non-blocking, and thread-safe. + * This [handler] can be invoked concurrently with the surrounding code. + * There is no guarantee on the execution context in which the [handler] will be invoked. + */ + public fun invokeOnCancellation(handler: CompletionHandler) + + /** + * Resumes this continuation with the specified [value] in the invoker thread without going through + * the [dispatch][CoroutineDispatcher.dispatch] function of the [CoroutineDispatcher] in the [context]. + * This function is designed to only be used by [CoroutineDispatcher] implementations. + * **It should not be used in general code**. + * + * **Note: This function is experimental.** Its signature general code may be changed in the future. + */ + @ExperimentalCoroutinesApi + public fun CoroutineDispatcher.resumeUndispatched(value: T) + + /** + * Resumes this continuation with the specified [exception] in the invoker thread without going through + * the [dispatch][CoroutineDispatcher.dispatch] function of the [CoroutineDispatcher] in the [context]. + * This function is designed to only be used by [CoroutineDispatcher] implementations. + * **It should not be used in general code**. + * + * **Note: This function is experimental.** Its signature general code may be changed in the future. + */ + @ExperimentalCoroutinesApi + public fun CoroutineDispatcher.resumeUndispatchedWithException(exception: Throwable) + + /** @suppress */ + @Deprecated( + "Use the overload that also accepts the `value` and the coroutine context in lambda", + level = DeprecationLevel.WARNING, + replaceWith = ReplaceWith("resume(value) { cause, _, _ -> onCancellation(cause) }") + ) // warning since 1.9.0, was experimental + public fun resume(value: T, onCancellation: ((cause: Throwable) -> Unit)?) + + /** + * Resumes this continuation with the specified [value], calling the specified [onCancellation] if and only if + * the [value] was not successfully used to resume the continuation. + * + * The [value] can be rejected in two cases (in both of which [onCancellation] will be called): + * - Cancellation happened before the handler was resumed; + * - The continuation was resumed successfully (before cancellation), but the coroutine's job was cancelled before + * it had a chance to run in its dispatcher, and so the suspended function threw an exception instead of returning + * this value. + * + * The installed [onCancellation] handler should not throw any exceptions. + * If it does, they will get caught, wrapped into a `CompletionHandlerException`, and + * processed as an uncaught exception in the context of the current coroutine + * (see [CoroutineExceptionHandler]). + * + * With this version of [resume], it's possible to pass resources that can not simply be left for the garbage + * collector (like file handles, sockets, etc.) and need to be closed explicitly: + * + * ``` + * continuation.resume(resourceToResumeWith) { _, resourceToClose, _ -> + * resourceToClose.close() + * } + * ``` + * + * [onCancellation] accepts three arguments: + * + * - `cause: Throwable` is the exception with which the continuation was cancelled. + * - `value` is exactly the same as the [value] passed to [resume] itself. + * In the example above, `resourceToResumeWith` is exactly the same as `resourceToClose`; in particular, + * one could call `resourceToResumeWith.close()` in the lambda for the same effect. + * The reason to reference `resourceToClose` anyway is to avoid a memory allocation due to the lambda + * capturing the `resourceToResumeWith` reference. + * - `context` is the [context] of this continuation. + * Like with `value`, the reason this is available as a lambda parameter, even though it is always possible to + * call [context] from the lambda instead, is to allow lambdas to capture less of their environment. + * + * A more complete example and further details are given in + * the documentation for the [suspendCancellableCoroutine] function. + * + * **Note**: The [onCancellation] handler must be fast, non-blocking, and thread-safe. + * It can be invoked concurrently with the surrounding code. + * There is no guarantee on the execution context of its invocation. + */ + public fun resume( + value: R, onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)? + ) +} + +/** + * A version of `invokeOnCancellation` that accepts a class as a handler instead of a lambda, but identical otherwise. + * This allows providing a custom [toString] instance that will look better during debugging. + */ +internal fun CancellableContinuation.invokeOnCancellation(handler: CancelHandler) = when (this) { + is CancellableContinuationImpl -> invokeOnCancellationInternal(handler) + else -> throw UnsupportedOperationException("third-party implementation of CancellableContinuation is not supported") +} + +/** + * Suspends the coroutine like [suspendCoroutine], but providing a [CancellableContinuation] to + * the [block]. This function throws a [CancellationException] if the [Job] of the coroutine is + * cancelled or completed while it is suspended, or if [CancellableContinuation.cancel] is invoked. + * + * A typical use of this function is to suspend a coroutine while waiting for a result + * from a single-shot callback API and to return the result to the caller. + * For multi-shot callback APIs see [callbackFlow][kotlinx.coroutines.flow.callbackFlow]. + * + * ``` + * suspend fun awaitCallback(): T = suspendCancellableCoroutine { continuation -> + * val callback = object : Callback { // Implementation of some callback interface + * override fun onCompleted(value: T) { + * // Resume coroutine with a value provided by the callback + * continuation.resume(value) + * } + * override fun onApiError(cause: Throwable) { + * // Resume coroutine with an exception provided by the callback + * continuation.resumeWithException(cause) + * } + * } + * // Register callback with an API + * api.register(callback) + * // Remove callback on cancellation + * continuation.invokeOnCancellation { api.unregister(callback) } + * // At this point the coroutine is suspended by suspendCancellableCoroutine until callback fires + * } + * ``` + * + * > The callback `register`/`unregister` methods provided by an external API must be thread-safe, because + * > `invokeOnCancellation` block can be called at any time due to asynchronous nature of cancellation, even + * > concurrently with the call of the callback. + * + * ### Prompt cancellation guarantee + * + * This function provides **prompt cancellation guarantee**. + * If the [Job] of the current coroutine was cancelled while this function was suspended it will not resume + * successfully, even if [CancellableContinuation.resume] was already invoked. + * + * The cancellation of the coroutine's job is generally asynchronous with respect to the suspended coroutine. + * The suspended coroutine is resumed with a call to its [Continuation.resumeWith] member function or to the + * [resume][Continuation.resume] extension function. + * However, when coroutine is resumed, it does not immediately start executing, but is passed to its + * [CoroutineDispatcher] to schedule its execution when dispatcher's resources become available for execution. + * The job's cancellation can happen before, after, and concurrently with the call to `resume`. In any + * case, prompt cancellation guarantees that the coroutine will not resume its code successfully. + * + * If the coroutine was resumed with an exception (for example, using [Continuation.resumeWithException] extension + * function) and cancelled, then the exception thrown by the `suspendCancellableCoroutine` function is determined + * by what happened first: exceptional resume or cancellation. + * + * ### Returning resources from a suspended coroutine + * + * As a result of the prompt cancellation guarantee, when a closeable resource + * (like open file or a handle to another native resource) is returned from a suspended coroutine as a value, + * it can be lost when the coroutine is cancelled. To ensure that the resource can be properly closed + * in this case, the [CancellableContinuation] interface provides two functions. + * + * - [invokeOnCancellation][CancellableContinuation.invokeOnCancellation] installs a handler that is called + * whenever a suspend coroutine is being cancelled. In addition to the example at the beginning, it can be + * used to ensure that a resource that was opened before the call to + * `suspendCancellableCoroutine` or in its body is closed in case of cancellation. + * + * ``` + * suspendCancellableCoroutine { continuation -> + * val resource = openResource() // Opens some resource + * continuation.invokeOnCancellation { + * resource.close() // Ensures the resource is closed on cancellation + * } + * // ... + * } + * ``` + * + * - [resume(value) { ... }][CancellableContinuation.resume] method on a [CancellableContinuation] takes + * an optional `onCancellation` block. It can be used when resuming with a resource that must be closed by + * the code that called the corresponding suspending function. + * + * ``` + * suspendCancellableCoroutine { continuation -> + * val callback = object : Callback { // Implementation of some callback interface + * // A callback provides a reference to some closeable resource + * override fun onCompleted(resource: T) { + * // Resume coroutine with a value provided by the callback and ensure the resource is closed in case + * // when the coroutine is cancelled before the caller gets a reference to the resource. + * continuation.resume(resource) { cause, resourceToClose, context -> + * resourceToClose.close() // Close the resource on cancellation + * // If we used `resource` instead of `resourceToClose`, this lambda would need to allocate a closure, + * // but with `resourceToClose`, the lambda does not capture any of its environment. + * } + * } + * // ... + * } + * ``` + * + * ### Implementation details and custom continuation interceptors + * + * The prompt cancellation guarantee is the result of a coordinated implementation inside `suspendCancellableCoroutine` + * function and the [CoroutineDispatcher] class. The coroutine dispatcher checks for the status of the [Job] immediately + * before continuing its normal execution and aborts this normal execution, calling all the corresponding + * cancellation handlers, if the job was cancelled. + * + * If a custom implementation of [ContinuationInterceptor] is used in a coroutine's context that does not extend + * [CoroutineDispatcher] class, then there is no prompt cancellation guarantee. A custom continuation interceptor + * can resume execution of a previously suspended coroutine even if its job was already cancelled. + */ +public suspend inline fun suspendCancellableCoroutine( + crossinline block: (CancellableContinuation) -> Unit +): T = + suspendCoroutineUninterceptedOrReturn { uCont -> + val cancellable = CancellableContinuationImpl(uCont.intercepted(), resumeMode = MODE_CANCELLABLE) + /* + * For non-atomic cancellation we setup parent-child relationship immediately + * in case when `block` blocks the current thread (e.g. Rx2 with trampoline scheduler), but + * properly supports cancellation. + */ + cancellable.initCancellability() + block(cancellable) + cancellable.getResult() + } + +/** + * Suspends the coroutine similar to [suspendCancellableCoroutine], but an instance of + * [CancellableContinuationImpl] is reused. + */ +internal suspend inline fun suspendCancellableCoroutineReusable( + crossinline block: (CancellableContinuationImpl) -> Unit +): T = suspendCoroutineUninterceptedOrReturn { uCont -> + val cancellable = getOrCreateCancellableContinuation(uCont.intercepted()) + try { + block(cancellable) + } catch (e: Throwable) { + // Here we catch any unexpected exception from user-supplied block (e.g. invariant violation) + // and release claimed continuation in order to leave it in a reasonable state (see #3613) + cancellable.releaseClaimedReusableContinuation() + throw e + } + cancellable.getResult() +} + +internal fun getOrCreateCancellableContinuation(delegate: Continuation): CancellableContinuationImpl { + // If used outside our dispatcher + if (delegate !is DispatchedContinuation) { + return CancellableContinuationImpl(delegate, MODE_CANCELLABLE) + } + /* + * Attempt to claim reusable instance. + * + * suspendCancellableCoroutineReusable { // <- claimed + * // Any asynchronous cancellation is "postponed" while this block + * // is being executed + * } // postponed cancellation is checked here. + * + * Claim can fail for the following reasons: + * 1) Someone tried to make idempotent resume. + * Idempotent resume is internal (used only by us) and is used only in `select`, + * thus leaking CC instance for indefinite time. + * 2) Continuation was cancelled. Then we should prevent any further reuse and bail out. + */ + return delegate.claimReusableCancellableContinuation()?.takeIf { it.resetStateReusable() } + ?: return CancellableContinuationImpl(delegate, MODE_CANCELLABLE_REUSABLE) +} + +/** + * Disposes the specified [handle] when this continuation is cancelled. + * + * This is a shortcut for the following code with slightly more efficient implementation (one fewer object created): + * ``` + * invokeOnCancellation { handle.dispose() } + * ``` + * + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public fun CancellableContinuation<*>.disposeOnCancellation(handle: DisposableHandle): Unit = + invokeOnCancellation(handler = DisposeOnCancel(handle)) + +private class DisposeOnCancel(private val handle: DisposableHandle) : CancelHandler { + override fun invoke(cause: Throwable?) = handle.dispose() + override fun toString(): String = "DisposeOnCancel[$handle]" +} diff --git a/kotlinx-coroutines-core/common/src/CancellableContinuationImpl.kt b/kotlinx-coroutines-core/common/src/CancellableContinuationImpl.kt new file mode 100644 index 0000000000..3dc07f1e0f --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CancellableContinuationImpl.kt @@ -0,0 +1,700 @@ +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlin.jvm.* + +private const val UNDECIDED = 0 +private const val SUSPENDED = 1 +private const val RESUMED = 2 + +private const val DECISION_SHIFT = 29 +private const val INDEX_MASK = (1 shl DECISION_SHIFT) - 1 +private const val NO_INDEX = INDEX_MASK + +private inline val Int.decision get() = this shr DECISION_SHIFT +private inline val Int.index get() = this and INDEX_MASK +@Suppress("NOTHING_TO_INLINE") +private inline fun decisionAndIndex(decision: Int, index: Int) = (decision shl DECISION_SHIFT) + index + +@JvmField +internal val RESUME_TOKEN = Symbol("RESUME_TOKEN") + +/** + * @suppress **This is unstable API and it is subject to change.** + */ +@OptIn(InternalForInheritanceCoroutinesApi::class) +@PublishedApi +internal open class CancellableContinuationImpl( + final override val delegate: Continuation, + resumeMode: Int +) : DispatchedTask(resumeMode), CancellableContinuation, CoroutineStackFrame, Waiter { + init { + assert { resumeMode != MODE_UNINITIALIZED } // invalid mode for CancellableContinuationImpl + } + + public override val context: CoroutineContext = delegate.context + + /* + * Implementation notes + * + * CancellableContinuationImpl is a subset of Job with following limitations: + * 1) It can have only cancellation listener (no "on cancelling") + * 2) It always invokes cancellation listener if it's cancelled (no 'invokeImmediately') + * 3) It can have at most one cancellation listener + * 4) Its cancellation listeners cannot be deregistered + * As a consequence it has much simpler state machine, more lightweight machinery and + * less dependencies. + */ + + /** decision state machine + + +-----------+ trySuspend +-----------+ + | UNDECIDED | -------------> | SUSPENDED | + +-----------+ +-----------+ + | + | tryResume + V + +-----------+ + | RESUMED | + +-----------+ + + Note: both tryResume and trySuspend can be invoked at most once, first invocation wins. + If the cancellation handler is specified via a [Segment] instance and the index in it + (so [Segment.onCancellation] should be called), the [_decisionAndIndex] field may store + this index additionally to the "decision" value. + */ + private val _decisionAndIndex = atomic(decisionAndIndex(UNDECIDED, NO_INDEX)) + + /* + === Internal states === + name state class public state description + ------ ------------ ------------ ----------- + ACTIVE Active : Active active, no listeners + SINGLE_A CancelHandler : Active active, one cancellation listener + CANCELLED CancelledContinuation: Cancelled cancelled (final state) + COMPLETED any : Completed produced some result or threw an exception (final state) + */ + private val _state = atomic(Active) + + /* + * This field has a concurrent rendezvous in the following scenario: + * + * - installParentHandle publishes this instance on T1 + * + * T1 writes: + * - handle = installed; right after the installation + * - Shortly after: if (isComplete) handle = NonDisposableHandle + * + * Any other T writes if the parent job is cancelled in detachChild: + * - handle = NonDisposableHandle + * + * We want to preserve a strict invariant on parentHandle transition, allowing only three of them: + * null -> anyHandle + * anyHandle -> NonDisposableHandle + * null -> NonDisposableHandle + * + * With a guarantee that after disposal the only state handle may end up in is NonDisposableHandle + */ + private val _parentHandle = atomic(null) + private val parentHandle: DisposableHandle? + get() = _parentHandle.value + + internal val state: Any? get() = _state.value + + public override val isActive: Boolean get() = state is NotCompleted + + public override val isCompleted: Boolean get() = state !is NotCompleted + + public override val isCancelled: Boolean get() = state is CancelledContinuation + + // We cannot invoke `state.toString()` since it may cause a circular dependency + private val stateDebugRepresentation get() = when(state) { + is NotCompleted -> "Active" + is CancelledContinuation -> "Cancelled" + else -> "Completed" + } + + public override fun initCancellability() { + /* + * Invariant: at the moment of invocation, `this` has not yet + * leaked to user code and no one is able to invoke `resume` or `cancel` + * on it yet. Also, this function is not invoked for reusable continuations. + */ + val handle = installParentHandle() + ?: return // fast path -- don't do anything without parent + // now check our state _after_ registering, could have completed while we were registering, + // but only if parent was cancelled. Parent could be in a "cancelling" state for a while, + // so we are helping it and cleaning the node ourselves + if (isCompleted) { + // Can be invoked concurrently in 'parentCancelled', no problems here + handle.dispose() + _parentHandle.value = NonDisposableHandle + } + } + + private fun isReusable(): Boolean = resumeMode.isReusableMode && (delegate as DispatchedContinuation<*>).isReusable() + + /** + * Resets cancellability state in order to [suspendCancellableCoroutineReusable] to work. + * Invariant: used only by [suspendCancellableCoroutineReusable] in [REUSABLE_CLAIMED] state. + */ + @JvmName("resetStateReusable") // Prettier stack traces + internal fun resetStateReusable(): Boolean { + assert { resumeMode == MODE_CANCELLABLE_REUSABLE } + assert { parentHandle !== NonDisposableHandle } + val state = _state.value + assert { state !is NotCompleted } + if (state is CompletedContinuation<*> && state.idempotentResume != null) { + // Cannot reuse continuation that was resumed with idempotent marker + detachChild() + return false + } + _decisionAndIndex.value = decisionAndIndex(UNDECIDED, NO_INDEX) + _state.value = Active + return true + } + + public override val callerFrame: CoroutineStackFrame? + get() = delegate as? CoroutineStackFrame + + public override fun getStackTraceElement(): StackTraceElement? = null + + override fun takeState(): Any? = state + + // Note: takeState does not clear the state so we don't use takenState + // and we use the actual current state where in CAS-loop + override fun cancelCompletedResult(takenState: Any?, cause: Throwable): Unit = _state.loop { state -> + when (state) { + is NotCompleted -> error("Not completed") + is CompletedExceptionally -> return // already completed exception or cancelled, nothing to do + is CompletedContinuation<*> -> { + check(!state.cancelled) { "Must be called at most once" } + val update = state.copy(cancelCause = cause) + if (_state.compareAndSet(state, update)) { + state.invokeHandlers(this, cause) + return // done + } + } + else -> { + // completed normally without marker class, promote to CompletedContinuation in case + // if invokeOnCancellation if called later + if (_state.compareAndSet(state, CompletedContinuation(state, cancelCause = cause))) { + return // done + } + } + } + } + + /* + * Attempt to postpone cancellation for reusable cancellable continuation + */ + private fun cancelLater(cause: Throwable): Boolean { + // Ensure that we are postponing cancellation to the right reusable instance + if (!isReusable()) return false + val dispatched = delegate as DispatchedContinuation<*> + return dispatched.postponeCancellation(cause) + } + + public override fun cancel(cause: Throwable?): Boolean { + _state.loop { state -> + if (state !is NotCompleted) return false // false if already complete or cancelling + // Active -- update to final state + val update = CancelledContinuation(this, cause, handled = state is CancelHandler || state is Segment<*>) + if (!_state.compareAndSet(state, update)) return@loop // retry on cas failure + // Invoke cancel handler if it was present + when (state) { + is CancelHandler -> callCancelHandler(state, cause) + is Segment<*> -> callSegmentOnCancellation(state, cause) + } + // Complete state update + detachChildIfNonReusable() + dispatchResume(resumeMode) // no need for additional cancellation checks + return true + } + } + + internal fun parentCancelled(cause: Throwable) { + if (cancelLater(cause)) return + cancel(cause) + // Even if cancellation has failed, we should detach child to avoid potential leak + detachChildIfNonReusable() + } + + private inline fun callCancelHandlerSafely(block: () -> Unit) { + try { + block() + } catch (ex: Throwable) { + // Handler should never fail, if it does -- it is an unhandled exception + handleCoroutineException( + context, + CompletionHandlerException("Exception in invokeOnCancellation handler for $this", ex) + ) + } + } + + fun callCancelHandler(handler: CancelHandler, cause: Throwable?) = + callCancelHandlerSafely { handler.invoke(cause) } + + private fun callSegmentOnCancellation(segment: Segment<*>, cause: Throwable?) { + val index = _decisionAndIndex.value.index + check(index != NO_INDEX) { "The index for Segment.onCancellation(..) is broken" } + callCancelHandlerSafely { segment.onCancellation(index, cause, context) } + } + + fun callOnCancellation( + onCancellation: (cause: Throwable, value: R, context: CoroutineContext) -> Unit, + cause: Throwable, + value: R + ) { + try { + onCancellation.invoke(cause, value, context) + } catch (ex: Throwable) { + // Handler should never fail, if it does -- it is an unhandled exception + handleCoroutineException( + context, + CompletionHandlerException("Exception in resume onCancellation handler for $this", ex) + ) + } + } + + /** + * It is used when parent is cancelled to get the cancellation cause for this continuation. + */ + open fun getContinuationCancellationCause(parent: Job): Throwable = + parent.getCancellationException() + + private fun trySuspend(): Boolean { + _decisionAndIndex.loop { cur -> + when (cur.decision) { + UNDECIDED -> if (this._decisionAndIndex.compareAndSet(cur, decisionAndIndex(SUSPENDED, cur.index))) return true + RESUMED -> return false + else -> error("Already suspended") + } + } + } + + private fun tryResume(): Boolean { + _decisionAndIndex.loop { cur -> + when (cur.decision) { + UNDECIDED -> if (this._decisionAndIndex.compareAndSet(cur, decisionAndIndex(RESUMED, cur.index))) return true + SUSPENDED -> return false + else -> error("Already resumed") + } + } + } + + @PublishedApi + internal fun getResult(): Any? { + val isReusable = isReusable() + // trySuspend may fail either if 'block' has resumed/cancelled a continuation, + // or we got async cancellation from parent. + if (trySuspend()) { + /* + * Invariant: parentHandle is `null` *only* for reusable continuations. + * We were neither resumed nor cancelled, time to suspend. + * But first we have to install parent cancellation handle (if we didn't yet), + * so CC could be properly resumed on parent cancellation. + * + * This read has benign data-race with write of 'NonDisposableHandle' + * in 'detachChildIfNotReusable'. + */ + if (parentHandle == null) { + installParentHandle() + } + /* + * Release the continuation after installing the handle (if needed). + * If we were successful, then do nothing, it's ok to reuse the instance now. + * Otherwise, dispose the handle by ourselves. + */ + if (isReusable) { + releaseClaimedReusableContinuation() + } + return COROUTINE_SUSPENDED + } + // otherwise, onCompletionInternal was already invoked & invoked tryResume, and the result is in the state + if (isReusable) { + // release claimed reusable continuation for the future reuse + releaseClaimedReusableContinuation() + } + val state = this.state + if (state is CompletedExceptionally) throw recoverStackTrace(state.cause, this) + // if the parent job was already cancelled, then throw the corresponding cancellation exception + // otherwise, there is a race if suspendCancellableCoroutine { cont -> ... } does cont.resume(...) + // before the block returns. This getResult would return a result as opposed to cancellation + // exception that should have happened if the continuation is dispatched for execution later. + if (resumeMode.isCancellableMode) { + val job = context[Job] + if (job != null && !job.isActive) { + val cause = job.getCancellationException() + cancelCompletedResult(state, cause) + throw recoverStackTrace(cause, this) + } + } + return getSuccessfulResult(state) + } + + private fun installParentHandle(): DisposableHandle? { + val parent = context[Job] ?: return null // don't do anything without a parent + // Install the handle + val handle = parent.invokeOnCompletion(handler = ChildContinuation(this)) + _parentHandle.compareAndSet(null, handle) + return handle + } + + /** + * Tries to release reusable continuation. It can fail is there was an asynchronous cancellation, + * in which case it detaches from the parent and cancels this continuation. + */ + internal fun releaseClaimedReusableContinuation() { + // Cannot be cast if e.g. invoked from `installParentHandleReusable` for context without dispatchers, but with Job in it + val cancellationCause = (delegate as? DispatchedContinuation<*>)?.tryReleaseClaimedContinuation(this) ?: return + detachChild() + cancel(cancellationCause) + } + + override fun resumeWith(result: Result) = + resumeImpl(result.toState(this), resumeMode) + + @Suppress("OVERRIDE_DEPRECATION") + override fun resume(value: T, onCancellation: ((cause: Throwable) -> Unit)?) = + resumeImpl(value, resumeMode, onCancellation?.let { { cause, _, _ -> onCancellation(cause) } }) + + override fun resume( + value: R, + onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)? + ) = + resumeImpl(value, resumeMode, onCancellation) + + /** + * An optimized version for the code below that does not allocate + * a cancellation handler object and efficiently stores the specified + * [segment] and [index] in this [CancellableContinuationImpl]. + * + * The only difference is that `segment.onCancellation(..)` is never + * called if this continuation is already completed; + * + * ``` + * invokeOnCancellation { cause -> + * segment.onCancellation(index, cause) + * } + * ``` + */ + override fun invokeOnCancellation(segment: Segment<*>, index: Int) { + _decisionAndIndex.update { + check(it.index == NO_INDEX) { + "invokeOnCancellation should be called at most once" + } + decisionAndIndex(it.decision, index) + } + invokeOnCancellationImpl(segment) + } + + override fun invokeOnCancellation(handler: CompletionHandler) = + invokeOnCancellation(CancelHandler.UserSupplied(handler)) + + internal fun invokeOnCancellationInternal(handler: CancelHandler) = invokeOnCancellationImpl(handler) + + private fun invokeOnCancellationImpl(handler: Any) { + assert { handler is CancelHandler || handler is Segment<*> } + _state.loop { state -> + when (state) { + is Active -> { + if (_state.compareAndSet(state, handler)) return // quit on cas success + } + is CancelHandler, is Segment<*> -> multipleHandlersError(handler, state) + is CompletedExceptionally -> { + /* + * Continuation was already cancelled or completed exceptionally. + * NOTE: multiple invokeOnCancellation calls with different handlers are not allowed, + * so we check to make sure handler was installed just once. + */ + if (!state.makeHandled()) multipleHandlersError(handler, state) + /* + * Call the handler only if it was cancelled (not called when completed exceptionally). + * :KLUDGE: We have to invoke a handler in platform-specific way via `invokeIt` extension, + * because we play type tricks on Kotlin/JS and handler is not necessarily a function there + */ + if (state is CancelledContinuation) { + val cause: Throwable? = (state as? CompletedExceptionally)?.cause + if (handler is CancelHandler) { + callCancelHandler(handler, cause) + } else { + val segment = handler as Segment<*> + callSegmentOnCancellation(segment, cause) + } + } + return + } + + is CompletedContinuation<*> -> { + /* + * Continuation was already completed, and might already have cancel handler. + */ + if (state.cancelHandler != null) multipleHandlersError(handler, state) + // Segment.invokeOnCancellation(..) does NOT need to be called on completed continuation. + if (handler is Segment<*>) return + handler as CancelHandler + if (state.cancelled) { + // Was already cancelled while being dispatched -- invoke the handler directly + callCancelHandler(handler, state.cancelCause) + return + } + val update = state.copy(cancelHandler = handler) + if (_state.compareAndSet(state, update)) return // quit on cas success + } + else -> { + /* + * Continuation was already completed normally, but might get cancelled while being dispatched. + * Change its state to CompletedContinuation, unless we have Segment which + * does not need to be called in this case. + */ + if (handler is Segment<*>) return + handler as CancelHandler + val update = CompletedContinuation(state, cancelHandler = handler) + if (_state.compareAndSet(state, update)) return // quit on cas success + } + } + } + } + + private fun multipleHandlersError(handler: Any, state: Any?) { + error("It's prohibited to register multiple handlers, tried to register $handler, already has $state") + } + + private fun dispatchResume(mode: Int) { + if (tryResume()) return // completed before getResult invocation -- bail out + // otherwise, getResult has already commenced, i.e. completed later or in other thread + dispatch(mode) + } + + private fun resumedState( + state: NotCompleted, + proposedUpdate: R, + resumeMode: Int, + onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)?, + idempotent: Any? + ): Any? = when { + proposedUpdate is CompletedExceptionally -> { + assert { idempotent == null } // there are no idempotent exceptional resumes + assert { onCancellation == null } // only successful results can be cancelled + proposedUpdate + } + !resumeMode.isCancellableMode && idempotent == null -> proposedUpdate // cannot be cancelled in process, all is fine + onCancellation != null || state is CancelHandler || idempotent != null -> + // mark as CompletedContinuation if special cases are present: + // Cancellation handlers that shall be called after resume or idempotent resume + CompletedContinuation(proposedUpdate, state as? CancelHandler, onCancellation, idempotent) + else -> proposedUpdate // simple case -- use the value directly + } + + internal fun resumeImpl( + proposedUpdate: R, + resumeMode: Int, + onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)? = null + ) { + _state.loop { state -> + when (state) { + is NotCompleted -> { + val update = resumedState(state, proposedUpdate, resumeMode, onCancellation, idempotent = null) + if (!_state.compareAndSet(state, update)) return@loop // retry on cas failure + detachChildIfNonReusable() + dispatchResume(resumeMode) // dispatch resume, but it might get cancelled in process + return // done + } + + is CancelledContinuation -> { + /* + * If continuation was cancelled, then resume attempt must be ignored, + * because cancellation is asynchronous and may race with resume. + * Racy exceptions will be lost, too. + */ + if (state.makeResumed()) { // check if trying to resume one (otherwise error) + // call onCancellation + onCancellation?.let { callOnCancellation(it, state.cause, proposedUpdate) } + return // done + } + } + } + alreadyResumedError(proposedUpdate) // otherwise, an error (second resume attempt) + } + } + + /** + * Similar to [tryResume], but does not actually completes resume (needs [completeResume] call). + * Returns [RESUME_TOKEN] when resumed, `null` when it was already resumed or cancelled. + */ + private fun tryResumeImpl( + proposedUpdate: R, + idempotent: Any?, + onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)? + ): Symbol? { + _state.loop { state -> + when (state) { + is NotCompleted -> { + val update = resumedState(state, proposedUpdate, resumeMode, onCancellation, idempotent) + if (!_state.compareAndSet(state, update)) return@loop // retry on cas failure + detachChildIfNonReusable() + return RESUME_TOKEN + } + is CompletedContinuation<*> -> { + return if (idempotent != null && state.idempotentResume === idempotent) { + assert { state.result == proposedUpdate } // "Non-idempotent resume" + RESUME_TOKEN // resumed with the same token -- ok + } else { + null // resumed with a different token or non-idempotent -- too late + } + } + else -> return null // cannot resume -- not active anymore + } + } + } + + private fun alreadyResumedError(proposedUpdate: Any?): Nothing { + error("Already resumed, but proposed with update $proposedUpdate") + } + + // Unregister from parent job + private fun detachChildIfNonReusable() { + // If instance is reusable, do not detach on every reuse, #releaseInterceptedContinuation will do it for us in the end + if (!isReusable()) detachChild() + } + + /** + * Detaches from the parent. + */ + internal fun detachChild() { + val handle = parentHandle ?: return + handle.dispose() + _parentHandle.value = NonDisposableHandle + } + + // Note: Always returns RESUME_TOKEN | null + override fun tryResume(value: T, idempotent: Any?): Any? = + tryResumeImpl(value, idempotent, onCancellation = null) + + override fun tryResume( + value: R, + idempotent: Any?, + onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)? + ): Any? = + tryResumeImpl(value, idempotent, onCancellation) + + override fun tryResumeWithException(exception: Throwable): Any? = + tryResumeImpl(CompletedExceptionally(exception), idempotent = null, onCancellation = null) + + // note: token is always RESUME_TOKEN + override fun completeResume(token: Any) { + assert { token === RESUME_TOKEN } + dispatchResume(resumeMode) + } + + override fun CoroutineDispatcher.resumeUndispatched(value: T) { + val dc = delegate as? DispatchedContinuation + resumeImpl(value, if (dc?.dispatcher === this) MODE_UNDISPATCHED else resumeMode) + } + + override fun CoroutineDispatcher.resumeUndispatchedWithException(exception: Throwable) { + val dc = delegate as? DispatchedContinuation + resumeImpl(CompletedExceptionally(exception), if (dc?.dispatcher === this) MODE_UNDISPATCHED else resumeMode) + } + + @Suppress("UNCHECKED_CAST") + override fun getSuccessfulResult(state: Any?): T = + when (state) { + is CompletedContinuation<*> -> state.result as T + else -> state as T + } + + // The exceptional state in CancellableContinuationImpl is stored directly and it is not recovered yet. + // The stacktrace recovery is invoked here. + override fun getExceptionalResult(state: Any?): Throwable? = + super.getExceptionalResult(state)?.let { recoverStackTrace(it, delegate) } + + // For nicer debugging + public override fun toString(): String = + "${nameString()}(${delegate.toDebugString()}){$stateDebugRepresentation}@$hexAddress" + + protected open fun nameString(): String = + "CancellableContinuation" + +} + +// Marker for active continuation +internal interface NotCompleted + +private object Active : NotCompleted { + override fun toString(): String = "Active" +} + +/** + * Essentially the same as just a function from `Throwable?` to `Unit`. + * The only thing implementors can do is call [invoke]. + * The reason this abstraction exists is to allow providing a readable [toString] in the list of completion handlers + * as seen from the debugger. + * Use [UserSupplied] to create an instance from a lambda. + * We can't avoid defining a separate type, because on JS, you can't inherit from a function type. + */ +internal interface CancelHandler : NotCompleted { + /** + * Signals cancellation. + * + * This function: + * - Does not throw any exceptions. + * Violating this rule in an implementation leads to [handleUncaughtCoroutineException] being called with a + * [CompletionHandlerException] wrapping the thrown exception. + * - Is fast, non-blocking, and thread-safe. + * - Can be invoked concurrently with the surrounding code. + * - Can be invoked from any context. + * + * The meaning of `cause` that is passed to the handler is: + * - It is `null` if the continuation was cancelled directly via [CancellableContinuation.cancel] without a `cause`. + * - It is an instance of [CancellationException] if the continuation was _normally_ cancelled from the outside. + * **It should not be treated as an error**. In particular, it should not be reported to error logs. + * - Otherwise, the continuation had cancelled with an _error_. + */ + fun invoke(cause: Throwable?) + + /** + * A lambda passed from outside the coroutine machinery. + * + * See the requirements for [CancelHandler.invoke] when implementing this function. + */ + class UserSupplied(private val handler: (cause: Throwable?) -> Unit) : CancelHandler { + /** @suppress */ + override fun invoke(cause: Throwable?) { handler(cause) } + + override fun toString() = "CancelHandler.UserSupplied[${handler.classSimpleName}@$hexAddress]" + } +} + +// Completed with additional metadata +private data class CompletedContinuation( + @JvmField val result: R, + // installed via `invokeOnCancellation` + @JvmField val cancelHandler: CancelHandler? = null, + // installed via the `resume` block + @JvmField val onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)? = null, + @JvmField val idempotentResume: Any? = null, + @JvmField val cancelCause: Throwable? = null +) { + val cancelled: Boolean get() = cancelCause != null + + fun invokeHandlers(cont: CancellableContinuationImpl<*>, cause: Throwable) { + cancelHandler?.let { cont.callCancelHandler(it, cause) } + onCancellation?.let { cont.callOnCancellation(it, cause, result) } + } +} + +// Same as ChildHandleNode, but for cancellable continuation +private class ChildContinuation( + @JvmField val child: CancellableContinuationImpl<*> +) : JobNode() { + override val onCancelling get() = true + + override fun invoke(cause: Throwable?) { + child.parentCancelled(child.getContinuationCancellationCause(job)) + } +} diff --git a/kotlinx-coroutines-core/common/src/CloseableCoroutineDispatcher.kt b/kotlinx-coroutines-core/common/src/CloseableCoroutineDispatcher.kt new file mode 100644 index 0000000000..cc56121f02 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CloseableCoroutineDispatcher.kt @@ -0,0 +1,24 @@ +package kotlinx.coroutines + +/** + * [CoroutineDispatcher] that provides a method to close it, + * causing the rejection of any new tasks and cleanup of all underlying resources + * associated with the current dispatcher. + * Examples of closeable dispatchers are dispatchers backed by `java.lang.Executor` and + * by `kotlin.native.Worker`. + * + * **The `CloseableCoroutineDispatcher` class is not stable for inheritance in 3rd party libraries**, as new methods + * might be added to this interface in the future, but is stable for use. + */ +@ExperimentalCoroutinesApi +public expect abstract class CloseableCoroutineDispatcher() : CoroutineDispatcher, AutoCloseable { + + /** + * Initiate the closing sequence of the coroutine dispatcher. + * After a successful call to [close], no new tasks will be accepted to be [dispatched][dispatch]. + * The previously-submitted tasks will still be run, but [close] is not guaranteed to wait for them to finish. + * + * Invocations of `close` are idempotent and thread-safe. + */ + public abstract override fun close() +} diff --git a/kotlinx-coroutines-core/common/src/CompletableDeferred.kt b/kotlinx-coroutines-core/common/src/CompletableDeferred.kt new file mode 100644 index 0000000000..2788ce8298 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CompletableDeferred.kt @@ -0,0 +1,90 @@ +@file:Suppress("DEPRECATION_ERROR") + +package kotlinx.coroutines + +import kotlinx.coroutines.selects.* + +/** + * A [Deferred] that can be completed via public functions [complete] or [cancel][Job.cancel]. + * + * Note that the [complete] function returns `false` when this deferred value is already complete or completing, + * while [cancel][Job.cancel] returns `true` as long as the deferred is still _cancelling_ and the corresponding + * exception is incorporated into the final [completion exception][getCompletionExceptionOrNull]. + * + * An instance of completable deferred can be created by `CompletableDeferred()` function in _active_ state. + * + * All functions on this interface are **thread-safe** and can + * be safely invoked from concurrent coroutines without external synchronization. + */ +@OptIn(ExperimentalSubclassOptIn::class) +@SubclassOptInRequired(InternalForInheritanceCoroutinesApi::class) +public interface CompletableDeferred : Deferred { + /** + * Completes this deferred value with a given [value]. The result is `true` if this deferred was + * completed as a result of this invocation and `false` otherwise (if it was already completed). + * + * Subsequent invocations of this function have no effect and always produce `false`. + * + * This function transitions this deferred into _completed_ state if it was not completed or cancelled yet. + * However, if this deferred has children, then it transitions into _completing_ state and becomes _complete_ + * once all its children are [complete][isCompleted]. See [Job] for details. + */ + public fun complete(value: T): Boolean + + /** + * Completes this deferred value exceptionally with a given [exception]. The result is `true` if this deferred was + * completed as a result of this invocation and `false` otherwise (if it was already completed). + * + * Subsequent invocations of this function have no effect and always produce `false`. + * + * This function transitions this deferred into _cancelled_ state if it was not completed or cancelled yet. + * However, that if this deferred has children, then it transitions into _cancelling_ state and becomes _cancelled_ + * once all its children are [complete][isCompleted]. See [Job] for details. + */ + public fun completeExceptionally(exception: Throwable): Boolean +} + +/** + * Completes this deferred value with the value or exception in the given [result]. Returns `true` if this deferred + * was completed as a result of this invocation and `false` otherwise (if it was already completed). + * + * Subsequent invocations of this function have no effect and always produce `false`. + * + * This function transitions this deferred in the same ways described by [CompletableDeferred.complete] and + * [CompletableDeferred.completeExceptionally]. + */ +public fun CompletableDeferred.completeWith(result: Result): Boolean = + result.fold({ complete(it) }, { completeExceptionally(it) }) + +/** + * Creates a [CompletableDeferred] in an _active_ state. + * It is optionally a child of a [parent] job. + */ +@Suppress("FunctionName") +public fun CompletableDeferred(parent: Job? = null): CompletableDeferred = CompletableDeferredImpl(parent) + +/** + * Creates an already _completed_ [CompletableDeferred] with a given [value]. + */ +@Suppress("FunctionName") +public fun CompletableDeferred(value: T): CompletableDeferred = CompletableDeferredImpl(null).apply { complete(value) } + +/** + * Concrete implementation of [CompletableDeferred]. + */ +@OptIn(InternalForInheritanceCoroutinesApi::class) +@Suppress("UNCHECKED_CAST") +private class CompletableDeferredImpl( + parent: Job? +) : JobSupport(true), CompletableDeferred { + init { initParentJob(parent) } + override val onCancelComplete get() = true + override fun getCompleted(): T = getCompletedInternal() as T + override suspend fun await(): T = awaitInternal() as T + override val onAwait: SelectClause1 get() = onAwaitInternal as SelectClause1 + + override fun complete(value: T): Boolean = + makeCompleting(value) + override fun completeExceptionally(exception: Throwable): Boolean = + makeCompleting(CompletedExceptionally(exception)) +} diff --git a/kotlinx-coroutines-core/common/src/CompletableJob.kt b/kotlinx-coroutines-core/common/src/CompletableJob.kt new file mode 100644 index 0000000000..a793acc40c --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CompletableJob.kt @@ -0,0 +1,44 @@ +package kotlinx.coroutines + +/** + * A job that can be completed using [complete()] function. + * It is returned by [Job()][Job] and [SupervisorJob()][SupervisorJob] constructor functions. + * + * All functions on this interface are **thread-safe** and can + * be safely invoked from concurrent coroutines without external synchronization. + * + * **The `CompletableJob` interface is not stable for inheritance in 3rd party libraries**, + * as new methods might be added to this interface in the future, but is stable for use. + */ +@OptIn(ExperimentalSubclassOptIn::class) +@SubclassOptInRequired(InternalForInheritanceCoroutinesApi::class) +public interface CompletableJob : Job { + /** + * Completes this job. The result is `true` if this job was completed as a result of this invocation and + * `false` otherwise (if it was already completed). + * + * Subsequent invocations of this function have no effect and always produce `false`. + * + * This function transitions this job into _completed_ state if it was not completed or cancelled yet. + * However, that if this job has children, then it transitions into _completing_ state and becomes _complete_ + * once all its children are [complete][isCompleted]. See [Job] for details. + */ + public fun complete(): Boolean + + /** + * Completes this job exceptionally with a given [exception]. The result is `true` if this job was + * completed as a result of this invocation and `false` otherwise (if it was already completed). + * [exception] parameter is used as an additional debug information that is not handled by any exception handlers. + * + * Subsequent invocations of this function have no effect and always produce `false`. + * + * This function transitions this job into the _cancelled_ state if it has not been _completed_ or _cancelled_ yet. + * However, if this job has children, then it transitions into the _cancelling_ state and becomes _cancelled_ + * once all its children are [complete][isCompleted]. See [Job] for details. + * + * It is the responsibility of the caller to properly handle and report the given [exception]. + * All the job’s children will receive a [CancellationException] with + * the [exception] as a cause for the sake of diagnosis. + */ + public fun completeExceptionally(exception: Throwable): Boolean +} diff --git a/kotlinx-coroutines-core/common/src/CompletionHandler.common.kt b/kotlinx-coroutines-core/common/src/CompletionHandler.common.kt new file mode 100644 index 0000000000..ea56b15734 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CompletionHandler.common.kt @@ -0,0 +1,27 @@ +package kotlinx.coroutines + +/** + * Handler for [Job.invokeOnCompletion] and [CancellableContinuation.invokeOnCancellation]. + * + * The meaning of `cause` that is passed to the handler is: + * - It is `null` if the job has completed normally or the continuation was cancelled without a `cause`. + * - It is an instance of [CancellationException] if the job or the continuation was cancelled _normally_. + * **It should not be treated as an error**. In particular, it should not be reported to error logs. + * - Otherwise, the job or the continuation had _failed_. + * + * A function used for this should not throw any exceptions. + * If it does, they will get caught, wrapped into [CompletionHandlerException], and then either + * - passed to [handleCoroutineException] for [CancellableContinuation.invokeOnCancellation] + * and, for [Job] instances that are coroutines, [Job.invokeOnCompletion], or + * - for [Job] instances that are not coroutines, simply thrown, potentially crashing unrelated code. + * + * Functions used for this must be fast, non-blocking, and thread-safe. + * This handler can be invoked concurrently with the surrounding code. + * There is no guarantee on the execution context in which the function is invoked. + * + * **Note**: This type is a part of internal machinery that supports parent-child hierarchies + * and allows for implementation of suspending functions that wait on the Job's state. + * This type should not be used in general application code. + */ +// TODO: deprecate. This doesn't seem better than a simple function type. +public typealias CompletionHandler = (cause: Throwable?) -> Unit diff --git a/kotlinx-coroutines-core/common/src/CompletionState.kt b/kotlinx-coroutines-core/common/src/CompletionState.kt new file mode 100644 index 0000000000..1b05bdbb73 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CompletionState.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* + +internal fun Result.toState(): Any? = getOrElse { CompletedExceptionally(it) } + +internal fun Result.toState(caller: CancellableContinuation<*>): Any? = + getOrElse { CompletedExceptionally(recoverStackTrace(it, caller)) } + +@Suppress("RESULT_CLASS_IN_RETURN_TYPE", "UNCHECKED_CAST") +internal fun recoverResult(state: Any?, uCont: Continuation): Result = + if (state is CompletedExceptionally) + Result.failure(recoverStackTrace(state.cause, uCont)) + else + Result.success(state as T) + +/** + * Class for an internal state of a job that was cancelled (completed exceptionally). + * + * @param cause the exceptional completion cause. It's either original exceptional cause + * or artificial [CancellationException] if no cause was provided + */ +internal open class CompletedExceptionally( + @JvmField val cause: Throwable, + handled: Boolean = false +) { + private val _handled = atomic(handled) + val handled: Boolean get() = _handled.value + fun makeHandled(): Boolean = _handled.compareAndSet(false, true) + override fun toString(): String = "$classSimpleName[$cause]" +} + +/** + * A specific subclass of [CompletedExceptionally] for cancelled [AbstractContinuation]. + * + * @param continuation the continuation that was cancelled. + * @param cause the exceptional completion cause. If `cause` is null, then a [CancellationException] + * if created on first access to [exception] property. + */ +internal class CancelledContinuation( + continuation: Continuation<*>, + cause: Throwable?, + handled: Boolean +) : CompletedExceptionally(cause ?: CancellationException("Continuation $continuation was cancelled normally"), handled) { + private val _resumed = atomic(false) + fun makeResumed(): Boolean = _resumed.compareAndSet(false, true) +} diff --git a/kotlinx-coroutines-core/common/src/CoroutineContext.common.kt b/kotlinx-coroutines-core/common/src/CoroutineContext.common.kt new file mode 100644 index 0000000000..48e59fe3a9 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CoroutineContext.common.kt @@ -0,0 +1,27 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* + +/** + * Creates a context for a new coroutine. It installs [Dispatchers.Default] when no other dispatcher or + * [ContinuationInterceptor] is specified and adds optional support for debugging facilities (when turned on) + * and copyable-thread-local facilities on JVM. + */ +public expect fun CoroutineScope.newCoroutineContext(context: CoroutineContext): CoroutineContext + +/** + * Creates a context for coroutine builder functions that do not launch a new coroutine, e.g. [withContext]. + * @suppress + */ +@InternalCoroutinesApi +public expect fun CoroutineContext.newCoroutineContext(addedContext: CoroutineContext): CoroutineContext + +@PublishedApi // to have unmangled name when using from other modules via suppress +@Suppress("PropertyName") +internal expect val DefaultDelay: Delay + +// countOrElement -- pre-cached value for ThreadContext.kt +internal expect inline fun withCoroutineContext(context: CoroutineContext, countOrElement: Any?, block: () -> T): T +internal expect inline fun withContinuationContext(continuation: Continuation<*>, countOrElement: Any?, block: () -> T): T +internal expect fun Continuation<*>.toDebugString(): String +internal expect val CoroutineContext.coroutineName: String? diff --git a/kotlinx-coroutines-core/common/src/CoroutineDispatcher.kt b/kotlinx-coroutines-core/common/src/CoroutineDispatcher.kt new file mode 100644 index 0000000000..340737b1f6 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CoroutineDispatcher.kt @@ -0,0 +1,268 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +/** + * Base class to be extended by all coroutine dispatcher implementations. + * + * If `kotlinx-coroutines` is used, it is recommended to avoid [ContinuationInterceptor] instances that are not + * [CoroutineDispatcher] implementations, as [CoroutineDispatcher] ensures that the + * debugging facilities in the [newCoroutineContext] function work properly. + * + * ## Predefined dispatchers + * + * The following standard implementations are provided by `kotlinx.coroutines` as properties on + * the [Dispatchers] object: + * + * - [Dispatchers.Default] is used by all standard builders if no dispatcher or any other [ContinuationInterceptor] + * is specified in their context. + * It uses a common pool of shared background threads. + * This is an appropriate choice for compute-intensive coroutines that consume CPU resources. + * - `Dispatchers.IO` (available on the JVM and Native targets) + * uses a shared pool of on-demand created threads and is designed for offloading of IO-intensive _blocking_ + * operations (like file I/O and blocking socket I/O). + * - [Dispatchers.Main] represents the UI thread if one is available. + * - [Dispatchers.Unconfined] starts coroutine execution in the current call-frame until the first suspension, + * at which point the coroutine builder function returns. + * When the coroutine is resumed, the thread from which it is resumed will run the coroutine code until the next + * suspension, and so on. + * **The `Unconfined` dispatcher should not normally be used in code**. + * - Calling [limitedParallelism] on any dispatcher creates a view of the dispatcher that limits the parallelism + * to the given value. + * This allows creating private thread pools without spawning new threads. + * For example, `Dispatchers.IO.limitedParallelism(4)` creates a dispatcher that allows running at most + * 4 tasks in parallel, reusing the existing IO dispatcher threads. + * - When thread pools completely separate from [Dispatchers.Default] and [Dispatchers.IO] are required, + * they can be created with `newSingleThreadContext` and `newFixedThreadPoolContext` on the JVM and Native targets. + * - An arbitrary `java.util.concurrent.Executor` can be converted to a dispatcher with the + * `asCoroutineDispatcher` extension function. + * + * ## Dispatch procedure + * + * Typically, a dispatch procedure is performed as follows: + * + * - First, [isDispatchNeeded] is invoked to determine whether the coroutine should be dispatched + * or is already in the right context. + * - If [isDispatchNeeded] returns `true`, the coroutine is dispatched using the [dispatch] method. + * It may take a while for the dispatcher to start the task, + * but the [dispatch] method itself may return immediately, before the task has even begun to execute. + * - If no dispatch is needed (which is the case for [Dispatchers.Main.immediate][MainCoroutineDispatcher.immediate] + * when already on the main thread and for [Dispatchers.Unconfined]), + * [dispatch] is typically not called, + * and the coroutine is resumed in the thread performing the dispatch procedure, + * forming an event loop to prevent stack overflows. + * See [Dispatchers.Unconfined] for a description of event loops. + * + * This behavior may be different on the very first dispatch procedure for a given coroutine, depending on the + * [CoroutineStart] parameter of the coroutine builder. + */ +public abstract class CoroutineDispatcher : + AbstractCoroutineContextElement(ContinuationInterceptor), ContinuationInterceptor { + + /** @suppress */ + @ExperimentalStdlibApi + public companion object Key : AbstractCoroutineContextKey( + ContinuationInterceptor, + { it as? CoroutineDispatcher }) + + /** + * Returns `true` if the execution of the coroutine should be performed with [dispatch] method. + * The default behavior for most dispatchers is to return `true`. + * + * If this method returns `false`, the coroutine is resumed immediately in the current thread, + * potentially forming an event-loop to prevent stack overflows. + * The event loop is an advanced topic and its implications can be found in [Dispatchers.Unconfined] documentation. + * + * The [context] parameter represents the context of the coroutine that is being dispatched, + * or [EmptyCoroutineContext] if a non-coroutine-specific [Runnable] is dispatched instead. + * + * A dispatcher can override this method to provide a performance optimization and avoid paying a cost of an unnecessary dispatch. + * E.g. [MainCoroutineDispatcher.immediate] checks whether we are already in the required UI thread in this method and avoids + * an additional dispatch when it is not required. + * + * While this approach can be more efficient, it is not chosen by default to provide a consistent dispatching behaviour + * so that users won't observe unexpected and non-consistent order of events by default. + * + * Coroutine builders like [launch][CoroutineScope.launch] and [async][CoroutineScope.async] accept an optional [CoroutineStart] + * parameter that allows one to optionally choose the [undispatched][CoroutineStart.UNDISPATCHED] behavior to start coroutine immediately, + * but to be resumed only in the provided dispatcher. + * + * This method should generally be exception-safe. An exception thrown from this method + * may leave the coroutines that use this dispatcher in the inconsistent and hard to debug state. + * + * @see dispatch + * @see Dispatchers.Unconfined + */ + public open fun isDispatchNeeded(context: CoroutineContext): Boolean = true + + /** + * Creates a view of the current dispatcher that limits the parallelism to the given [value][parallelism]. + * The resulting view uses the original dispatcher for execution but with the guarantee that + * no more than [parallelism] coroutines are executed at the same time. + * + * This method does not impose restrictions on the number of views or the total sum of parallelism values, + * each view controls its own parallelism independently with the guarantee that the effective parallelism + * of all views cannot exceed the actual parallelism of the original dispatcher. + * + * The resulting dispatcher does not guarantee that the coroutines will always be dispatched on the same + * subset of threads, it only guarantees that at most [parallelism] coroutines are executed at the same time, + * and reuses threads from the original dispatchers. + * It does not constitute a resource -- it is a _view_ of the underlying dispatcher that can be thrown away + * and is not required to be closed. + * + * ### Example of usage + * ``` + * // Background dispatcher for the application + * val dispatcher = newFixedThreadPoolContext(4, "App Background") + * // At most 2 threads will be processing images as it is really slow and CPU-intensive + * val imageProcessingDispatcher = dispatcher.limitedParallelism(2, "Image processor") + * // At most 3 threads will be processing JSON to avoid image processing starvation + * val jsonProcessingDispatcher = dispatcher.limitedParallelism(3, "Json processor") + * // At most 1 thread will be doing IO + * val fileWriterDispatcher = dispatcher.limitedParallelism(1, "File writer") + * ``` + * Note how in this example the application has an executor with 4 threads, but the total sum of all limits + * is 6. Still, at most 4 coroutines can be executed simultaneously as each view limits only its own parallelism, + * and at most 4 threads can exist in the system. + * + * Note that this example was structured in such a way that it illustrates the parallelism guarantees. + * In practice, it is usually better to use `Dispatchers.IO` or [Dispatchers.Default] instead of creating a + * `backgroundDispatcher`. + * + * ### `limitedParallelism(1)` pattern + * + * One of the common patterns is confining the execution of specific tasks to a sequential execution in background + * with `limitedParallelism(1)` invocation. + * For that purpose, the implementation guarantees that tasks are executed sequentially and that a happens-before relation + * is established between them: + * + * ``` + * val confined = Dispatchers.Default.limitedParallelism(1, "incrementDispatcher") + * var counter = 0 + * + * // Invoked from arbitrary coroutines + * launch(confined) { + * // This increment is sequential and race-free + * ++counter + * } + * ``` + * Note that there is no guarantee that the underlying system thread will always be the same. + * + * ### Dispatchers.IO + * + * `Dispatcher.IO` is considered _elastic_ for the purposes of limited parallelism -- the sum of + * views is not restricted by the capacity of `Dispatchers.IO`. + * It means that it is safe to replace `newFixedThreadPoolContext(nThreads)` with + * `Dispatchers.IO.limitedParallelism(nThreads)` w.r.t. available number of threads. + * See `Dispatchers.IO` documentation for more details. + * + * ### Restrictions and implementation details + * + * The default implementation of `limitedParallelism` does not support direct dispatchers, + * such as executing the given runnable in place during [dispatch] calls. + * Any dispatcher that may return `false` from [isDispatchNeeded] is considered direct. + * For direct dispatchers, it is recommended to override this method + * and provide a domain-specific implementation or to throw an [UnsupportedOperationException]. + * + * Implementations of this method are allowed to return `this` if the current dispatcher already satisfies the parallelism requirement. + * For example, `Dispatchers.Main.limitedParallelism(1)` returns `Dispatchers.Main`, because the main dispatcher is already single-threaded. + * + * @param name optional name for the resulting dispatcher string representation if a new dispatcher was created. + * Implementations are free to ignore this parameter. + * @throws IllegalArgumentException if the given [parallelism] is non-positive + * @throws UnsupportedOperationException if the current dispatcher does not support limited parallelism views + */ + public open fun limitedParallelism(parallelism: Int, name: String? = null): CoroutineDispatcher { + parallelism.checkParallelism() + return LimitedDispatcher(this, parallelism, name) + } + + // Was experimental since 1.6.0, deprecated since 1.8.x + @Deprecated("Deprecated for good. Override 'limitedParallelism(parallelism: Int, name: String?)' instead", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("limitedParallelism(parallelism, null)") + ) + public open fun limitedParallelism(parallelism: Int): CoroutineDispatcher = limitedParallelism(parallelism, null) + + /** + * Requests execution of a runnable [block]. + * The dispatcher guarantees that [block] will eventually execute, typically by dispatching it to a thread pool, + * using a dedicated thread, or just executing the block in place. + * The [context] parameter represents the context of the coroutine that is being dispatched, + * or [EmptyCoroutineContext] if a non-coroutine-specific [Runnable] is dispatched instead. + * Implementations may use [context] for additional context-specific information, + * such as priority, whether the dispatched coroutine can be invoked in place, + * coroutine name, and additional diagnostic elements. + * + * This method should guarantee that the given [block] will be eventually invoked, + * otherwise the system may reach a deadlock state and never leave it. + * The cancellation mechanism is transparent for [CoroutineDispatcher] and is managed by [block] internals. + * + * This method should generally be exception-safe. An exception thrown from this method + * may leave the coroutines that use this dispatcher in an inconsistent and hard-to-debug state. + * It is assumed that if any exceptions do get thrown from this method, then [block] will not be executed. + * + * This method must not immediately call [block]. Doing so may result in `StackOverflowError` + * when `dispatch` is invoked repeatedly, for example when [yield] is called in a loop. + * In order to execute a block in place, it is required to return `false` from [isDispatchNeeded] + * and delegate the `dispatch` implementation to `Dispatchers.Unconfined.dispatch` in such cases. + * To support this, the coroutines machinery ensures in-place execution and forms an event-loop to + * avoid unbound recursion. + * + * @see isDispatchNeeded + * @see Dispatchers.Unconfined + */ + public abstract fun dispatch(context: CoroutineContext, block: Runnable) + + /** + * Dispatches execution of a runnable `block` onto another thread in the given `context` + * with a hint for the dispatcher that the current dispatch is triggered by a [yield] call, so that the execution of this + * continuation may be delayed in favor of already dispatched coroutines. + * + * Though the `yield` marker may be passed as a part of [context], this + * is a separate method for performance reasons. + * + * Implementation note: this entry-point is used for `Dispatchers.IO` and [Dispatchers.Default] + * unerlying implementations, see overrides for this method. + * + * @suppress **This an internal API and should not be used from general code.** + */ + @InternalCoroutinesApi + public open fun dispatchYield(context: CoroutineContext, block: Runnable): Unit = safeDispatch(context, block) + + /** + * Returns a continuation that wraps the provided [continuation], thus intercepting all resumptions. + * + * This method should generally be exception-safe. An exception thrown from this method + * may leave the coroutines that use this dispatcher in the inconsistent and hard to debug state. + */ + public final override fun interceptContinuation(continuation: Continuation): Continuation = + DispatchedContinuation(this, continuation) + + public final override fun releaseInterceptedContinuation(continuation: Continuation<*>) { + /* + * Unconditional cast is safe here: we return only DispatchedContinuation from `interceptContinuation`, + * any ClassCastException can only indicate compiler bug + */ + val dispatched = continuation as DispatchedContinuation<*> + dispatched.release() + } + + /** + * @suppress **Error**: Operator '+' on two CoroutineDispatcher objects is meaningless. + * CoroutineDispatcher is a coroutine context element and `+` is a set-sum operator for coroutine contexts. + * The dispatcher to the right of `+` just replaces the dispatcher to the left. + */ + @Suppress("DeprecatedCallableAddReplaceWith") + @Deprecated( + message = "Operator '+' on two CoroutineDispatcher objects is meaningless. " + + "CoroutineDispatcher is a coroutine context element and `+` is a set-sum operator for coroutine contexts. " + + "The dispatcher to the right of `+` just replaces the dispatcher to the left.", + level = DeprecationLevel.ERROR + ) + public operator fun plus(other: CoroutineDispatcher): CoroutineDispatcher = other + + /** @suppress for nicer debugging */ + override fun toString(): String = "$classSimpleName@$hexAddress" +} diff --git a/kotlinx-coroutines-core/common/src/CoroutineExceptionHandler.kt b/kotlinx-coroutines-core/common/src/CoroutineExceptionHandler.kt new file mode 100644 index 0000000000..0899eb6fb6 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CoroutineExceptionHandler.kt @@ -0,0 +1,108 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +/** + * Helper function for coroutine builder implementations to handle uncaught and unexpected exceptions in coroutines, + * that could not be otherwise handled in a normal way through structured concurrency, saving them to a future, and + * cannot be rethrown. This is a last resort handler to prevent lost exceptions. + * + * If there is [CoroutineExceptionHandler] in the context, then it is used. If it throws an exception during handling + * or is absent, all instances of [CoroutineExceptionHandler] found via [ServiceLoader] and + * [Thread.uncaughtExceptionHandler] are invoked. + * + * @suppress **This is internal API and it is subject to change.** + */ +@InternalCoroutinesApi +public fun handleCoroutineException(context: CoroutineContext, exception: Throwable) { + val reportException = if (exception is DispatchException) exception.cause else exception + // Invoke an exception handler from the context if present + try { + context[CoroutineExceptionHandler]?.let { + it.handleException(context, reportException) + return + } + } catch (t: Throwable) { + handleUncaughtCoroutineException(context, handlerException(reportException, t)) + return + } + // If a handler is not present in the context or an exception was thrown, fallback to the global handler + handleUncaughtCoroutineException(context, reportException) +} + +internal fun handlerException(originalException: Throwable, thrownException: Throwable): Throwable { + if (originalException === thrownException) return originalException + return RuntimeException("Exception while trying to handle coroutine exception", thrownException).apply { + addSuppressed(originalException) + } +} + +/** + * Creates a [CoroutineExceptionHandler] instance. + * @param handler a function which handles exception thrown by a coroutine + */ +@Suppress("FunctionName") +public inline fun CoroutineExceptionHandler(crossinline handler: (CoroutineContext, Throwable) -> Unit): CoroutineExceptionHandler = + object : AbstractCoroutineContextElement(CoroutineExceptionHandler), CoroutineExceptionHandler { + override fun handleException(context: CoroutineContext, exception: Throwable) = + handler.invoke(context, exception) + } + +/** + * An optional element in the coroutine context to handle **uncaught** exceptions. + * + * Normally, uncaught exceptions can only result from _root_ coroutines created using the [launch][CoroutineScope.launch] builder. + * All _children_ coroutines (coroutines created in the context of another [Job]) delegate handling of their + * exceptions to their parent coroutine, which also delegates to the parent, and so on until the root, + * so the `CoroutineExceptionHandler` installed in their context is never used. + * Coroutines running with [SupervisorJob] do not propagate exceptions to their parent and are treated like root coroutines. + * A coroutine that was created using [async][CoroutineScope.async] always catches all its exceptions and represents them + * in the resulting [Deferred] object, so it cannot result in uncaught exceptions. + * + * ### Handling coroutine exceptions + * + * `CoroutineExceptionHandler` is a last-resort mechanism for global "catch all" behavior. + * You cannot recover from the exception in the `CoroutineExceptionHandler`. The coroutine had already completed + * with the corresponding exception when the handler is called. Normally, the handler is used to + * log the exception, show some kind of error message, terminate, and/or restart the application. + * + * If you need to handle exception in a specific part of the code, it is recommended to use `try`/`catch` around + * the corresponding code inside your coroutine. This way you can prevent completion of the coroutine + * with the exception (exception is now _caught_), retry the operation, and/or take other arbitrary actions: + * + * ``` + * scope.launch { // launch child coroutine in a scope + * try { + * // do something + * } catch (e: Throwable) { + * // handle exception + * } + * } + * ``` + * + * ### Uncaught exceptions with no handler + * + * When no handler is installed, exception are handled in the following way: + * - If exception is [CancellationException], it is ignored, as these exceptions are used to cancel coroutines. + * - Otherwise, if there is a [Job] in the context, then [Job.cancel] is invoked. + * - Otherwise, as a last resort, the exception is processed in a platform-specific manner: + * - On JVM, all instances of [CoroutineExceptionHandler] found via [ServiceLoader], as well as + * the current thread's [Thread.uncaughtExceptionHandler], are invoked. + * - On Native, the whole application crashes with the exception. + * - On JS, the exception is logged via the Console API. + * + * [CoroutineExceptionHandler] can be invoked from an arbitrary thread. + */ +public interface CoroutineExceptionHandler : CoroutineContext.Element { + /** + * Key for [CoroutineExceptionHandler] instance in the coroutine context. + */ + public companion object Key : CoroutineContext.Key + + /** + * Handles uncaught [exception] in the given [context]. It is invoked + * if coroutine has an uncaught exception. + */ + public fun handleException(context: CoroutineContext, exception: Throwable) +} diff --git a/kotlinx-coroutines-core/common/src/CoroutineName.kt b/kotlinx-coroutines-core/common/src/CoroutineName.kt new file mode 100644 index 0000000000..82b45c87ba --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CoroutineName.kt @@ -0,0 +1,25 @@ +package kotlinx.coroutines + +import kotlin.coroutines.AbstractCoroutineContextElement +import kotlin.coroutines.CoroutineContext + +/** + * User-specified name of coroutine. This name is used in debugging mode. + * See [newCoroutineContext][CoroutineScope.newCoroutineContext] for the description of coroutine debugging facilities. + */ +public data class CoroutineName( + /** + * User-defined coroutine name. + */ + val name: String +) : AbstractCoroutineContextElement(CoroutineName) { + /** + * Key for [CoroutineName] instance in the coroutine context. + */ + public companion object Key : CoroutineContext.Key + + /** + * Returns a string representation of the object. + */ + override fun toString(): String = "CoroutineName($name)" +} diff --git a/kotlinx-coroutines-core/common/src/CoroutineScope.kt b/kotlinx-coroutines-core/common/src/CoroutineScope.kt new file mode 100644 index 0000000000..9c1bd02728 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CoroutineScope.kt @@ -0,0 +1,375 @@ +@file:OptIn(ExperimentalContracts::class) +@file:Suppress("LEAKED_IN_PLACE_LAMBDA", "WRONG_INVOCATION_KIND") + +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.intrinsics.* +import kotlin.contracts.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +/** + * Defines a scope for new coroutines. Every **coroutine builder** (like [launch], [async], etc.) + * is an extension on [CoroutineScope] and inherits its [coroutineContext][CoroutineScope.coroutineContext] + * to automatically propagate all its elements and cancellation. + * + * The best ways to obtain a standalone instance of the scope are [CoroutineScope()] and [MainScope()] factory functions, + * taking care to cancel these coroutine scopes when they are no longer needed (see the section on custom usage below for + * explanation and example). + * + * Additional context elements can be appended to the scope using the [plus][CoroutineScope.plus] operator. + * + * ### Convention for structured concurrency + * + * Manual implementation of this interface is not recommended, implementation by delegation should be preferred instead. + * By convention, the [context of a scope][CoroutineScope.coroutineContext] should contain an instance of a + * [job][Job] to enforce the discipline of **structured concurrency** with propagation of cancellation. + * + * Every coroutine builder (like [launch], [async], and others) + * and every scoping function (like [coroutineScope] and [withContext]) provides _its own_ scope + * with its own [Job] instance into the inner block of code it runs. + * By convention, they all wait for all the coroutines inside their block to complete before completing themselves, + * thus enforcing the structured concurrency. See [Job] documentation for more details. + * + * ### Android usage + * + * Android has first-party support for coroutine scope in all entities with the lifecycle. + * See [the corresponding documentation](https://developer.android.com/topic/libraries/architecture/coroutines#lifecyclescope). + * + * ### Custom usage + * + * `CoroutineScope` should be declared as a property on entities with a well-defined lifecycle that are + * responsible for launching child coroutines. The corresponding instance of `CoroutineScope` shall be created + * with either `CoroutineScope()` or `MainScope()`: + * + * - `CoroutineScope()` uses the [context][CoroutineContext] provided to it as a parameter for its coroutines + * and adds a [Job] if one is not provided as part of the context. + * - `MainScope()` uses [Dispatchers.Main] for its coroutines and has a [SupervisorJob]. + * + * **The key part of custom usage of `CoroutineScope` is cancelling it at the end of the lifecycle.** + * The [CoroutineScope.cancel] extension function shall be used when the entity that was launching coroutines + * is no longer needed. It cancels all the coroutines that might still be running on behalf of it. + * + * For example: + * + * ``` + * class MyUIClass { + * val scope = MainScope() // the scope of MyUIClass, uses Dispatchers.Main + * + * fun destroy() { // destroys an instance of MyUIClass + * scope.cancel() // cancels all coroutines launched in this scope + * // ... do the rest of cleanup here ... + * } + * + * /* + * * Note: if this instance is destroyed or any of the launched coroutines + * * in this method throws an exception, then all nested coroutines are cancelled. + * */ + * fun showSomeData() = scope.launch { // launched in the main thread + * // ... here we can use suspending functions or coroutine builders with other dispatchers + * draw(data) // draw in the main thread + * } + * } + * ``` + */ +public interface CoroutineScope { + /** + * The context of this scope. + * Context is encapsulated by the scope and used for implementation of coroutine builders that are extensions on the scope. + * Accessing this property in general code is not recommended for any purposes except accessing the [Job] instance for advanced usages. + * + * By convention, should contain an instance of a [job][Job] to enforce structured concurrency. + */ + public val coroutineContext: CoroutineContext +} + +/** + * Adds the specified coroutine context to this scope, overriding existing elements in the current + * scope's context with the corresponding keys. + * + * This is a shorthand for `CoroutineScope(thisScope.coroutineContext + context)` and can be used as + * a combinator with existing constructors: + * ``` + * class MyActivity { + * val uiScope = MainScope() + CoroutineName("MyActivity") + * } + * ``` + */ +public operator fun CoroutineScope.plus(context: CoroutineContext): CoroutineScope = + ContextScope(coroutineContext + context) + +/** + * Creates the main [CoroutineScope] for UI components. + * + * Example of use: + * ``` + * class MyAndroidActivity { + * private val scope = MainScope() + * + * override fun onDestroy() { + * super.onDestroy() + * scope.cancel() + * } + * } + * ``` + * + * The resulting scope has [SupervisorJob] and [Dispatchers.Main] context elements. + * If you want to append additional elements to the main scope, use [CoroutineScope.plus] operator: + * `val scope = MainScope() + CoroutineName("MyActivity")`. + */ +@Suppress("FunctionName") +public fun MainScope(): CoroutineScope = ContextScope(SupervisorJob() + Dispatchers.Main) + +/** + * Returns `true` when the current [Job] is still active (has not completed and was not cancelled yet). + * + * Coroutine cancellation [is cooperative](https://kotlinlang.org/docs/cancellation-and-timeouts.html#cancellation-is-cooperative) + * and normally, it's checked if a coroutine is cancelled when it *suspends*, for example, + * when trying to read from a [channel][kotlinx.coroutines.channels.Channel] that is empty. + * + * Sometimes, a coroutine does not need to perform suspending operations, but still wants to be cooperative + * and respect cancellation. + * + * The [isActive] property is inteded to be used for scenarios like this: + * ``` + * val watchdogDispatcher = Dispatchers.IO.limitParallelism(1) + * fun backgroundWork() { + * println("Doing bookkeeping in the background in a non-suspending manner") + * Thread.sleep(100L) // Sleep 100ms + * } + * // Part of some non-trivial CoroutineScope-confined lifecycle + * launch(watchdogDispatcher) { + * while (isActive) { + * // Repetitively do some background work that is non-suspending + * backgroundWork() + * } + * } + * ``` + * + * This function returns `true` if there is no [job][Job] in the scope's [coroutineContext][CoroutineScope.coroutineContext]. + * This property is a shortcut for `coroutineContext.isActive` in the scope when + * [CoroutineScope] is available. + * See [coroutineContext][kotlin.coroutines.coroutineContext], + * [isActive][kotlinx.coroutines.isActive] and [Job.isActive]. + */ +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +public val CoroutineScope.isActive: Boolean + get() = coroutineContext[Job]?.isActive ?: true + +/** + * A global [CoroutineScope] not bound to any job. + * Global scope is used to launch top-level coroutines that operate + * throughout the application's lifetime and are not canceled prematurely. + * + * Active coroutines launched in `GlobalScope` do not keep the process alive. They are like daemon threads. + * + * This is a **delicate** API. It is easy to accidentally create resource or memory leaks when + * `GlobalScope` is used. A coroutine launched in `GlobalScope` is not subject to the principle of structured + * concurrency, so if it hangs or gets delayed due to a problem (e.g., due to a slow network), it will stay working + * and consuming resources. For example, consider the following code: + * + * ``` + * fun loadConfiguration() { + * GlobalScope.launch { + * val config = fetchConfigFromServer() // network request + * updateConfiguration(config) + * } + * } + * ``` + * + * A call to `loadConfiguration` creates a coroutine in the `GlobalScope` that works in the background without any + * provision to cancel it or to wait for its completion. If a network is slow, it keeps waiting in the background, + * consuming resources. Repeated calls to `loadConfiguration` will consume more and more resources. + * + * ### Possible replacements + * + * In many circumstances, uses of 'GlobalScope' should be removed, + * with the containing operation marked as 'suspend', for example: + * + * ``` + * suspend fun loadConfiguration() { + * val config = fetchConfigFromServer() // network request + * updateConfiguration(config) + * } + * ``` + * + * In cases when `GlobalScope.launch` was used to launch multiple concurrent operations, the corresponding + * operations shall be grouped with [coroutineScope] instead: + * + * ``` + * // concurrently load configuration and data + * suspend fun loadConfigurationAndData() { + * coroutineScope { + * launch { loadConfiguration() } + * launch { loadData() } + * } + * } + * ``` + * + * In top-level code, when launching a concurrent operation from a non-suspending context, an appropriately + * confined instance of [CoroutineScope] shall be used instead of `GlobalScope`. See docs on [CoroutineScope] for + * details. + * + * ### GlobalScope vs. Custom CoroutineScope + * + * Do not replace `GlobalScope.launch { ... }` with `CoroutineScope().launch { ... }` constructor function call. + * The latter has the same pitfalls as `GlobalScope`. See [CoroutineScope] documentation on the intended usage of + * `CoroutineScope()` constructor function. + * + * ### Legitimate use-cases + * + * There are limited circumstances under which `GlobalScope` can be legitimately and safely used, such as top-level background + * processes that must stay active for the whole duration of the application's lifetime. Because of that, any use + * of `GlobalScope` requires an explicit opt-in with `@OptIn(DelicateCoroutinesApi::class)`, like this: + * + * ``` + * // A global coroutine to log statistics every second, must be always active + * @OptIn(DelicateCoroutinesApi::class) + * val globalScopeReporter = GlobalScope.launch { + * while (true) { + * delay(1000) + * logStatistics() + * } + * } + * ``` + */ +@DelicateCoroutinesApi +public object GlobalScope : CoroutineScope { + /** + * Returns [EmptyCoroutineContext]. + */ + override val coroutineContext: CoroutineContext + get() = EmptyCoroutineContext +} + +/** + * Creates a [CoroutineScope] and calls the specified suspend block with this scope. + * The provided scope inherits its [coroutineContext][CoroutineScope.coroutineContext] from the outer scope, using the + * [Job] from that context as the parent for a new [Job]. + * + * This function is designed for _concurrent decomposition_ of work. When any child coroutine in this scope fails, + * this scope fails, cancelling all the other children (for a different behavior, see [supervisorScope]). + * This function returns as soon as the given block and all its child coroutines are completed. + * A usage of a scope looks like this: + * + * ``` + * suspend fun showSomeData() = coroutineScope { + * val data = async(Dispatchers.IO) { // <- extension on current scope + * ... load some UI data for the Main thread ... + * } + * + * withContext(Dispatchers.Main) { + * doSomeWork() + * val result = data.await() + * display(result) + * } + * } + * ``` + * + * The scope in this example has the following semantics: + * 1) `showSomeData` returns as soon as the data is loaded and displayed in the UI. + * 2) If `doSomeWork` throws an exception, then the `async` task is cancelled and `showSomeData` rethrows that exception. + * 3) If the outer scope of `showSomeData` is cancelled, both started `async` and `withContext` blocks are cancelled. + * 4) If the `async` block fails, `withContext` will be cancelled. + * + * The method may throw a [CancellationException] if the current job was cancelled externally, + * rethrow the exception thrown by [block], or throw an unhandled [Throwable] if there is one + * (for example, from a crashed coroutine that was started with [launch][CoroutineScope.launch] in this scope). + */ +public suspend fun coroutineScope(block: suspend CoroutineScope.() -> R): R { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + return suspendCoroutineUninterceptedOrReturn { uCont -> + val coroutine = ScopeCoroutine(uCont.context, uCont) + coroutine.startUndispatchedOrReturn(coroutine, block) + } +} + +/** + * Creates a [CoroutineScope] that wraps the given coroutine [context]. + * + * If the given [context] does not contain a [Job] element, then a default `Job()` is created. + * This way, failure of any child coroutine in this scope or [cancellation][CoroutineScope.cancel] of the scope itself + * cancels all the scope's children, just like inside [coroutineScope] block. + */ +@Suppress("FunctionName") +public fun CoroutineScope(context: CoroutineContext): CoroutineScope = + ContextScope(if (context[Job] != null) context else context + Job()) + +/** + * Cancels this scope, including its job and all its children with an optional cancellation [cause]. + * A cause can be used to specify an error message or to provide other details on + * a cancellation reason for debugging purposes. + * Throws [IllegalStateException] if the scope does not have a job in it. + */ +public fun CoroutineScope.cancel(cause: CancellationException? = null) { + val job = coroutineContext[Job] ?: error("Scope cannot be cancelled because it does not have a job: $this") + job.cancel(cause) +} + +/** + * Cancels this scope, including its job and all its children with a specified diagnostic error [message]. + * A [cause] can be specified to provide additional details on a cancellation reason for debugging purposes. + * Throws [IllegalStateException] if the scope does not have a job in it. + */ +public fun CoroutineScope.cancel(message: String, cause: Throwable? = null): Unit = cancel(CancellationException(message, cause)) + +/** + * Throws the [CancellationException] that was the scope's cancellation cause if the scope is no longer [active][CoroutineScope.isActive]. + * + * Coroutine cancellation [is cooperative](https://kotlinlang.org/docs/cancellation-and-timeouts.html#cancellation-is-cooperative) + * and normally, it's checked if a coroutine is cancelled when it *suspends*, for example, + * when trying to read from a [channel][kotlinx.coroutines.channels.Channel] that is empty. + * + * Sometimes, a coroutine does not need to perform suspending operations, but still wants to be cooperative + * and respect cancellation. + * + * [ensureActive] function is inteded to be used for these scenarios and immediately bubble up the cancellation exception: + * ``` + * val watchdogDispatcher = Dispatchers.IO.limitParallelism(1) + * fun backgroundWork() { + * println("Doing bookkeeping in the background in a non-suspending manner") + * Thread.sleep(100L) // Sleep 100ms + * } + * fun postBackgroundCleanup() = println("Doing something else") + * // Part of some non-trivial CoroutineScope-confined lifecycle + * launch(watchdogDispatcher) { + * while (true) { + * // Repeatatively do some background work that is non-suspending + * backgroundWork() + * ensureActive() // Bail out if the scope was cancelled + * postBackgroundCleanup() // Won't be invoked if the scope was cancelled + * } + * } + * ``` + * This function does not do anything if there is no [Job] in the scope's [coroutineContext][CoroutineScope.coroutineContext]. + * + * @see CoroutineScope.isActive + * @see CoroutineContext.ensureActive + */ +public fun CoroutineScope.ensureActive(): Unit = coroutineContext.ensureActive() + +/** + * Returns the current [CoroutineContext] retrieved by using [kotlin.coroutines.coroutineContext]. + * This function is an alias to avoid name clash with [CoroutineScope.coroutineContext]: + * + * ``` + * // ANTIPATTERN! DO NOT WRITE SUCH A CODE + * suspend fun CoroutineScope.suspendFunWithScope() { + * // Name of the CoroutineScope.coroutineContext in 'this' position, same as `this.coroutineContext` + * println(coroutineContext[CoroutineName]) + * // Name of the context that invoked this suspend function, same as `kotlin.coroutines.coroutineContext` + * println(currentCoroutineContext()[CoroutineName]) + * } + * + * withContext(CoroutineName("Caller")) { + * // Will print 'CoroutineName("Receiver")' and 'CoroutineName("Caller")' + * CoroutineScope("Receiver").suspendFunWithScope() + * } + * ``` + * + * This function should always be preferred over [kotlin.coroutines.coroutineContext] property even when there is no explicit clash. + */ +public suspend inline fun currentCoroutineContext(): CoroutineContext = coroutineContext diff --git a/kotlinx-coroutines-core/common/src/CoroutineStart.kt b/kotlinx-coroutines-core/common/src/CoroutineStart.kt new file mode 100644 index 0000000000..c4c4cea723 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/CoroutineStart.kt @@ -0,0 +1,371 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.intrinsics.* +import kotlin.coroutines.* + +/** + * Defines start options for coroutines builders. + * + * It is used in the `start` parameter of coroutine builder functions like + * [launch][CoroutineScope.launch] and [async][CoroutineScope.async] + * to describe when and how the coroutine should be dispatched initially. + * + * This parameter only affects how the coroutine behaves until the code of its body starts executing. + * After that, cancellability and dispatching are defined by the behavior of the invoked suspending functions. + * + * The summary of coroutine start options is: + * - [DEFAULT] immediately schedules the coroutine for execution according to its context. + * - [LAZY] delays the moment of the initial dispatch until the result of the coroutine is needed. + * - [ATOMIC] prevents the coroutine from being cancelled before it starts, ensuring that its code will start + * executing in any case. + * - [UNDISPATCHED] immediately executes the coroutine until its first suspension point _in the current thread_. + */ +public enum class CoroutineStart { + /** + * Immediately schedules the coroutine for execution according to its context. This is usually the default option. + * + * [DEFAULT] uses the default dispatch procedure described in the [CoroutineDispatcher] documentation. + * + * If the coroutine's [Job] is cancelled before it started executing, then it will not start its + * execution at all and will be considered [cancelled][Job.isCancelled]. + * + * Examples: + * + * ``` + * // Example of starting a new coroutine that goes through a dispatch + * runBlocking { + * println("1. About to start a new coroutine.") + * // Dispatch the job to execute later. + * // The parent coroutine's dispatcher is inherited by default. + * // In this case, it's the single thread backing `runBlocking`. + * launch { // CoroutineStart.DEFAULT is launch's default start mode + * println("3. When the thread is available, we start the coroutine") + * } + * println("2. The thread keeps doing other work after launching the coroutine") + * } + * ``` + * + * ``` + * // Example of starting a new coroutine that doesn't go through a dispatch initially + * runBlocking { + * println("1. About to start a coroutine not needing a dispatch.") + * // Dispatch the job to execute. + * // `Dispatchers.Unconfined` is explicitly chosen. + * launch(Dispatchers.Unconfined) { // CoroutineStart.DEFAULT is the launch's default start mode + * println("2. The body will be executed immediately") + * delay(50.milliseconds) // give up the thread to the outer coroutine + * println("4. When the thread is next available, this coroutine proceeds further") + * } + * println("3. After the initial suspension, the thread does other work.") + * } + * ``` + * + * ``` + * // Example of cancelling coroutines before they start executing. + * runBlocking { + * // dispatch the job to execute on this thread later + * launch { // CoroutineStart.DEFAULT is the launch's default start mode + * println("This code will never execute") + * } + * cancel() // cancels the current coroutine scope and its children + * launch(Dispatchers.Unconfined) { + * println("This code will never execute") + * } + * println("This code will execute.") + * } + * ``` + */ + DEFAULT, + + /** + * Starts the coroutine lazily, only when it is needed. + * + * Starting a coroutine with [LAZY] only creates the coroutine, but does not schedule it for execution. + * When the completion of the coroutine is first awaited + * (for example, via [Job.join]) or explicitly [started][Job.start], + * the dispatch procedure described in the [CoroutineDispatcher] documentation is performed in the thread + * that did it. + * + * The details of what counts as waiting can be found in the documentation of the corresponding coroutine builders + * like [launch][CoroutineScope.launch] and [async][CoroutineScope.async]. + * + * If the coroutine's [Job] is cancelled before it started executing, then it will not start its + * execution at all and will be considered [cancelled][Job.isCancelled]. + * + * **Pitfall**: launching a coroutine with [LAZY] without awaiting or cancelling it at any point means that it will + * never be completed, leading to deadlocks and resource leaks. + * For example, the following code will deadlock, since [coroutineScope] waits for all of its child coroutines to + * complete: + * ``` + * // This code hangs! + * coroutineScope { + * launch(start = CoroutineStart.LAZY) { } + * } + * ``` + * + * The behavior of [LAZY] can be described with the following examples: + * + * ``` + * // Example of lazily starting a new coroutine that goes through a dispatch + * runBlocking { + * println("1. About to start a new coroutine.") + * // Create a job to execute on `Dispatchers.Default` later. + * val job = launch(Dispatchers.Default, start = CoroutineStart.LAZY) { + * println("3. Only now does the coroutine start.") + * } + * delay(10.milliseconds) // try to give the coroutine some time to run + * println("2. The coroutine still has not started. Now, we join it.") + * job.join() + * } + * ``` + * + * ``` + * // Example of lazily starting a new coroutine that doesn't go through a dispatch initially + * runBlocking { + * println("1. About to lazily start a new coroutine.") + * // Create a job to execute on `Dispatchers.Unconfined` later. + * val lazyJob = launch(Dispatchers.Unconfined, start = CoroutineStart.LAZY) { + * println("3. The coroutine starts on the thread that called `join`.") + * } + * // We start the job on another thread for illustrative purposes + * launch(Dispatchers.Default) { + * println("2. We start the lazyJob.") + * job.start() // runs lazyJob's code in-place + * println("4. Only now does the `start` call return.") + * } + * } + * ``` + * + * ## Alternatives + * + * The effects of [LAZY] can usually be achieved more idiomatically without it. + * + * When a coroutine is started with [LAZY] and is stored in a property, + * it may be a better choice to use [lazy] instead: + * + * ``` + * // instead of `val page = scope.async(start = CoroutineStart.LAZY) { getPage() }`, do + * val page by lazy { scope.async { getPage() } } + * ``` + * + * This way, the child coroutine is not created at all unless it is needed. + * Note that with this, any access to this variable will start the coroutine, + * even something like `page.invokeOnCompletion { }` or `page.isActive`. + * + * If a coroutine is started with [LAZY] and then unconditionally started, + * it is more idiomatic to create the coroutine in the exact place where it is started: + * + * ``` + * // instead of `val job = scope.launch(start = CoroutineStart.LAZY) { }; job.start()`, do + * scope.launch { } + * ``` + */ + LAZY, + + /** + * Atomically (i.e., in a non-cancellable way) schedules the coroutine for execution according to its context. + * + * This is similar to [DEFAULT], but the coroutine is guaranteed to start executing even if it was cancelled. + * This only affects the behavior until the body of the coroutine starts executing; + * inside the body, cancellation will work as usual. + * + * Like [ATOMIC], [UNDISPATCHED], too, ensures that coroutines will be started in any case. + * The difference is that, instead of immediately starting them on the same thread, + * [ATOMIC] performs the full dispatch procedure just as [DEFAULT] does. + * + * Because of this, we can use [ATOMIC] in cases where we want to be certain that some code eventually runs + * and uses a specific dispatcher to do that. + * + * Example: + * ``` + * val mutex = Mutex() + * + * mutex.lock() // lock the mutex outside the coroutine + * // ... // initial portion of the work, protected by the mutex + * val job = launch(start = CoroutineStart.ATOMIC) { + * // the work must continue in a coroutine, but still under the mutex + * println("Coroutine running!") + * try { + * // this `try` block will be entered in any case because of ATOMIC + * println("Starting task...") + * delay(10.milliseconds) // throws due to cancellation + * println("Finished task.") + * } finally { + * mutex.unlock() // correctly release the mutex + * } + * } + * + * job.cancelAndJoin() // we immediately cancel the coroutine. + * mutex.withLock { + * println("The lock has been returned correctly!") + * } + * ``` + * + * Here, we used [ATOMIC] to ensure that a mutex that was acquired outside the coroutine does get released + * even if cancellation happens between `lock()` and `launch`. + * As a result, the mutex will always be released. + * + * The behavior of [ATOMIC] can be described with the following examples: + * + * ``` + * // Example of cancelling atomically started coroutines + * runBlocking { + * println("1. Atomically starting a coroutine that goes through a dispatch.") + * launch(start = CoroutineStart.ATOMIC) { + * check(!isActive) // attempting to suspend later will throw + * println("4. The coroutine was cancelled (isActive = $isActive), but starts anyway.") + * try { + * delay(10.milliseconds) // will throw: the coroutine is cancelled + * println("This code will never run.") + * } catch (e: CancellationException) { + * println("5. Cancellation at later points still works.") + * throw e + * } + * } + * println("2. Cancelling this coroutine and all of its children.") + * cancel() + * launch(Dispatchers.Unconfined, start = CoroutineStart.ATOMIC) { + * check(!isActive) // attempting to suspend will throw + * println("3. An undispatched coroutine starts.") + * } + * ensureActive() // we can even crash the current coroutine. + * } + * ``` + * + * This is a **delicate** API. The coroutine starts execution even if its [Job] is cancelled before starting. + * However, the resources used within a coroutine may rely on the cancellation mechanism, + * and cannot be used after the [Job] cancellation. For instance, in Android development, updating a UI element + * is not allowed if the coroutine's scope, which is tied to the element's lifecycle, has been cancelled. + */ + @DelicateCoroutinesApi + ATOMIC, + + /** + * Immediately executes the coroutine until its first suspension point _in the current thread_. + * + * Starting a coroutine using [UNDISPATCHED] is similar to using [Dispatchers.Unconfined] with [DEFAULT], except: + * - Resumptions from later suspensions will properly use the actual dispatcher from the coroutine's context. + * Only the code until the first suspension point will be executed immediately. + * - Even if the coroutine was cancelled already, its code will still start running, similar to [ATOMIC]. + * - The coroutine will not form an event loop. See [Dispatchers.Unconfined] for an explanation of event loops. + * + * This set of behaviors makes [UNDISPATCHED] well-suited for cases where the coroutine has a distinct + * initialization phase whose side effects we want to rely on later. + * + * Example: + * ``` + * var tasks = 0 + * repeat(3) { + * launch(start = CoroutineStart.UNDISPATCHED) { + * tasks++ + * try { + * println("Waiting for a reply...") + * delay(50.milliseconds) + * println("Got a reply!") + * } finally { + * tasks-- + * } + * } + * } + * // Because of UNDISPATCHED, + * // we know that the tasks already ran to their first suspension point, + * // so this number is non-zero initially. + * while (tasks > 0) { + * println("currently active: $tasks") + * delay(10.milliseconds) + * } + * ``` + * + * Here, we implement a publisher-subscriber interaction, where [UNDISPATCHED] ensures that the + * subscribers do get registered before the publisher first checks if it can stop emitting values due to + * the lack of subscribers. + * + * ``` + * // Constant usage of stack space + * fun CoroutineScope.factorialWithUnconfined(n: Int): Deferred = + * async(Dispatchers.Unconfined) { + * if (n > 0) { + * n * factorialWithUnconfined(n - 1).await() + * } else { + * 1 // replace with `error()` to see the stacktrace + * } + * } + * + * // Linearly increasing usage of stack space + * fun CoroutineScope.factorialWithUndispatched(n: Int): Deferred = + * async(start = CoroutineStart.UNDISPATCHED) { + * if (n > 0) { + * n * factorialWithUndispatched(n - 1).await() + * } else { + * 1 // replace with `error()` to see the stacktrace + * } + * } + * ``` + * + * Calling `factorialWithUnconfined` from this example will result in a constant-size stack, + * whereas `factorialWithUndispatched` will lead to `n` recursively nested calls, + * resulting in a stack overflow for large values of `n`. + * + * The behavior of [UNDISPATCHED] can be described with the following examples: + * + * ``` + * runBlocking { + * println("1. About to start a new coroutine.") + * launch(Dispatchers.Default, start = CoroutineStart.UNDISPATCHED) { + * println("2. The coroutine is immediately started in the same thread.") + * delay(10.milliseconds) + * println("4. The execution continues in a Dispatchers.Default thread.") + * } + * println("3. Execution of the outer coroutine only continues later.") + * } + * ``` + * + * ``` + * // Cancellation does not prevent the coroutine from being started + * runBlocking { + * println("1. First, we cancel this scope.") + * cancel() + * println("2. Now, we start a new UNDISPATCHED child.") + * launch(start = CoroutineStart.UNDISPATCHED) { + * check(!isActive) // the child is already cancelled + * println("3. We entered the coroutine despite being cancelled.") + * } + * println("4. Execution of the outer coroutine only continues later.") + * } + * ``` + * + * **Pitfall**: unlike [Dispatchers.Unconfined] and [MainCoroutineDispatcher.immediate], nested undispatched + * coroutines do not form an event loop that otherwise prevents potential stack overflow in case of unlimited + * nesting. This property is necessary for the use case of guaranteed initialization, but may be undesirable in + * other cases. + * See [Dispatchers.Unconfined] for an explanation of event loops. + */ + UNDISPATCHED; + + /** + * Starts the corresponding block with receiver as a coroutine with this coroutine start strategy. + * + * - [DEFAULT] uses [startCoroutineCancellable]. + * - [ATOMIC] uses [startCoroutine]. + * - [UNDISPATCHED] uses [startCoroutineUndispatched]. + * - [LAZY] does nothing. + * + * @suppress **This an internal API and should not be used from general code.** + */ + @InternalCoroutinesApi + public operator fun invoke(block: suspend R.() -> T, receiver: R, completion: Continuation): Unit = + when (this) { + DEFAULT -> block.startCoroutineCancellable(receiver, completion) + ATOMIC -> block.startCoroutine(receiver, completion) + UNDISPATCHED -> block.startCoroutineUndispatched(receiver, completion) + LAZY -> Unit // will start lazily + } + + /** + * Returns `true` when [LAZY]. + * + * @suppress **This an internal API and should not be used from general code.** + */ + @InternalCoroutinesApi + public val isLazy: Boolean get() = this === LAZY +} diff --git a/kotlinx-coroutines-core/common/src/Debug.common.kt b/kotlinx-coroutines-core/common/src/Debug.common.kt new file mode 100644 index 0000000000..60b54cf600 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Debug.common.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines + +internal expect val DEBUG: Boolean +internal expect val Any.hexAddress: String +internal expect val Any.classSimpleName: String +internal expect fun assert(value: () -> Boolean) + +/** + * Throwable which can be cloned during stacktrace recovery in a class-specific way. + * For additional information about stacktrace recovery see [STACKTRACE_RECOVERY_PROPERTY_NAME] + * + * Example of usage: + * ``` + * class BadResponseCodeException(val responseCode: Int) : Exception(), CopyableThrowable { + * + * override fun createCopy(): BadResponseCodeException { + * val result = BadResponseCodeException(responseCode) + * result.initCause(this) + * return result + * } + * ``` + * + * Copy mechanism is used only on JVM, but it might be convenient to implement it in common exceptions, + * so on JVM their stacktraces will be properly recovered. + */ +@ExperimentalCoroutinesApi // Since 1.2.0, no ETA on stability +public interface CopyableThrowable where T : Throwable, T : CopyableThrowable { + + /** + * Creates a copy of the current instance. + * + * For better debuggability, it is recommended to use original exception as [cause][Throwable.cause] of the resulting one. + * Stacktrace of copied exception will be overwritten by stacktrace recovery machinery by [Throwable.setStackTrace] call. + * An exception can opt-out of copying by returning `null` from this function. + * Suppressed exceptions of the original exception should not be copied in order to avoid circular exceptions. + * + * This function is allowed to create a copy with a modified [message][Throwable.message], but it should be noted + * that the copy can be later recovered as well and message modification code should handle this situation correctly + * (e.g. by also storing the original message and checking it) to produce a human-readable result. + */ + public fun createCopy(): T? +} diff --git a/kotlinx-coroutines-core/common/src/Deferred.kt b/kotlinx-coroutines-core/common/src/Deferred.kt new file mode 100644 index 0000000000..0404cdd4ee --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Deferred.kt @@ -0,0 +1,102 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.selects.* + +/** + * Deferred value is a non-blocking cancellable future — it is a [Job] with a result. + * + * It is created with the [async][CoroutineScope.async] coroutine builder or via the constructor of [CompletableDeferred] class. + * It is in [active][isActive] state while the value is being computed. + * + * `Deferred` has the same state machine as the [Job] with additional convenience methods to retrieve + * the successful or failed result of the computation that was carried out. The result of the deferred is + * available when it is [completed][isCompleted] and can be retrieved by [await] method, which throws + * an exception if the deferred had failed. + * Note that a _cancelled_ deferred is also considered as completed. + * The corresponding exception can be retrieved via [getCompletionExceptionOrNull] from a completed instance of deferred. + * + * Usually, a deferred value is created in _active_ state (it is created and started). + * However, the [async][CoroutineScope.async] coroutine builder has an optional `start` parameter that creates a deferred value in _new_ state + * when this parameter is set to [CoroutineStart.LAZY]. + * Such a deferred can be made _active_ by invoking [start], [join], or [await]. + * + * A deferred value is a [Job]. A job in the + * [coroutineContext](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.coroutines/coroutine-context.html) + * of [async][CoroutineScope.async] builder represents the coroutine itself. + * + * All functions on this interface and on all interfaces derived from it are **thread-safe** and can + * be safely invoked from concurrent coroutines without external synchronization. + */ +@OptIn(ExperimentalSubclassOptIn::class) +@SubclassOptInRequired(InternalForInheritanceCoroutinesApi::class) +public interface Deferred : Job { + + /** + * Awaits for completion of this value without blocking the thread and returns the resulting value or throws + * the exception if the deferred was cancelled. + * + * Unless the calling coroutine is cancelled, [await] will return the same result on each invocation: + * if the [Deferred] completed successfully, [await] will return the same value every time; + * if the [Deferred] completed exceptionally, [await] will rethrow the same exception. + * + * This suspending function is itself cancellable: if the [Job] of the current coroutine is cancelled or completed + * while this suspending function is waiting, this function immediately resumes with [CancellationException]. + * + * This means that [await] can throw [CancellationException] in two cases: + * - if the coroutine in which [await] was called got cancelled, + * - or if the [Deferred] itself got completed with a [CancellationException]. + * + * In both cases, the [CancellationException] will cancel the coroutine calling [await], unless it's caught. + * The following idiom may be helpful to avoid this: + * ``` + * try { + * deferred.await() + * } catch (e: CancellationException) { + * currentCoroutineContext().ensureActive() // throws if the current coroutine was cancelled + * processException(e) // if this line executes, the exception is the result of `await` itself + * } + * ``` + * + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + * + * This function can be used in [select] invocations with an [onAwait] clause. + * Use [isCompleted] to check for completion of this deferred value without waiting, and + * [join] to wait for completion without returning the result. + */ + public suspend fun await(): T + + /** + * Clause using the [await] suspending function as a [select] clause. + * It selects with the deferred value when the [Deferred] completes. + * If [Deferred] completes with an exception, the whole the [select] invocation fails with the same exception. + * Note that, if [Deferred] completed with a [CancellationException], throwing it may have unintended + * consequences. See [await] for details. + */ + public val onAwait: SelectClause1 + + /** + * Returns *completed* result or throws [IllegalStateException] if this deferred value has not + * [completed][isCompleted] yet. It throws the corresponding exception if this deferred was [cancelled][isCancelled]. + * + * This function is designed to be used from [invokeOnCompletion] handlers, when there is an absolute certainty that + * the value is already complete. See also [getCompletionExceptionOrNull]. + * + * **Note: This is an experimental api.** This function may be removed or renamed in the future. + */ + @ExperimentalCoroutinesApi + public fun getCompleted(): T + + /** + * Returns *completion exception* result if this deferred was [cancelled][isCancelled] and has [completed][isCompleted], + * `null` if it had completed normally, or throws [IllegalStateException] if this deferred value has not + * [completed][isCompleted] yet. + * + * This function is designed to be used from [invokeOnCompletion] handlers, when there is an absolute certainty that + * the value is already complete. See also [getCompleted]. + * + * **Note: This is an experimental api.** This function may be removed or renamed in the future. + */ + @ExperimentalCoroutinesApi + public fun getCompletionExceptionOrNull(): Throwable? +} diff --git a/kotlinx-coroutines-core/common/src/Delay.kt b/kotlinx-coroutines-core/common/src/Delay.kt new file mode 100644 index 0000000000..67d3d16bb1 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Delay.kt @@ -0,0 +1,158 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.selects.* +import kotlin.coroutines.* +import kotlin.time.* +import kotlin.time.Duration.Companion.nanoseconds + +/** + * This dispatcher _feature_ is implemented by [CoroutineDispatcher] implementations that natively support + * scheduled execution of tasks. + * + * Implementation of this interface affects operation of + * [delay][kotlinx.coroutines.delay] and [withTimeout] functions. + * + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public interface Delay { + + /** @suppress **/ + @Deprecated( + message = "Deprecated without replacement as an internal method never intended for public use", + level = DeprecationLevel.ERROR + ) // Error since 1.6.0 + public suspend fun delay(time: Long) { + if (time <= 0) return // don't delay + return suspendCancellableCoroutine { scheduleResumeAfterDelay(time, it) } + } + + /** + * Schedules resume of a specified [continuation] after a specified delay [timeMillis]. + * + * Continuation **must be scheduled** to resume even if it is already cancelled, because a cancellation is just + * an exception that the coroutine that used `delay` might wanted to catch and process. It might + * need to close some resources in its `finally` blocks, for example. + * + * This implementation is supposed to use dispatcher's native ability for scheduled execution in its thread(s). + * In order to avoid an extra delay of execution, the following code shall be used to resume this + * [continuation] when the code is already executing in the appropriate thread: + * + * ```kotlin + * with(continuation) { resumeUndispatchedWith(Unit) } + * ``` + */ + public fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) + + /** + * Schedules invocation of a specified [block] after a specified delay [timeMillis]. + * The resulting [DisposableHandle] can be used to [dispose][DisposableHandle.dispose] of this invocation + * request if it is not needed anymore. + */ + public fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + DefaultDelay.invokeOnTimeout(timeMillis, block, context) +} + +/** + * Enhanced [Delay] interface that provides additional diagnostics for [withTimeout]. + * Is going to be removed once there is proper JVM-default support. + * Then we'll be able put this function into [Delay] without breaking binary compatibility. + */ +@InternalCoroutinesApi +internal interface DelayWithTimeoutDiagnostics : Delay { + /** + * Returns a string that explains that the timeout has occurred, and explains what can be done about it. + */ + fun timeoutMessage(timeout: Duration): String +} + +/** + * Suspends until cancellation, in which case it will throw a [CancellationException]. + * + * This function returns [Nothing], so it can be used in any coroutine, + * regardless of the required return type. + * + * Usage example in callback adapting code: + * + * ```kotlin + * fun currentTemperature(): Flow = callbackFlow { + * val callback = SensorCallback { degreesCelsius: Double -> + * trySend(Temperature.celsius(degreesCelsius)) + * } + * try { + * registerSensorCallback(callback) + * awaitCancellation() // Suspends to keep getting updates until cancellation. + * } finally { + * unregisterSensorCallback(callback) + * } + * } + * ``` + * + * Usage example in (non declarative) UI code: + * + * ```kotlin + * suspend fun showStuffUntilCancelled(content: Stuff): Nothing { + * someSubView.text = content.title + * anotherSubView.text = content.description + * someView.visibleInScope { + * awaitCancellation() // Suspends so the view stays visible. + * } + * } + * ``` + */ +public suspend fun awaitCancellation(): Nothing = suspendCancellableCoroutine {} + +/** + * Delays coroutine for at least the given time without blocking a thread and resumes it after a specified time. + * If the given [timeMillis] is non-positive, this function returns immediately. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + * + * If you want to delay forever (until cancellation), consider using [awaitCancellation] instead. + * + * Note that delay can be used in [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. + * + * Implementation note: how exactly time is tracked is an implementation detail of [CoroutineDispatcher] in the context. + * @param timeMillis time in milliseconds. + */ +public suspend fun delay(timeMillis: Long) { + if (timeMillis <= 0) return // don't delay + return suspendCancellableCoroutine sc@ { cont: CancellableContinuation -> + // if timeMillis == Long.MAX_VALUE then just wait forever like awaitCancellation, don't schedule. + if (timeMillis < Long.MAX_VALUE) { + cont.context.delay.scheduleResumeAfterDelay(timeMillis, cont) + } + } +} + +/** + * Delays coroutine for at least the given [duration] without blocking a thread and resumes it after the specified time. + * If the given [duration] is non-positive, this function returns immediately. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + * + * If you want to delay forever (until cancellation), consider using [awaitCancellation] instead. + * + * Note that delay can be used in [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. + * + * Implementation note: how exactly time is tracked is an implementation detail of [CoroutineDispatcher] in the context. + */ +public suspend fun delay(duration: Duration): Unit = delay(duration.toDelayMillis()) + +/** Returns [Delay] implementation of the given context */ +internal val CoroutineContext.delay: Delay get() = get(ContinuationInterceptor) as? Delay ?: DefaultDelay + +/** + * Convert this duration to its millisecond value. Durations which have a nanosecond component less than + * a single millisecond will be rounded up to the next largest millisecond. + */ +internal fun Duration.toDelayMillis(): Long = when (isPositive()) { + true -> plus(999_999L.nanoseconds).inWholeMilliseconds + false -> 0L +} diff --git a/kotlinx-coroutines-core/common/src/Dispatchers.common.kt b/kotlinx-coroutines-core/common/src/Dispatchers.common.kt new file mode 100644 index 0000000000..c499a47f92 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Dispatchers.common.kt @@ -0,0 +1,73 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* + +/** + * Groups various implementations of [CoroutineDispatcher]. + */ +public expect object Dispatchers { + /** + * The default [CoroutineDispatcher] that is used by all standard builders like + * [launch][CoroutineScope.launch], [async][CoroutineScope.async], etc. + * if neither a dispatcher nor any other [ContinuationInterceptor] is specified in their context. + * + * It is backed by a shared pool of threads on JVM and Native. By default, the maximum number of threads used + * by this dispatcher is equal to the number of CPU cores, but is at least two. + */ + public val Default: CoroutineDispatcher + + /** + * A coroutine dispatcher that is confined to the Main thread operating with UI objects. + * Usually such dispatchers are single-threaded. + * + * Access to this property may throw an [IllegalStateException] if no main dispatchers are present in the classpath. + * + * Depending on platform and classpath, it can be mapped to different dispatchers: + * - On JVM it is either the Android main thread dispatcher, JavaFx, or Swing EDT dispatcher. It is chosen by the + * [`ServiceLoader`](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html). + * - On JS it is equivalent to the [Default] dispatcher with [immediate][MainCoroutineDispatcher.immediate] support. + * - On Native Darwin-based targets, it is a dispatcher backed by Darwin's main queue. + * - On other Native targets, it is not available. + * - `Dispatchers.setMain` from the `kotlinx-coroutines-test` artifact can replace the main dispatcher with a mock one for testing. + * + * In order to work with the `Main` dispatcher on the JVM, the following artifact should be added to the project runtime dependencies: + * - `kotlinx-coroutines-android` — for Android Main thread dispatcher + * - `kotlinx-coroutines-javafx` — for JavaFx Application thread dispatcher + * - `kotlinx-coroutines-swing` — for Swing EDT dispatcher + */ + public val Main: MainCoroutineDispatcher + + /** + * A coroutine dispatcher that is not confined to any specific thread. + * It executes the initial continuation of a coroutine in the current call-frame + * and lets the coroutine resume in whatever thread that is used by the corresponding suspending function, without + * mandating any specific threading policy. Nested coroutines launched in this dispatcher form an event-loop to avoid + * stack overflows. + * + * ### Event loop + * Event loop semantics is a purely internal concept and has no guarantees on the order of execution + * except that all queued coroutines will be executed on the current thread in the lexical scope of the outermost + * unconfined coroutine. + * + * For example, the following code: + * ``` + * withContext(Dispatchers.Unconfined) { + * println(1) + * launch(Dispatchers.Unconfined) { // Nested unconfined + * println(2) + * } + * println(3) + * } + * println("Done") + * ``` + * Can print both "1 2 3" and "1 3 2". This is an implementation detail that can be changed. + * However, it is guaranteed that "Done" will only be printed once the code in both `withContext` and `launch` completes. + * + * If you need your coroutine to be confined to a particular thread or a thread-pool after resumption, + * but still want to execute it in the current call-frame until its first suspension, you can use + * an optional [CoroutineStart] parameter in coroutine builders like + * [launch][CoroutineScope.launch] and [async][CoroutineScope.async] setting it to + * the value of [CoroutineStart.UNDISPATCHED]. + */ + public val Unconfined: CoroutineDispatcher +} diff --git a/kotlinx-coroutines-core/common/src/EventLoop.common.kt b/kotlinx-coroutines-core/common/src/EventLoop.common.kt new file mode 100644 index 0000000000..84291a1b69 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/EventLoop.common.kt @@ -0,0 +1,546 @@ +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlinx.coroutines.internal.* +import kotlin.concurrent.Volatile +import kotlin.coroutines.* +import kotlin.jvm.* + +/** + * Extended by [CoroutineDispatcher] implementations that have event loop inside and can + * be asked to process next event from their event queue. + * + * It may optionally implement [Delay] interface and support time-scheduled tasks. + * It is created or pigged back onto (see [ThreadLocalEventLoop]) + * by `runBlocking` and by [Dispatchers.Unconfined]. + * + * @suppress **This an internal API and should not be used from general code.** + */ +internal abstract class EventLoop : CoroutineDispatcher() { + /** + * Counts the number of nested `runBlocking` and [Dispatchers.Unconfined] that use this event loop. + */ + private var useCount = 0L + + /** + * Set to true on any use by `runBlocking`, because it potentially leaks this loop to other threads, so + * this instance must be properly shutdown. We don't need to shutdown event loop that was used solely + * by [Dispatchers.Unconfined] -- it can be left as [ThreadLocalEventLoop] and reused next time. + */ + private var shared = false + + /** + * Queue used by [Dispatchers.Unconfined] tasks. + * These tasks are thread-local for performance and take precedence over the rest of the queue. + */ + private var unconfinedQueue: ArrayDeque>? = null + + /** + * Processes next event in this event loop. + * + * The result of this function is to be interpreted like this: + * - `<= 0` -- there are potentially more events for immediate processing; + * - `> 0` -- a number of nanoseconds to wait for next scheduled event; + * - [Long.MAX_VALUE] -- no more events. + * + * **NOTE**: Must be invoked only from the event loop's thread + * (no check for performance reasons, may be added in the future). + */ + open fun processNextEvent(): Long { + if (!processUnconfinedEvent()) return Long.MAX_VALUE + return 0 + } + + protected open val isEmpty: Boolean get() = isUnconfinedQueueEmpty + + protected open val nextTime: Long + get() { + val queue = unconfinedQueue ?: return Long.MAX_VALUE + return if (queue.isEmpty()) Long.MAX_VALUE else 0L + } + + fun processUnconfinedEvent(): Boolean { + val queue = unconfinedQueue ?: return false + val task = queue.removeFirstOrNull() ?: return false + task.run() + return true + } + /** + * Returns `true` if the invoking `runBlocking(context) { ... }` that was passed this event loop in its context + * parameter should call [processNextEvent] for this event loop (otherwise, it will process thread-local one). + * By default, event loop implementation is thread-local and should not processed in the context + * (current thread's event loop should be processed instead). + */ + open fun shouldBeProcessedFromContext(): Boolean = false + + /** + * Dispatches task whose dispatcher returned `false` from [CoroutineDispatcher.isDispatchNeeded] + * into the current event loop. + */ + fun dispatchUnconfined(task: DispatchedTask<*>) { + val queue = unconfinedQueue ?: + ArrayDeque>().also { unconfinedQueue = it } + queue.addLast(task) + } + + val isActive: Boolean + get() = useCount > 0 + + val isUnconfinedLoopActive: Boolean + get() = useCount >= delta(unconfined = true) + + // May only be used from the event loop's thread + val isUnconfinedQueueEmpty: Boolean + get() = unconfinedQueue?.isEmpty() ?: true + + private fun delta(unconfined: Boolean) = + if (unconfined) (1L shl 32) else 1L + + fun incrementUseCount(unconfined: Boolean = false) { + useCount += delta(unconfined) + if (!unconfined) shared = true + } + + fun decrementUseCount(unconfined: Boolean = false) { + useCount -= delta(unconfined) + if (useCount > 0) return + assert { useCount == 0L } // "Extra decrementUseCount" + if (shared) { + // shut it down and remove from ThreadLocalEventLoop + shutdown() + } + } + + final override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + parallelism.checkParallelism() + return namedOrThis(name) // Single-threaded, short-circuit + } + + open fun shutdown() {} +} + +internal object ThreadLocalEventLoop { + private val ref = commonThreadLocal(Symbol("ThreadLocalEventLoop")) + + internal val eventLoop: EventLoop + get() = ref.get() ?: createEventLoop().also { ref.set(it) } + + internal fun currentOrNull(): EventLoop? = + ref.get() + + internal fun resetEventLoop() { + ref.set(null) + } + + internal fun setEventLoop(eventLoop: EventLoop) { + ref.set(eventLoop) + } +} + +private val DISPOSED_TASK = Symbol("REMOVED_TASK") + +// results for scheduleImpl +private const val SCHEDULE_OK = 0 +private const val SCHEDULE_COMPLETED = 1 +private const val SCHEDULE_DISPOSED = 2 + +private const val MS_TO_NS = 1_000_000L +private const val MAX_MS = Long.MAX_VALUE / MS_TO_NS + +/** + * First-line overflow protection -- limit maximal delay. + * Delays longer than this one (~146 years) are considered to be delayed "forever". + */ +private const val MAX_DELAY_NS = Long.MAX_VALUE / 2 + +internal fun delayToNanos(timeMillis: Long): Long = when { + timeMillis <= 0 -> 0L + timeMillis >= MAX_MS -> Long.MAX_VALUE + else -> timeMillis * MS_TO_NS +} + +internal fun delayNanosToMillis(timeNanos: Long): Long = + timeNanos / MS_TO_NS + +private val CLOSED_EMPTY = Symbol("CLOSED_EMPTY") + +private typealias Queue = LockFreeTaskQueueCore + +internal expect abstract class EventLoopImplPlatform() : EventLoop { + // Called to unpark this event loop's thread + protected fun unpark() + + // Called to reschedule to DefaultExecutor when this event loop is complete + protected fun reschedule(now: Long, delayedTask: EventLoopImplBase.DelayedTask) +} + +internal abstract class EventLoopImplBase: EventLoopImplPlatform(), Delay { + // null | CLOSED_EMPTY | task | Queue + private val _queue = atomic(null) + + // Allocated only only once + private val _delayed = atomic(null) + + private val _isCompleted = atomic(false) + private var isCompleted + get() = _isCompleted.value + set(value) { _isCompleted.value = value } + + override val isEmpty: Boolean get() { + if (!isUnconfinedQueueEmpty) return false + val delayed = _delayed.value + if (delayed != null && !delayed.isEmpty) return false + return when (val queue = _queue.value) { + null -> true + is Queue<*> -> queue.isEmpty + else -> queue === CLOSED_EMPTY + } + } + + override val nextTime: Long + get() { + if (super.nextTime == 0L) return 0L + val queue = _queue.value + when { + queue === null -> {} // empty queue -- proceed + queue is Queue<*> -> if (!queue.isEmpty) return 0 // non-empty queue + queue === CLOSED_EMPTY -> return Long.MAX_VALUE // no more events -- closed + else -> return 0 // non-empty queue + } + val nextDelayedTask = _delayed.value?.peek() ?: return Long.MAX_VALUE + return (nextDelayedTask.nanoTime - nanoTime()).coerceAtLeast(0) + } + + override fun shutdown() { + // Clean up thread-local reference here -- this event loop is shutting down + ThreadLocalEventLoop.resetEventLoop() + // We should signal that this event loop should not accept any more tasks + // and process queued events (that could have been added after last processNextEvent) + isCompleted = true + closeQueue() + // complete processing of all queued tasks + while (processNextEvent() <= 0) { /* spin */ } + // reschedule the rest of delayed tasks + rescheduleAllDelayed() + } + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val timeNanos = delayToNanos(timeMillis) + if (timeNanos < MAX_DELAY_NS) { + val now = nanoTime() + DelayedResumeTask(now + timeNanos, continuation).also { task -> + /* + * Order is important here: first we schedule the heap and only then + * publish it to continuation. Otherwise, `DelayedResumeTask` would + * have to know how to be disposed of even when it wasn't scheduled yet. + */ + schedule(now, task) + continuation.disposeOnCancellation(task) + } + } + } + + protected fun scheduleInvokeOnTimeout(timeMillis: Long, block: Runnable): DisposableHandle { + val timeNanos = delayToNanos(timeMillis) + return if (timeNanos < MAX_DELAY_NS) { + val now = nanoTime() + DelayedRunnableTask(now + timeNanos, block).also { task -> + schedule(now, task) + } + } else { + NonDisposableHandle + } + } + + override fun processNextEvent(): Long { + // unconfined events take priority + if (processUnconfinedEvent()) return 0 + // queue all delayed tasks that are due to be executed + enqueueDelayedTasks() + // then process one event from queue + val task = dequeue() + if (task != null) { + platformAutoreleasePool { task.run() } + return 0 + } + return nextTime + } + + final override fun dispatch(context: CoroutineContext, block: Runnable) = enqueue(block) + + open fun enqueue(task: Runnable) { + // are there some delayed tasks that should execute before this one? If so, move them to the queue first. + enqueueDelayedTasks() + if (enqueueImpl(task)) { + // todo: we should unpark only when this delayed task became first in the queue + unpark() + } else { + DefaultExecutor.enqueue(task) + } + } + + @Suppress("UNCHECKED_CAST") + private fun enqueueImpl(task: Runnable): Boolean { + _queue.loop { queue -> + if (isCompleted) return false // fail fast if already completed, may still add, but queues will close + when (queue) { + null -> if (_queue.compareAndSet(null, task)) return true + is Queue<*> -> { + when ((queue as Queue).addLast(task)) { + Queue.ADD_SUCCESS -> return true + Queue.ADD_CLOSED -> return false + Queue.ADD_FROZEN -> _queue.compareAndSet(queue, queue.next()) + } + } + else -> when { + queue === CLOSED_EMPTY -> return false + else -> { + // update to full-blown queue to add one more + val newQueue = Queue(Queue.INITIAL_CAPACITY, singleConsumer = true) + newQueue.addLast(queue as Runnable) + newQueue.addLast(task) + if (_queue.compareAndSet(queue, newQueue)) return true + } + } + } + } + } + + @Suppress("UNCHECKED_CAST") + private fun dequeue(): Runnable? { + _queue.loop { queue -> + when (queue) { + null -> return null + is Queue<*> -> { + val result = (queue as Queue).removeFirstOrNull() + if (result !== Queue.REMOVE_FROZEN) return result as Runnable? + _queue.compareAndSet(queue, queue.next()) + } + else -> when { + queue === CLOSED_EMPTY -> return null + else -> if (_queue.compareAndSet(queue, null)) return queue as Runnable + } + } + } + } + + /** Move all delayed tasks that are due to the main queue. */ + private fun enqueueDelayedTasks() { + val delayed = _delayed.value + if (delayed != null && !delayed.isEmpty) { + val now = nanoTime() + while (true) { + // make sure that moving from delayed to queue removes from delayed only after it is added to queue + // to make sure that 'isEmpty' and `nextTime` that check both of them + // do not transiently report that both delayed and queue are empty during move + delayed.removeFirstIf { + if (it.timeToExecute(now)) { + enqueueImpl(it) + } else + false + } ?: break // quit loop when nothing more to remove or enqueueImpl returns false on "isComplete" + } + } + } + + private fun closeQueue() { + assert { isCompleted } + _queue.loop { queue -> + when (queue) { + null -> if (_queue.compareAndSet(null, CLOSED_EMPTY)) return + is Queue<*> -> { + queue.close() + return + } + else -> when { + queue === CLOSED_EMPTY -> return + else -> { + // update to full-blown queue to close + val newQueue = Queue(Queue.INITIAL_CAPACITY, singleConsumer = true) + newQueue.addLast(queue as Runnable) + if (_queue.compareAndSet(queue, newQueue)) return + } + } + } + } + + } + + fun schedule(now: Long, delayedTask: DelayedTask) { + when (scheduleImpl(now, delayedTask)) { + SCHEDULE_OK -> if (shouldUnpark(delayedTask)) unpark() + SCHEDULE_COMPLETED -> reschedule(now, delayedTask) + SCHEDULE_DISPOSED -> {} // do nothing -- task was already disposed + else -> error("unexpected result") + } + } + + private fun shouldUnpark(task: DelayedTask): Boolean = _delayed.value?.peek() === task + + private fun scheduleImpl(now: Long, delayedTask: DelayedTask): Int { + if (isCompleted) return SCHEDULE_COMPLETED + val delayedQueue = _delayed.value ?: run { + _delayed.compareAndSet(null, DelayedTaskQueue(now)) + _delayed.value!! + } + return delayedTask.scheduleTask(now, delayedQueue, this) + } + + // It performs "hard" shutdown for test cleanup purposes + protected fun resetAll() { + _queue.value = null + _delayed.value = null + } + + // This is a "soft" (normal) shutdown + private fun rescheduleAllDelayed() { + val now = nanoTime() + while (true) { + /* + * `removeFirstOrNull` below is the only operation on DelayedTask & ThreadSafeHeap that is not + * synchronized on DelayedTask itself. All other operation are synchronized both on + * DelayedTask & ThreadSafeHeap instances (in this order). It is still safe, because `dispose` + * first removes DelayedTask from the heap (under synchronization) then + * assign "_heap = DISPOSED_TASK", so there cannot be ever a race to _heap reference update. + */ + val delayedTask = _delayed.value?.removeFirstOrNull() ?: break + reschedule(now, delayedTask) + } + } + + internal abstract class DelayedTask( + /** + * This field can be only modified in [scheduleTask] before putting this DelayedTask + * into heap to avoid overflow and corruption of heap data structure. + */ + @JvmField var nanoTime: Long + ) : Runnable, Comparable, DisposableHandle, ThreadSafeHeapNode, SynchronizedObject() { + @Volatile + private var _heap: Any? = null // null | ThreadSafeHeap | DISPOSED_TASK + + override var heap: ThreadSafeHeap<*>? + get() = _heap as? ThreadSafeHeap<*> + set(value) { + require(_heap !== DISPOSED_TASK) // this can never happen, it is always checked before adding/removing + _heap = value + } + + override var index: Int = -1 + + override fun compareTo(other: DelayedTask): Int { + val dTime = nanoTime - other.nanoTime + return when { + dTime > 0 -> 1 + dTime < 0 -> -1 + else -> 0 + } + } + + fun timeToExecute(now: Long): Boolean = now - nanoTime >= 0L + + fun scheduleTask(now: Long, delayed: DelayedTaskQueue, eventLoop: EventLoopImplBase): Int = synchronized(this) { + if (_heap === DISPOSED_TASK) return SCHEDULE_DISPOSED // don't add -- was already disposed + delayed.addLastIf(this) { firstTask -> + if (eventLoop.isCompleted) return SCHEDULE_COMPLETED // non-local return from scheduleTask + /** + * We are about to add new task and we have to make sure that [DelayedTaskQueue] + * invariant is maintained. The code in this lambda is additionally executed under + * the lock of [DelayedTaskQueue] and working with [DelayedTaskQueue.timeNow] here is thread-safe. + */ + if (firstTask == null) { + /** + * When adding the first delayed task we simply update queue's [DelayedTaskQueue.timeNow] to + * the current now time even if that means "going backwards in time". This makes the structure + * self-correcting in spite of wild jumps in `nanoTime()` measurements once all delayed tasks + * are removed from the delayed queue for execution. + */ + delayed.timeNow = now + } else { + /** + * Carefully update [DelayedTaskQueue.timeNow] so that it does not sweep past first's tasks time + * and only goes forward in time. We cannot let it go backwards in time or invariant can be + * violated for tasks that were already scheduled. + */ + val firstTime = firstTask.nanoTime + // compute min(now, firstTime) using a wrap-safe check + val minTime = if (firstTime - now >= 0) now else firstTime + // update timeNow only when going forward in time + if (minTime - delayed.timeNow > 0) delayed.timeNow = minTime + } + /** + * Here [DelayedTaskQueue.timeNow] was already modified and we have to double-check that newly added + * task does not violate [DelayedTaskQueue] invariant because of that. Note also that this scheduleTask + * function can be called to reschedule from one queue to another and this might be another reason + * where new task's time might now violate invariant. + * We correct invariant violation (if any) by simply changing this task's time to now. + */ + if (nanoTime - delayed.timeNow < 0) nanoTime = delayed.timeNow + true + } + return SCHEDULE_OK + } + + final override fun dispose(): Unit = synchronized(this) { + val heap = _heap + if (heap === DISPOSED_TASK) return // already disposed + (heap as? DelayedTaskQueue)?.remove(this) // remove if it is in heap (first) + _heap = DISPOSED_TASK // never add again to any heap + } + + override fun toString(): String = "Delayed[nanos=$nanoTime]" + } + + private inner class DelayedResumeTask( + nanoTime: Long, + private val cont: CancellableContinuation + ) : DelayedTask(nanoTime) { + override fun run() { with(cont) { resumeUndispatched(Unit) } } + override fun toString(): String = super.toString() + cont.toString() + } + + private class DelayedRunnableTask( + nanoTime: Long, + private val block: Runnable + ) : DelayedTask(nanoTime) { + override fun run() { block.run() } + override fun toString(): String = super.toString() + block.toString() + } + + /** + * Delayed task queue maintains stable time-comparision invariant despite potential wraparounds in + * long nano time measurements by maintaining last observed [timeNow]. It protects the integrity of the + * heap data structure in spite of potential non-monotonicity of `nanoTime()` source. + * The invariant is that for every scheduled [DelayedTask]: + * + * ``` + * delayedTask.nanoTime - timeNow >= 0 + * ``` + * + * So the comparison of scheduled tasks via [DelayedTask.compareTo] is always stable as + * scheduled [DelayedTask.nanoTime] can be at most [Long.MAX_VALUE] apart. This invariant is maintained when + * new tasks are added by [DelayedTask.scheduleTask] function and it cannot be violated when tasks are removed + * (so there is nothing special to do there). + */ + internal class DelayedTaskQueue( + @JvmField var timeNow: Long + ) : ThreadSafeHeap() +} + +internal expect fun createEventLoop(): EventLoop + +internal expect fun nanoTime(): Long + +internal expect object DefaultExecutor { + fun enqueue(task: Runnable) +} + +/** + * Used by Darwin targets to wrap a [Runnable.run] call in an Objective-C Autorelease Pool. It is a no-op on JVM, JS and + * non-Darwin native targets. + * + * Coroutines on Darwin targets can call into the Objective-C world, where a callee may push a to-be-returned object to + * the Autorelease Pool, so as to avoid a premature ARC release before it reaches the caller. This means the pool must + * be eventually drained to avoid leaks. Since Kotlin Coroutines does not use [NSRunLoop], which provides automatic + * pool management, it must manage the pool creation and pool drainage manually. + */ +internal expect inline fun platformAutoreleasePool(crossinline block: () -> Unit) diff --git a/kotlinx-coroutines-core/common/src/Exceptions.common.kt b/kotlinx-coroutines-core/common/src/Exceptions.common.kt new file mode 100644 index 0000000000..e19c36f273 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Exceptions.common.kt @@ -0,0 +1,26 @@ +package kotlinx.coroutines + +/** + * This exception gets thrown if an exception is caught while processing [CompletionHandler] invocation for [Job]. + * + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public class CompletionHandlerException(message: String, cause: Throwable) : RuntimeException(message, cause) + +public expect open class CancellationException(message: String?) : IllegalStateException + +public expect fun CancellationException(message: String?, cause: Throwable?) : CancellationException + +internal expect class JobCancellationException( + message: String, + cause: Throwable?, + job: Job +) : CancellationException { + internal val job: Job +} + +internal class CoroutinesInternalError(message: String, cause: Throwable) : Error(message, cause) + +// For use in tests +internal expect val RECOVER_STACK_TRACES: Boolean diff --git a/kotlinx-coroutines-core/common/src/Guidance.kt b/kotlinx-coroutines-core/common/src/Guidance.kt new file mode 100644 index 0000000000..455ab9153d --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Guidance.kt @@ -0,0 +1,38 @@ +package kotlinx.coroutines + +import kotlin.coroutines.CoroutineContext +import kotlin.coroutines.EmptyCoroutineContext + +/** + * @suppress this is a function that should help users who are trying to use 'launch' + * without the corresponding coroutine scope. It is not supposed to be called. + */ +@Deprecated("'launch' can not be called without the corresponding coroutine scope. " + + "Consider wrapping 'launch' in 'coroutineScope { }', using 'runBlocking { }', " + + "or using some other 'CoroutineScope'", level = DeprecationLevel.ERROR) +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +@kotlin.internal.LowPriorityInOverloadResolution +public fun launch( + context: CoroutineContext = EmptyCoroutineContext, + start: CoroutineStart = CoroutineStart.DEFAULT, + block: suspend CoroutineScope.() -> Unit +): Job { + throw UnsupportedOperationException("Should never be called, was introduced to help with incomplete code") +} + +/** + * @suppress this is a function that should help users who are trying to use 'launch' + * without the corresponding coroutine scope. It is not supposed to be called. + */ +@Deprecated("'async' can not be called without the corresponding coroutine scope. " + + "Consider wrapping 'async' in 'coroutineScope { }', using 'runBlocking { }', " + + "or using some other 'CoroutineScope'", level = DeprecationLevel.ERROR) +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +@kotlin.internal.LowPriorityInOverloadResolution +public fun async( + context: CoroutineContext = EmptyCoroutineContext, + start: CoroutineStart = CoroutineStart.DEFAULT, + block: suspend CoroutineScope.() -> T +): Deferred { + throw UnsupportedOperationException("Should never be called, was introduced to help with incomplete code") +} diff --git a/kotlinx-coroutines-core/common/src/Job.kt b/kotlinx-coroutines-core/common/src/Job.kt new file mode 100644 index 0000000000..512699e29d --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Job.kt @@ -0,0 +1,691 @@ +@file:JvmMultifileClass +@file:JvmName("JobKt") +@file:Suppress("DEPRECATION_ERROR", "RedundantUnitReturnType") + +package kotlinx.coroutines + +import kotlinx.coroutines.selects.* +import kotlin.coroutines.* +import kotlin.jvm.* + +// --------------- core job interfaces --------------- + +/** + * A background job. + * Conceptually, a job is a cancellable thing with a lifecycle that + * concludes in its completion. + * + * Jobs can be arranged into parent-child hierarchies where the cancellation + * of a parent leads to the immediate cancellation of all its [children] recursively. + * Failure of a child with an exception other than [CancellationException] immediately cancels its parent and, + * consequently, all its other children. + * This behavior can be customized using [SupervisorJob]. + * + * The most basic instances of the `Job` interface are created like this: + * + * - A **coroutine job** is created with the [launch][CoroutineScope.launch] coroutine builder. + * It runs a specified block of code and completes upon completion of this block. + * - **[CompletableJob]** is created with a `Job()` factory function. + * It is completed by calling [CompletableJob.complete]. + * + * Conceptually, an execution of a job does not produce a result value. + * Jobs are launched solely for their + * side effects. + * See the [Deferred] interface for a job that produces a result. + * + * ### Job states + * + * A job has the following states: + * + * | **State** | [isActive] | [isCompleted] | [isCancelled] | + * | -------------------------------- | ---------- | ------------- | ------------- | + * | _New_ (optional initial state) | `false` | `false` | `false` | + * | _Active_ (default initial state) | `true` | `false` | `false` | + * | _Completing_ (transient state) | `true` | `false` | `false` | + * | _Cancelling_ (transient state) | `false` | `false` | `true` | + * | _Cancelled_ (final state) | `false` | `true` | `true` | + * | _Completed_ (final state) | `false` | `true` | `false` | + * + * + * Note that these states are mentioned in italics below to make them easier to distinguish. + * + * Usually, a job is created in the _active_ state (it is created and started). + * However, coroutine builders + * that provide an optional `start` parameter create a coroutine in the _new_ state when this parameter is set to + * [CoroutineStart.LAZY]. + * Such a job can be made _active_ by invoking [start] or [join]. + * + * A job is in the _active_ state while the coroutine is working or until the [CompletableJob] completes, + * fails, or is cancelled. + * + * Failure of an _active_ job with an exception transitions the state to the _cancelling_ state. + * A job can be cancelled at any time with the [cancel] function that forces it to transition to + * the _cancelling_ state immediately. + * The job becomes _cancelled_ when it finishes executing its work and + * all its children complete. + * + * Completion of an _active_ coroutine's body or a call to [CompletableJob.complete] transitions the job to + * the _completing_ state. + * It waits in the _completing_ state for all its children to complete before + * transitioning to the _completed_ state. + * Note that _completing_ state is purely internal to the job. + * For an outside observer, a _completing_ job is still + * active, while internally it is waiting for its children. + * + * ``` + * wait children + * +-----+ start +--------+ complete +-------------+ finish +-----------+ + * | New | -----> | Active | ---------> | Completing | -------> | Completed | + * +-----+ +--------+ +-------------+ +-----------+ + * | cancel / fail | + * | +----------------+ + * | | + * V V + * +------------+ finish +-----------+ + * | Cancelling | --------------------------------> | Cancelled | + * +------------+ +-----------+ + * ``` + * + * A `Job` instance in the + * [coroutineContext](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.coroutines/coroutine-context.html) + * represents the coroutine itself. + * + * ### Cancellation cause + * + * A coroutine job is said to _complete exceptionally_ when its body throws an exception; + * a [CompletableJob] is completed exceptionally by calling [CompletableJob.completeExceptionally]. + * An exceptionally completed job is cancelled, + * and the corresponding exception becomes the _cancellation cause_ of the job. + * + * Normal cancellation of a job is distinguished from its failure by the exception + * that caused its cancellation. + * A coroutine that throws a [CancellationException] is considered to be _cancelled_ normally. + * If a different exception causes the cancellation, then the job has _failed_. + * When a job has _failed_, its parent gets cancelled with the same type of exception, + * thus ensuring transparency in delegating parts of the job to its children. + * + * Note, that the [cancel] function on a job only accepts a [CancellationException] as a cancellation cause, thus + * calling [cancel] always results in a normal cancellation of a job, which does not lead to cancellation + * of its parent. + * This way, a parent can [cancel] his children (cancelling all their children recursively, too) + * without cancelling himself. + * + * ### Concurrency and synchronization + * + * All functions on this interface and on all interfaces derived from it are **thread-safe** and can + * be safely invoked from concurrent coroutines without external synchronization. + */ +@OptIn(ExperimentalSubclassOptIn::class) +@SubclassOptInRequired(InternalForInheritanceCoroutinesApi::class) +public interface Job : CoroutineContext.Element { + /** + * Key for [Job] instance in the coroutine context. + */ + public companion object Key : CoroutineContext.Key + + // ------------ state query ------------ + + /** + * Returns the parent of the current job if the parent-child relationship + * is established or `null` if the job has no parent or was successfully completed. + * + * Accesses to this property are not idempotent, the property becomes `null` as soon + * as the job is transitioned to its final state, whether it is cancelled or completed, + * and all job children are completed. + * + * For a coroutine, its corresponding job completes as soon as the coroutine itself + * and all its children are complete. + * + * @see [Job] state transitions for additional details. + */ + @ExperimentalCoroutinesApi + public val parent: Job? + + /** + * Returns `true` when this job is active -- it was already started and has not completed nor was cancelled yet. + * The job that is waiting for its [children] to complete is still considered to be active if it + * was not cancelled nor failed. + * + * See [Job] documentation for more details on job states. + */ + public val isActive: Boolean + + /** + * Returns `true` when this job has completed for any reason. A job that was cancelled or failed + * and has finished its execution is also considered complete. Job becomes complete only after + * all its [children] complete. + * + * See [Job] documentation for more details on job states. + */ + public val isCompleted: Boolean + + /** + * Returns `true` if this job was cancelled for any reason, either by explicit invocation of [cancel] or + * because it had failed or its child or parent was cancelled. + * In the general case, it does not imply that the + * job has already [completed][isCompleted], because it may still be finishing whatever it was doing and + * waiting for its [children] to complete. + * + * See [Job] documentation for more details on cancellation and failures. + */ + public val isCancelled: Boolean + + /** + * Returns [CancellationException] that signals the completion of this job. This function is + * used by [cancellable][suspendCancellableCoroutine] suspending functions. They throw exception + * returned by this function when they suspend in the context of this job and this job becomes _complete_. + * + * This function returns the original [cancel] cause of this job if that `cause` was an instance of + * [CancellationException]. Otherwise (if this job was cancelled with a cause of a different type, or + * was cancelled without a cause, or had completed normally), an instance of [CancellationException] is + * returned. The [CancellationException.cause] of the resulting [CancellationException] references + * the original cancellation cause that was passed to [cancel] function. + * + * This function throws [IllegalStateException] when invoked on a job that is still active. + * + * @suppress **This an internal API and should not be used from general code.** + */ + @InternalCoroutinesApi + public fun getCancellationException(): CancellationException + + // ------------ state update ------------ + + /** + * Starts coroutine related to this job (if any) if it was not started yet. + * The result is `true` if this invocation actually started coroutine or `false` + * if it was already started or completed. + */ + public fun start(): Boolean + + + /** + * Cancels this job with an optional cancellation [cause]. + * A cause can be used to specify an error message or to provide other details on + * the cancellation reason for debugging purposes. + * See [Job] documentation for full explanation of cancellation machinery. + */ + public fun cancel(cause: CancellationException? = null) + + /** + * @suppress This method implements old version of JVM ABI. Use [cancel]. + */ + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") + public fun cancel(): Unit = cancel(null) + + /** + * @suppress This method has bad semantics when cause is not a [CancellationException]. Use [cancel]. + */ + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") + public fun cancel(cause: Throwable? = null): Boolean + + // ------------ parent-child ------------ + + /** + * Returns a sequence of this job's children. + * + * A job becomes a child of this job when it is constructed with this job in its + * [CoroutineContext] or using an explicit `parent` parameter. + * + * A parent-child relation has the following effect: + * + * - Cancellation of parent with [cancel] or its exceptional completion (failure) + * immediately cancels all its children. + * - Parent cannot complete until all its children are complete. Parent waits for all its children to + * complete in _completing_ or _cancelling_ state. + * - Uncaught exception in a child, by default, cancels parent. This applies even to + * children created with [async][CoroutineScope.async] and other future-like + * coroutine builders, even though their exceptions are caught and are encapsulated in their result. + * This default behavior can be overridden with [SupervisorJob]. + */ + public val children: Sequence + + /** + * Attaches child job so that this job becomes its parent and + * returns a handle that should be used to detach it. + * + * A parent-child relation has the following effect: + * - Cancellation of parent with [cancel] or its exceptional completion (failure) + * immediately cancels all its children. + * - Parent cannot complete until all its children are complete. Parent waits for all its children to + * complete in _completing_ or _cancelling_ states. + * + * **A child must store the resulting [ChildHandle] and [dispose][DisposableHandle.dispose] the attachment + * to its parent on its own completion.** + * + * Coroutine builders and job factory functions that accept `parent` [CoroutineContext] parameter + * lookup a [Job] instance in the parent context and use this function to attach themselves as a child. + * They also store a reference to the resulting [ChildHandle] and dispose a handle when they complete. + * + * @suppress This is an internal API. This method is too error prone for public API. + */ + // ChildJob and ChildHandle are made internal on purpose to further deter 3rd-party impl of Job + @InternalCoroutinesApi + public fun attachChild(child: ChildJob): ChildHandle + + // ------------ state waiting ------------ + + /** + * Suspends the coroutine until this job is complete. This invocation resumes normally (without exception) + * when the job is complete for any reason and the [Job] of the invoking coroutine is still [active][isActive]. + * This function also [starts][Job.start] the corresponding coroutine if the [Job] was still in _new_ state. + * + * Note that the job becomes complete only when all its children are complete. + * + * This suspending function is cancellable and **always** checks for a cancellation of the invoking coroutine's Job. + * If the [Job] of the invoking coroutine is cancelled or completed when this + * suspending function is invoked or while it is suspended, this function + * throws [CancellationException]. + * + * In particular, it means that a parent coroutine invoking `join` on a child coroutine throws + * [CancellationException] if the child had failed, since a failure of a child coroutine cancels parent by default, + * unless the child was launched from within [supervisorScope]. + * + * This function can be used in [select] invocation with [onJoin] clause. + * Use [isCompleted] to check for a completion of this job without waiting. + * + * There is [cancelAndJoin] function that combines an invocation of [cancel] and `join`. + */ + public suspend fun join() + + /** + * Clause for [select] expression of [join] suspending function that selects when the job is complete. + * This clause never fails, even if the job completes exceptionally. + */ + public val onJoin: SelectClause0 + + // ------------ low-level state-notification ------------ + + /** + * Registers handler that is **synchronously** invoked once on completion of this job. + * When the job is already complete, then the handler is immediately invoked + * with the job's exception or cancellation cause or `null`. Otherwise, the handler will be invoked once when this + * job is complete. + * + * The meaning of `cause` that is passed to the handler: + * - Cause is `null` when the job has completed normally. + * - Cause is an instance of [CancellationException] when the job was cancelled _normally_. + * **It should not be treated as an error**. In particular, it should not be reported to error logs. + * - Otherwise, the job had _failed_. + * + * The resulting [DisposableHandle] can be used to [dispose][DisposableHandle.dispose] the + * registration of this handler and release its memory if its invocation is no longer needed. + * There is no need to dispose the handler after completion of this job. The references to + * all the handlers are released when this job completes. + * + * Installed [handler] should not throw any exceptions. If it does, they will get caught, + * wrapped into [CompletionHandlerException], and rethrown, potentially causing crash of unrelated code. + * + * **Note**: Implementation of `CompletionHandler` must be fast, non-blocking, and thread-safe. + * This handler can be invoked concurrently with the surrounding code. + * There is no guarantee on the execution context in which the [handler] is invoked. + */ + public fun invokeOnCompletion(handler: CompletionHandler): DisposableHandle + + /** + * Kept for preserving compatibility. Shouldn't be used by anyone. + * @suppress + */ + @InternalCoroutinesApi + public fun invokeOnCompletion( + onCancelling: Boolean = false, + invokeImmediately: Boolean = true, + handler: CompletionHandler): DisposableHandle + + // ------------ unstable internal API ------------ + + /** + * @suppress **Error**: Operator '+' on two Job objects is meaningless. + * Job is a coroutine context element and `+` is a set-sum operator for coroutine contexts. + * The job to the right of `+` just replaces the job the left of `+`. + */ + @Suppress("DeprecatedCallableAddReplaceWith") + @Deprecated(message = "Operator '+' on two Job objects is meaningless. " + + "Job is a coroutine context element and `+` is a set-sum operator for coroutine contexts. " + + "The job to the right of `+` just replaces the job the left of `+`.", + level = DeprecationLevel.ERROR) + public operator fun plus(other: Job): Job = other +} + +/** + * Registers a handler that is **synchronously** invoked once on cancellation or completion of this job. + * + * If the handler would have been invoked earlier if it was registered at that time, then it is invoked immediately, + * unless [invokeImmediately] is set to `false`. + * + * The meaning of `cause` that is passed to the handler is: + * - It is `null` if the job has completed normally. + * - It is an instance of [CancellationException] if the job was cancelled _normally_. + * **It should not be treated as an error**. In particular, it should not be reported to error logs. + * - Otherwise, the job had _failed_. + * + * The resulting [DisposableHandle] can be used to [dispose][DisposableHandle.dispose] of the registration of this + * handler and release its memory if its invocation is no longer needed. + * There is no need to dispose of the handler after completion of this job. The references to + * all the handlers are released when this job completes. + */ +internal fun Job.invokeOnCompletion( + invokeImmediately: Boolean = true, + handler: JobNode, +): DisposableHandle = when (this) { + is JobSupport -> invokeOnCompletionInternal(invokeImmediately, handler) + else -> invokeOnCompletion(handler.onCancelling, invokeImmediately, handler::invoke) +} + +/** + * Creates a job object in an active state. + * A failure of any child of this job immediately causes this job to fail, too, and cancels the rest of its children. + * + * To handle children failure independently of each other use [SupervisorJob]. + * + * If [parent] job is specified, then this job becomes a child job of its parent and + * is cancelled when its parent fails or is cancelled. All this job's children are cancelled in this case, too. + * + * Conceptually, the resulting job works in the same way as the job created by the `launch { body }` invocation + * (see [launch]), but without any code in the body. It is active until cancelled or completed. Invocation of + * [CompletableJob.complete] or [CompletableJob.completeExceptionally] corresponds to the successful or + * failed completion of the body of the coroutine. + * + * @param parent an optional parent job. + */ +@Suppress("FunctionName") +public fun Job(parent: Job? = null): CompletableJob = JobImpl(parent) + +/** @suppress Binary compatibility only */ +@Suppress("FunctionName") +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") +@JvmName("Job") +public fun Job0(parent: Job? = null): Job = Job(parent) + +/** + * A handle to an allocated object that can be disposed to make it eligible for garbage collection. + */ +public fun interface DisposableHandle { + /** + * Disposes the corresponding object, making it eligible for garbage collection. + * Repeated invocation of this function has no effect. + */ + public fun dispose() +} + +// -------------------- Parent-child communication -------------------- + +/** + * A reference that parent receives from its child so that it can report its cancellation. + * + * @suppress **This is unstable API and it is subject to change.** + */ +@InternalCoroutinesApi +@Deprecated(level = DeprecationLevel.ERROR, message = "This is internal API and may be removed in the future releases") +@OptIn(InternalForInheritanceCoroutinesApi::class) +public interface ChildJob : Job { + /** + * Parent is cancelling its child by invoking this method. + * Child finds the cancellation cause using [ParentJob.getChildJobCancellationCause]. + * This method does nothing is the child is already being cancelled. + * + * @suppress **This is unstable API and it is subject to change.** + */ + @InternalCoroutinesApi + public fun parentCancelled(parentJob: ParentJob) +} + +/** + * A reference that child receives from its parent when it is being cancelled by the parent. + * + * @suppress **This is unstable API and it is subject to change.** + */ +@InternalCoroutinesApi +@Deprecated(level = DeprecationLevel.ERROR, message = "This is internal API and may be removed in the future releases") +@OptIn(InternalForInheritanceCoroutinesApi::class) +public interface ParentJob : Job { + /** + * Child job is using this method to learn its cancellation cause when the parent cancels it with [ChildJob.parentCancelled]. + * This method is invoked only if the child was not already being cancelled. + * + * Note that [CancellationException] is the method's return type: if child is cancelled by its parent, + * then the original exception is **already** handled by either the parent or the original source of failure. + * + * @suppress **This is unstable API and it is subject to change.** + */ + @InternalCoroutinesApi + public fun getChildJobCancellationCause(): CancellationException +} + +/** + * A handle that child keep onto its parent so that it is able to report its cancellation. + * + * @suppress **This is unstable API and it is subject to change.** + */ +@InternalCoroutinesApi +@Deprecated(level = DeprecationLevel.ERROR, message = "This is internal API and may be removed in the future releases") +public interface ChildHandle : DisposableHandle { + + /** + * Returns the parent of the current parent-child relationship. + * @suppress **This is unstable API and it is subject to change.** + */ + @InternalCoroutinesApi + public val parent: Job? + + /** + * Child is cancelling its parent by invoking this method. + * This method is invoked by the child twice. The first time child report its root cause as soon as possible, + * so that all its siblings and the parent can start cancelling their work asap. The second time + * child invokes this method when it had aggregated and determined its final cancellation cause. + * + * @suppress **This is unstable API and it is subject to change.** + */ + @InternalCoroutinesApi + public fun childCancelled(cause: Throwable): Boolean +} + +// -------------------- Job extensions -------------------- + +/** + * Disposes a specified [handle] when this job is complete. + * + * This is a shortcut for the following code with slightly more efficient implementation (one fewer object created). + * ``` + * invokeOnCompletion { handle.dispose() } + * ``` + */ +internal fun Job.disposeOnCompletion(handle: DisposableHandle): DisposableHandle = + invokeOnCompletion(handler = DisposeOnCompletion(handle)) + +/** + * Cancels the job and suspends the invoking coroutine until the cancelled job is complete. + * + * This suspending function is cancellable and **always** checks for a cancellation of the invoking coroutine's Job. + * If the [Job] of the invoking coroutine is cancelled or completed when this + * suspending function is invoked or while it is suspended, this function + * throws [CancellationException]. + * + * In particular, it means that a parent coroutine invoking `cancelAndJoin` on a child coroutine throws + * [CancellationException] if the child had failed, since a failure of a child coroutine cancels parent by default, + * unless the child was launched from within [supervisorScope]. + * + * This is a shortcut for the invocation of [cancel][Job.cancel] followed by [join][Job.join]. + */ +public suspend fun Job.cancelAndJoin() { + cancel() + return join() +} + +/** + * Cancels all [children][Job.children] jobs of this coroutine using [Job.cancel] for all of them + * with an optional cancellation [cause]. + * Unlike [Job.cancel] on this job as a whole, the state of this job itself is not affected. + */ +public fun Job.cancelChildren(cause: CancellationException? = null) { + children.forEach { it.cancel(cause) } +} + +/** + * @suppress This method implements old version of JVM ABI. Use [cancel]. + */ +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") +public fun Job.cancelChildren(): Unit = cancelChildren(null) + +/** + * @suppress This method has bad semantics when cause is not a [CancellationException]. Use [Job.cancelChildren]. + */ +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") +public fun Job.cancelChildren(cause: Throwable? = null) { + children.forEach { (it as? JobSupport)?.cancelInternal(cause.orCancellation(this)) } +} + +// -------------------- CoroutineContext extensions -------------------- + +/** + * Returns `true` when the [Job] of the coroutine in this context is still active + * (has not completed and was not cancelled yet) or the context does not have a [Job] in it. + * + * Check this property in long-running computation loops to support cancellation + * when [CoroutineScope.isActive] is not available: + * + * ``` + * while (coroutineContext.isActive) { + * // do some computation + * } + * ``` + * + * The `coroutineContext.isActive` expression is a shortcut for `get(Job)?.isActive ?: true`. + * See [Job.isActive]. + */ +public val CoroutineContext.isActive: Boolean + get() = get(Job)?.isActive ?: true + +/** + * Cancels [Job] of this context with an optional cancellation cause. + * See [Job.cancel] for details. + */ +public fun CoroutineContext.cancel(cause: CancellationException? = null) { + this[Job]?.cancel(cause) +} + +/** + * @suppress This method implements old version of JVM ABI. Use [CoroutineContext.cancel]. + */ +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") +public fun CoroutineContext.cancel(): Unit = cancel(null) + +/** + * Ensures that current job is [active][Job.isActive]. + * If the job is no longer active, throws [CancellationException]. + * If the job was cancelled, thrown exception contains the original cancellation cause. + * + * This method is a drop-in replacement for the following code, but with more precise exception: + * ``` + * if (!job.isActive) { + * throw CancellationException() + * } + * ``` + */ +public fun Job.ensureActive(): Unit { + if (!isActive) throw getCancellationException() +} + +/** + * Ensures that job in the current context is [active][Job.isActive]. + * + * If the job is no longer active, throws [CancellationException]. + * If the job was cancelled, thrown exception contains the original cancellation cause. + * This function does not do anything if there is no [Job] in the context, since such a coroutine cannot be cancelled. + * + * This method is a drop-in replacement for the following code, but with more precise exception: + * ``` + * if (!isActive) { + * throw CancellationException() + * } + * ``` + */ +public fun CoroutineContext.ensureActive() { + get(Job)?.ensureActive() +} + +/** + * Cancels current job, including all its children with a specified diagnostic error [message]. + * A [cause] can be specified to provide additional details on a cancellation reason for debugging purposes. + */ +public fun Job.cancel(message: String, cause: Throwable? = null): Unit = cancel(CancellationException(message, cause)) + +/** + * @suppress This method has bad semantics when cause is not a [CancellationException]. Use [CoroutineContext.cancel]. + */ +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") +public fun CoroutineContext.cancel(cause: Throwable? = null): Boolean { + val job = this[Job] as? JobSupport ?: return false + job.cancelInternal(cause.orCancellation(job)) + return true +} + +/** + * Cancels all children of the [Job] in this context, without touching the state of this job itself + * with an optional cancellation cause. See [Job.cancel]. + * It does not do anything if there is no job in the context or it has no children. + */ +public fun CoroutineContext.cancelChildren(cause: CancellationException? = null) { + this[Job]?.children?.forEach { it.cancel(cause) } +} + +/** + * @suppress This method implements old version of JVM ABI. Use [CoroutineContext.cancelChildren]. + */ +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") +public fun CoroutineContext.cancelChildren(): Unit = cancelChildren(null) + +/** + * Retrieves the current [Job] instance from the given [CoroutineContext] or + * throws [IllegalStateException] if no job is present in the context. + * + * This method is a short-cut for `coroutineContext[Job]!!` and should be used only when it is known in advance that + * the context does have instance of the job in it. + */ +public val CoroutineContext.job: Job get() = get(Job) ?: error("Current context doesn't contain Job in it: $this") + +/** + * @suppress This method has bad semantics when cause is not a [CancellationException]. Use [CoroutineContext.cancelChildren]. + */ +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") +public fun CoroutineContext.cancelChildren(cause: Throwable? = null) { + val job = this[Job] ?: return + job.children.forEach { (it as? JobSupport)?.cancelInternal(cause.orCancellation(job)) } +} + +private fun Throwable?.orCancellation(job: Job): Throwable = this ?: JobCancellationException("Job was cancelled", null, job) + +/** + * No-op implementation of [DisposableHandle]. + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public object NonDisposableHandle : DisposableHandle, ChildHandle { + + override val parent: Job? get() = null + + /** + * Does not do anything. + * @suppress + */ + override fun dispose() {} + + /** + * Returns `false`. + * @suppress + */ + override fun childCancelled(cause: Throwable): Boolean = false + + /** + * Returns "NonDisposableHandle" string. + * @suppress + */ + override fun toString(): String = "NonDisposableHandle" +} + +private class DisposeOnCompletion( + private val handle: DisposableHandle +) : JobNode() { + override val onCancelling get() = false + + override fun invoke(cause: Throwable?) = handle.dispose() +} diff --git a/kotlinx-coroutines-core/common/src/JobSupport.kt b/kotlinx-coroutines-core/common/src/JobSupport.kt new file mode 100644 index 0000000000..81ff9d4811 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/JobSupport.kt @@ -0,0 +1,1582 @@ +@file:Suppress("DEPRECATION_ERROR") + +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlin.js.* +import kotlin.jvm.* + +/** + * A concrete implementation of [Job]. It is optionally a child to a parent job. + * + * This is an open class designed for extension by more specific classes that might augment the + * state and mare store addition state information for completed jobs, like their result values. + * + * @param active when `true` the job is created in _active_ state, when `false` in _new_ state. See [Job] for details. + * @suppress **This is unstable API and it is subject to change.** + */ +@OptIn(InternalForInheritanceCoroutinesApi::class) +@Deprecated(level = DeprecationLevel.ERROR, message = "This is internal API and may be removed in the future releases") +public open class JobSupport constructor(active: Boolean) : Job, ChildJob, ParentJob { + final override val key: CoroutineContext.Key<*> get() = Job + + /* + === Internal states === + + name state class public state description + ------ ------------ ------------ ----------- + EMPTY_N EmptyNew : New no listeners + EMPTY_A EmptyActive : Active no listeners + SINGLE JobNode : Active a single listener + SINGLE+ JobNode : Active a single listener + NodeList added as its next + LIST_N InactiveNodeList : New a list of listeners (promoted once, does not got back to EmptyNew) + LIST_A NodeList : Active a list of listeners (promoted once, does not got back to JobNode/EmptyActive) + COMPLETING Finishing : Completing has a list of listeners (promoted once from LIST_*) + CANCELLING Finishing : Cancelling -- " -- + FINAL_C Cancelled : Cancelled Cancelled (final state) + FINAL_R : Completed produced some result + + === Transitions === + + New states Active states Inactive states + + +---------+ +---------+ } + | EMPTY_N | ----> | EMPTY_A | ----+ } Empty states + +---------+ +---------+ | } + | | | ^ | +----------+ + | | | | +--> | FINAL_* | + | | V | | +----------+ + | | +---------+ | } + | | | SINGLE | ----+ } JobNode states + | | +---------+ | } + | | | | } + | | V | } + | | +---------+ | } + | +-------> | SINGLE+ | ----+ } + | +---------+ | } + | | | + V V | + +---------+ +---------+ | } + | LIST_N | ----> | LIST_A | ----+ } [Inactive]NodeList states + +---------+ +---------+ | } + | | | | | + | | +--------+ | | + | | | V | + | | | +------------+ | +------------+ } + | +-------> | COMPLETING | --+-- | CANCELLING | } Finishing states + | | +------------+ +------------+ } + | | | ^ + | | | | + +--------+---------+--------------------+ + + + This state machine and its transition matrix are optimized for the common case when a job is created in active + state (EMPTY_A), at most one completion listener is added to it during its life-time, and it completes + successfully without children (in this case it directly goes from EMPTY_A or SINGLE state to FINAL_R + state without going to COMPLETING state) + + Note that the actual `_state` variable can also be a reference to atomic operation descriptor `OpDescriptor` + + ---------- TIMELINE of state changes and notification in Job lifecycle ---------- + + | The longest possible chain of events in shown, shorter versions cut-through intermediate states, + | while still performing all the notifications in this order. + + + Job object is created + ## NEW: state == EMPTY_NEW | is InactiveNodeList + + initParentJob / initParentJobInternal (invokes attachChild on its parent, initializes parentHandle) + ~ waits for start + >> start / join / await invoked + ## ACTIVE: state == EMPTY_ACTIVE | is JobNode | is NodeList + + onStart (lazy coroutine is started) + ~ active coroutine is working (or scheduled to execution) + >> childCancelled / cancelImpl invoked + ## CANCELLING: state is Finishing, state.rootCause != null + ------ cancelling listeners are not admitted anymore, invokeOnCompletion(onCancelling=true) returns NonDisposableHandle + ------ new children get immediately cancelled, but are still admitted to the list + + onCancelling + + notifyCancelling (invoke all cancelling listeners -- cancel all children, suspended functions resume with exception) + + cancelParent (rootCause of cancellation is communicated to the parent, parent is cancelled, too) + ~ waits for completion of coroutine body + >> makeCompleting / makeCompletingOnce invoked + ## COMPLETING: state is Finishing, state.isCompleting == true + ------ new children are not admitted anymore, attachChild returns NonDisposableHandle + ~ waits for children + >> last child completes + - computes the final exception + ## SEALED: state is Finishing, state.isSealed == true + ------ cancel/childCancelled returns false (cannot handle exceptions anymore) + + cancelParent (final exception is communicated to the parent, parent incorporates it) + + handleJobException ("launch" StandaloneCoroutine invokes CoroutineExceptionHandler) + ## COMPLETE: state !is Incomplete (CompletedExceptionally | Cancelled) + ------ completion listeners are not admitted anymore, invokeOnCompletion returns NonDisposableHandle + + parentHandle.dispose + + notifyCompletion (invoke all completion listeners) + + onCompletionInternal / onCompleted / onCancelled + + --------------------------------------------------------------------------------- + */ + + // Note: use shared objects while we have no listeners + private val _state = atomic(if (active) EMPTY_ACTIVE else EMPTY_NEW) + + private val _parentHandle = atomic(null) + internal var parentHandle: ChildHandle? + get() = _parentHandle.value + set(value) { _parentHandle.value = value } + + override val parent: Job? + get() = parentHandle?.parent + + // ------------ initialization ------------ + + /** + * Initializes parent job. + * It shall be invoked at most once after construction after all other initialization. + */ + protected fun initParentJob(parent: Job?) { + assert { parentHandle == null } + if (parent == null) { + parentHandle = NonDisposableHandle + return + } + parent.start() // make sure the parent is started + val handle = parent.attachChild(this) + parentHandle = handle + // now check our state _after_ registering (see tryFinalizeSimpleState order of actions) + if (isCompleted) { + handle.dispose() + parentHandle = NonDisposableHandle // release it just in case, to aid GC + } + } + + // ------------ state query ------------ + /** + * Returns current state of this job. + * If final state of the job is [Incomplete], then it is boxed into [IncompleteStateBox] + * and should be [unboxed][unboxState] before returning to user code. + */ + internal val state: Any? get() = _state.value + + /** + * @suppress **This is unstable API and it is subject to change.** + */ + private inline fun loopOnState(block: (Any?) -> Unit): Nothing { + while (true) { + block(state) + } + } + + public override val isActive: Boolean get() { + val state = this.state + return state is Incomplete && state.isActive + } + + public final override val isCompleted: Boolean get() = state !is Incomplete + + public final override val isCancelled: Boolean get() { + val state = this.state + return state is CompletedExceptionally || (state is Finishing && state.isCancelling) + } + + // ------------ state update ------------ + + // Finalizes Finishing -> Completed (terminal state) transition. + // ## IMPORTANT INVARIANT: Only one thread can be concurrently invoking this method. + // Returns final state that was created and updated to + private fun finalizeFinishingState(state: Finishing, proposedUpdate: Any?): Any? { + /* + * Note: proposed state can be Incomplete, e.g. + * async { + * something.invokeOnCompletion {} // <- returns handle which implements Incomplete under the hood + * } + */ + assert { this.state === state } // consistency check -- it cannot change + assert { !state.isSealed } // consistency check -- cannot be sealed yet + assert { state.isCompleting } // consistency check -- must be marked as completing + val proposedException = (proposedUpdate as? CompletedExceptionally)?.cause + // Create the final exception and seal the state so that no more exceptions can be added + val wasCancelling: Boolean + val finalException = synchronized(state) { + wasCancelling = state.isCancelling + val exceptions = state.sealLocked(proposedException) + val finalCause = getFinalRootCause(state, exceptions) + if (finalCause != null) addSuppressedExceptions(finalCause, exceptions) + finalCause + } + // Create the final state object + val finalState = when { + // was not cancelled (no exception) -> use proposed update value + finalException == null -> proposedUpdate + // small optimization when we can used proposeUpdate object as is on cancellation + finalException === proposedException -> proposedUpdate + // cancelled job final state + else -> CompletedExceptionally(finalException) + } + // Now handle the final exception + if (finalException != null) { + val handled = cancelParent(finalException) || handleJobException(finalException) + if (handled) (finalState as CompletedExceptionally).makeHandled() + } + // Process state updates for the final state before the state of the Job is actually set to the final state + // to avoid races where outside observer may see the job in the final state, yet exception is not handled yet. + if (!wasCancelling) onCancelling(finalException) + onCompletionInternal(finalState) + // Then CAS to completed state -> it must succeed + val casSuccess = _state.compareAndSet(state, finalState.boxIncomplete()) + assert { casSuccess } + // And process all post-completion actions + completeStateFinalization(state, finalState) + return finalState + } + + private fun getFinalRootCause(state: Finishing, exceptions: List): Throwable? { + // A case of no exceptions + if (exceptions.isEmpty()) { + // materialize cancellation exception if it was not materialized yet + if (state.isCancelling) return defaultCancellationException() + return null + } + /* + * 1) If we have non-CE, use it as root cause + * 2) If our original cause was TCE, use *non-original* TCE because of the special nature of TCE + * - It is a CE, so it's not reported by children + * - The first instance (cancellation cause) is created by timeout coroutine and has no meaningful stacktrace + * - The potential second instance is thrown by withTimeout lexical block itself, then it has recovered stacktrace + * 3) Just return the very first CE + */ + val firstNonCancellation = exceptions.firstOrNull { it !is CancellationException } + if (firstNonCancellation != null) return firstNonCancellation + val first = exceptions[0] + if (first is TimeoutCancellationException) { + val detailedTimeoutException = exceptions.firstOrNull { it !== first && it is TimeoutCancellationException } + if (detailedTimeoutException != null) return detailedTimeoutException + } + return first + } + + private fun addSuppressedExceptions(rootCause: Throwable, exceptions: List) { + if (exceptions.size <= 1) return // nothing more to do here + val seenExceptions = identitySet(exceptions.size) + /* + * Note that root cause may be a recovered exception as well. + * To avoid cycles we unwrap the root cause and check for self-suppression against unwrapped cause, + * but add suppressed exceptions to the recovered root cause (as it is our final exception) + */ + val unwrappedCause = unwrap(rootCause) + for (exception in exceptions) { + val unwrapped = unwrap(exception) + if (unwrapped !== rootCause && unwrapped !== unwrappedCause && + unwrapped !is CancellationException && seenExceptions.add(unwrapped)) { + rootCause.addSuppressed(unwrapped) + } + } + } + + // fast-path method to finalize normally completed coroutines without children + // returns true if complete, and afterCompletion(update) shall be called + private fun tryFinalizeSimpleState(state: Incomplete, update: Any?): Boolean { + assert { state is Empty || state is JobNode } // only simple state without lists where children can concurrently add + assert { update !is CompletedExceptionally } // only for normal completion + if (!_state.compareAndSet(state, update.boxIncomplete())) return false + onCancelling(null) // simple state is not a failure + onCompletionInternal(update) + completeStateFinalization(state, update) + return true + } + + // suppressed == true when any exceptions were suppressed while building the final completion cause + private fun completeStateFinalization(state: Incomplete, update: Any?) { + /* + * Now the job in THE FINAL state. We need to properly handle the resulting state. + * Order of various invocations here is important. + * + * 1) Unregister from parent job. + */ + parentHandle?.let { + it.dispose() // volatile read parentHandle _after_ state was updated + parentHandle = NonDisposableHandle // release it just in case, to aid GC + } + val cause = (update as? CompletedExceptionally)?.cause + /* + * 2) Invoke completion handlers: .join(), callbacks etc. + * It's important to invoke them only AFTER exception handling and everything else, see #208 + */ + if (state is JobNode) { // SINGLE/SINGLE+ state -- one completion handler (common case) + try { + state.invoke(cause) + } catch (ex: Throwable) { + handleOnCompletionException(CompletionHandlerException("Exception in completion handler $state for $this", ex)) + } + } else { + state.list?.notifyCompletion(cause) + } + } + + private fun notifyCancelling(list: NodeList, cause: Throwable) { + // first cancel our own children + onCancelling(cause) + list.close(LIST_CANCELLATION_PERMISSION) + notifyHandlers(list, cause) { it.onCancelling } + // then cancel parent + cancelParent(cause) // tentative cancellation -- does not matter if there is no parent + } + + /** + * The method that is invoked when the job is cancelled to possibly propagate cancellation to the parent. + * Returns `true` if the parent is responsible for handling the exception, `false` otherwise. + * + * Invariant: never returns `false` for instances of [CancellationException], otherwise such exception + * may leak to the [CoroutineExceptionHandler]. + */ + private fun cancelParent(cause: Throwable): Boolean { + // Is scoped coroutine -- don't propagate, will be rethrown + if (isScopedCoroutine) return true + + /* CancellationException is considered "normal" and parent usually is not cancelled when child produces it. + * This allow parent to cancel its children (normally) without being cancelled itself, unless + * child crashes and produce some other exception during its completion. + */ + val isCancellation = cause is CancellationException + val parent = parentHandle + // No parent -- ignore CE, report other exceptions. + if (parent === null || parent === NonDisposableHandle) { + return isCancellation + } + + // Notify parent but don't forget to check cancellation + return parent.childCancelled(cause) || isCancellation + } + + private fun NodeList.notifyCompletion(cause: Throwable?) { + close(LIST_ON_COMPLETION_PERMISSION) + notifyHandlers(this, cause) { true } + } + + private inline fun notifyHandlers(list: NodeList, cause: Throwable?, predicate: (JobNode) -> Boolean) { + var exception: Throwable? = null + list.forEach { node -> + if (node is JobNode && predicate(node)) { + try { + node.invoke(cause) + } catch (ex: Throwable) { + exception?.apply { addSuppressed(ex) } ?: run { + exception = CompletionHandlerException("Exception in completion handler $node for $this", ex) + } + } + } + } + exception?.let { handleOnCompletionException(it) } + } + + public final override fun start(): Boolean { + loopOnState { state -> + when (startInternal(state)) { + FALSE -> return false + TRUE -> return true + } + } + } + + // returns: RETRY/FALSE/TRUE: + // FALSE when not new, + // TRUE when started + // RETRY when need to retry + private fun startInternal(state: Any?): Int { + when (state) { + is Empty -> { // EMPTY_X state -- no completion handlers + if (state.isActive) return FALSE // already active + if (!_state.compareAndSet(state, EMPTY_ACTIVE)) return RETRY + onStart() + return TRUE + } + is InactiveNodeList -> { // LIST state -- inactive with a list of completion handlers + if (!_state.compareAndSet(state, state.list)) return RETRY + onStart() + return TRUE + } + else -> return FALSE // not a new state + } + } + + /** + * Override to provide the actual [start] action. + * This function is invoked exactly once when non-active coroutine is [started][start]. + */ + protected open fun onStart() {} + + public final override fun getCancellationException(): CancellationException = + when (val state = this.state) { + is Finishing -> state.rootCause?.toCancellationException("$classSimpleName is cancelling") + ?: error("Job is still new or active: $this") + is Incomplete -> error("Job is still new or active: $this") + is CompletedExceptionally -> state.cause.toCancellationException() + else -> JobCancellationException("$classSimpleName has completed normally", null, this) + } + + protected fun Throwable.toCancellationException(message: String? = null): CancellationException = + this as? CancellationException ?: defaultCancellationException(message, this) + + /** + * Returns the cause that signals the completion of this job -- it returns the original + * [cancel] cause, [CancellationException] or **`null` if this job had completed normally**. + * This function throws [IllegalStateException] when invoked for an job that has not [completed][isCompleted] nor + * is being cancelled yet. + */ + protected val completionCause: Throwable? + get() = when (val state = state) { + is Finishing -> state.rootCause + ?: error("Job is still new or active: $this") + is Incomplete -> error("Job is still new or active: $this") + is CompletedExceptionally -> state.cause + else -> null + } + + /** + * Returns `true` when [completionCause] exception was handled by parent coroutine. + */ + protected val completionCauseHandled: Boolean + get() = state.let { it is CompletedExceptionally && it.handled } + + public final override fun invokeOnCompletion(handler: CompletionHandler): DisposableHandle = + invokeOnCompletionInternal( + invokeImmediately = true, + node = InvokeOnCompletion(handler), + ) + + public final override fun invokeOnCompletion(onCancelling: Boolean, invokeImmediately: Boolean, handler: CompletionHandler): DisposableHandle = + invokeOnCompletionInternal( + invokeImmediately = invokeImmediately, + node = if (onCancelling) { + InvokeOnCancelling(handler) + } else { + InvokeOnCompletion(handler) + } + ) + + internal fun invokeOnCompletionInternal( + invokeImmediately: Boolean, + node: JobNode + ): DisposableHandle { + node.job = this + // Create node upfront -- for common cases it just initializes JobNode.job field, + // for user-defined handlers it allocates a JobNode object that we might not need, but this is Ok. + val added = tryPutNodeIntoList(node) { state, list -> + if (node.onCancelling) { + /** + * We are querying whether the job was already cancelled when we entered this block. + * We can't naively attempt to add the node to the list, because a lot of time could pass between + * notifying the cancellation handlers (and thus closing the list, forcing us to retry) + * and reaching a final state. + * + * Alternatively, we could also try to add the node to the list first and then read the latest state + * to check for an exception, but that logic would need to manually handle the final state, which is + * less straightforward. + */ + val rootCause = (state as? Finishing)?.rootCause + if (rootCause == null) { + /** + * There is no known root cause yet, so we can add the node to the list of state handlers. + * + * If this call fails, because of the bitmask, this means one of the two happened: + * - [notifyCancelling] was already called. + * This means that the job is already being cancelled: otherwise, with what exception would we + * notify the handler? + * So, we can retry the operation: either the state is already final, or the `rootCause` check + * above will give a different result. + * - [notifyCompletion] was already called. + * This means that the job is already complete. + * We can retry the operation and will observe the final state. + */ + list.addLast(node, LIST_CANCELLATION_PERMISSION or LIST_ON_COMPLETION_PERMISSION) + } else { + /** + * The root cause is known, so we can invoke the handler immediately and avoid adding it. + */ + if (invokeImmediately) node.invoke(rootCause) + return NonDisposableHandle + } + } else { + /** + * The non-[onCancelling]-handlers are interested in completions only, so it's safe to add them at + * any time before [notifyCompletion] is called (which closes the list). + * + * If the list *is* closed, on a retry, we'll observe the final state, as [notifyCompletion] is only + * called after the state transition. + */ + list.addLast(node, LIST_ON_COMPLETION_PERMISSION) + } + } + when { + added -> return node + invokeImmediately -> node.invoke((state as? CompletedExceptionally)?.cause) + } + return NonDisposableHandle + } + + /** + * Puts [node] into the current state's list of completion handlers. + * + * Returns `false` if the state is already complete and doesn't accept new handlers. + * Returns `true` if the handler was successfully added to the list. + * + * [tryAdd] is invoked when the state is [Incomplete] and the list is not `null`, to decide on the specific + * behavior in this case. It must return + * - `true` if the element was successfully added to the list + * - `false` if the operation needs to be retried + */ + private inline fun tryPutNodeIntoList( + node: JobNode, + tryAdd: (Incomplete, NodeList) -> Boolean + ): Boolean { + loopOnState { state -> + when (state) { + is Empty -> { // EMPTY_X state -- no completion handlers + if (state.isActive) { + // try to move to the SINGLE state + if (_state.compareAndSet(state, node)) return true + } else + promoteEmptyToNodeList(state) // that way we can add listener for non-active coroutine + } + is Incomplete -> when (val list = state.list) { + null -> promoteSingleToNodeList(state as JobNode) + else -> if (tryAdd(state, list)) return true + } + else -> return false + } + } + } + + private fun promoteEmptyToNodeList(state: Empty) { + // try to promote it to LIST state with the corresponding state + val list = NodeList() + val update = if (state.isActive) list else InactiveNodeList(list) + _state.compareAndSet(state, update) + } + + private fun promoteSingleToNodeList(state: JobNode) { + // try to promote it to list (SINGLE+ state) + state.addOneIfEmpty(NodeList()) + // it must be in SINGLE+ state or state has changed (node could have need removed from state) + val list = state.nextNode // either our NodeList or somebody else won the race, updated state + // just attempt converting it to list if state is still the same, then we'll continue lock-free loop + _state.compareAndSet(state, list) + } + + public final override suspend fun join() { + if (!joinInternal()) { // fast-path no wait + coroutineContext.ensureActive() + return // do not suspend + } + return joinSuspend() // slow-path wait + } + + private fun joinInternal(): Boolean { + loopOnState { state -> + if (state !is Incomplete) return false // not active anymore (complete) -- no need to wait + if (startInternal(state) >= 0) return true // wait unless need to retry + } + } + + private suspend fun joinSuspend() = suspendCancellableCoroutine { cont -> + // We have to invoke join() handler only on cancellation, on completion we will be resumed regularly without handlers + cont.disposeOnCancellation(invokeOnCompletion(handler = ResumeOnCompletion(cont))) + } + + @Suppress("UNCHECKED_CAST") + public final override val onJoin: SelectClause0 + get() = SelectClause0Impl( + clauseObject = this@JobSupport, + regFunc = JobSupport::registerSelectForOnJoin as RegistrationFunction + ) + + @Suppress("UNUSED_PARAMETER") + private fun registerSelectForOnJoin(select: SelectInstance<*>, ignoredParam: Any?) { + if (!joinInternal()) { + select.selectInRegistrationPhase(Unit) + return + } + val disposableHandle = invokeOnCompletion(handler = SelectOnJoinCompletionHandler(select)) + select.disposeOnCompletion(disposableHandle) + } + + private inner class SelectOnJoinCompletionHandler( + private val select: SelectInstance<*> + ) : JobNode() { + override val onCancelling: Boolean get() = false + override fun invoke(cause: Throwable?) { + select.trySelect(this@JobSupport, Unit) + } + } + + /** + * @suppress **This is unstable API and it is subject to change.** + */ + internal fun removeNode(node: JobNode) { + // remove logic depends on the state of the job + loopOnState { state -> + when (state) { + is JobNode -> { // SINGE/SINGLE+ state -- one completion handler + if (state !== node) return // a different job node --> we were already removed + // try remove and revert back to empty state + if (_state.compareAndSet(state, EMPTY_ACTIVE)) return + } + is Incomplete -> { // may have a list of completion handlers + // remove node from the list if there is a list + if (state.list != null) node.remove() + return + } + else -> return // it is complete and does not have any completion handlers + } + } + } + + /** + * Returns `true` for job that do not have "body block" to complete and should immediately go into + * completing state and start waiting for children. + * + * @suppress **This is unstable API and it is subject to change.** + */ + internal open val onCancelComplete: Boolean get() = false + + // external cancel with cause, never invoked implicitly from internal machinery + public override fun cancel(cause: CancellationException?) { + cancelInternal(cause ?: defaultCancellationException()) + } + + protected open fun cancellationExceptionMessage(): String = "Job was cancelled" + + // HIDDEN in Job interface. Invoked only by legacy compiled code. + // external cancel with (optional) cause, never invoked implicitly from internal machinery + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Added since 1.2.0 for binary compatibility with versions <= 1.1.x") + public override fun cancel(cause: Throwable?): Boolean { + cancelInternal(cause?.toCancellationException() ?: defaultCancellationException()) + return true + } + + // It is overridden in channel-linked implementation + public open fun cancelInternal(cause: Throwable) { + cancelImpl(cause) + } + + // Parent is cancelling child + public final override fun parentCancelled(parentJob: ParentJob) { + cancelImpl(parentJob) + } + + /** + * Child was cancelled with a cause. + * In this method parent decides whether it cancels itself (e.g. on a critical failure) and whether it handles the exception of the child. + * It is overridden in supervisor implementations to completely ignore any child cancellation. + * Returns `true` if exception is handled, `false` otherwise (then caller is responsible for handling an exception) + * + * Invariant: never returns `false` for instances of [CancellationException], otherwise such exception + * may leak to the [CoroutineExceptionHandler]. + */ + public open fun childCancelled(cause: Throwable): Boolean { + if (cause is CancellationException) return true + return cancelImpl(cause) && handlesException + } + + /** + * Makes this [Job] cancelled with a specified [cause]. + * It is used in [AbstractCoroutine]-derived classes when there is an internal failure. + */ + public fun cancelCoroutine(cause: Throwable?): Boolean = cancelImpl(cause) + + // cause is Throwable or ParentJob when cancelChild was invoked + // returns true is exception was handled, false otherwise + internal fun cancelImpl(cause: Any?): Boolean { + var finalState: Any? = COMPLETING_ALREADY + if (onCancelComplete) { + // make sure it is completing, if cancelMakeCompleting returns state it means it had make it + // completing and had recorded exception + finalState = cancelMakeCompleting(cause) + if (finalState === COMPLETING_WAITING_CHILDREN) return true + } + if (finalState === COMPLETING_ALREADY) { + finalState = makeCancelling(cause) + } + return when { + finalState === COMPLETING_ALREADY -> true + finalState === COMPLETING_WAITING_CHILDREN -> true + finalState === TOO_LATE_TO_CANCEL -> false + else -> { + afterCompletion(finalState) + true + } + } + } + + // cause is Throwable or ParentJob when cancelChild was invoked + // It contains a loop and never returns COMPLETING_RETRY, can return + // COMPLETING_ALREADY -- if already completed/completing + // COMPLETING_WAITING_CHILDREN -- if started waiting for children + // final state -- when completed, for call to afterCompletion + private fun cancelMakeCompleting(cause: Any?): Any? { + loopOnState { state -> + if (state !is Incomplete || state is Finishing && state.isCompleting) { + // already completed/completing, do not even create exception to propose update + return COMPLETING_ALREADY + } + val proposedUpdate = CompletedExceptionally(createCauseException(cause)) + val finalState = tryMakeCompleting(state, proposedUpdate) + if (finalState !== COMPLETING_RETRY) return finalState + } + } + + @Suppress("NOTHING_TO_INLINE") // Save a stack frame + internal inline fun defaultCancellationException(message: String? = null, cause: Throwable? = null) = + JobCancellationException(message ?: cancellationExceptionMessage(), cause, this) + + override fun getChildJobCancellationCause(): CancellationException { + // determine root cancellation cause of this job (why is it cancelling its children?) + val state = this.state + val rootCause = when (state) { + is Finishing -> state.rootCause + is CompletedExceptionally -> state.cause + is Incomplete -> error("Cannot be cancelling child in this state: $state") + else -> null // create exception with the below code on normal completion + } + return (rootCause as? CancellationException) ?: JobCancellationException("Parent job is ${stateString(state)}", rootCause, this) + } + + // cause is Throwable or ParentJob when cancelChild was invoked + private fun createCauseException(cause: Any?): Throwable = when (cause) { + is Throwable? -> cause ?: defaultCancellationException() + else -> (cause as ParentJob).getChildJobCancellationCause() + } + + // transitions to Cancelling state + // cause is Throwable or ParentJob when cancelChild was invoked + // It contains a loop and never returns COMPLETING_RETRY, can return + // COMPLETING_ALREADY -- if already completing or successfully made cancelling, added exception + // COMPLETING_WAITING_CHILDREN -- if started waiting for children, added exception + // TOO_LATE_TO_CANCEL -- too late to cancel, did not add exception + // final state -- when completed, for call to afterCompletion + private fun makeCancelling(cause: Any?): Any? { + var causeExceptionCache: Throwable? = null // lazily init result of createCauseException(cause) + loopOnState { state -> + when (state) { + is Finishing -> { // already finishing -- collect exceptions + val notifyRootCause = synchronized(state) { + if (state.isSealed) return TOO_LATE_TO_CANCEL // already sealed -- cannot add exception nor mark cancelled + // add exception, do nothing is parent is cancelling child that is already being cancelled + val wasCancelling = state.isCancelling // will notify if was not cancelling + // Materialize missing exception if it is the first exception (otherwise -- don't) + if (cause != null || !wasCancelling) { + val causeException = causeExceptionCache ?: createCauseException(cause).also { causeExceptionCache = it } + state.addExceptionLocked(causeException) + } + // take cause for notification if was not in cancelling state before + state.rootCause.takeIf { !wasCancelling } + } + notifyRootCause?.let { notifyCancelling(state.list, it) } + return COMPLETING_ALREADY + } + is Incomplete -> { + // Not yet finishing -- try to make it cancelling + val causeException = causeExceptionCache ?: createCauseException(cause).also { causeExceptionCache = it } + if (state.isActive) { + // active state becomes cancelling + if (tryMakeCancelling(state, causeException)) return COMPLETING_ALREADY + } else { + // non active state starts completing + val finalState = tryMakeCompleting(state, CompletedExceptionally(causeException)) + when { + finalState === COMPLETING_ALREADY -> error("Cannot happen in $state") + finalState === COMPLETING_RETRY -> return@loopOnState + else -> return finalState + } + } + } + else -> return TOO_LATE_TO_CANCEL // already complete + } + } + } + + // Performs promotion of incomplete coroutine state to NodeList for the purpose of + // converting coroutine state to Cancelling, returns null when need to retry + private fun getOrPromoteCancellingList(state: Incomplete): NodeList? = state.list ?: + when (state) { + is Empty -> NodeList() // we can allocate new empty list that'll get integrated into Cancelling state + is JobNode -> { + // SINGLE/SINGLE+ must be promoted to NodeList first, because otherwise we cannot + // correctly capture a reference to it + promoteSingleToNodeList(state) + null // retry + } + else -> error("State should have list: $state") + } + + // try make new Cancelling state on the condition that we're still in the expected state + private fun tryMakeCancelling(state: Incomplete, rootCause: Throwable): Boolean { + assert { state !is Finishing } // only for non-finishing states + assert { state.isActive } // only for active states + // get state's list or else promote to list to correctly operate on child lists + val list = getOrPromoteCancellingList(state) ?: return false + // Create cancelling state (with rootCause!) + val cancelling = Finishing(list, false, rootCause) + if (!_state.compareAndSet(state, cancelling)) return false + // Notify listeners + notifyCancelling(list, rootCause) + return true + } + + /** + * Completes this job. Used by [CompletableDeferred.complete] (and exceptionally) + * and by [JobImpl.cancel]. It returns `false` on repeated invocation + * (when this job is already completing). + */ + internal fun makeCompleting(proposedUpdate: Any?): Boolean { + loopOnState { state -> + val finalState = tryMakeCompleting(state, proposedUpdate) + when { + finalState === COMPLETING_ALREADY -> return false + finalState === COMPLETING_WAITING_CHILDREN -> return true + finalState === COMPLETING_RETRY -> return@loopOnState + else -> { + afterCompletion(finalState) + return true + } + } + } + } + + /** + * Completes this job. Used by [AbstractCoroutine.resume]. + * It throws [IllegalStateException] on repeated invocation (when this job is already completing). + * Returns: + * - [COMPLETING_WAITING_CHILDREN] if started waiting for children. + * - Final state otherwise (caller should do [afterCompletion]) + */ + internal fun makeCompletingOnce(proposedUpdate: Any?): Any? { + loopOnState { state -> + val finalState = tryMakeCompleting(state, proposedUpdate) + when { + finalState === COMPLETING_ALREADY -> + throw IllegalStateException( + "Job $this is already complete or completing, " + + "but is being completed with $proposedUpdate", proposedUpdate.exceptionOrNull + ) + finalState === COMPLETING_RETRY -> return@loopOnState + else -> return finalState // COMPLETING_WAITING_CHILDREN or final state + } + } + } + + // Returns one of COMPLETING symbols or final state: + // COMPLETING_ALREADY -- when already complete or completing + // COMPLETING_RETRY -- when need to retry due to interference + // COMPLETING_WAITING_CHILDREN -- when made completing and is waiting for children + // final state -- when completed, for call to afterCompletion + private fun tryMakeCompleting(state: Any?, proposedUpdate: Any?): Any? { + if (state !is Incomplete) + return COMPLETING_ALREADY + /* + * FAST PATH -- no children to wait for && simple state (no list) && not cancelling => can complete immediately + * Cancellation (failures) always have to go through Finishing state to serialize exception handling. + * Otherwise, there can be a race between (completed state -> handled exception and newly attached child/join) + * which may miss unhandled exception. + */ + if ((state is Empty || state is JobNode) && state !is ChildHandleNode && proposedUpdate !is CompletedExceptionally) { + if (tryFinalizeSimpleState(state, proposedUpdate)) { + // Completed successfully on fast path -- return updated state + return proposedUpdate + } + return COMPLETING_RETRY + } + // The separate slow-path function to simplify profiling + return tryMakeCompletingSlowPath(state, proposedUpdate) + } + + // Returns one of COMPLETING symbols or final state: + // COMPLETING_ALREADY -- when already complete or completing + // COMPLETING_RETRY -- when need to retry due to interference + // COMPLETING_WAITING_CHILDREN -- when made completing and is waiting for children + // final state -- when completed, for call to afterCompletion + private fun tryMakeCompletingSlowPath(state: Incomplete, proposedUpdate: Any?): Any? { + // get state's list or else promote to list to correctly operate on child lists + val list = getOrPromoteCancellingList(state) ?: return COMPLETING_RETRY + // promote to Finishing state if we are not in it yet + // This promotion has to be atomic w.r.t to state change, so that a coroutine that is not active yet + // atomically transition to finishing & completing state + val finishing = state as? Finishing ?: Finishing(list, false, null) + // must synchronize updates to finishing state + val notifyRootCause: Throwable? + synchronized(finishing) { + // check if this state is already completing + if (finishing.isCompleting) return COMPLETING_ALREADY + // mark as completing + finishing.isCompleting = true + // if we need to promote to finishing, then atomically do it here. + // We do it as early is possible while still holding the lock. This ensures that we cancelImpl asap + // (if somebody else is faster) and we synchronize all the threads on this finishing lock asap. + if (finishing !== state) { + if (!_state.compareAndSet(state, finishing)) return COMPLETING_RETRY + } + // ## IMPORTANT INVARIANT: Only one thread (that had set isCompleting) can go past this point + assert { !finishing.isSealed } // cannot be sealed + // add new proposed exception to the finishing state + val wasCancelling = finishing.isCancelling + (proposedUpdate as? CompletedExceptionally)?.let { finishing.addExceptionLocked(it.cause) } + // If it just becomes cancelling --> must process cancelling notifications + notifyRootCause = finishing.rootCause.takeIf { !wasCancelling } + } + // process cancelling notification here -- it cancels all the children _before_ we start to wait them (sic!!!) + notifyRootCause?.let { notifyCancelling(list, it) } + // now wait for children + // we can't close the list yet: while there are active children, adding new ones is still allowed. + val child = list.nextChild() + if (child != null && tryWaitForChild(finishing, child, proposedUpdate)) + return COMPLETING_WAITING_CHILDREN + // turns out, there are no children to await, so we close the list. + list.close(LIST_CHILD_PERMISSION) + // some children could have sneaked into the list, so we try waiting for them again. + // it would be more correct to re-open the list (otherwise, we get non-linearizable behavior), + // but it's too difficult with the current lock-free list implementation. + val anotherChild = list.nextChild() + if (anotherChild != null && tryWaitForChild(finishing, anotherChild, proposedUpdate)) + return COMPLETING_WAITING_CHILDREN + // otherwise -- we have not children left (all were already cancelled?) + return finalizeFinishingState(finishing, proposedUpdate) + } + + private val Any?.exceptionOrNull: Throwable? + get() = (this as? CompletedExceptionally)?.cause + + // return false when there is no more incomplete children to wait + // ## IMPORTANT INVARIANT: Only one thread can be concurrently invoking this method. + private tailrec fun tryWaitForChild(state: Finishing, child: ChildHandleNode, proposedUpdate: Any?): Boolean { + val handle = child.childJob.invokeOnCompletion( + invokeImmediately = false, + handler = ChildCompletion(this, state, child, proposedUpdate) + ) + if (handle !== NonDisposableHandle) return true // child is not complete and we've started waiting for it + val nextChild = child.nextChild() ?: return false + return tryWaitForChild(state, nextChild, proposedUpdate) + } + + // ## IMPORTANT INVARIANT: Only one thread can be concurrently invoking this method. + private fun continueCompleting(state: Finishing, lastChild: ChildHandleNode, proposedUpdate: Any?) { + assert { this.state === state } // consistency check -- it cannot change while we are waiting for children + // figure out if we need to wait for the next child + val waitChild = lastChild.nextChild() + // try to wait for the next child + if (waitChild != null && tryWaitForChild(state, waitChild, proposedUpdate)) return // waiting for next child + // no more children to await, so *maybe* we can complete the job; for that, we stop accepting new children. + // potentially, the list can be closed for children more than once: if we detect that there are no more + // children, attempt to close the list, and then new children sneak in, this whole logic will be + // repeated, including closing the list. + state.list.close(LIST_CHILD_PERMISSION) + // did any new children sneak in? + val waitChildAgain = lastChild.nextChild() + if (waitChildAgain != null && tryWaitForChild(state, waitChildAgain, proposedUpdate)) { + // yes, so now we have to wait for them! + // ideally, we should re-open the list, + // but it's too difficult with the current lock-free list implementation, + // so we'll live with non-linearizable behavior for now. + return + } + // no more children, now we are sure; try to update the state + val finalState = finalizeFinishingState(state, proposedUpdate) + afterCompletion(finalState) + } + + private fun LockFreeLinkedListNode.nextChild(): ChildHandleNode? { + var cur = this + while (cur.isRemoved) cur = cur.prevNode // rollback to prev non-removed (or list head) + while (true) { + cur = cur.nextNode + if (cur.isRemoved) continue + if (cur is ChildHandleNode) return cur + if (cur is NodeList) return null // checked all -- no more children + } + } + + public final override val children: Sequence get() = sequence { + when (val state = this@JobSupport.state) { + is ChildHandleNode -> yield(state.childJob) + is Incomplete -> state.list?.let { list -> + list.forEach { if (it is ChildHandleNode) yield(it.childJob) } + } + } + } + + @Suppress("OverridingDeprecatedMember") + public final override fun attachChild(child: ChildJob): ChildHandle { + /* + * Note: This function attaches a special ChildHandleNode node object. This node object + * is handled in a special way on completion on the coroutine (we wait for all of them) and also + * can't be added simply with `invokeOnCompletionInternal` -- we add this node to the list even + * if the job is already cancelling. + * It's required to properly await all children before completion and provide a linearizable hierarchy view: + * If the child is attached when the job is already being cancelled, such a child will receive + * an immediate notification on cancellation, + * but the parent *will* wait for that child before completion and will handle its exception. + */ + val node = ChildHandleNode(child).also { it.job = this } + val added = tryPutNodeIntoList(node) { _, list -> + // First, try to add a child along the cancellation handlers + val addedBeforeCancellation = list.addLast( + node, + LIST_ON_COMPLETION_PERMISSION or LIST_CHILD_PERMISSION or LIST_CANCELLATION_PERMISSION + ) + if (addedBeforeCancellation) { + // The child managed to be added before the parent started to cancel or complete. Success. + true + } else { + /* Either cancellation or completion already happened, the child was not added. + * Now we need to try adding it just for completion. */ + val addedBeforeCompletion = list.addLast( + node, + LIST_CHILD_PERMISSION or LIST_ON_COMPLETION_PERMISSION + ) + /* + * Whether or not we managed to add the child before the parent completed, we need to investigate: + * why didn't we manage to add it before cancellation? + * If it's because cancellation happened in the meantime, we need to notify the child about it. + * We check the latest state because the original state with which we started may not have had + * the information about the cancellation yet. + */ + val rootCause = when (val latestState = this.state) { + is Finishing -> { + // The state is still incomplete, so we need to notify the child about the completion cause. + latestState.rootCause + } + else -> { + /** Since the list is already closed for [onCancelling], the job is either Finishing or + * already completed. We need to notify the child about the completion cause. */ + assert { latestState !is Incomplete } + (latestState as? CompletedExceptionally)?.cause + } + } + /** + * We must cancel the child if the parent was cancelled already, even if we successfully attached, + * as this child didn't make it before [notifyCancelling] and won't be notified that it should be + * cancelled. + * + * And if the parent wasn't cancelled and the previous [LockFreeLinkedListNode.addLast] failed because + * the job is in its final state already, we won't be able to attach anyway, so we must just invoke + * the handler and return. + */ + node.invoke(rootCause) + if (addedBeforeCompletion) { + /** The root cause can't be null: since the earlier addition to the list failed, this means that + * the job was already cancelled or completed. */ + assert { rootCause != null } + true + } else { + /** No sense in retrying: we know it won't succeed, and we already invoked the handler. */ + return NonDisposableHandle + } + } + } + if (added) return node + /** We can only end up here if [tryPutNodeIntoList] detected a final state. */ + node.invoke((state as? CompletedExceptionally)?.cause) + return NonDisposableHandle + } + + /** + * Override to process any exceptions that were encountered while invoking completion handlers + * installed via [invokeOnCompletion]. + * + * @suppress **This is unstable API and it is subject to change.** + */ + internal open fun handleOnCompletionException(exception: Throwable) { + throw exception + } + + /** + * This function is invoked once as soon as this job is being cancelled for any reason or completes, + * similarly to [invokeOnCompletion] with `onCancelling` set to `true`. + * + * The meaning of [cause] parameter: + * - Cause is `null` when the job has completed normally. + * - Cause is an instance of [CancellationException] when the job was cancelled _normally_. + * **It should not be treated as an error**. In particular, it should not be reported to error logs. + * - Otherwise, the job had been cancelled or failed with exception. + * + * The specified [cause] is not the final cancellation cause of this job. + * A job may produce other exceptions while it is failing and the final cause might be different. + * + * @suppress **This is unstable API and it is subject to change.* + */ + protected open fun onCancelling(cause: Throwable?) {} + + /** + * Returns `true` for scoped coroutines. + * Scoped coroutine is a coroutine that is executed sequentially within the enclosing scope without any concurrency. + * Scoped coroutines always handle any exception happened within -- they just rethrow it to the enclosing scope. + * Examples of scoped coroutines are `coroutineScope`, `withTimeout` and `runBlocking`. + */ + protected open val isScopedCoroutine: Boolean get() = false + + /** + * Returns `true` for jobs that handle their exceptions or integrate them into the job's result via [onCompletionInternal]. + * A valid implementation of this getter should recursively check parent as well before returning `false`. + * + * The only instance of the [Job] that does not handle its exceptions is [JobImpl] and its subclass [SupervisorJobImpl]. + * @suppress **This is unstable API and it is subject to change.* + */ + internal open val handlesException: Boolean get() = true + + /** + * Handles the final job [exception] that was not handled by the parent coroutine. + * Returns `true` if it handles exception (so handling at later stages is not needed). + * It is designed to be overridden by launch-like coroutines + * (`StandaloneCoroutine` and `ActorCoroutine`) that don't have a result type + * that can represent exceptions. + * + * This method is invoked **exactly once** when the final exception of the job is determined + * and before it becomes complete. At the moment of invocation the job and all its children are complete. + */ + protected open fun handleJobException(exception: Throwable): Boolean = false + + /** + * Override for completion actions that need to update some external object depending on job's state, + * right before all the waiters for coroutine's completion are notified. + * + * @param state the final state. + * + * @suppress **This is unstable API and it is subject to change.** + */ + protected open fun onCompletionInternal(state: Any?) {} + + /** + * Override for the very last action on job's completion to resume the rest of the code in + * scoped coroutines. It is called when this job is externally completed in an unknown + * context and thus should resume with a default mode. + * + * @suppress **This is unstable API and it is subject to change.** + */ + protected open fun afterCompletion(state: Any?) {} + + // for nicer debugging + public override fun toString(): String = + "${toDebugString()}@$hexAddress" + + @InternalCoroutinesApi + public fun toDebugString(): String = "${nameString()}{${stateString(state)}}" + + /** + * @suppress **This is unstable API and it is subject to change.** + */ + internal open fun nameString(): String = classSimpleName + + private fun stateString(state: Any?): String = when (state) { + is Finishing -> when { + state.isCancelling -> "Cancelling" + state.isCompleting -> "Completing" + else -> "Active" + } + is Incomplete -> if (state.isActive) "Active" else "New" + is CompletedExceptionally -> "Cancelled" + else -> "Completed" + } + + // Completing & Cancelling states, + // All updates are guarded by synchronized(this), reads are volatile + @Suppress("UNCHECKED_CAST") + private class Finishing( + override val list: NodeList, + isCompleting: Boolean, + rootCause: Throwable? + ) : SynchronizedObject(), Incomplete { + private val _isCompleting = atomic(isCompleting) + var isCompleting: Boolean + get() = _isCompleting.value + set(value) { _isCompleting.value = value } + + private val _rootCause = atomic(rootCause) + var rootCause: Throwable? // NOTE: rootCause is kept even when SEALED + get() = _rootCause.value + set(value) { _rootCause.value = value } + + private val _exceptionsHolder = atomic(null) + private var exceptionsHolder: Any? // Contains null | Throwable | ArrayList | SEALED + get() = _exceptionsHolder.value + set(value) { _exceptionsHolder.value = value } + + // Note: cannot be modified when sealed + val isSealed: Boolean get() = exceptionsHolder === SEALED + val isCancelling: Boolean get() = rootCause != null + override val isActive: Boolean get() = rootCause == null // !isCancelling + + // Seals current state and returns list of exceptions + // guarded by `synchronized(this)` + fun sealLocked(proposedException: Throwable?): List { + val list = when(val eh = exceptionsHolder) { // volatile read + null -> allocateList() + is Throwable -> allocateList().also { it.add(eh) } + is ArrayList<*> -> eh as ArrayList + else -> error("State is $eh") // already sealed -- cannot happen + } + val rootCause = this.rootCause // volatile read + rootCause?.let { list.add(0, it) } // note -- rootCause goes to the beginning + if (proposedException != null && proposedException != rootCause) list.add(proposedException) + exceptionsHolder = SEALED + return list + } + + // guarded by `synchronized(this)` + fun addExceptionLocked(exception: Throwable) { + val rootCause = this.rootCause // volatile read + if (rootCause == null) { + this.rootCause = exception + return + } + if (exception === rootCause) return // nothing to do + when (val eh = exceptionsHolder) { // volatile read + null -> exceptionsHolder = exception + is Throwable -> { + if (exception === eh) return // nothing to do + exceptionsHolder = allocateList().apply { + add(eh) + add(exception) + + } + } + is ArrayList<*> -> (eh as ArrayList).add(exception) + else -> error("State is $eh") // already sealed -- cannot happen + } + } + + private fun allocateList() = ArrayList(4) + + override fun toString(): String = + "Finishing[cancelling=$isCancelling, completing=$isCompleting, rootCause=$rootCause, exceptions=$exceptionsHolder, list=$list]" + } + + private val Incomplete.isCancelling: Boolean + get() = this is Finishing && isCancelling + + // Used by parent that is waiting for child completion + private class ChildCompletion( + private val parent: JobSupport, + private val state: Finishing, + private val child: ChildHandleNode, + private val proposedUpdate: Any? + ) : JobNode() { + override val onCancelling get() = false + override fun invoke(cause: Throwable?) { + parent.continueCompleting(state, child, proposedUpdate) + } + } + + private class AwaitContinuation( + delegate: Continuation, + private val job: JobSupport + ) : CancellableContinuationImpl(delegate, MODE_CANCELLABLE) { + override fun getContinuationCancellationCause(parent: Job): Throwable { + val state = job.state + /* + * When the job we are waiting for had already completely completed exceptionally or + * is failing, we shall use its root/completion cause for await's result. + */ + if (state is Finishing) state.rootCause?.let { return it } + if (state is CompletedExceptionally) return state.cause + return parent.getCancellationException() + } + + override fun nameString(): String = + "AwaitContinuation" + } + + /* + * ================================================================================================= + * This is ready-to-use implementation for Deferred interface. + * However, it is not type-safe. Conceptually it just exposes the value of the underlying + * completed state as `Any?` + * ================================================================================================= + */ + + public val isCompletedExceptionally: Boolean get() = state is CompletedExceptionally + + public fun getCompletionExceptionOrNull(): Throwable? { + val state = this.state + check(state !is Incomplete) { "This job has not completed yet" } + return state.exceptionOrNull + } + + /** + * @suppress **This is unstable API and it is subject to change.** + */ + internal fun getCompletedInternal(): Any? { + val state = this.state + check(state !is Incomplete) { "This job has not completed yet" } + if (state is CompletedExceptionally) throw state.cause + return state.unboxState() + } + + /** + * @suppress **This is unstable API and it is subject to change.** + */ + protected suspend fun awaitInternal(): Any? { + // fast-path -- check state (avoid extra object creation) + while (true) { // lock-free loop on state + val state = this.state + if (state !is Incomplete) { + // already complete -- just return result + if (state is CompletedExceptionally) { // Slow path to recover stacktrace + recoverAndThrow(state.cause) + } + return state.unboxState() + + } + if (startInternal(state) >= 0) break // break unless needs to retry + } + return awaitSuspend() // slow-path + } + + private suspend fun awaitSuspend(): Any? = suspendCoroutineUninterceptedOrReturn { uCont -> + /* + * Custom code here, so that parent coroutine that is using await + * on its child deferred (async) coroutine would throw the exception that this child had + * thrown and not a JobCancellationException. + */ + val cont = AwaitContinuation(uCont.intercepted(), this) + // we are mimicking suspendCancellableCoroutine here and call initCancellability, too. + cont.initCancellability() + cont.disposeOnCancellation(invokeOnCompletion(handler = ResumeAwaitOnCompletion(cont))) + cont.getResult() + } + + @Suppress("UNCHECKED_CAST") + protected val onAwaitInternal: SelectClause1<*> get() = SelectClause1Impl( + clauseObject = this@JobSupport, + regFunc = JobSupport::onAwaitInternalRegFunc as RegistrationFunction, + processResFunc = JobSupport::onAwaitInternalProcessResFunc as ProcessResultFunction + ) + + @Suppress("UNUSED_PARAMETER") + private fun onAwaitInternalRegFunc(select: SelectInstance<*>, ignoredParam: Any?) { + while (true) { + val state = this.state + if (state !is Incomplete) { + val result = if (state is CompletedExceptionally) state else state.unboxState() + select.selectInRegistrationPhase(result) + return + } + if (startInternal(state) >= 0) break // break unless needs to retry + } + val disposableHandle = invokeOnCompletion(handler = SelectOnAwaitCompletionHandler(select)) + select.disposeOnCompletion(disposableHandle) + } + + @Suppress("UNUSED_PARAMETER") + private fun onAwaitInternalProcessResFunc(ignoredParam: Any?, result: Any?): Any? { + if (result is CompletedExceptionally) throw result.cause + return result + } + + private inner class SelectOnAwaitCompletionHandler( + private val select: SelectInstance<*> + ) : JobNode() { + override val onCancelling get() = false + override fun invoke(cause: Throwable?) { + val state = this@JobSupport.state + val result = if (state is CompletedExceptionally) state else state.unboxState() + select.trySelect(this@JobSupport, result) + } + } +} + +/* + * Class to represent object as the final state of the Job + */ +private class IncompleteStateBox(@JvmField val state: Incomplete) +internal fun Any?.boxIncomplete(): Any? = if (this is Incomplete) IncompleteStateBox(this) else this +internal fun Any?.unboxState(): Any? = (this as? IncompleteStateBox)?.state ?: this + +// --------------- helper classes & constants for job implementation + +private val COMPLETING_ALREADY = Symbol("COMPLETING_ALREADY") +@JvmField +internal val COMPLETING_WAITING_CHILDREN = Symbol("COMPLETING_WAITING_CHILDREN") +private val COMPLETING_RETRY = Symbol("COMPLETING_RETRY") +private val TOO_LATE_TO_CANCEL = Symbol("TOO_LATE_TO_CANCEL") + +private const val RETRY = -1 +private const val FALSE = 0 +private const val TRUE = 1 + +private val SEALED = Symbol("SEALED") +private val EMPTY_NEW = Empty(false) +private val EMPTY_ACTIVE = Empty(true) + +// bit mask +private const val LIST_ON_COMPLETION_PERMISSION = 1 +private const val LIST_CHILD_PERMISSION = 2 +private const val LIST_CANCELLATION_PERMISSION = 4 + +private class Empty(override val isActive: Boolean) : Incomplete { + override val list: NodeList? get() = null + override fun toString(): String = "Empty{${if (isActive) "Active" else "New" }}" +} + +@OptIn(InternalForInheritanceCoroutinesApi::class) +@PublishedApi // for a custom job in the test module +internal open class JobImpl(parent: Job?) : JobSupport(true), CompletableJob { + init { initParentJob(parent) } + override val onCancelComplete get() = true + /* + * Check whether parent is able to handle exceptions as well. + * With this check, an exception in that pattern will be handled once: + * ``` + * launch { + * val child = Job(coroutineContext[Job]) + * launch(child) { throw ... } + * } + * ``` + */ + override val handlesException: Boolean = handlesException() + override fun complete() = makeCompleting(Unit) + override fun completeExceptionally(exception: Throwable): Boolean = + makeCompleting(CompletedExceptionally(exception)) + + @JsName("handlesExceptionF") + private fun handlesException(): Boolean { + var parentJob = (parentHandle as? ChildHandleNode)?.job ?: return false + while (true) { + if (parentJob.handlesException) return true + parentJob = (parentJob.parentHandle as? ChildHandleNode)?.job ?: return false + } + } +} + +// -------- invokeOnCompletion nodes + +internal interface Incomplete { + val isActive: Boolean + val list: NodeList? // is null only for Empty and JobNode incomplete state objects +} + +internal abstract class JobNode : LockFreeLinkedListNode(), DisposableHandle, Incomplete { + /** + * Initialized by [JobSupport.invokeOnCompletionInternal]. + */ + lateinit var job: JobSupport + + /** + * If `false`, [invoke] will be called once the job is cancelled or is complete. + * If `true`, [invoke] is invoked as soon as the job becomes _cancelling_ instead, and if that doesn't happen, + * it will be called once the job is cancelled or is complete. + */ + abstract val onCancelling: Boolean + override val isActive: Boolean get() = true + override val list: NodeList? get() = null + + override fun dispose() = job.removeNode(this) + override fun toString() = "$classSimpleName@$hexAddress[job@${job.hexAddress}]" + /** + * Signals completion. + * + * This function: + * - Does not throw any exceptions. + * For [Job] instances that are coroutines, exceptions thrown by this function will be caught, wrapped into + * [CompletionHandlerException], and passed to [handleCoroutineException], but for those that are not coroutines, + * they will just be rethrown, potentially crashing unrelated code. + * - Is fast, non-blocking, and thread-safe. + * - Can be invoked concurrently with the surrounding code. + * - Can be invoked from any context. + * + * The meaning of `cause` that is passed to the handler is: + * - It is `null` if the job has completed normally. + * - It is an instance of [CancellationException] if the job was cancelled _normally_. + * **It should not be treated as an error**. In particular, it should not be reported to error logs. + * - Otherwise, the job had _failed_. + * + * [CompletionHandler] is the user-visible interface for supplying custom implementations of [invoke] + * (see [InvokeOnCompletion] and [InvokeOnCancelling]). + */ + abstract fun invoke(cause: Throwable?) +} + +internal class NodeList : LockFreeLinkedListHead(), Incomplete { + override val isActive: Boolean get() = true + override val list: NodeList get() = this + + fun getString(state: String) = buildString { + append("List{") + append(state) + append("}[") + var first = true + this@NodeList.forEach { node -> + if (node is JobNode) { + if (first) first = false else append(", ") + append(node) + } + } + append("]") + } + + override fun toString(): String = + if (DEBUG) getString("Active") else super.toString() +} + +private class InactiveNodeList( + override val list: NodeList +) : Incomplete { + override val isActive: Boolean get() = false + override fun toString(): String = if (DEBUG) list.getString("New") else super.toString() +} + +private class InvokeOnCompletion( + private val handler: CompletionHandler +) : JobNode() { + override val onCancelling get() = false + override fun invoke(cause: Throwable?) = handler.invoke(cause) +} + +private class ResumeOnCompletion( + private val continuation: Continuation +) : JobNode() { + override val onCancelling get() = false + override fun invoke(cause: Throwable?) = continuation.resume(Unit) +} + +private class ResumeAwaitOnCompletion( + private val continuation: CancellableContinuationImpl +) : JobNode() { + override val onCancelling get() = false + override fun invoke(cause: Throwable?) { + val state = job.state + assert { state !is Incomplete } + if (state is CompletedExceptionally) { + // Resume with with the corresponding exception to preserve it + continuation.resumeWithException(state.cause) + } else { + // Resuming with value in a cancellable way (AwaitContinuation is configured for this mode). + @Suppress("UNCHECKED_CAST") + continuation.resume(state.unboxState() as T) + } + } +} + +// -------- invokeOnCancellation nodes + +private class InvokeOnCancelling( + private val handler: CompletionHandler +) : JobNode() { + // delegate handler shall be invoked at most once, so here is an additional flag + private val _invoked = atomic(false) + override val onCancelling get() = true + override fun invoke(cause: Throwable?) { + if (_invoked.compareAndSet(expect = false, update = true)) handler.invoke(cause) + } +} + +private class ChildHandleNode( + @JvmField val childJob: ChildJob +) : JobNode(), ChildHandle { + override val parent: Job get() = job + override val onCancelling: Boolean get() = true + override fun invoke(cause: Throwable?) = childJob.parentCancelled(job) + override fun childCancelled(cause: Throwable): Boolean = job.childCancelled(cause) +} diff --git a/kotlinx-coroutines-core/common/src/MainCoroutineDispatcher.kt b/kotlinx-coroutines-core/common/src/MainCoroutineDispatcher.kt new file mode 100644 index 0000000000..a6adc38796 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/MainCoroutineDispatcher.kt @@ -0,0 +1,73 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* + +/** + * Base class for special [CoroutineDispatcher] which is confined to application "Main" or "UI" thread + * and used for any UI-based activities. Instance of `MainDispatcher` can be obtained by [Dispatchers.Main]. + * + * Platform may or may not provide instance of `MainDispatcher`, see documentation to [Dispatchers.Main] + */ +public abstract class MainCoroutineDispatcher : CoroutineDispatcher() { + + /** + * Returns dispatcher that executes coroutines immediately when it is already in the right context + * (e.g. current looper is the same as this handler's looper) without an additional [re-dispatch][CoroutineDispatcher.dispatch]. + * + * Immediate dispatcher is safe from stack overflows and in case of nested invocations forms event-loop similar to [Dispatchers.Unconfined]. + * The event loop is an advanced topic and its implications can be found in [Dispatchers.Unconfined] documentation. + * The formed event-loop is shared with [Dispatchers.Unconfined] and other immediate dispatchers, potentially overlapping tasks between them. + * + * Example of usage: + * ``` + * suspend fun updateUiElement(val text: String) { + * /* + * * If it is known that updateUiElement can be invoked both from the Main thread and from other threads, + * * `immediate` dispatcher is used as a performance optimization to avoid unnecessary dispatch. + * * + * * In that case, when `updateUiElement` is invoked from the Main thread, `uiElement.text` will be + * * invoked immediately without any dispatching, otherwise, the `Dispatchers.Main` dispatch cycle will be triggered. + * */ + * withContext(Dispatchers.Main.immediate) { + * uiElement.text = text + * } + * // Do context-independent logic such as logging + * } + * ``` + * + * Method may throw [UnsupportedOperationException] if immediate dispatching is not supported by current dispatcher, + * please refer to specific dispatcher documentation. + * + * [Dispatchers.Main] supports immediate execution for Android, JavaFx and Swing platforms. + */ + public abstract val immediate: MainCoroutineDispatcher + + /** + * Returns a name of this main dispatcher for debugging purposes. This implementation returns + * `Dispatchers.Main` or `Dispatchers.Main.immediate` if it is the same as the corresponding + * reference in [Dispatchers] or a short class-name representation with address otherwise. + */ + override fun toString(): String = toStringInternalImpl() ?: "$classSimpleName@$hexAddress" + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + parallelism.checkParallelism() + // MainCoroutineDispatcher is single-threaded -- short-circuit any attempts to limit it + return namedOrThis(name) + } + + /** + * Internal method for more specific [toString] implementations. It returns non-null + * string if this dispatcher is set in the platform as the main one. + * @suppress + */ + @InternalCoroutinesApi + protected fun toStringInternalImpl(): String? { + val main = Dispatchers.Main + if (this === main) return "Dispatchers.Main" + val immediate = + try { main.immediate } + catch (e: UnsupportedOperationException) { null } + if (this === immediate) return "Dispatchers.Main.immediate" + return null + } +} diff --git a/kotlinx-coroutines-core/common/src/NonCancellable.kt b/kotlinx-coroutines-core/common/src/NonCancellable.kt new file mode 100644 index 0000000000..25c4f6f9d9 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/NonCancellable.kt @@ -0,0 +1,139 @@ +@file:Suppress("DEPRECATION_ERROR") + +package kotlinx.coroutines + +import kotlinx.coroutines.selects.* +import kotlin.coroutines.* + +/** + * A non-cancelable job that is always [active][Job.isActive]. It is designed for [withContext] function + * to prevent cancellation of code blocks that need to be executed without cancellation. + * + * Use it like this: + * ``` + * withContext(NonCancellable) { + * // this code will not be cancelled + * } + * ``` + * + * **WARNING**: This object is not designed to be used with [launch], [async], and other coroutine builders. + * if you write `launch(NonCancellable) { ... }` then not only the newly launched job will not be cancelled + * when the parent is cancelled, the whole parent-child relation between parent and child is severed. + * The parent will not wait for the child's completion, nor will be cancelled when the child crashed. + */ +@OptIn(InternalForInheritanceCoroutinesApi::class) +@Suppress("DeprecatedCallableAddReplaceWith") +public object NonCancellable : AbstractCoroutineContextElement(Job), Job { + + private const val message = "NonCancellable can be used only as an argument for 'withContext', direct usages of its API are prohibited" + + /** + * Always returns `null`. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override val parent: Job? + get() = null + + /** + * Always returns `true`. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override val isActive: Boolean + get() = true + + /** + * Always returns `false`. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override val isCompleted: Boolean get() = false + + /** + * Always returns `false`. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override val isCancelled: Boolean get() = false + + /** + * Always returns `false`. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override fun start(): Boolean = false + + /** + * Always throws [UnsupportedOperationException]. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override suspend fun join() { + throw UnsupportedOperationException("This job is always active") + } + + /** + * Always throws [UnsupportedOperationException]. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override val onJoin: SelectClause0 + get() = throw UnsupportedOperationException("This job is always active") + + /** + * Always throws [IllegalStateException]. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override fun getCancellationException(): CancellationException = throw IllegalStateException("This job is always active") + + /** + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override fun invokeOnCompletion(handler: CompletionHandler): DisposableHandle = + NonDisposableHandle + + /** + * Always returns no-op handle. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override fun invokeOnCompletion(onCancelling: Boolean, invokeImmediately: Boolean, handler: CompletionHandler): DisposableHandle = + NonDisposableHandle + + /** + * Does nothing. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override fun cancel(cause: CancellationException?) {} + + /** + * Always returns `false`. + * @suppress This method has bad semantics when cause is not a [CancellationException]. Use [cancel]. + */ + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") + override fun cancel(cause: Throwable?): Boolean = false // never handles exceptions + + /** + * Always returns [emptySequence]. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override val children: Sequence + get() = emptySequence() + + /** + * Always returns [NonDisposableHandle] and does not do anything. + * @suppress **This an internal API and should not be used from general code.** + */ + @Deprecated(level = DeprecationLevel.WARNING, message = message) + override fun attachChild(child: ChildJob): ChildHandle = NonDisposableHandle + + /** @suppress */ + override fun toString(): String { + return "NonCancellable" + } +} diff --git a/kotlinx-coroutines-core/common/src/Runnable.common.kt b/kotlinx-coroutines-core/common/src/Runnable.common.kt new file mode 100644 index 0000000000..d8a6304e55 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Runnable.common.kt @@ -0,0 +1,14 @@ +package kotlinx.coroutines + +/** + * A runnable task for [CoroutineDispatcher.dispatch]. + * + * It is equivalent to the type `() -> Unit`, but on the JVM, it is represented as a `java.lang.Runnable`, + * making it easier to wrap the interfaces that expect `java.lang.Runnable` into a [CoroutineDispatcher]. + */ +public expect fun interface Runnable { + /** + * @suppress + */ + public fun run() +} diff --git a/kotlinx-coroutines-core/common/src/SchedulerTask.common.kt b/kotlinx-coroutines-core/common/src/SchedulerTask.common.kt new file mode 100644 index 0000000000..e950dcb538 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/SchedulerTask.common.kt @@ -0,0 +1,16 @@ +package kotlinx.coroutines + +/** + * A [Runnable] that's especially optimized for running in [Dispatchers.Default] on the JVM. + * + * Replacing a [SchedulerTask] with a [Runnable] should not lead to any change in observable behavior. + * + * An arbitrary [Runnable], once it is dispatched by [Dispatchers.Default], gets wrapped into a class that + * stores the submission time, the execution context, etc. + * For [Runnable] instances that we know are only going to be executed in dispatch procedures, we can avoid the + * overhead of separately allocating a wrapper, and instead have the [Runnable] contain the required fields + * on construction. + * + * When running outside the standard dispatchers, these new fields are just dead weight. + */ +internal expect abstract class SchedulerTask internal constructor() : Runnable diff --git a/kotlinx-coroutines-core/common/src/Supervisor.kt b/kotlinx-coroutines-core/common/src/Supervisor.kt new file mode 100644 index 0000000000..deec713e86 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Supervisor.kt @@ -0,0 +1,69 @@ +@file:OptIn(ExperimentalContracts::class) +@file:Suppress("LEAKED_IN_PLACE_LAMBDA", "WRONG_INVOCATION_KIND") + +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.intrinsics.* +import kotlin.contracts.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlin.jvm.* + +/** + * Creates a _supervisor_ job object in an active state. + * Children of a supervisor job can fail independently of each other. + * + * A failure or cancellation of a child does not cause the supervisor job to fail and does not affect its other children, + * so a supervisor can implement a custom policy for handling failures of its children: + * + * - A failure of a child job that was created using [launch][CoroutineScope.launch] can be handled via [CoroutineExceptionHandler] in the context. + * - A failure of a child job that was created using [async][CoroutineScope.async] can be handled via [Deferred.await] on the resulting deferred value. + * + * If a [parent] job is specified, then this supervisor job becomes a child job of the [parent] and is cancelled when the + * parent fails or is cancelled. All this supervisor's children are cancelled in this case, too. + */ +@Suppress("FunctionName") +public fun SupervisorJob(parent: Job? = null) : CompletableJob = SupervisorJobImpl(parent) + +/** @suppress Binary compatibility only */ +@Suppress("FunctionName") +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") +@JvmName("SupervisorJob") +public fun SupervisorJob0(parent: Job? = null) : Job = SupervisorJob(parent) + +/** + * Creates a [CoroutineScope] with [SupervisorJob] and calls the specified suspend [block] with this scope. + * The provided scope inherits its [coroutineContext][CoroutineScope.coroutineContext] from the outer scope, using the + * [Job] from that context as the parent for the new [SupervisorJob]. + * This function returns as soon as the given block and all its child coroutines are completed. + * + * Unlike [coroutineScope], a failure of a child does not cause this scope to fail and does not affect its other children, + * so a custom policy for handling failures of its children can be implemented. See [SupervisorJob] for additional details. + * + * If an exception happened in [block], then the supervisor job is failed and all its children are cancelled. + * If the current coroutine was cancelled, then both the supervisor job itself and all its children are cancelled. + * + * The method may throw a [CancellationException] if the current job was cancelled externally, + * or rethrow an exception thrown by the given [block]. + */ +public suspend fun supervisorScope(block: suspend CoroutineScope.() -> R): R { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + return suspendCoroutineUninterceptedOrReturn { uCont -> + val coroutine = SupervisorCoroutine(uCont.context, uCont) + coroutine.startUndispatchedOrReturn(coroutine, block) + } +} + +private class SupervisorJobImpl(parent: Job?) : JobImpl(parent) { + override fun childCancelled(cause: Throwable): Boolean = false +} + +private class SupervisorCoroutine( + context: CoroutineContext, + uCont: Continuation +) : ScopeCoroutine(context, uCont) { + override fun childCancelled(cause: Throwable): Boolean = false +} diff --git a/kotlinx-coroutines-core/common/src/Timeout.kt b/kotlinx-coroutines-core/common/src/Timeout.kt new file mode 100644 index 0000000000..65e68ba299 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Timeout.kt @@ -0,0 +1,190 @@ +@file:OptIn(ExperimentalContracts::class) +@file:Suppress("LEAKED_IN_PLACE_LAMBDA", "WRONG_INVOCATION_KIND") + +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.intrinsics.* +import kotlinx.coroutines.selects.* +import kotlin.contracts.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlin.jvm.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds + +/** + * Runs a given suspending [block] of code inside a coroutine with a specified [timeout][timeMillis] and throws + * a [TimeoutCancellationException] if the timeout was exceeded. + * If the given [timeMillis] is non-positive, [TimeoutCancellationException] is thrown immediately. + * + * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of + * the cancellable suspending function inside the block throws a [TimeoutCancellationException]. + * + * The sibling function that does not throw an exception on timeout is [withTimeoutOrNull]. + * Note that the timeout action can be specified for a [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. + * + * **The timeout event is asynchronous with respect to the code running in the block** and may happen at any time, + * even right before the return from inside the timeout [block]. Keep this in mind if you open or acquire some + * resource inside the [block] that needs closing or release outside the block. + * See the + * [Asynchronous timeout and resources](https://kotlinlang.org/docs/reference/coroutines/cancellation-and-timeouts.html#asynchronous-timeout-and-resources) + * section of the coroutines guide for details. + * + * > Implementation note: how the time is tracked exactly is an implementation detail of the context's [CoroutineDispatcher]. + * + * @param timeMillis timeout time in milliseconds. + */ +public suspend fun withTimeout(timeMillis: Long, block: suspend CoroutineScope.() -> T): T { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + if (timeMillis <= 0L) throw TimeoutCancellationException("Timed out immediately") + return suspendCoroutineUninterceptedOrReturn { uCont -> + setupTimeout(TimeoutCoroutine(timeMillis, uCont), block) + } +} + +/** + * Runs a given suspending [block] of code inside a coroutine with the specified [timeout] and throws + * a [TimeoutCancellationException] if the timeout was exceeded. + * If the given [timeout] is non-positive, [TimeoutCancellationException] is thrown immediately. + * + * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of + * the cancellable suspending function inside the block throws a [TimeoutCancellationException]. + * + * The sibling function that does not throw an exception on timeout is [withTimeoutOrNull]. + * Note that the timeout action can be specified for a [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. + * + * **The timeout event is asynchronous with respect to the code running in the block** and may happen at any time, + * even right before the return from inside the timeout [block]. Keep this in mind if you open or acquire some + * resource inside the [block] that needs closing or release outside the block. + * See the + * [Asynchronous timeout and resources](https://kotlinlang.org/docs/reference/coroutines/cancellation-and-timeouts.html#asynchronous-timeout-and-resources) + * section of the coroutines guide for details. + * + * > Implementation note: how the time is tracked exactly is an implementation detail of the context's [CoroutineDispatcher]. + */ +public suspend fun withTimeout(timeout: Duration, block: suspend CoroutineScope.() -> T): T { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + return withTimeout(timeout.toDelayMillis(), block) +} + +/** + * Runs a given suspending block of code inside a coroutine with a specified [timeout][timeMillis] and returns + * `null` if this timeout was exceeded. + * If the given [timeMillis] is non-positive, `null` is returned immediately. + * + * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of + * cancellable suspending function inside the block throws a [TimeoutCancellationException]. + * + * The sibling function that throws an exception on timeout is [withTimeout]. + * Note that the timeout action can be specified for a [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. + * + * **The timeout event is asynchronous with respect to the code running in the block** and may happen at any time, + * even right before the return from inside the timeout [block]. Keep this in mind if you open or acquire some + * resource inside the [block] that needs closing or release outside the block. + * See the + * [Asynchronous timeout and resources](https://kotlinlang.org/docs/reference/coroutines/cancellation-and-timeouts.html#asynchronous-timeout-and-resources) + * section of the coroutines guide for details. + * + * > Implementation note: how the time is tracked exactly is an implementation detail of the context's [CoroutineDispatcher]. + * + * @param timeMillis timeout time in milliseconds. + */ +public suspend fun withTimeoutOrNull(timeMillis: Long, block: suspend CoroutineScope.() -> T): T? { + if (timeMillis <= 0L) return null + + var coroutine: TimeoutCoroutine? = null + try { + return suspendCoroutineUninterceptedOrReturn { uCont -> + val timeoutCoroutine = TimeoutCoroutine(timeMillis, uCont) + coroutine = timeoutCoroutine + setupTimeout(timeoutCoroutine, block) + } + } catch (e: TimeoutCancellationException) { + // Return null if it's our exception, otherwise propagate it upstream (e.g. in case of nested withTimeouts) + if (e.coroutine === coroutine) { + return null + } + throw e + } +} + +/** + * Runs a given suspending block of code inside a coroutine with the specified [timeout] and returns + * `null` if this timeout was exceeded. + * If the given [timeout] is non-positive, `null` is returned immediately. + * + * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of + * cancellable suspending function inside the block throws a [TimeoutCancellationException]. + * + * The sibling function that throws an exception on timeout is [withTimeout]. + * Note that the timeout action can be specified for a [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. + * + * **The timeout event is asynchronous with respect to the code running in the block** and may happen at any time, + * even right before the return from inside the timeout [block]. Keep this in mind if you open or acquire some + * resource inside the [block] that needs closing or release outside the block. + * See the + * [Asynchronous timeout and resources](https://kotlinlang.org/docs/reference/coroutines/cancellation-and-timeouts.html#asynchronous-timeout-and-resources) + * section of the coroutines guide for details. + * + * > Implementation note: how the time is tracked exactly is an implementation detail of the context's [CoroutineDispatcher]. + */ +public suspend fun withTimeoutOrNull(timeout: Duration, block: suspend CoroutineScope.() -> T): T? = + withTimeoutOrNull(timeout.toDelayMillis(), block) + +private fun setupTimeout( + coroutine: TimeoutCoroutine, + block: suspend CoroutineScope.() -> T +): Any? { + // schedule cancellation of this coroutine on time + val cont = coroutine.uCont + val context = cont.context + coroutine.disposeOnCompletion(context.delay.invokeOnTimeout(coroutine.time, coroutine, coroutine.context)) + // restart the block using a new coroutine with a new job, + // however, start it undispatched, because we already are in the proper context + return coroutine.startUndispatchedOrReturnIgnoreTimeout(coroutine, block) +} + +private class TimeoutCoroutine( + @JvmField val time: Long, + uCont: Continuation // unintercepted continuation +) : ScopeCoroutine(uCont.context, uCont), Runnable { + override fun run() { + cancelCoroutine(TimeoutCancellationException(time, context.delay, this)) + } + + override fun nameString(): String = + "${super.nameString()}(timeMillis=$time)" +} + +/** + * This exception is thrown by [withTimeout] to indicate timeout. + */ +public class TimeoutCancellationException internal constructor( + message: String, + @JvmField @Transient internal val coroutine: Job? +) : CancellationException(message), CopyableThrowable { + /** + * Creates a timeout exception with the given message. + * This constructor is needed for exception stack-traces recovery. + */ + internal constructor(message: String) : this(message, null) + + // message is never null in fact + override fun createCopy(): TimeoutCancellationException = + TimeoutCancellationException(message ?: "", coroutine).also { it.initCause(this) } +} + +internal fun TimeoutCancellationException( + time: Long, + delay: Delay, + coroutine: Job +) : TimeoutCancellationException { + val message = (delay as? DelayWithTimeoutDiagnostics)?.timeoutMessage(time.milliseconds) + ?: "Timed out waiting for $time ms" + return TimeoutCancellationException(message, coroutine) +} diff --git a/kotlinx-coroutines-core/common/src/Unconfined.kt b/kotlinx-coroutines-core/common/src/Unconfined.kt new file mode 100644 index 0000000000..2e16f951b8 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Unconfined.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* +import kotlin.jvm.* + +/** + * A coroutine dispatcher that is not confined to any specific thread. + */ +internal object Unconfined : CoroutineDispatcher() { + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + throw UnsupportedOperationException("limitedParallelism is not supported for Dispatchers.Unconfined") + } + + override fun isDispatchNeeded(context: CoroutineContext): Boolean = false + + override fun dispatch(context: CoroutineContext, block: Runnable) { + /** It can only be called by the [yield] function. See also code of [yield] function. */ + val yieldContext = context[YieldContext] + if (yieldContext != null) { + // report to "yield" that it is an unconfined dispatcher and don't call "block.run()" + yieldContext.dispatcherWasUnconfined = true + return + } + throw UnsupportedOperationException("Dispatchers.Unconfined.dispatch function can only be used by the yield function. " + + "If you wrap Unconfined dispatcher in your code, make sure you properly delegate " + + "isDispatchNeeded and dispatch calls.") + } + + override fun toString(): String = "Dispatchers.Unconfined" +} + +/** + * Used to detect calls to [Unconfined.dispatch] from [yield] function. + */ +@PublishedApi +internal class YieldContext : AbstractCoroutineContextElement(Key) { + companion object Key : CoroutineContext.Key + + @JvmField + var dispatcherWasUnconfined = false +} diff --git a/kotlinx-coroutines-core/common/src/Waiter.kt b/kotlinx-coroutines-core/common/src/Waiter.kt new file mode 100644 index 0000000000..cc037d9d21 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Waiter.kt @@ -0,0 +1,17 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.Segment +import kotlinx.coroutines.selects.* + +/** + * All waiters (such as [CancellableContinuationImpl] and [SelectInstance]) in synchronization and + * communication primitives, should implement this interface to make the code faster and easier to read. + */ +internal interface Waiter { + /** + * When this waiter is cancelled, [Segment.onCancellation] with + * the specified [segment] and [index] should be called. + * This function installs the corresponding cancellation handler. + */ + fun invokeOnCancellation(segment: Segment<*>, index: Int) +} diff --git a/kotlinx-coroutines-core/common/src/Yield.kt b/kotlinx-coroutines-core/common/src/Yield.kt new file mode 100644 index 0000000000..36487501f9 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/Yield.kt @@ -0,0 +1,166 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.intrinsics.* + +/** + * Suspends this coroutine and immediately schedules it for further execution. + * + * A coroutine runs uninterrupted on a thread until the coroutine suspends, + * giving other coroutines a chance to use that thread for their computations. + * Normally, coroutines suspend whenever they wait for something to happen: + * for example, trying to receive a value from a channel that's currently empty will suspend. + * Sometimes, a coroutine does not need to wait for anything, + * but we still want it to give other coroutines a chance to run. + * Calling [yield] has this effect: + * + * ``` + * fun updateProgressBar(value: Int, marker: String) { + * print(marker) + * } + * val singleThreadedDispatcher = Dispatchers.Default.limitedParallelism(1) + * withContext(singleThreadedDispatcher) { + * launch { + * repeat(5) { + * updateProgressBar(it, "A") + * yield() + * } + * } + * launch { + * repeat(5) { + * updateProgressBar(it, "B") + * yield() + * } + * } + * } + * ``` + * + * In this example, without the [yield], first, `A` would run its five stages of work to completion, and only then + * would `B` even start executing. + * With both `yield` calls, the coroutines share the single thread with each other after each stage of work. + * This is useful when several coroutines running on the same thread (or thread pool) must regularly publish + * their results for the program to stay responsive. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while + * [yield] is invoked or while waiting for dispatch, it immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + * + * **Note**: if there is only a single coroutine executing on the current dispatcher, + * it is possible that [yield] will not actually suspend. + * However, even in that case, the [check for cancellation][ensureActive] still happens. + * + * **Note**: if there is no [CoroutineDispatcher] in the context, it does not suspend. + * + * ## Pitfall: using `yield` to wait for something to happen + * + * Using `yield` for anything except a way to ensure responsiveness is often a problem. + * When possible, it is recommended to structure the code in terms of coroutines waiting for some events instead of + * yielding. + * Below, we list the common problems involving [yield] and outline how to avoid them. + * + * ### Case 1: using `yield` to ensure a specific interleaving of actions + * + * ``` + * val singleThreadedDispatcher = Dispatchers.Default.limitedParallelism(1) + * withContext(singleThreadedDispatcher) { + * var value: Int? = null + * val job = launch { // a new coroutine on the same dispatcher + * // yield() // uncomment to see the crash + * value = 42 + * println("2. Value provided") + * } + * check(value == null) + * println("No value yet!") + * println("1. Awaiting the value...") + * // ANTIPATTERN! DO NOT WRITE SUCH CODE! + * yield() // allow the other coroutine to run + * // job.join() // would work more reliably in this scenario! + * check(value != null) + * println("3. Obtained $value") + * } + * ``` + * + * Here, [yield] allows `singleThreadedDispatcher` to execute the task that ultimately provides the `value`. + * Without the [yield], the `value != null` check would be executed directly after `Awaiting the value` is printed. + * However, if the value-producing coroutine is modified to suspend before providing the value, this will + * no longer work; explicitly waiting for the coroutine to finish via [Job.join] instead is robust against such changes. + * + * Therefore, it is an antipattern to use `yield` to synchronize code across several coroutines. + * + * ### Case 2: using `yield` in a loop to wait for something to happen + * + * ``` + * val singleThreadedDispatcher = Dispatchers.Default.limitedParallelism(1) + * withContext(singleThreadedDispatcher) { + * var value: Int? = null + * val job = launch { // a new coroutine on the same dispatcher + * delay(1.seconds) + * value = 42 + * } + * // ANTIPATTERN! DO NOT WRITE SUCH CODE! + * while (value == null) { + * yield() // allow the other coroutines to run + * } + * println("Obtained $value") + * } + * ``` + * + * This example will lead to correct results no matter how much the value-producing coroutine suspends, + * but it is still flawed. + * For the one second that it takes for the other coroutine to obtain the value, + * `value == null` would be constantly re-checked, leading to unjustified resource consumption. + * + * In this specific case, [CompletableDeferred] can be used instead: + * + * ``` + * val singleThreadedDispatcher = Dispatchers.Default.limitedParallelism(1) + * withContext(singleThreadedDispatcher) { + * val deferred = CompletableDeferred() + * val job = launch { // a new coroutine on the same dispatcher + * delay(1.seconds) + * deferred.complete(42) + * } + * val value = deferred.await() + * println("Obtained $value") + * } + * ``` + * + * `while (channel.isEmpty) { yield() }; channel.receive()` can be replaced with just `channel.receive()`; + * `while (job.isActive) { yield() }` can be replaced with [`job.join()`][Job.join]; + * in both cases, this will avoid the unnecessary work of checking the loop conditions. + * In general, seek ways to allow a coroutine to stay suspended until it actually has useful work to do. + * + * ## Implementation details + * + * Some coroutine dispatchers include optimizations that make yielding different from normal suspensions. + * For example, when yielding, [Dispatchers.Unconfined] checks whether there are any other coroutines in the event + * loop where the current coroutine executes; if not, the sole coroutine continues to execute without suspending. + * Also, `Dispatchers.IO` and `Dispatchers.Default` on the JVM tweak the scheduling behavior to improve liveness + * when `yield()` is used in a loop. + * + * For custom implementations of [CoroutineDispatcher], this function checks [CoroutineDispatcher.isDispatchNeeded] and + * then invokes [CoroutineDispatcher.dispatch] regardless of the result; no way is provided to change this behavior. + */ +public suspend fun yield(): Unit = suspendCoroutineUninterceptedOrReturn sc@ { uCont -> + val context = uCont.context + context.ensureActive() + val cont = uCont.intercepted() as? DispatchedContinuation ?: return@sc Unit + if (cont.dispatcher.safeIsDispatchNeeded(context)) { + // this is a regular dispatcher -- do simple dispatchYield + cont.dispatchYield(context, Unit) + } else { + // This is either an "immediate" dispatcher or the Unconfined dispatcher + // This code detects the Unconfined dispatcher even if it was wrapped into another dispatcher + val yieldContext = YieldContext() + cont.dispatchYield(context + yieldContext, Unit) + // Special case for the unconfined dispatcher that can yield only in existing unconfined loop + if (yieldContext.dispatcherWasUnconfined) { + // Means that the Unconfined dispatcher got the call, but did not do anything. + // See also code of "Unconfined.dispatch" function. + return@sc if (cont.yieldUndispatched()) COROUTINE_SUSPENDED else Unit + } + // Otherwise, it was some other dispatcher that successfully dispatched the coroutine + } + COROUTINE_SUSPENDED +} diff --git a/kotlinx-coroutines-core/common/src/channels/Broadcast.kt b/kotlinx-coroutines-core/common/src/channels/Broadcast.kt new file mode 100644 index 0000000000..f5d14263ea --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/Broadcast.kt @@ -0,0 +1,123 @@ +@file:Suppress("DEPRECATION", "DEPRECATION_ERROR") + +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.Channel.Factory.CONFLATED +import kotlinx.coroutines.channels.Channel.Factory.UNLIMITED +import kotlinx.coroutines.intrinsics.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +/** + * @suppress obsolete since 1.5.0, WARNING since 1.7.0, ERROR since 1.9.0 + */ +@ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.ERROR, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") +public fun ReceiveChannel.broadcast( + capacity: Int = 1, + start: CoroutineStart = CoroutineStart.LAZY +): BroadcastChannel { + val scope = GlobalScope + Dispatchers.Unconfined + CoroutineExceptionHandler { _, _ -> } + val channel = this + // We can run this coroutine in the context that ignores all exceptions, because of `onCompletion = consume()` + // which passes all exceptions upstream to the source ReceiveChannel + return scope.broadcast(capacity = capacity, start = start, onCompletion = { cancelConsumed(it) }) { + for (e in channel) { + send(e) + } + } +} + +/** + * @suppress obsolete since 1.5.0, WARNING since 1.7.0, ERROR since 1.9.0 + */ +@ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.ERROR, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") +public fun CoroutineScope.broadcast( + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = 1, + start: CoroutineStart = CoroutineStart.LAZY, + onCompletion: CompletionHandler? = null, + @BuilderInference block: suspend ProducerScope.() -> Unit +): BroadcastChannel { + val newContext = newCoroutineContext(context) + val channel = BroadcastChannel(capacity) + val coroutine = if (start.isLazy) + LazyBroadcastCoroutine(newContext, channel, block) else + BroadcastCoroutine(newContext, channel, active = true) + if (onCompletion != null) coroutine.invokeOnCompletion(handler = onCompletion) + coroutine.start(start, coroutine, block) + return coroutine +} + +private open class BroadcastCoroutine( + parentContext: CoroutineContext, + protected val _channel: BroadcastChannel, + active: Boolean +) : AbstractCoroutine(parentContext, initParentJob = false, active = active), + ProducerScope, BroadcastChannel by _channel { + + init { + initParentJob(parentContext[Job]) + } + + override val isActive: Boolean get() = super.isActive + + override val channel: SendChannel + get() = this + + @Suppress("MULTIPLE_DEFAULTS_INHERITED_FROM_SUPERTYPES_DEPRECATION_WARNING") // do not remove the MULTIPLE_DEFAULTS suppression: required in K2 + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") + final override fun cancel(cause: Throwable?): Boolean { + cancelInternal(cause ?: defaultCancellationException()) + return true + } + + @Suppress("MULTIPLE_DEFAULTS_INHERITED_FROM_SUPERTYPES_DEPRECATION_WARNING") // do not remove the MULTIPLE_DEFAULTS suppression: required in K2 + final override fun cancel(cause: CancellationException?) { + cancelInternal(cause ?: defaultCancellationException()) + } + + override fun cancelInternal(cause: Throwable) { + val exception = cause.toCancellationException() + _channel.cancel(exception) // cancel the channel + cancelCoroutine(exception) // cancel the job + } + + override fun onCompleted(value: Unit) { + _channel.close() + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + val processed = _channel.close(cause) + if (!processed && !handled) handleCoroutineException(context, cause) + } + + // The BroadcastChannel could be also closed + override fun close(cause: Throwable?): Boolean { + val result = _channel.close(cause) + start() // start coroutine if it was not started yet + return result + } +} + +private class LazyBroadcastCoroutine( + parentContext: CoroutineContext, + channel: BroadcastChannel, + block: suspend ProducerScope.() -> Unit +) : BroadcastCoroutine(parentContext, channel, active = false) { + private val continuation = block.createCoroutineUnintercepted(this, this) + + override fun openSubscription(): ReceiveChannel { + // open subscription _first_ + val subscription = _channel.openSubscription() + // then start coroutine + start() + return subscription + } + + override fun onStart() { + continuation.startCoroutineCancellable(this) + } +} diff --git a/kotlinx-coroutines-core/common/src/channels/BroadcastChannel.kt b/kotlinx-coroutines-core/common/src/channels/BroadcastChannel.kt new file mode 100644 index 0000000000..1f0739109e --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/BroadcastChannel.kt @@ -0,0 +1,362 @@ +@file:Suppress("FunctionName", "DEPRECATION", "DEPRECATION_ERROR") + +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.BufferOverflow.* +import kotlinx.coroutines.channels.Channel.Factory.BUFFERED +import kotlinx.coroutines.channels.Channel.Factory.CHANNEL_DEFAULT_CAPACITY +import kotlinx.coroutines.channels.Channel.Factory.CONFLATED +import kotlinx.coroutines.channels.Channel.Factory.UNLIMITED +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* + +/** + * @suppress obsolete since 1.5.0, WARNING since 1.7.0, ERROR since 1.9.0 + */ +@ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.ERROR, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") +public interface BroadcastChannel : SendChannel { + /** + * @suppress + */ + public fun openSubscription(): ReceiveChannel + + /** + * @suppress + */ + public fun cancel(cause: CancellationException? = null) + + /** + * @suppress + */ + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Binary compatibility only") + public fun cancel(cause: Throwable? = null): Boolean +} + +/** + * @suppress obsolete since 1.5.0, WARNING since 1.7.0, ERROR since 1.9.0 + */ +@ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.ERROR, message = "BroadcastChannel is deprecated in the favour of SharedFlow and StateFlow, and is no longer supported") +public fun BroadcastChannel(capacity: Int): BroadcastChannel = + when (capacity) { + 0 -> throw IllegalArgumentException("Unsupported 0 capacity for BroadcastChannel") + UNLIMITED -> throw IllegalArgumentException("Unsupported UNLIMITED capacity for BroadcastChannel") + CONFLATED -> ConflatedBroadcastChannel() + BUFFERED -> BroadcastChannelImpl(CHANNEL_DEFAULT_CAPACITY) + else -> BroadcastChannelImpl(capacity) + } + +/** + * @suppress obsolete since 1.5.0, WARNING since 1.7.0, ERROR since 1.9.0 + */ +@ObsoleteCoroutinesApi +@Deprecated(level = DeprecationLevel.ERROR, message = "ConflatedBroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") +public class ConflatedBroadcastChannel private constructor( + private val broadcast: BroadcastChannelImpl +) : BroadcastChannel by broadcast { + public constructor(): this(BroadcastChannelImpl(capacity = CONFLATED)) + /** + * @suppress + */ + public constructor(value: E) : this() { + trySend(value) + } + + /** + * @suppress + */ + public val value: E get() = broadcast.value + + /** + * @suppress + */ + public val valueOrNull: E? get() = broadcast.valueOrNull +} + +/** + * A common implementation for both the broadcast channel with a buffer of fixed [capacity] + * and the conflated broadcast channel (see [ConflatedBroadcastChannel]). + * + * **Note**, that elements that are sent to this channel while there are no + * [openSubscription] subscribers are immediately lost. + * + * This channel is created by `BroadcastChannel(capacity)` factory function invocation. + */ +@Suppress("MULTIPLE_DEFAULTS_INHERITED_FROM_SUPERTYPES_DEPRECATION_WARNING", "MULTIPLE_DEFAULTS_INHERITED_FROM_SUPERTYPES_WHEN_NO_EXPLICIT_OVERRIDE_DEPRECATION_WARNING") // do not remove the MULTIPLE_DEFAULTS suppression: required in K2 +internal class BroadcastChannelImpl( + /** + * Buffer capacity; [Channel.CONFLATED] when this broadcast is conflated. + */ + val capacity: Int +) : BufferedChannel(capacity = Channel.RENDEZVOUS, onUndeliveredElement = null), BroadcastChannel { + init { + require(capacity >= 1 || capacity == CONFLATED) { + "BroadcastChannel capacity must be positive or Channel.CONFLATED, but $capacity was specified" + } + } + + // This implementation uses coarse-grained synchronization, + // as, reputedly, it is the simplest synchronization scheme. + // All operations are protected by this lock. + private val lock = ReentrantLock() + // The list of subscribers; all accesses should be protected by lock. + // Each change must create a new list instance to avoid `ConcurrentModificationException`. + private var subscribers: List> = emptyList() + // When this broadcast is conflated, this field stores the last sent element. + // If this channel is empty or not conflated, it stores a special `NO_ELEMENT` marker. + private var lastConflatedElement: Any? = NO_ELEMENT // NO_ELEMENT or E + + // ########################### + // # Subscription Management # + // ########################### + + override fun openSubscription(): ReceiveChannel = lock.withLock { // protected by lock + // Is this broadcast conflated or buffered? + // Create the corresponding subscription channel. + val s = if (capacity == CONFLATED) SubscriberConflated() else SubscriberBuffered() + // If this broadcast is already closed or cancelled, + // and the last sent element is not available in case + // this broadcast is conflated, close the created + // subscriber immediately and return it. + if (isClosedForSend && lastConflatedElement === NO_ELEMENT) { + s.close(closeCause) + return s + } + // Is this broadcast conflated? If so, send + // the last sent element to the subscriber. + if (lastConflatedElement !== NO_ELEMENT) { + s.trySend(value) + } + // Add the subscriber to the list and return it. + subscribers += s + s + } + + private fun removeSubscriber(s: ReceiveChannel) = lock.withLock { // protected by lock + subscribers = subscribers.filter { it !== s } + } + + // ############################# + // # The `send(..)` Operations # + // ############################# + + /** + * Sends the specified element to all subscribers. + * + * **!!! THIS IMPLEMENTATION IS NOT LINEARIZABLE !!!** + * + * As the operation should send the element to multiple + * subscribers simultaneously, it is non-trivial to + * implement it in an atomic way. Specifically, this + * would require a special implementation that does + * not transfer the element until all parties are able + * to resume it (this `send(..)` can be cancelled + * or the broadcast can become closed in the meantime). + * As broadcasts are obsolete, we keep this implementation + * as simple as possible, allowing non-linearizability + * in corner cases. + */ + override suspend fun send(element: E) { + val subs = lock.withLock { // protected by lock + // Is this channel closed for send? + if (isClosedForSend) throw sendException + // Update the last sent element if this broadcast is conflated. + if (capacity == CONFLATED) lastConflatedElement = element + // Get a reference to the list of subscribers under the lock. + subscribers + } + // The lock has been released. Send the element to the + // subscribers one-by-one, and finish immediately + // when this broadcast discovered in the closed state. + // Note that this implementation is non-linearizable; + // see this method documentation for details. + subs.forEach { + // We use special function to send the element, + // which returns `true` on success and `false` + // if the subscriber is closed. + val success = it.sendBroadcast(element) + // The sending attempt has failed. + // Check whether the broadcast is closed. + if (!success && isClosedForSend) throw sendException + } + } + + override fun trySend(element: E): ChannelResult = lock.withLock { // protected by lock + // Is this channel closed for send? + if (isClosedForSend) return super.trySend(element) + // Check whether the plain `send(..)` operation + // should suspend and fail in this case. + val shouldSuspend = subscribers.any { it.shouldSendSuspend() } + if (shouldSuspend) return ChannelResult.failure() + // Update the last sent element if this broadcast is conflated. + if (capacity == CONFLATED) lastConflatedElement = element + // Send the element to all subscribers. + // It is guaranteed that the attempt cannot fail, + // as both the broadcast closing and subscription + // cancellation are guarded by lock, which is held + // by the current operation. + subscribers.forEach { it.trySend(element) } + // Finish with success. + return ChannelResult.success(Unit) + } + + // ########################################### + // # The `select` Expression: onSend { ... } # + // ########################################### + + override fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // It is extremely complicated to support sending via `select` for broadcasts, + // as the operation should wait on multiple subscribers simultaneously. + // At the same time, broadcasts are obsolete, so we need a simple implementation + // that works somehow. Here is a tricky work-around. First, we launch a new + // coroutine that performs plain `send(..)` operation and tries to complete + // this `select` via `trySelect`, independently on whether it is in the + // registration or in the waiting phase. On success, the operation finishes. + // On failure, if another clause is already selected or the `select` operation + // has been cancelled, we observe non-linearizable behaviour, as this `onSend` + // clause is completed as well. However, we believe that such a non-linearizability + // is fine for obsolete API. The last case is when the `select` operation is still + // in the registration case, so this `onSend` clause should be re-registered. + // The idea is that we keep information that this `onSend` clause is already selected + // and finish immediately. + @Suppress("UNCHECKED_CAST") + element as E + // First, check whether this `onSend` clause is already + // selected, finishing immediately in this case. + lock.withLock { + val result = onSendInternalResult.remove(select) + if (result != null) { // already selected! + // `result` is either `Unit` ot `CHANNEL_CLOSED`. + select.selectInRegistrationPhase(result) + return + } + } + // Start a new coroutine that performs plain `send(..)` + // and tries to select this `onSend` clause at the end. + CoroutineScope(select.context).launch(start = CoroutineStart.UNDISPATCHED) { + val success: Boolean = try { + send(element) + // The element has been successfully sent! + true + } catch (t: Throwable) { + // This broadcast must be closed. However, it is possible that + // an unrelated exception, such as `OutOfMemoryError` has been thrown. + // This implementation checks that the channel is actually closed, + // re-throwing the caught exception otherwise. + if (isClosedForSend && (t is ClosedSendChannelException || sendException === t)) false + else throw t + } + // Mark this `onSend` clause as selected and + // try to complete the `select` operation. + lock.withLock { + // Status of this `onSend` clause should not be presented yet. + assert { onSendInternalResult[select] == null } + // Success or fail? Put the corresponding result. + onSendInternalResult[select] = if (success) Unit else CHANNEL_CLOSED + // Try to select this `onSend` clause. + select as SelectImplementation<*> + val trySelectResult = select.trySelectDetailed(this@BroadcastChannelImpl, Unit) + if (trySelectResult !== TrySelectDetailedResult.REREGISTER) { + // In case of re-registration (this `select` was still + // in the registration phase), the algorithm will invoke + // `registerSelectForSend`. As we stored an information that + // this `onSend` clause is already selected (in `onSendInternalResult`), + // the algorithm, will complete immediately. Otherwise, to avoid memory + // leaks, we must remove this information from the hashmap. + onSendInternalResult.remove(select) + } + } + + } + } + private val onSendInternalResult = HashMap, Any?>() // select -> Unit or CHANNEL_CLOSED + + // ############################ + // # Closing and Cancellation # + // ############################ + + override fun close(cause: Throwable?): Boolean = lock.withLock { // protected by lock + // Close all subscriptions first. + subscribers.forEach { it.close(cause) } + // Remove all subscriptions that do not contain + // buffered elements or waiting send-s to avoid + // memory leaks. We must keep other subscriptions + // in case `broadcast.cancel(..)` is called. + subscribers = subscribers.filter { it.hasElements() } + // Delegate to the parent implementation. + super.close(cause) + } + + override fun cancelImpl(cause: Throwable?): Boolean = lock.withLock { // protected by lock + // Cancel all subscriptions. As part of cancellation procedure, + // subscriptions automatically remove themselves from this broadcast. + subscribers.forEach { it.cancelImpl(cause) } + // For the conflated implementation, clear the last sent element. + lastConflatedElement = NO_ELEMENT + // Finally, delegate to the parent implementation. + super.cancelImpl(cause) + } + + override val isClosedForSend: Boolean + // Protect by lock to synchronize with `close(..)` / `cancel(..)`. + get() = lock.withLock { super.isClosedForSend } + + // ############################## + // # Subscriber Implementations # + // ############################## + + private inner class SubscriberBuffered : BufferedChannel(capacity = capacity) { + public override fun cancelImpl(cause: Throwable?): Boolean = lock.withLock { + // Remove this subscriber from the broadcast on cancellation. + removeSubscriber(this@SubscriberBuffered ) + super.cancelImpl(cause) + } + } + + private inner class SubscriberConflated : ConflatedBufferedChannel(capacity = 1, onBufferOverflow = DROP_OLDEST) { + public override fun cancelImpl(cause: Throwable?): Boolean { + // Remove this subscriber from the broadcast on cancellation. + removeSubscriber(this@SubscriberConflated ) + return super.cancelImpl(cause) + } + } + + // ######################################## + // # ConflatedBroadcastChannel Operations # + // ######################################## + + @Suppress("UNCHECKED_CAST") + val value: E get() = lock.withLock { + // Is this channel closed for sending? + if (isClosedForSend) { + throw closeCause ?: IllegalStateException("This broadcast channel is closed") + } + // Is there sent element? + if (lastConflatedElement === NO_ELEMENT) error("No value") + // Return the last sent element. + lastConflatedElement as E + } + + @Suppress("UNCHECKED_CAST") + val valueOrNull: E? get() = lock.withLock { + // Is this channel closed for sending? + if (isClosedForReceive) null + // Is there sent element? + else if (lastConflatedElement === NO_ELEMENT) null + // Return the last sent element. + else lastConflatedElement as E + } + + // ################# + // # For Debugging # + // ################# + + override fun toString() = + (if (lastConflatedElement !== NO_ELEMENT) "CONFLATED_ELEMENT=$lastConflatedElement; " else "") + + "BROADCAST=<${super.toString()}>; " + + "SUBSCRIBERS=${subscribers.joinToString(separator = ";", prefix = "<", postfix = ">")}" +} + +private val NO_ELEMENT = Symbol("NO_ELEMENT") diff --git a/kotlinx-coroutines-core/common/src/channels/BufferOverflow.kt b/kotlinx-coroutines-core/common/src/channels/BufferOverflow.kt new file mode 100644 index 0000000000..652f8d7c1e --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/BufferOverflow.kt @@ -0,0 +1,39 @@ +package kotlinx.coroutines.channels + +/** + * A strategy for buffer overflow handling in [channels][Channel] and [flows][kotlinx.coroutines.flow.Flow] that + * controls what is going to be sacrificed on buffer overflow: + * + * - [SUSPEND] — the upstream that is [sending][SendChannel.send] or + * is [emitting][kotlinx.coroutines.flow.FlowCollector.emit] a value is **suspended** while the buffer is full. + * - [DROP_OLDEST] — **the oldest** value in the buffer is dropped on overflow, and the new value is added, + * all without suspending. + * - [DROP_LATEST] — the buffer remains unchanged on overflow, and the value that we were going to add + * gets discarded, all without suspending. + */ +public enum class BufferOverflow { + /** + * Suspend until free space appears in the buffer. + * + * Use this to create backpressure, forcing the producers to slow down creation of new values in response to + * consumers not being able to process the incoming values in time. + * [SUSPEND] is a good choice when all elements must eventually be processed. + */ + SUSPEND, + + /** + * Drop **the oldest** value in the buffer on overflow, add the new value to the buffer, do not suspend. + * + * Use this in scenarios when only the last few values are important and skipping the processing of severely + * outdated ones is desirable. + */ + DROP_OLDEST, + + /** + * Leave the buffer unchanged on overflow, dropping the value that we were going to add, do not suspend. + * + * This option can be used in rare advanced scenarios where all elements that are expected to enter the buffer are + * equal, so it is not important which of them get thrown away. + */ + DROP_LATEST +} diff --git a/kotlinx-coroutines-core/common/src/channels/BufferedChannel.kt b/kotlinx-coroutines-core/common/src/channels/BufferedChannel.kt new file mode 100644 index 0000000000..f94a9e9970 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/BufferedChannel.kt @@ -0,0 +1,3115 @@ +@file:Suppress("PrivatePropertyName") + +package kotlinx.coroutines.channels + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.ChannelResult.Companion.closed +import kotlinx.coroutines.channels.ChannelResult.Companion.failure +import kotlinx.coroutines.channels.ChannelResult.Companion.success +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlinx.coroutines.selects.TrySelectDetailedResult.* +import kotlin.coroutines.* +import kotlin.js.* +import kotlin.jvm.* +import kotlin.math.* +import kotlin.reflect.* + +/** + * The buffered channel implementation, which also serves as a rendezvous channel when the capacity is zero. + * The high-level structure bases on a conceptually infinite array for storing elements and waiting requests, + * separate counters of [send] and [receive] invocations that were ever performed, and an additional counter + * that indicates the end of the logical buffer by counting the number of array cells it ever contained. + * The key idea is that both [send] and [receive] start by incrementing their counters, assigning the array cell + * referenced by the counter. In case of rendezvous channels, the operation either suspends and stores its continuation + * in the cell or makes a rendezvous with the opposite request. Each cell can be processed by exactly one [send] and + * one [receive]. As for buffered channels, [send]-s can also add elements without suspension if the logical buffer + * contains the cell, while the [receive] operation updates the end of the buffer when its synchronization finishes. + * + * Please see the ["Fast and Scalable Channels in Kotlin Coroutines"](https://arxiv.org/abs/2211.04986) + * paper by Nikita Koval, Roman Elizarov, and Dan Alistarh for the detailed algorithm description. + */ +internal open class BufferedChannel( + /** + * Channel capacity; `Channel.RENDEZVOUS` for rendezvous channel + * and `Channel.UNLIMITED` for unlimited capacity. + */ + private val capacity: Int, + @JvmField + internal val onUndeliveredElement: OnUndeliveredElement? = null +) : Channel { + init { + require(capacity >= 0) { "Invalid channel capacity: $capacity, should be >=0" } + // This implementation has second `init`. + } + + // Maintenance note: use `Buffered1ChannelLincheckTest` to check hypotheses. + + /* + The counters indicate the total numbers of send, receive, and buffer expansion calls + ever performed. The counters are incremented in the beginning of the corresponding + operation; thus, acquiring a unique (for the operation type) cell to process. + The segments reference to the last working one for each operation type. + + Notably, the counter for send is combined with the channel closing status + for synchronization simplicity and performance reasons. + + The logical end of the buffer is initialized with the channel capacity. + If the channel is rendezvous or unlimited, the counter equals `BUFFER_END_RENDEZVOUS` + or `BUFFER_END_RENDEZVOUS`, respectively, and never updates. The `bufferEndSegment` + point to a special `NULL_SEGMENT` in this case. + */ + private val sendersAndCloseStatus = atomic(0L) + private val receivers = atomic(0L) + private val bufferEnd = atomic(initialBufferEnd(capacity)) + + internal val sendersCounter: Long get() = sendersAndCloseStatus.value.sendersCounter + internal val receiversCounter: Long get() = receivers.value + private val bufferEndCounter: Long get() = bufferEnd.value + + /* + Additionally to the counters above, we need an extra one that + tracks the number of cells processed by `expandBuffer()`. + When a receiver aborts, the corresponding cell might be + physically removed from the data structure to avoid memory + leaks, while it still can be unprocessed by `expandBuffer()`. + In this case, `expandBuffer()` cannot know whether the + removed cell contained sender or receiver and, therefore, + cannot proceed. To solve the race, we ensure that cells + correspond to cancelled receivers cannot be physically + removed until the cell is processed. + This additional counter enables the synchronization, + */ + private val completedExpandBuffersAndPauseFlag = atomic(bufferEndCounter) + + private val isRendezvousOrUnlimited + get() = bufferEndCounter.let { it == BUFFER_END_RENDEZVOUS || it == BUFFER_END_UNLIMITED } + + private val sendSegment: AtomicRef> + private val receiveSegment: AtomicRef> + private val bufferEndSegment: AtomicRef> + + init { + @Suppress("LeakingThis") + val firstSegment = ChannelSegment(id = 0, prev = null, channel = this, pointers = 3) + sendSegment = atomic(firstSegment) + receiveSegment = atomic(firstSegment) + // If this channel is rendezvous or has unlimited capacity, the algorithm never + // invokes the buffer expansion procedure, and the corresponding segment reference + // points to a special `NULL_SEGMENT` one and never updates. + @Suppress("UNCHECKED_CAST") + bufferEndSegment = atomic(if (isRendezvousOrUnlimited) (NULL_SEGMENT as ChannelSegment) else firstSegment) + } + + // ######################### + // ## The send operations ## + // ######################### + + override suspend fun send(element: E): Unit = + sendImpl( // <-- this is an inline function + element = element, + // Do not create a continuation until it is required; + // it is created later via [onNoWaiterSuspend], if needed. + waiter = null, + // Finish immediately if a rendezvous happens + // or the element has been buffered. + onRendezvousOrBuffered = {}, + // As no waiter is provided, suspension is impossible. + onSuspend = { _, _ -> assert { false } }, + // According to the `send(e)` contract, we need to call + // `onUndeliveredElement(..)` handler and throw an exception + // if the channel is already closed. + onClosed = { onClosedSend(element) }, + // When `send(e)` decides to suspend, the corresponding + // `onNoWaiterSuspend` function that creates a continuation + // is called. The tail-call optimization is applied here. + onNoWaiterSuspend = { segm, i, elem, s -> sendOnNoWaiterSuspend(segm, i, elem, s) } + ) + + // NB: return type could've been Nothing, but it breaks TCO + private suspend fun onClosedSend(element: E): Unit = suspendCancellableCoroutine { continuation -> + onUndeliveredElement?.callUndeliveredElementCatchingException(element)?.let { + // If it crashes, add send exception as suppressed for better diagnostics + it.addSuppressed(sendException) + continuation.resumeWithStackTrace(it) + return@suspendCancellableCoroutine + } + continuation.resumeWithStackTrace(sendException) + } + + private suspend fun sendOnNoWaiterSuspend( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /** The element to be inserted. */ + element: E, + /** The global index of the cell. */ + s: Long + ) = suspendCancellableCoroutineReusable sc@{ cont -> + sendImplOnNoWaiter( // <-- this is an inline function + segment = segment, index = index, element = element, s = s, + // Store the created continuation as a waiter. + waiter = cont, + // If a rendezvous happens or the element has been buffered, + // resume the continuation and finish. In case of prompt + // cancellation, it is guaranteed that the element + // has been already buffered or passed to receiver. + onRendezvousOrBuffered = { cont.resume(Unit) }, + // If the channel is closed, call `onUndeliveredElement(..)` and complete the + // continuation with the corresponding exception. + onClosed = { onClosedSendOnNoWaiterSuspend(element, cont) }, + ) + } + + private fun Waiter.prepareSenderForSuspension( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int + ) { + // To distinguish cancelled senders and receivers, + // senders equip the index value with an additional marker, + // adding `SEGMENT_SIZE` to the value. + invokeOnCancellation(segment, index + SEGMENT_SIZE) + } + + private fun onClosedSendOnNoWaiterSuspend(element: E, cont: CancellableContinuation) { + onUndeliveredElement?.callUndeliveredElement(element, cont.context) + cont.resumeWithException(recoverStackTrace(sendException, cont)) + } + + override fun trySend(element: E): ChannelResult { + // Do not try to send the element if the plain `send(e)` operation would suspend. + if (shouldSendSuspend(sendersAndCloseStatus.value)) return failure() + // This channel either has waiting receivers or is closed. + // Let's try to send the element! + // The logic is similar to the plain `send(e)` operation, with + // the only difference that we install `INTERRUPTED_SEND` in case + // the operation decides to suspend. + return sendImpl( // <-- this is an inline function + element = element, + // Store an already interrupted sender in case of suspension. + waiter = INTERRUPTED_SEND, + // Finish successfully when a rendezvous happens + // or the element has been buffered. + onRendezvousOrBuffered = { success(Unit) }, + // On suspension, the `INTERRUPTED_SEND` token has been installed, + // and this `trySend(e)` must fail. According to the contract, + // we do not need to call the [onUndeliveredElement] handler. + onSuspend = { segm, _ -> + segm.onSlotCleaned() + failure() + }, + // If the channel is closed, return the corresponding result. + onClosed = { closed(sendException) } + ) + } + + /** + * This is a special `send(e)` implementation that returns `true` if the element + * has been successfully sent, and `false` if the channel is closed. + * + * In case of coroutine cancellation, the element may be undelivered -- + * the [onUndeliveredElement] feature is unsupported in this implementation. + * + */ + internal open suspend fun sendBroadcast(element: E): Boolean = suspendCancellableCoroutine { cont -> + check(onUndeliveredElement == null) { + "the `onUndeliveredElement` feature is unsupported for `sendBroadcast(e)`" + } + sendImpl( + element = element, + waiter = SendBroadcast(cont), + onRendezvousOrBuffered = { cont.resume(true) }, + onSuspend = { _, _ -> }, + onClosed = { cont.resume(false) } + ) + } + + /** + * Specifies waiting [sendBroadcast] operation. + */ + private class SendBroadcast( + val cont: CancellableContinuation + ) : Waiter by cont as CancellableContinuationImpl + + /** + * Abstract send implementation. + */ + private inline fun sendImpl( + /* The element to be sent. */ + element: E, + /* The waiter to be stored in case of suspension, + or `null` if the waiter is not created yet. + In the latter case, when the algorithm decides + to suspend, [onNoWaiterSuspend] is called. */ + waiter: Any?, + /* This lambda is invoked when the element has been + buffered or a rendezvous with a receiver happens. */ + onRendezvousOrBuffered: () -> R, + /* This lambda is called when the operation suspends in the + cell specified by the segment and the index in it. */ + onSuspend: (segm: ChannelSegment, i: Int) -> R, + /* This lambda is called when the channel + is observed in the closed state. */ + onClosed: () -> R, + /* This lambda is called when the operation decides + to suspend, but the waiter is not provided (equals `null`). + It should create a waiter and delegate to `sendImplOnNoWaiter`. */ + onNoWaiterSuspend: ( + segm: ChannelSegment, + i: Int, + element: E, + s: Long + ) -> R = { _, _, _, _ -> error("unexpected") } + ): R { + // Read the segment reference before the counter increment; + // it is crucial to be able to find the required segment later. + var segment = sendSegment.value + while (true) { + // Atomically increment the `senders` counter and obtain the + // value right before the increment along with the close status. + val sendersAndCloseStatusCur = sendersAndCloseStatus.getAndIncrement() + val s = sendersAndCloseStatusCur.sendersCounter + // Is this channel already closed? Keep the information. + val closed = sendersAndCloseStatusCur.isClosedForSend0 + // Count the required segment id and the cell index in it. + val id = s / SEGMENT_SIZE + val i = (s % SEGMENT_SIZE).toInt() + // Try to find the required segment if the initially obtained + // one (in the beginning of this function) has lower id. + if (segment.id != id) { + // Find the required segment. + segment = findSegmentSend(id, segment) ?: + // The required segment has not been found. + // Finish immediately if this channel is closed, + // restarting the operation otherwise. + // In the latter case, the required segment was full + // of interrupted waiters and, therefore, removed + // physically to avoid memory leaks. + if (closed) { + return onClosed() + } else { + continue + } + } + // Update the cell according to the algorithm. Importantly, when + // the channel is already closed, storing a waiter is illegal, so + // the algorithm stores the `INTERRUPTED_SEND` token in this case. + when (updateCellSend(segment, i, element, s, waiter, closed)) { + RESULT_RENDEZVOUS -> { + // A rendezvous with a receiver has happened. + // The previous segments are no longer needed + // for the upcoming requests, so the algorithm + // resets the link to the previous segment. + segment.cleanPrev() + return onRendezvousOrBuffered() + } + RESULT_BUFFERED -> { + // The element has been buffered. + return onRendezvousOrBuffered() + } + RESULT_SUSPEND -> { + // The operation has decided to suspend and installed the + // specified waiter. If the channel was already closed, + // and the `INTERRUPTED_SEND` token has been installed as a waiter, + // this request finishes with the `onClosed()` action. + if (closed) { + segment.onSlotCleaned() + return onClosed() + } + (waiter as? Waiter)?.prepareSenderForSuspension(segment, i) + return onSuspend(segment, i) + } + RESULT_CLOSED -> { + // This channel is closed. + // In case this segment is already or going to be + // processed by a receiver, ensure that all the + // previous segments are unreachable. + if (s < receiversCounter) segment.cleanPrev() + return onClosed() + } + RESULT_FAILED -> { + // Either the cell stores an interrupted receiver, + // or it was poisoned by a concurrent receiver. + // In both cases, all the previous segments are already processed, + segment.cleanPrev() + continue + } + RESULT_SUSPEND_NO_WAITER -> { + // The operation has decided to suspend, + // but no waiter has been provided. + return onNoWaiterSuspend(segment, i, element, s) + } + } + } + } + + // Note: this function is temporarily moved from ConflatedBufferedChannel to BufferedChannel class, because of this issue: KT-65554. + // For now, an inline function, which invokes atomic operations, may only be called within a parent class. + protected fun trySendDropOldest(element: E): ChannelResult = + sendImpl( // <-- this is an inline function + element = element, + // Put the element into the logical buffer even + // if this channel is already full, the `onSuspend` + // callback below extract the first (oldest) element. + waiter = BUFFERED, + // Finish successfully when a rendezvous has happened + // or the element has been buffered. + onRendezvousOrBuffered = { return success(Unit) }, + // In case the algorithm decided to suspend, the element + // was added to the buffer. However, as the buffer is now + // overflowed, the first (oldest) element has to be extracted. + onSuspend = { segm, i -> + dropFirstElementUntilTheSpecifiedCellIsInTheBuffer(segm.id * SEGMENT_SIZE + i) + return success(Unit) + }, + // If the channel is closed, return the corresponding result. + onClosed = { return closed(sendException) } + ) + + private inline fun sendImplOnNoWaiter( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The element to be sent. */ + element: E, + /* The global index of the cell. */ + s: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Waiter, + /* This lambda is invoked when the element has been + buffered or a rendezvous with a receiver happens.*/ + onRendezvousOrBuffered: () -> Unit, + /* This lambda is called when the channel + is observed in the closed state. */ + onClosed: () -> Unit, + ) { + // Update the cell again, now with the non-null waiter, + // restarting the operation from the beginning on failure. + // Check the `sendImpl(..)` function for the comments. + when (updateCellSend(segment, index, element, s, waiter, false)) { + RESULT_RENDEZVOUS -> { + segment.cleanPrev() + onRendezvousOrBuffered() + } + RESULT_BUFFERED -> { + onRendezvousOrBuffered() + } + RESULT_SUSPEND -> { + waiter.prepareSenderForSuspension(segment, index) + } + RESULT_CLOSED -> { + if (s < receiversCounter) segment.cleanPrev() + onClosed() + } + RESULT_FAILED -> { + segment.cleanPrev() + sendImpl( + element = element, + waiter = waiter, + onRendezvousOrBuffered = onRendezvousOrBuffered, + onSuspend = { _, _ -> }, + onClosed = onClosed, + ) + } + else -> error("unexpected") + } + } + + private fun updateCellSend( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The element to be sent. */ + element: E, + /* The global index of the cell. */ + s: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Any?, + closed: Boolean + ): Int { + // This is a fast-path of `updateCellSendSlow(..)`. + // + // First, the algorithm stores the element, + // performing the synchronization after that. + // This way, receivers safely retrieve the + // element, following the safe publication pattern. + segment.storeElement(index, element) + if (closed) return updateCellSendSlow(segment, index, element, s, waiter, closed) + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty. + state === null -> { + // If the element should be buffered, or a rendezvous should happen + // while the receiver is still coming, try to buffer the element. + // Otherwise, try to store the specified waiter in the cell. + if (bufferOrRendezvousSend(s)) { + // Move the cell state to `BUFFERED`. + if (segment.casState(index, null, BUFFERED)) { + // The element has been successfully buffered, finish. + return RESULT_BUFFERED + } + } else { + // This `send(e)` operation should suspend. + // However, in case the channel has already + // been observed closed, `INTERRUPTED_SEND` + // is installed instead. + if (waiter == null) { + // The waiter is not specified; return the corresponding result. + return RESULT_SUSPEND_NO_WAITER + } else { + // Try to install the waiter. + if (segment.casState(index, null, waiter)) return RESULT_SUSPEND + } + } + } + // A waiting receiver is stored in the cell. + state is Waiter -> { + // As the element will be passed directly to the waiter, + // the algorithm cleans the element slot in the cell. + segment.cleanElement(index) + // Try to make a rendezvous with the suspended receiver. + return if (state.tryResumeReceiver(element)) { + // Rendezvous! Move the cell state to `DONE_RCV` and finish. + segment.setState(index, DONE_RCV) + onReceiveDequeued() + RESULT_RENDEZVOUS + } else { + // The resumption has failed. Update the cell state correspondingly + // and clean the element field. It is also possible for a concurrent + // cancellation handler to update the cell state; we can safely + // ignore these updates. + if (segment.getAndSetState(index, INTERRUPTED_RCV) !== INTERRUPTED_RCV) { + segment.onCancelledRequest(index, true) + } + RESULT_FAILED + } + } + } + return updateCellSendSlow(segment, index, element, s, waiter, closed) + } + + /** + * Updates the working cell of an abstract send operation. + */ + private fun updateCellSendSlow( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The element to be sent. */ + element: E, + /* The global index of the cell. */ + s: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Any?, + closed: Boolean + ): Int { + // Then, the cell state should be updated according to + // its state machine; see the paper mentioned in the very + // beginning for the cell life-cycle and the algorithm details. + while (true) { + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty. + state === null -> { + // If the element should be buffered, or a rendezvous should happen + // while the receiver is still coming, try to buffer the element. + // Otherwise, try to store the specified waiter in the cell. + if (bufferOrRendezvousSend(s) && !closed) { + // Move the cell state to `BUFFERED`. + if (segment.casState(index, null, BUFFERED)) { + // The element has been successfully buffered, finish. + return RESULT_BUFFERED + } + } else { + // This `send(e)` operation should suspend. + // However, in case the channel has already + // been observed closed, `INTERRUPTED_SEND` + // is installed instead. + when { + // The channel is closed + closed -> if (segment.casState(index, null, INTERRUPTED_SEND)) { + segment.onCancelledRequest(index, false) + return RESULT_CLOSED + } + // The waiter is not specified; return the corresponding result. + waiter == null -> return RESULT_SUSPEND_NO_WAITER + // Try to install the waiter. + else -> if (segment.casState(index, null, waiter)) return RESULT_SUSPEND + } + } + } + // This cell is in the logical buffer. + state === IN_BUFFER -> { + // Try to buffer the element. + if (segment.casState(index, state, BUFFERED)) { + // The element has been successfully buffered, finish. + return RESULT_BUFFERED + } + } + // The cell stores a cancelled receiver. + state === INTERRUPTED_RCV -> { + // Clean the element slot to avoid memory leaks and finish. + segment.cleanElement(index) + return RESULT_FAILED + } + // The cell is poisoned by a concurrent receive. + state === POISONED -> { + // Clean the element slot to avoid memory leaks and finish. + segment.cleanElement(index) + return RESULT_FAILED + } + // The channel is already closed. + state === CHANNEL_CLOSED -> { + // Clean the element slot to avoid memory leaks, + // ensure that the closing/cancellation procedure + // has been completed, and finish. + segment.cleanElement(index) + completeCloseOrCancel() + return RESULT_CLOSED + } + // A waiting receiver is stored in the cell. + else -> { + assert { state is Waiter || state is WaiterEB } + // As the element will be passed directly to the waiter, + // the algorithm cleans the element slot in the cell. + segment.cleanElement(index) + // Unwrap the waiting receiver from `WaiterEB` if needed. + // As a receiver is stored in the cell, the buffer expansion + // procedure would finish, so senders simply ignore the "EB" marker. + val receiver = if (state is WaiterEB) state.waiter else state + // Try to make a rendezvous with the suspended receiver. + return if (receiver.tryResumeReceiver(element)) { + // Rendezvous! Move the cell state to `DONE_RCV` and finish. + segment.setState(index, DONE_RCV) + onReceiveDequeued() + RESULT_RENDEZVOUS + } else { + // The resumption has failed. Update the cell state correspondingly + // and clean the element field. It is also possible for a concurrent + // `expandBuffer()` or the cancellation handler to update the cell state; + // we can safely ignore these updates as senders do not help `expandBuffer()`. + if (segment.getAndSetState(index, INTERRUPTED_RCV) !== INTERRUPTED_RCV) { + segment.onCancelledRequest(index, true) + } + RESULT_FAILED + } + } + } + } + } + + /** + * Checks whether a [send] invocation is bound to suspend if it is called + * with the specified [sendersAndCloseStatus], [receivers], and [bufferEnd] + * values. When this channel is already closed, the function returns `false`. + * + * Specifically, [send] suspends if the channel is not unlimited, + * the number of receivers is greater than then index of the working cell of the + * potential [send] invocation, and the buffer does not cover this cell + * in case of buffered channel. + * When the channel is already closed, [send] does not suspend. + */ + @JsName("shouldSendSuspend0") + private fun shouldSendSuspend(curSendersAndCloseStatus: Long): Boolean { + // Does not suspend if the channel is already closed. + if (curSendersAndCloseStatus.isClosedForSend0) return false + // Does not suspend if a rendezvous may happen or the buffer is not full. + return !bufferOrRendezvousSend(curSendersAndCloseStatus.sendersCounter) + } + + /** + * Returns `true` when the specified [send] should place + * its element to the working cell without suspension. + */ + private fun bufferOrRendezvousSend(curSenders: Long): Boolean = + curSenders < bufferEndCounter || curSenders < receiversCounter + capacity + + /** + * Checks whether a [send] invocation is bound to suspend if it is called + * with the current counter and close status values. See [shouldSendSuspend] for details. + * + * Note that this implementation is _false positive_ in case of rendezvous channels, + * so it can return `false` when a [send] invocation is bound to suspend. Specifically, + * the counter of `receive()` operations may indicate that there is a waiting receiver, + * while it has already been cancelled, so the potential rendezvous is bound to fail. + */ + internal open fun shouldSendSuspend(): Boolean = shouldSendSuspend(sendersAndCloseStatus.value) + + /** + * Tries to resume this receiver with the specified [element] as a result. + * Returns `true` on success and `false` otherwise. + */ + @Suppress("UNCHECKED_CAST") + private fun Any.tryResumeReceiver(element: E): Boolean = when(this) { + is SelectInstance<*> -> { // `onReceiveXXX` select clause + trySelect(this@BufferedChannel, element) + } + is ReceiveCatching<*> -> { + this as ReceiveCatching + cont.tryResume0(success(element), onUndeliveredElement?.bindCancellationFunResult()) + } + is BufferedChannel<*>.BufferedChannelIterator -> { + this as BufferedChannel.BufferedChannelIterator + tryResumeHasNext(element) + } + is CancellableContinuation<*> -> { // `receive()` + this as CancellableContinuation + tryResume0(element, onUndeliveredElement?.bindCancellationFun()) + } + else -> error("Unexpected receiver type: $this") + } + + // ########################## + // # The receive operations # + // ########################## + + /** + * This function is invoked when a receiver is added as a waiter in this channel. + */ + protected open fun onReceiveEnqueued() {} + + /** + * This function is invoked when a waiting receiver is no longer stored in this channel; + * independently on whether it is caused by rendezvous, cancellation, or channel closing. + */ + protected open fun onReceiveDequeued() {} + + override suspend fun receive(): E = + receiveImpl( // <-- this is an inline function + // Do not create a continuation until it is required; + // it is created later via [onNoWaiterSuspend], if needed. + waiter = null, + // Return the received element on successful retrieval from + // the buffer or rendezvous with a suspended sender. + // Also, inform `BufferedChannel` extensions that + // synchronization of this receive operation is completed. + onElementRetrieved = { element -> + return element + }, + // As no waiter is provided, suspension is impossible. + onSuspend = { _, _, _ -> error("unexpected") }, + // Throw an exception if the channel is already closed. + onClosed = { throw recoverStackTrace(receiveException) }, + // If `receive()` decides to suspend, the corresponding + // `suspend` function that creates a continuation is called. + // The tail-call optimization is applied here. + onNoWaiterSuspend = { segm, i, r -> receiveOnNoWaiterSuspend(segm, i, r) } + ) + + private suspend fun receiveOnNoWaiterSuspend( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The global index of the cell. */ + r: Long + ) = suspendCancellableCoroutineReusable { cont -> + receiveImplOnNoWaiter( // <-- this is an inline function + segment = segment, index = index, r = r, + // Store the created continuation as a waiter. + waiter = cont, + // In case of successful element retrieval, resume + // the continuation with the element and inform the + // `BufferedChannel` extensions that the synchronization + // is completed. Importantly, the receiver coroutine + // may be cancelled after it is successfully resumed but + // not dispatched yet. In case `onUndeliveredElement` is + // specified, we need to invoke it in the latter case. + onElementRetrieved = { element -> + val onCancellation = onUndeliveredElement?.bindCancellationFun() + cont.resume(element, onCancellation) + }, + onClosed = { onClosedReceiveOnNoWaiterSuspend(cont) }, + ) + } + + private fun Waiter.prepareReceiverForSuspension(segment: ChannelSegment, index: Int) { + onReceiveEnqueued() + invokeOnCancellation(segment, index) + } + + private fun onClosedReceiveOnNoWaiterSuspend(cont: CancellableContinuation) { + cont.resumeWithException(receiveException) + } + + /* + The implementation is exactly the same as of `receive()`, + with the only difference that this function returns a `ChannelResult` + instance and does not throw exception explicitly in case the channel + is already closed for receiving. Please refer the plain `receive()` + implementation for the comments. + */ + override suspend fun receiveCatching(): ChannelResult = + receiveImpl( // <-- this is an inline function + waiter = null, + onElementRetrieved = { element -> + success(element) + }, + onSuspend = { _, _, _ -> error("unexpected") }, + onClosed = { closed(closeCause) }, + onNoWaiterSuspend = { segm, i, r -> receiveCatchingOnNoWaiterSuspend(segm, i, r) } + ) + + private suspend fun receiveCatchingOnNoWaiterSuspend( + segment: ChannelSegment, + index: Int, + r: Long + ) = suspendCancellableCoroutineReusable { cont -> + val waiter = ReceiveCatching(cont as CancellableContinuationImpl>) + receiveImplOnNoWaiter( + segment, index, r, + waiter = waiter, + onElementRetrieved = { element -> + cont.resume(success(element), onUndeliveredElement?.bindCancellationFunResult()) + }, + onClosed = { onClosedReceiveCatchingOnNoWaiterSuspend(cont) } + ) + } + + private fun onClosedReceiveCatchingOnNoWaiterSuspend(cont: CancellableContinuation>) { + cont.resume(closed(closeCause)) + } + + override fun tryReceive(): ChannelResult { + // Read the `receivers` counter first. + val r = receivers.value + val sendersAndCloseStatusCur = sendersAndCloseStatus.value + // Is this channel closed for receive? + if (sendersAndCloseStatusCur.isClosedForReceive0) { + return closed(closeCause) + } + // Do not try to receive an element if the plain `receive()` operation would suspend. + val s = sendersAndCloseStatusCur.sendersCounter + if (r >= s) return failure() + // Let's try to retrieve an element! + // The logic is similar to the plain `receive()` operation, with + // the only difference that we store `INTERRUPTED_RCV` in case + // the operation decides to suspend. This way, we can leverage + // the unconditional `Fetch-and-Add` instruction. + // One may consider storing `INTERRUPTED_RCV` instead of an actual waiter + // on suspension (a.k.a. "no elements to retrieve") as a short-cut of + // "suspending and cancelling immediately". + return receiveImpl( // <-- this is an inline function + // Store an already interrupted receiver in case of suspension. + waiter = INTERRUPTED_RCV, + // Finish when an element is successfully retrieved. + onElementRetrieved = { element -> success(element) }, + // On suspension, the `INTERRUPTED_RCV` token has been + // installed, and this `tryReceive()` must fail. + onSuspend = { segm, _, globalIndex -> + // Emulate "cancelled" receive, thus invoking 'waitExpandBufferCompletion' manually, + // because effectively there were no cancellation + waitExpandBufferCompletion(globalIndex) + segm.onSlotCleaned() + failure() + }, + // If the channel is closed, return the corresponding result. + onClosed = { closed(closeCause) } + ) + } + + /** + * Extracts the first element from this channel until the cell with the specified + * index is moved to the logical buffer. This is a key procedure for the _conflated_ + * channel implementation, see [ConflatedBufferedChannel] with the [BufferOverflow.DROP_OLDEST] + * strategy on buffer overflowing. + */ + protected fun dropFirstElementUntilTheSpecifiedCellIsInTheBuffer(globalCellIndex: Long) { + assert { isConflatedDropOldest } + // Read the segment reference before the counter increment; + // it is crucial to be able to find the required segment later. + var segment = receiveSegment.value + while (true) { + // Read the receivers counter to check whether the specified cell is already in the buffer + // or should be moved to the buffer in a short time, due to the already started `receive()`. + val r = this.receivers.value + if (globalCellIndex < max(r + capacity, bufferEndCounter)) return + // The cell is outside the buffer. Try to extract the first element + // if the `receivers` counter has not been changed. + if (!this.receivers.compareAndSet(r, r + 1)) continue + // Count the required segment id and the cell index in it. + val id = r / SEGMENT_SIZE + val i = (r % SEGMENT_SIZE).toInt() + // Try to find the required segment if the initially obtained + // segment (in the beginning of this function) has lower id. + if (segment.id != id) { + // Find the required segment, restarting the operation if it has not been found. + segment = findSegmentReceive(id, segment) ?: + // The required segment has not been found. It is possible that the channel is already + // closed for receiving, so the linked list of segments is closed as well. + // In the latter case, the operation will finish eventually after incrementing + // the `receivers` counter sufficient times. Note that it is impossible to check + // whether this channel is closed for receiving (we do this in `receive`), + // as it may call this function when helping to complete closing the channel. + continue + } + // Update the cell according to the cell life-cycle. + val updCellResult = updateCellReceive(segment, i, r, null) + when { + updCellResult === FAILED -> { + // The cell is poisoned; restart from the beginning. + // To avoid memory leaks, we also need to reset + // the `prev` pointer of the working segment. + if (r < sendersCounter) segment.cleanPrev() + } + else -> { // element + // A buffered element was retrieved from the cell. + // Clean the reference to the previous segment. + segment.cleanPrev() + @Suppress("UNCHECKED_CAST") + onUndeliveredElement?.callUndeliveredElementCatchingException(updCellResult as E)?.let { throw it } + } + } + } + } + + /** + * Abstract receive implementation. + */ + private inline fun receiveImpl( + /* The waiter to be stored in case of suspension, + or `null` if the waiter is not created yet. + In the latter case, if the algorithm decides + to suspend, [onNoWaiterSuspend] is called. */ + waiter: Any?, + /* This lambda is invoked when an element has been + successfully retrieved, either from the buffer or + by making a rendezvous with a suspended sender. */ + onElementRetrieved: (element: E) -> R, + /* This lambda is called when the operation suspends in the cell + specified by the segment and its global and in-segment indices. */ + onSuspend: (segm: ChannelSegment, i: Int, r: Long) -> R, + /* This lambda is called when the channel is observed + in the closed state and no waiting sender is found, + which means that it is closed for receiving. */ + onClosed: () -> R, + /* This lambda is called when the operation decides + to suspend, but the waiter is not provided (equals `null`). + It should create a waiter and delegate to `sendImplOnNoWaiter`. */ + onNoWaiterSuspend: ( + segm: ChannelSegment, + i: Int, + r: Long + ) -> R = { _, _, _ -> error("unexpected") } + ): R { + // Read the segment reference before the counter increment; + // it is crucial to be able to find the required segment later. + var segment = receiveSegment.value + while (true) { + // Similar to the `send(e)` operation, `receive()` first checks + // whether the channel is already closed for receiving. + if (isClosedForReceive) return onClosed() + // Atomically increments the `receivers` counter + // and obtain the value right before the increment. + val r = this.receivers.getAndIncrement() + // Count the required segment id and the cell index in it. + val id = r / SEGMENT_SIZE + val i = (r % SEGMENT_SIZE).toInt() + // Try to find the required segment if the initially obtained + // segment (in the beginning of this function) has lower id. + if (segment.id != id) { + // Find the required segment, restarting the operation if it has not been found. + segment = findSegmentReceive(id, segment) ?: + // The required segment is not found. It is possible that the channel is already + // closed for receiving, so the linked list of segments is closed as well. + // In the latter case, the operation fails with the corresponding check at the beginning. + continue + } + // Update the cell according to the cell life-cycle. + val updCellResult = updateCellReceive(segment, i, r, waiter) + return when { + updCellResult === SUSPEND -> { + // The operation has decided to suspend and + // stored the specified waiter in the cell. + (waiter as? Waiter)?.prepareReceiverForSuspension(segment, i) + onSuspend(segment, i, r) + } + updCellResult === FAILED -> { + // The operation has tried to make a rendezvous + // but failed: either the opposite request has + // already been cancelled or the cell is poisoned. + // Restart from the beginning in this case. + // To avoid memory leaks, we also need to reset + // the `prev` pointer of the working segment. + if (r < sendersCounter) segment.cleanPrev() + continue + } + updCellResult === SUSPEND_NO_WAITER -> { + // The operation has decided to suspend, + // but no waiter has been provided. + onNoWaiterSuspend(segment, i, r) + } + else -> { // element + // Either a buffered element was retrieved from the cell + // or a rendezvous with a waiting sender has happened. + // Clean the reference to the previous segment before finishing. + segment.cleanPrev() + @Suppress("UNCHECKED_CAST") + onElementRetrieved(updCellResult as E) + } + } + } + } + + private inline fun receiveImplOnNoWaiter( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The global index of the cell. */ + r: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Waiter, + /* This lambda is invoked when an element has been + successfully retrieved, either from the buffer or + by making a rendezvous with a suspended sender. */ + onElementRetrieved: (element: E) -> Unit, + /* This lambda is called when the channel is observed + in the closed state and no waiting senders is found, + which means that it is closed for receiving. */ + onClosed: () -> Unit + ) { + // Update the cell with the non-null waiter, + // restarting from the beginning on failure. + // Check the `receiveImpl(..)` function for the comments. + val updCellResult = updateCellReceive(segment, index, r, waiter) + when { + updCellResult === SUSPEND -> { + waiter.prepareReceiverForSuspension(segment, index) + } + updCellResult === FAILED -> { + if (r < sendersCounter) segment.cleanPrev() + receiveImpl( + waiter = waiter, + onElementRetrieved = onElementRetrieved, + onSuspend = { _, _, _ -> }, + onClosed = onClosed + ) + } + else -> { + segment.cleanPrev() + @Suppress("UNCHECKED_CAST") + onElementRetrieved(updCellResult as E) + } + } + } + + private fun updateCellReceive( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The global index of the cell. */ + r: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Any?, + ): Any? { + // This is a fast-path of `updateCellReceiveSlow(..)`. + // + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty. + state === null -> { + // If a rendezvous must happen, the operation does not wait + // until the cell stores a buffered element or a suspended + // sender, poisoning the cell and restarting instead. + // Otherwise, try to store the specified waiter in the cell. + val senders = sendersAndCloseStatus.value.sendersCounter + if (r >= senders) { + // This `receive()` operation should suspend. + if (waiter === null) { + // The waiter is not specified; + // return the corresponding result. + return SUSPEND_NO_WAITER + } + // Try to install the waiter. + if (segment.casState(index, state, waiter)) { + // The waiter has been successfully installed. + // Invoke the `expandBuffer()` procedure and finish. + expandBuffer() + return SUSPEND + } + } + } + // The cell stores a buffered element. + state === BUFFERED -> if (segment.casState(index, state, DONE_RCV)) { + // Retrieve the element and expand the buffer. + expandBuffer() + return segment.retrieveElement(index) + } + } + return updateCellReceiveSlow(segment, index, r, waiter) + } + + private fun updateCellReceiveSlow( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The global index of the cell. */ + r: Long, + /* The waiter to be stored in case of suspension. */ + waiter: Any?, + ): Any? { + // The cell state should be updated according to its state machine; + // see the paper mentioned in the very beginning for the algorithm details. + while (true) { + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty. + state === null || state === IN_BUFFER -> { + // If a rendezvous must happen, the operation does not wait + // until the cell stores a buffered element or a suspended + // sender, poisoning the cell and restarting instead. + // Otherwise, try to store the specified waiter in the cell. + val senders = sendersAndCloseStatus.value.sendersCounter + if (r < senders) { + // The cell is already covered by sender, + // so a rendezvous must happen. Unfortunately, + // the cell is empty, so the operation poisons it. + if (segment.casState(index, state, POISONED)) { + // When the cell becomes poisoned, it is essentially + // the same as storing an already cancelled receiver. + // Thus, the `expandBuffer()` procedure should be invoked. + expandBuffer() + return FAILED + } + } else { + // This `receive()` operation should suspend. + if (waiter === null) { + // The waiter is not specified; + // return the corresponding result. + return SUSPEND_NO_WAITER + } + // Try to install the waiter. + if (segment.casState(index, state, waiter)) { + // The waiter has been successfully installed. + // Invoke the `expandBuffer()` procedure and finish. + expandBuffer() + return SUSPEND + } + } + } + // The cell stores a buffered element. + state === BUFFERED -> if (segment.casState(index, state, DONE_RCV)) { + // Retrieve the element and expand the buffer. + expandBuffer() + return segment.retrieveElement(index) + } + // The cell stores an interrupted sender. + state === INTERRUPTED_SEND -> return FAILED + // The cell is already poisoned by a concurrent + // `hasElements` call. Restart in this case. + state === POISONED -> return FAILED + // This channel is already closed. + state === CHANNEL_CLOSED -> { + // Although the channel is closed, it is still required + // to call the `expandBuffer()` procedure to keep + // `waitForExpandBufferCompletion()` correct. + expandBuffer() + return FAILED + } + // A concurrent `expandBuffer()` is resuming a + // suspended sender. Wait in a spin-loop until + // the resumption attempt completes: the cell + // state must change to either `BUFFERED` or + // `INTERRUPTED_SEND`. + state === RESUMING_BY_EB -> continue + // The cell stores a suspended sender; try to resume it. + else -> { + // To synchronize with expandBuffer(), the algorithm + // first moves the cell to an intermediate `S_RESUMING_BY_RCV` + // state, updating it to either `BUFFERED` (on success) or + // `INTERRUPTED_SEND` (on failure). + if (segment.casState(index, state, RESUMING_BY_RCV)) { + // Has a concurrent `expandBuffer()` delegated its completion? + val helpExpandBuffer = state is WaiterEB + // Extract the sender if needed and try to resume it. + val sender = if (state is WaiterEB) state.waiter else state + return if (sender.tryResumeSender(segment, index)) { + // The sender has been resumed successfully! + // Update the cell state correspondingly, + // expand the buffer, and return the element + // stored in the cell. + // In case a concurrent `expandBuffer()` has delegated + // its completion, the procedure should finish, as the + // sender is resumed. Thus, no further action is required. + segment.setState(index, DONE_RCV) + expandBuffer() + segment.retrieveElement(index) + } else { + // The resumption has failed. Update the cell correspondingly. + // In case a concurrent `expandBuffer()` has delegated + // its completion, the procedure should skip this cell, so + // `expandBuffer()` should be called once again. + segment.setState(index, INTERRUPTED_SEND) + segment.onCancelledRequest(index, false) + if (helpExpandBuffer) expandBuffer() + FAILED + } + } + } + } + } + } + + private fun Any.tryResumeSender(segment: ChannelSegment, index: Int): Boolean = when (this) { + is CancellableContinuation<*> -> { // suspended `send(e)` operation + @Suppress("UNCHECKED_CAST") + this as CancellableContinuation + tryResume0(Unit) + } + is SelectInstance<*> -> { + this as SelectImplementation<*> + val trySelectResult = trySelectDetailed(clauseObject = this@BufferedChannel, result = Unit) + // Clean the element slot to avoid memory leaks + // if this `select` clause should be re-registered. + if (trySelectResult === REREGISTER) segment.cleanElement(index) + // Was the resumption successful? + trySelectResult === SUCCESSFUL + } + is SendBroadcast -> cont.tryResume0(true) // // suspended `sendBroadcast(e)` operation + else -> error("Unexpected waiter: $this") + } + + // ################################ + // # The expandBuffer() procedure # + // ################################ + + private fun expandBuffer() { + // Do not need to take any action if + // this channel is rendezvous or unlimited. + if (isRendezvousOrUnlimited) return + // Read the current segment of + // the `expandBuffer()` procedure. + var segment = bufferEndSegment.value + // Try to expand the buffer until succeed. + try_again@ while (true) { + // Increment the logical end of the buffer. + // The `b`-th cell is going to be added to the buffer. + val b = bufferEnd.getAndIncrement() + val id = b / SEGMENT_SIZE + // After that, read the current `senders` counter. + // In case its value is lower than `b`, the `send(e)` + // invocation that will work with this `b`-th cell + // will detect that the cell is already a part of the + // buffer when comparing with the `bufferEnd` counter. + // However, `bufferEndSegment` may reference an outdated + // segment, which should be updated to avoid memory leaks. + val s = sendersCounter + if (s <= b) { + // Should `bufferEndSegment` be moved forward to avoid memory leaks? + if (segment.id < id && segment.next != null) + moveSegmentBufferEndToSpecifiedOrLast(id, segment) + // Increment the number of completed `expandBuffer()`-s and finish. + incCompletedExpandBufferAttempts() + return + } + // Is `bufferEndSegment` outdated or is the segment with the required id already removed? + // Find the required segment, creating new ones if needed. + if (segment.id != id) { + segment = findSegmentBufferEnd(id, segment, b) + // Restart if the required segment is removed, or + // the linked list of segments is already closed, + // and the required one will never be created. + // Please note that `findSegmentBuffer(..)` updates + // the number of completed `expandBuffer()` attempt + // in this case. + ?: continue@try_again + } + // Try to add the cell to the logical buffer, + // updating the cell state according to the state-machine. + val i = (b % SEGMENT_SIZE).toInt() + if (updateCellExpandBuffer(segment, i, b)) { + // The cell has been added to the logical buffer! + // Increment the number of completed `expandBuffer()`-s and finish. + // + // Note that it is possible to increment the number of + // completed `expandBuffer()` attempts earlier, right + // after the segment is obtained. We find this change + // counter-intuitive and prefer to avoid it. + incCompletedExpandBufferAttempts() + return + } else { + // The cell has not been added to the buffer. + // Increment the number of completed `expandBuffer()` + // attempts and restart. + incCompletedExpandBufferAttempts() + continue@try_again + } + } + } + + private fun updateCellExpandBuffer( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The global index of the cell. */ + b: Long + ): Boolean { + // This is a fast-path of `updateCellExpandBufferSlow(..)`. + // + // Read the current cell state. + val state = segment.getState(index) + if (state is Waiter) { + // Usually, a sender is stored in the cell. + // However, it is possible for a concurrent + // receiver to be already suspended there. + // Try to distinguish whether the waiter is a + // sender by comparing the global cell index with + // the `receivers` counter. In case the cell is not + // covered by a receiver, a sender is stored in the cell. + if (b >= receivers.value) { + // The cell stores a suspended sender. Try to resume it. + // To synchronize with a concurrent `receive()`, the algorithm + // first moves the cell state to an intermediate `RESUMING_BY_EB` + // state, updating it to either `BUFFERED` (on successful resumption) + // or `INTERRUPTED_SEND` (on failure). + if (segment.casState(index, state, RESUMING_BY_EB)) { + return if (state.tryResumeSender(segment, index)) { + // The sender has been resumed successfully! + // Move the cell to the logical buffer and finish. + segment.setState(index, BUFFERED) + true + } else { + // The resumption has failed. + segment.setState(index, INTERRUPTED_SEND) + segment.onCancelledRequest(index, false) + false + } + } + } + } + return updateCellExpandBufferSlow(segment, index, b) + } + + private fun updateCellExpandBufferSlow( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The global index of the cell. */ + b: Long + ): Boolean { + // Update the cell state according to its state machine. + // See the paper mentioned in the very beginning for + // the cell life-cycle and the algorithm details. + while (true) { + // Read the current cell state. + val state = segment.getState(index) + when { + // A suspended waiter, sender or receiver. + state is Waiter -> { + // Usually, a sender is stored in the cell. + // However, it is possible for a concurrent + // receiver to be already suspended there. + // Try to distinguish whether the waiter is a + // sender by comparing the global cell index with + // the `receivers` counter. In case the cell is not + // covered by a receiver, a sender is stored in the cell. + if (b < receivers.value) { + // The algorithm cannot distinguish whether the + // suspended in the cell operation is sender or receiver. + // To make progress, `expandBuffer()` delegates its completion + // to an upcoming pairwise request, atomically wrapping + // the waiter in `WaiterEB`. In case a sender is stored + // in the cell, the upcoming receiver will call `expandBuffer()` + // if the sender resumption fails; thus, effectively, skipping + // this cell. Otherwise, if a receiver is stored in the cell, + // this `expandBuffer()` procedure must finish; therefore, + // sender ignore the `WaiterEB` wrapper. + if (segment.casState(index, state, WaiterEB(waiter = state))) + return true + } else { + // The cell stores a suspended sender. Try to resume it. + // To synchronize with a concurrent `receive()`, the algorithm + // first moves the cell state to an intermediate `RESUMING_BY_EB` + // state, updating it to either `BUFFERED` (on successful resumption) + // or `INTERRUPTED_SEND` (on failure). + if (segment.casState(index, state, RESUMING_BY_EB)) { + return if (state.tryResumeSender(segment, index)) { + // The sender has been resumed successfully! + // Move the cell to the logical buffer and finish. + segment.setState(index, BUFFERED) + true + } else { + // The resumption has failed. + segment.setState(index, INTERRUPTED_SEND) + segment.onCancelledRequest(index, false) + false + } + } + } + } + // The cell stores an interrupted sender, skip it. + state === INTERRUPTED_SEND -> return false + // The cell is empty, a concurrent sender is coming. + state === null -> { + // To inform a concurrent sender that this cell is + // already a part of the buffer, the algorithm moves + // it to a special `IN_BUFFER` state. + if (segment.casState(index, state, IN_BUFFER)) return true + } + // The cell is already a part of the buffer, finish. + state === BUFFERED -> return true + // The cell is already processed by a receiver, no further action is required. + state === POISONED || state === DONE_RCV || state === INTERRUPTED_RCV -> return true + // The channel is closed, all the following + // cells are already in the same state, finish. + state === CHANNEL_CLOSED -> return true + // A concurrent receiver is resuming the suspended sender. + // Wait in a spin-loop until it changes the cell state + // to either `DONE_RCV` or `INTERRUPTED_SEND`. + state === RESUMING_BY_RCV -> continue // spin wait + else -> error("Unexpected cell state: $state") + } + } + } + + /** + * Increments the counter of completed [expandBuffer] invocations. + * To guarantee starvation-freedom for [waitExpandBufferCompletion], + * which waits until the counters of started and completed [expandBuffer] calls + * coincide and become greater or equal to the specified value, + * [waitExpandBufferCompletion] may set a flag that pauses further progress. + */ + private fun incCompletedExpandBufferAttempts(nAttempts: Long = 1) { + // Increment the number of completed `expandBuffer()` calls. + completedExpandBuffersAndPauseFlag.addAndGet(nAttempts).also { + // Should further `expandBuffer()`-s be paused? + // If so, this thread should wait in a spin-loop + // until the flag is unset. + if (it.ebPauseExpandBuffers) { + @Suppress("ControlFlowWithEmptyBody") + while (completedExpandBuffersAndPauseFlag.value.ebPauseExpandBuffers) {} + } + } + } + + /** + * Waits in a spin-loop until the [expandBuffer] call that + * should process the [globalIndex]-th cell is completed. + * Essentially, it waits until the numbers of started ([bufferEnd]) + * and completed ([completedExpandBuffersAndPauseFlag]) [expandBuffer] + * attempts coincide and become equal or greater than [globalIndex]. + * To avoid starvation, this function may set a flag + * that pauses further progress. + */ + internal fun waitExpandBufferCompletion(globalIndex: Long) { + // Do nothing if this channel is rendezvous or unlimited; + // `expandBuffer()` is not used in these cases. + if (isRendezvousOrUnlimited) return + // Wait in an infinite loop until the number of started + // buffer expansion calls become not lower than the cell index. + @Suppress("ControlFlowWithEmptyBody") + while (bufferEndCounter <= globalIndex) {} + // Now it is guaranteed that the `expandBuffer()` call that + // should process the required cell has been started. + // Wait in a fixed-size spin-loop until the numbers of + // started and completed buffer expansion calls coincide. + repeat(EXPAND_BUFFER_COMPLETION_WAIT_ITERATIONS) { + // Read the number of started buffer expansion calls. + val b = bufferEndCounter + // Read the number of completed buffer expansion calls. + val ebCompleted = completedExpandBuffersAndPauseFlag.value.ebCompletedCounter + // Do the numbers of started and completed calls coincide? + // Note that we need to re-read the number of started `expandBuffer()` + // calls to obtain a correct snapshot. + // Here we wait to a precise match in order to ensure that **our matching expandBuffer()** + // completed. The only way to ensure that is to check that number of started expands == number of finished expands + if (b == ebCompleted && b == bufferEndCounter) return + } + // To avoid starvation, pause further `expandBuffer()` calls. + completedExpandBuffersAndPauseFlag.update { + constructEBCompletedAndPauseFlag(it.ebCompletedCounter, true) + } + // Now wait in an infinite spin-loop until the counters coincide. + while (true) { + // Read the number of started buffer expansion calls. + val b = bufferEndCounter + // Read the number of completed buffer expansion calls + // along with the flag that pauses further progress. + val ebCompletedAndBit = completedExpandBuffersAndPauseFlag.value + val ebCompleted = ebCompletedAndBit.ebCompletedCounter + val pauseExpandBuffers = ebCompletedAndBit.ebPauseExpandBuffers + // Do the numbers of started and completed calls coincide? + // Note that we need to re-read the number of started `expandBuffer()` + // calls to obtain a correct snapshot. + if (b == ebCompleted && b == bufferEndCounter) { + // Unset the flag, which pauses progress, and finish. + completedExpandBuffersAndPauseFlag.update { + constructEBCompletedAndPauseFlag(it.ebCompletedCounter, false) + } + return + } + // It is possible that a concurrent caller of this function + // has unset the flag, which pauses further progress to avoid + // starvation. In this case, set the flag back. + if (!pauseExpandBuffers) { + completedExpandBuffersAndPauseFlag.compareAndSet( + ebCompletedAndBit, + constructEBCompletedAndPauseFlag(ebCompleted, true) + ) + } + } + } + + + // ####################### + // ## Select Expression ## + // ####################### + + @Suppress("UNCHECKED_CAST") + override val onSend: SelectClause2> + get() = SelectClause2Impl( + clauseObject = this@BufferedChannel, + regFunc = BufferedChannel<*>::registerSelectForSend as RegistrationFunction, + processResFunc = BufferedChannel<*>::processResultSelectSend as ProcessResultFunction + ) + + @Suppress("UNCHECKED_CAST") + protected open fun registerSelectForSend(select: SelectInstance<*>, element: Any?) = + sendImpl( // <-- this is an inline function + element = element as E, + waiter = select, + onRendezvousOrBuffered = { select.selectInRegistrationPhase(Unit) }, + onSuspend = { _, _ -> }, + onClosed = { onClosedSelectOnSend(element, select) } + ) + + + private fun onClosedSelectOnSend(element: E, select: SelectInstance<*>) { + onUndeliveredElement?.callUndeliveredElement(element, select.context) + select.selectInRegistrationPhase(CHANNEL_CLOSED) + } + + @Suppress("UNUSED_PARAMETER", "RedundantNullableReturnType") + private fun processResultSelectSend(ignoredParam: Any?, selectResult: Any?): Any? = + if (selectResult === CHANNEL_CLOSED) throw sendException + else this + + @Suppress("UNCHECKED_CAST") + override val onReceive: SelectClause1 + get() = SelectClause1Impl( + clauseObject = this@BufferedChannel, + regFunc = BufferedChannel<*>::registerSelectForReceive as RegistrationFunction, + processResFunc = BufferedChannel<*>::processResultSelectReceive as ProcessResultFunction, + onCancellationConstructor = onUndeliveredElementReceiveCancellationConstructor + ) + + @Suppress("UNCHECKED_CAST") + override val onReceiveCatching: SelectClause1> + get() = SelectClause1Impl( + clauseObject = this@BufferedChannel, + regFunc = BufferedChannel<*>::registerSelectForReceive as RegistrationFunction, + processResFunc = BufferedChannel<*>::processResultSelectReceiveCatching as ProcessResultFunction, + onCancellationConstructor = onUndeliveredElementReceiveCancellationConstructor + ) + + @Suppress("OVERRIDE_DEPRECATION", "UNCHECKED_CAST") + override val onReceiveOrNull: SelectClause1 + get() = SelectClause1Impl( + clauseObject = this@BufferedChannel, + regFunc = BufferedChannel<*>::registerSelectForReceive as RegistrationFunction, + processResFunc = BufferedChannel<*>::processResultSelectReceiveOrNull as ProcessResultFunction, + onCancellationConstructor = onUndeliveredElementReceiveCancellationConstructor + ) + + @Suppress("UNUSED_PARAMETER") + private fun registerSelectForReceive(select: SelectInstance<*>, ignoredParam: Any?) = + receiveImpl( // <-- this is an inline function + waiter = select, + onElementRetrieved = { elem -> select.selectInRegistrationPhase(elem) }, + onSuspend = { _, _, _ -> }, + onClosed = { onClosedSelectOnReceive(select) } + ) + + private fun onClosedSelectOnReceive(select: SelectInstance<*>) { + select.selectInRegistrationPhase(CHANNEL_CLOSED) + } + + @Suppress("UNUSED_PARAMETER") + private fun processResultSelectReceive(ignoredParam: Any?, selectResult: Any?): Any? = + if (selectResult === CHANNEL_CLOSED) throw receiveException + else selectResult + + @Suppress("UNUSED_PARAMETER") + private fun processResultSelectReceiveOrNull(ignoredParam: Any?, selectResult: Any?): Any? = + if (selectResult === CHANNEL_CLOSED) { + if (closeCause == null) null + else throw receiveException + } else selectResult + + @Suppress("UNCHECKED_CAST", "UNUSED_PARAMETER", "RedundantNullableReturnType") + private fun processResultSelectReceiveCatching(ignoredParam: Any?, selectResult: Any?): Any? = + if (selectResult === CHANNEL_CLOSED) closed(closeCause) + else success(selectResult as E) + + @Suppress("UNCHECKED_CAST") + private val onUndeliveredElementReceiveCancellationConstructor: OnCancellationConstructor? = onUndeliveredElement?.let { + { select: SelectInstance<*>, _: Any?, element: Any? -> + { _, _, _ -> + if (element !== CHANNEL_CLOSED) onUndeliveredElement.callUndeliveredElement(element as E, select.context) + } + } + } + + // ###################### + // ## Iterator Support ## + // ###################### + + override fun iterator(): ChannelIterator = BufferedChannelIterator() + + /** + * The key idea is that an iterator is a special receiver type, + * which should be resumed differently to [receive] and [onReceive] + * operations, but can be served as a waiter in a way similar to + * [CancellableContinuation] and [SelectInstance]. + * + * Roughly, [hasNext] is a [receive] sibling, while [next] simply + * returns the already retrieved element and [hasNext] being idempotent. + * From the implementation side, [receiveResult] stores the element retrieved by [hasNext] + * (or a special [CHANNEL_CLOSED] token if the channel is closed). + * + * The [invoke] function is a [CancelHandler] implementation, + * which requires knowing the [segment] and the [index] in it + * that specify the location of the stored iterator. + * + * To resume the suspended [hasNext] call, a special [tryResumeHasNext] + * function should be used in a way similar to [CancellableContinuation.tryResume] + * and [SelectInstance.trySelect]. When the channel becomes closed, + * [tryResumeHasNextOnClosedChannel] should be used instead. + */ + private inner class BufferedChannelIterator : ChannelIterator, Waiter { + /** + * Stores the element retrieved by [hasNext] or + * a special [CHANNEL_CLOSED] token if this channel is closed. + * If [hasNext] has not been invoked yet, [NO_RECEIVE_RESULT] is stored. + */ + private var receiveResult: Any? = NO_RECEIVE_RESULT + + /** + * When [hasNext] suspends, this field stores the corresponding + * continuation. The [tryResumeHasNext] and [tryResumeHasNextOnClosedChannel] + * function resume this continuation when the [hasNext] invocation should complete. + * + * This property is the subject to bening data race: + * It is nulled-out on both completion and cancellation paths that + * could happen concurrently. + */ + @BenignDataRace + private var continuation: CancellableContinuationImpl? = null + + // `hasNext()` is just a special receive operation. + override suspend fun hasNext(): Boolean { + return if (this.receiveResult !== NO_RECEIVE_RESULT && this.receiveResult !== CHANNEL_CLOSED) { + true + } else receiveImpl( // <-- this is an inline function + // Do not create a continuation until it is required; + // it is created later via [onNoWaiterSuspend], if needed. + waiter = null, + // Store the received element in `receiveResult` on successful + // retrieval from the buffer or rendezvous with a suspended sender. + // Also, inform the `BufferedChannel` extensions that + // the synchronization of this receive operation is completed. + onElementRetrieved = { element -> + this.receiveResult = element + true + }, + // As no waiter is provided, suspension is impossible. + onSuspend = { _, _, _ -> error("unreachable") }, + // Return `false` or throw an exception if the channel is already closed. + onClosed = { onClosedHasNext() }, + // If `hasNext()` decides to suspend, the corresponding + // `suspend` function that creates a continuation is called. + // The tail-call optimization is applied here. + onNoWaiterSuspend = { segm, i, r -> return hasNextOnNoWaiterSuspend(segm, i, r) } + ) + } + + private fun onClosedHasNext(): Boolean { + this.receiveResult = CHANNEL_CLOSED + val cause = closeCause ?: return false + throw recoverStackTrace(cause) + } + + private suspend fun hasNextOnNoWaiterSuspend( + /* The working cell is specified by + the segment and the index in it. */ + segment: ChannelSegment, + index: Int, + /* The global index of the cell. */ + r: Long + ): Boolean = suspendCancellableCoroutineReusable { cont -> + this.continuation = cont + receiveImplOnNoWaiter( // <-- this is an inline function + segment = segment, index = index, r = r, + waiter = this, // store this iterator as a waiter + // In case of successful element retrieval, store + // it in `receiveResult` and resume the continuation. + // Importantly, the receiver coroutine may be cancelled + // after it is successfully resumed but not dispatched yet. + // In case `onUndeliveredElement` is present, we must + // invoke it in the latter case. + onElementRetrieved = { element -> + this.receiveResult = element + this.continuation = null + cont.resume(true, onUndeliveredElement?.bindCancellationFun(element)) + }, + onClosed = { onClosedHasNextNoWaiterSuspend() } + ) + } + + override fun invokeOnCancellation(segment: Segment<*>, index: Int) { + this.continuation?.invokeOnCancellation(segment, index) + } + + private fun onClosedHasNextNoWaiterSuspend() { + // Read the current continuation and clean + // the corresponding field to avoid memory leaks. + val cont = this.continuation!! + this.continuation = null + // Update the `hasNext()` internal result. + this.receiveResult = CHANNEL_CLOSED + // If this channel was closed without exception, + // `hasNext()` should return `false`; otherwise, + // it throws the closing exception. + val cause = closeCause + if (cause == null) { + cont.resume(false) + } else { + cont.resumeWithException(recoverStackTrace(cause, cont)) + } + } + + @Suppress("UNCHECKED_CAST") + override fun next(): E { + // Read the already received result, or [NO_RECEIVE_RESULT] if [hasNext] has not been invoked yet. + val result = receiveResult + check(result !== NO_RECEIVE_RESULT) { "`hasNext()` has not been invoked" } + receiveResult = NO_RECEIVE_RESULT + // Is this channel closed? + if (result === CHANNEL_CLOSED) throw recoverStackTrace(receiveException) + // Return the element. + return result as E + } + + fun tryResumeHasNext(element: E): Boolean { + // Read the current continuation and clean + // the corresponding field to avoid memory leaks. + val cont = this.continuation!! + this.continuation = null + // Store the retrieved element in `receiveResult`. + this.receiveResult = element + // Try to resume this `hasNext()`. Importantly, the receiver coroutine + // may be cancelled after it is successfully resumed but not dispatched yet. + // In case `onUndeliveredElement` is specified, we need to invoke it in the latter case. + return cont.tryResume0(true, onUndeliveredElement?.bindCancellationFun(element)) + } + + fun tryResumeHasNextOnClosedChannel() { + /* + * Read the current continuation of the suspended `hasNext()` call and clean the corresponding field to avoid memory leaks. + * While this nulling out is unnecessary, it eliminates memory leaks (through the continuation) + * if the channel iterator accidentally remains GC-reachable after the channel is closed. + */ + val cont = this.continuation!! + this.continuation = null + // Update the `hasNext()` internal result and inform + // `BufferedChannel` extensions that synchronization + // of this receive operation is completed. + this.receiveResult = CHANNEL_CLOSED + // If this channel was closed without exception, + // `hasNext()` should return `false`; otherwise, + // it throws the closing exception. + val cause = closeCause + if (cause == null) { + cont.resume(false) + } else { + cont.resumeWithException(recoverStackTrace(cause, cont)) + } + } + } + + // ############################## + // ## Closing and Cancellation ## + // ############################## + + /** + * Store the cause of closing this channel, either via [close] or [cancel] call. + * The closing cause can be set only once. + */ + private val _closeCause = atomic(NO_CLOSE_CAUSE) + // Should be called only if this channel is closed or cancelled. + protected val closeCause get() = _closeCause.value as Throwable? + + /** Returns the closing cause if it is non-null, or [ClosedSendChannelException] otherwise. */ + protected val sendException get() = closeCause ?: ClosedSendChannelException(DEFAULT_CLOSE_MESSAGE) + + /** Returns the closing cause if it is non-null, or [ClosedReceiveChannelException] otherwise. */ + private val receiveException get() = closeCause ?: ClosedReceiveChannelException(DEFAULT_CLOSE_MESSAGE) + + /** + Stores the closed handler installed by [invokeOnClose]. + To synchronize [invokeOnClose] and [close], two additional + marker states, [CLOSE_HANDLER_INVOKED] and [CLOSE_HANDLER_CLOSED] + are used. The resulting state diagram is presented below. + + +------+ install handler +---------+ close(..) +---------+ + | null |------------------>| handler |------------>| INVOKED | + +------+ +---------+ +---------+ + | + | close(..) +--------+ + +----------->| CLOSED | + +--------+ + */ + private val closeHandler = atomic(null) + + /** + * Invoked when channel is closed as the last action of [close] invocation. + * This method should be idempotent and can be called multiple times. + */ + protected open fun onClosedIdempotent() {} + + override fun close(cause: Throwable?): Boolean = + closeOrCancelImpl(cause, cancel = false) + + @Suppress("OVERRIDE_DEPRECATION") + final override fun cancel(cause: Throwable?): Boolean = cancelImpl(cause) + + @Suppress("OVERRIDE_DEPRECATION") + final override fun cancel() { cancelImpl(null) } + + final override fun cancel(cause: CancellationException?) { cancelImpl(cause) } + + internal open fun cancelImpl(cause: Throwable?): Boolean = + closeOrCancelImpl(cause ?: CancellationException("Channel was cancelled"), cancel = true) + + /** + * This is a common implementation for [close] and [cancel]. It first tries + * to install the specified cause; the invocation that successfully installs + * the cause returns `true` as a results of this function, while all further + * [close] and [cancel] calls return `false`. + * + * After the closing/cancellation cause is installed, the channel should be marked + * as closed or cancelled, which bounds further `send(e)`-s to fails. + * + * Then, [completeCloseOrCancel] is called, which cancels waiting `receive()` + * requests ([cancelSuspendedReceiveRequests]) and removes unprocessed elements + * ([removeUnprocessedElements]) in case this channel is cancelled. + * + * Finally, if this [closeOrCancelImpl] has installed the cause, therefore, + * has closed the channel, [closeHandler] and [onClosedIdempotent] should be invoked. + */ + protected open fun closeOrCancelImpl(cause: Throwable?, cancel: Boolean): Boolean { + // If this is a `cancel(..)` invocation, set a bit that the cancellation + // has been started. This is crucial for ensuring linearizability, + // when concurrent `close(..)` and `isClosedFor[Send,Receive]` operations + // help this `cancel(..)`. + if (cancel) markCancellationStarted() + // Try to install the specified cause. On success, this invocation will + // return `true` as a result; otherwise, it will complete with `false`. + val closedByThisOperation = _closeCause.compareAndSet(NO_CLOSE_CAUSE, cause) + // Mark this channel as closed or cancelled, depending on this operation type. + if (cancel) markCancelled() else markClosed() + // Complete the closing or cancellation procedure. + completeCloseOrCancel() + // Finally, if this operation has installed the cause, + // it should invoke the close handlers. + return closedByThisOperation.also { + onClosedIdempotent() + if (it) invokeCloseHandler() + } + } + + /** + * Invokes the installed close handler, + * updating the [closeHandler] state correspondingly. + */ + private fun invokeCloseHandler() { + val closeHandler = closeHandler.getAndUpdate { + if (it === null) { + // Inform concurrent `invokeOnClose` + // that this channel is already closed. + CLOSE_HANDLER_CLOSED + } else { + // Replace the handler with a special + // `INVOKED` marker to avoid memory leaks. + CLOSE_HANDLER_INVOKED + } + } ?: return // no handler was installed, finish. + // Invoke the handler. + @Suppress("UNCHECKED_CAST") + closeHandler as (cause: Throwable?) -> Unit + closeHandler(closeCause) + } + + override fun invokeOnClose(handler: (cause: Throwable?) -> Unit) { + // Try to install the handler, finishing on success. + if (closeHandler.compareAndSet(null, handler)) { + // Handler has been successfully set, finish the operation. + return + } + // Either another handler is already set, or this channel is closed. + // In the latter case, the current handler should be invoked. + // However, the implementation must ensure that at most one + // handler is called, throwing an `IllegalStateException` + // if another close handler has been invoked. + closeHandler.loop { cur -> + when { + cur === CLOSE_HANDLER_CLOSED -> { + // Try to update the state from `CLOSED` to `INVOKED`. + // This is crucial to guarantee that at most one handler can be called. + // On success, invoke the handler and finish. + if (closeHandler.compareAndSet(CLOSE_HANDLER_CLOSED, CLOSE_HANDLER_INVOKED)) { + handler(closeCause) + return + } + } + cur === CLOSE_HANDLER_INVOKED -> error("Another handler was already registered and successfully invoked") + else -> error("Another handler is already registered: $cur") + } + } + } + + /** + * Marks this channel as closed. + * In case [cancelImpl] has already been invoked, + * and this channel is marked with [CLOSE_STATUS_CANCELLATION_STARTED], + * this function marks the channel as cancelled. + * + * All operation that notice this channel in the closed state, + * must help to complete the closing via [completeCloseOrCancel]. + */ + private fun markClosed(): Unit = + sendersAndCloseStatus.update { cur -> + when (cur.sendersCloseStatus) { + CLOSE_STATUS_ACTIVE -> // the channel is neither closed nor cancelled + constructSendersAndCloseStatus(cur.sendersCounter, CLOSE_STATUS_CLOSED) + CLOSE_STATUS_CANCELLATION_STARTED -> // the channel is going to be cancelled + constructSendersAndCloseStatus(cur.sendersCounter, CLOSE_STATUS_CANCELLED) + else -> return // the channel is already marked as closed or cancelled. + } + } + + /** + * Marks this channel as cancelled. + * + * All operation that notice this channel in the cancelled state, + * must help to complete the cancellation via [completeCloseOrCancel]. + */ + private fun markCancelled(): Unit = + sendersAndCloseStatus.update { cur -> + constructSendersAndCloseStatus(cur.sendersCounter, CLOSE_STATUS_CANCELLED) + } + + /** + * When the cancellation procedure starts, it is critical + * to mark the closing status correspondingly. Thus, other + * operations, which may help to complete the cancellation, + * always correctly update the status to `CANCELLED`. + */ + private fun markCancellationStarted(): Unit = + sendersAndCloseStatus.update { cur -> + if (cur.sendersCloseStatus == CLOSE_STATUS_ACTIVE) + constructSendersAndCloseStatus(cur.sendersCounter, CLOSE_STATUS_CANCELLATION_STARTED) + else return // this channel is already closed or cancelled + } + + /** + * Completes the started [close] or [cancel] procedure. + */ + private fun completeCloseOrCancel() { + isClosedForSend // must finish the started close/cancel if one is detected. + } + + protected open val isConflatedDropOldest get() = false + + /** + * Completes the channel closing procedure. + */ + private fun completeClose(sendersCur: Long): ChannelSegment { + // Close the linked list for further segment addition, + // obtaining the last segment in the data structure. + val lastSegment = closeLinkedList() + // In the conflated channel implementation (with the DROP_OLDEST + // elements conflation strategy), it is critical to mark all empty + // cells as closed to prevent in-progress `send(e)`-s, which have not + // put their elements yet, completions after this channel is closed. + // Otherwise, it is possible for a `send(e)` to put an element when + // the buffer is already full, while a concurrent receiver may extract + // the oldest element. When the channel is not closed, we can linearize + // this `receive()` before the `send(e)`, but after the channel is closed, + // `send(e)` must fails. Marking all unprocessed cells as `CLOSED` solves the issue. + if (isConflatedDropOldest) { + val lastBufferedCellGlobalIndex = markAllEmptyCellsAsClosed(lastSegment) + if (lastBufferedCellGlobalIndex != -1L) + dropFirstElementUntilTheSpecifiedCellIsInTheBuffer(lastBufferedCellGlobalIndex) + } + // Resume waiting `receive()` requests, + // informing them that the channel is closed. + cancelSuspendedReceiveRequests(lastSegment, sendersCur) + // Return the last segment in the linked list as a result + // of this function; we need it in `completeCancel(..)`. + return lastSegment + } + + /** + * Completes the channel cancellation procedure. + */ + private fun completeCancel(sendersCur: Long) { + // First, ensure that this channel is closed, + // obtaining the last segment in the linked list. + val lastSegment = completeClose(sendersCur) + // Cancel suspended `send(e)` requests and + // remove buffered elements in the reverse order. + removeUnprocessedElements(lastSegment) + } + + /** + * Closes the underlying linked list of segments for further segment addition. + */ + private fun closeLinkedList(): ChannelSegment { + // Choose the last segment. + var lastSegment = bufferEndSegment.value + sendSegment.value.let { if (it.id > lastSegment.id) lastSegment = it } + receiveSegment.value.let { if (it.id > lastSegment.id) lastSegment = it } + // Close the linked list of segment for new segment addition + // and return the last segment in the linked list. + return lastSegment.close() + } + + /** + * This function marks all empty cells, in the `null` and [IN_BUFFER] state, + * as closed. Notably, it processes the cells from right to left, and finishes + * immediately when the processing cell is already covered by `receive()` or + * contains a buffered elements ([BUFFERED] state). + * + * This function returns the global index of the last buffered element, + * or `-1` if this channel does not contain buffered elements. + */ + private fun markAllEmptyCellsAsClosed(lastSegment: ChannelSegment): Long { + // Process the cells in reverse order, from right to left. + var segment = lastSegment + while (true) { + for (index in SEGMENT_SIZE - 1 downTo 0) { + // Is this cell already covered by `receive()`? + val globalIndex = segment.id * SEGMENT_SIZE + index + if (globalIndex < receiversCounter) return -1 + // Process the cell `segment[index]`. + cell_update@ while (true) { + val state = segment.getState(index) + when { + // The cell is empty. + state === null || state === IN_BUFFER -> { + // Inform a possibly upcoming sender that this channel is already closed. + if (segment.casState(index, state, CHANNEL_CLOSED)) { + segment.onSlotCleaned() + break@cell_update + } + } + // The cell stores a buffered element. + state === BUFFERED -> return globalIndex + // Skip this cell if it is not empty and does not store a buffered element. + else -> break@cell_update + } + } + } + // Process the next segment, finishing if the linked list ends. + segment = segment.prev ?: return -1 + } + } + + /** + * Cancels suspended `send(e)` requests and removes buffered elements + * starting from the last cell in the specified [lastSegment] (it must + * be the physical tail of the underlying linked list) and updating + * the cells in reverse order. + */ + private fun removeUnprocessedElements(lastSegment: ChannelSegment) { + // Read the `onUndeliveredElement` lambda at once. In case it + // throws an exception, this exception is handled and stored in + // the variable below. If multiple exceptions are thrown, the first + // one is stored in the variable, while the others are suppressed. + val onUndeliveredElement = onUndeliveredElement + var undeliveredElementException: UndeliveredElementException? = null // first cancel exception, others suppressed + // To perform synchronization correctly, it is critical to + // process the cells in reverse order, from right to left. + // However, according to the API, suspended senders should + // be cancelled in the order of their suspension. Therefore, + // we need to collect all of them and cancel in the reverse + // order after that. + var suspendedSenders = InlineList() + var segment = lastSegment + process_segments@ while (true) { + for (index in SEGMENT_SIZE - 1 downTo 0) { + // Process the cell `segment[index]`. + val globalIndex = segment.id * SEGMENT_SIZE + index + // Update the cell state. + update_cell@ while (true) { + // Read the current state of the cell. + val state = segment.getState(index) + when { + // The cell is already processed by a receiver. + state === DONE_RCV -> break@process_segments + // The cell stores a buffered element. + state === BUFFERED -> { + // Is the cell already covered by a receiver? + if (globalIndex < receiversCounter) break@process_segments + // Update the cell state to `CHANNEL_CLOSED`. + if (segment.casState(index, state, CHANNEL_CLOSED)) { + // If `onUndeliveredElement` lambda is non-null, call it. + if (onUndeliveredElement != null) { + val element = segment.getElement(index) + undeliveredElementException = onUndeliveredElement.callUndeliveredElementCatchingException(element, undeliveredElementException) + } + // Clean the element field and inform the segment + // that the slot is cleaned to avoid memory leaks. + segment.cleanElement(index) + segment.onSlotCleaned() + break@update_cell + } + } + // The cell is empty. + state === IN_BUFFER || state === null -> { + // Update the cell state to `CHANNEL_CLOSED`. + if (segment.casState(index, state, CHANNEL_CLOSED)) { + // Inform the segment that the slot is cleaned to avoid memory leaks. + segment.onSlotCleaned() + break@update_cell + } + } + // The cell stores a suspended waiter. + state is Waiter || state is WaiterEB -> { + // Is the cell already covered by a receiver? + if (globalIndex < receiversCounter) break@process_segments + // Obtain the sender. + val sender: Waiter = if (state is WaiterEB) state.waiter + else state as Waiter + // Update the cell state to `CHANNEL_CLOSED`. + if (segment.casState(index, state, CHANNEL_CLOSED)) { + // If `onUndeliveredElement` lambda is non-null, call it. + if (onUndeliveredElement != null) { + val element = segment.getElement(index) + undeliveredElementException = onUndeliveredElement.callUndeliveredElementCatchingException(element, undeliveredElementException) + } + // Save the sender for further cancellation. + suspendedSenders += sender + // Clean the element field and inform the segment + // that the slot is cleaned to avoid memory leaks. + segment.cleanElement(index) + segment.onSlotCleaned() + break@update_cell + } + } + // A concurrent receiver is resuming a suspended sender. + // As the cell is covered by a receiver, finish immediately. + state === RESUMING_BY_EB || state === RESUMING_BY_RCV -> break@process_segments + // A concurrent `expandBuffer()` is resuming a suspended sender. + // Wait in a spin-loop until the cell state changes. + state === RESUMING_BY_EB -> continue@update_cell + else -> break@update_cell + } + } + } + // Process the previous segment. + segment = segment.prev ?: break + } + // Cancel suspended senders in their order of addition to this channel. + suspendedSenders.forEachReversed { it.resumeSenderOnCancelledChannel() } + // Throw `UndeliveredElementException` at the end if there was one. + undeliveredElementException?.let { throw it } + } + + /** + * Cancels suspended `receive` requests from the end to the beginning, + * also moving empty cells to the `CHANNEL_CLOSED` state. + */ + private fun cancelSuspendedReceiveRequests(lastSegment: ChannelSegment, sendersCounter: Long) { + // To perform synchronization correctly, it is critical to + // extract suspended requests in the reverse order, + // from the end to the beginning. + // However, according to the API, they should be cancelled + // in the order of their suspension. Therefore, we need to + // collect the suspended requests first, cancelling them + // in the reverse order after that. + var suspendedReceivers = InlineList() + var segment: ChannelSegment? = lastSegment + process_segments@ while (segment != null) { + for (index in SEGMENT_SIZE - 1 downTo 0) { + // Is the cell already covered by a sender? Finish immediately in this case. + if (segment.id * SEGMENT_SIZE + index < sendersCounter) break@process_segments + // Try to move the cell state to `CHANNEL_CLOSED`. + cell_update@ while (true) { + val state = segment.getState(index) + when { + state === null || state === IN_BUFFER -> { + if (segment.casState(index, state, CHANNEL_CLOSED)) { + segment.onSlotCleaned() + break@cell_update + } + } + state is WaiterEB -> { + if (segment.casState(index, state, CHANNEL_CLOSED)) { + suspendedReceivers += state.waiter // save for cancellation. + segment.onCancelledRequest(index = index, receiver = true) + break@cell_update + } + } + state is Waiter -> { + if (segment.casState(index, state, CHANNEL_CLOSED)) { + suspendedReceivers += state // save for cancellation. + segment.onCancelledRequest(index = index, receiver = true) + break@cell_update + } + } + else -> break@cell_update // nothing to cancel. + } + } + } + // Process the previous segment. + segment = segment.prev + } + // Cancel the suspended requests in their order of addition to this channel. + suspendedReceivers.forEachReversed { it.resumeReceiverOnClosedChannel() } + } + + /** + * Resumes this receiver because this channel is closed. + * This function does not take any effect if the operation has already been resumed or cancelled. + */ + private fun Waiter.resumeReceiverOnClosedChannel() = resumeWaiterOnClosedChannel(receiver = true) + + /** + * Resumes this sender because this channel is cancelled. + * This function does not take any effect if the operation has already been resumed or cancelled. + */ + private fun Waiter.resumeSenderOnCancelledChannel() = resumeWaiterOnClosedChannel(receiver = false) + + private fun Waiter.resumeWaiterOnClosedChannel(receiver: Boolean) { + when (this) { + is SendBroadcast -> cont.resume(false) + is CancellableContinuation<*> -> resumeWithException(if (receiver) receiveException else sendException) + is ReceiveCatching<*> -> cont.resume(closed(closeCause)) + is BufferedChannel<*>.BufferedChannelIterator -> tryResumeHasNextOnClosedChannel() + is SelectInstance<*> -> trySelect(this@BufferedChannel, CHANNEL_CLOSED) + else -> error("Unexpected waiter: $this") + } + } + + @ExperimentalCoroutinesApi + override val isClosedForSend: Boolean + get() = sendersAndCloseStatus.value.isClosedForSend0 + + private val Long.isClosedForSend0 get() = + isClosed(this, isClosedForReceive = false) + + @ExperimentalCoroutinesApi + override val isClosedForReceive: Boolean + get() = sendersAndCloseStatus.value.isClosedForReceive0 + + private val Long.isClosedForReceive0 get() = + isClosed(this, isClosedForReceive = true) + + private fun isClosed( + sendersAndCloseStatusCur: Long, + isClosedForReceive: Boolean + ) = when (sendersAndCloseStatusCur.sendersCloseStatus) { + // This channel is active and has not been closed. + CLOSE_STATUS_ACTIVE -> false + // The cancellation procedure has been started but + // not linearized yet, so this channel should be + // considered as active. + CLOSE_STATUS_CANCELLATION_STARTED -> false + // This channel has been successfully closed. + // Help to complete the closing procedure to + // guarantee linearizability, and return `true` + // for senders or the flag whether there still + // exist elements to retrieve for receivers. + CLOSE_STATUS_CLOSED -> { + completeClose(sendersAndCloseStatusCur.sendersCounter) + // When `isClosedForReceive` is `false`, always return `true`. + // Otherwise, it is possible that the channel is closed but + // still has elements to retrieve. + if (isClosedForReceive) !hasElements() else true + } + // This channel has been successfully cancelled. + // Help to complete the cancellation procedure to + // guarantee linearizability and return `true`. + CLOSE_STATUS_CANCELLED -> { + completeCancel(sendersAndCloseStatusCur.sendersCounter) + true + } + else -> error("unexpected close status: ${sendersAndCloseStatusCur.sendersCloseStatus}") + } + + @ExperimentalCoroutinesApi + override val isEmpty: Boolean get() { + // This function should return `false` if + // this channel is closed for `receive`. + if (isClosedForReceive) return false + // Does this channel has elements to retrieve? + if (hasElements()) return false + // This channel does not have elements to retrieve; + // Check that it is still not closed for `receive`. + return !isClosedForReceive + } + + /** + * Checks whether this channel contains elements to retrieve. + * Unfortunately, simply comparing the counters is insufficient, + * as some cells can be in the `INTERRUPTED` state due to cancellation. + * This function tries to find the first "alive" element, + * updating the `receivers` counter to skip empty cells. + * + * The implementation is similar to `receive()`. + */ + internal fun hasElements(): Boolean { + while (true) { + // Read the segment before obtaining the `receivers` counter value. + var segment = receiveSegment.value + // Obtains the `receivers` and `senders` counter values. + val r = receiversCounter + val s = sendersCounter + // Is there a chance that this channel has elements? + if (s <= r) return false // no elements + // The `r`-th cell is covered by a sender; check whether it contains an element. + // First, try to find the required segment if the initially + // obtained segment (in the beginning of this function) has lower id. + val id = r / SEGMENT_SIZE + if (segment.id != id) { + // Try to find the required segment. + segment = findSegmentReceive(id, segment) ?: + // The required segment has not been found. Either it has already + // been removed, or the underlying linked list is already closed + // for segment additions. In the latter case, the channel is closed + // and does not contain elements, so this operation returns `false`. + // Otherwise, if the required segment is removed, the operation restarts. + if (receiveSegment.value.id < id) return false else continue + } + segment.cleanPrev() // all the previous segments are no longer needed. + // Does the `r`-th cell contain waiting sender or buffered element? + val i = (r % SEGMENT_SIZE).toInt() + if (isCellNonEmpty(segment, i, r)) return true + // The cell is empty. Update `receivers` counter and try again. + receivers.compareAndSet(r, r + 1) // if this CAS fails, the counter has already been updated. + } + } + + /** + * Checks whether this cell contains a buffered element or a waiting sender, + * returning `true` in this case. Otherwise, if this cell is empty + * (due to waiter cancellation, cell poisoning, or channel closing), + * this function returns `false`. + * + * Notably, this function must be called only if the cell is covered by a sender. + */ + private fun isCellNonEmpty( + segment: ChannelSegment, + index: Int, + globalIndex: Long + ): Boolean { + // The logic is similar to `updateCellReceive` with the only difference + // that this function neither changes the cell state nor retrieves the element. + while (true) { + // Read the current cell state. + val state = segment.getState(index) + when { + // The cell is empty but a sender is coming. + state === null || state === IN_BUFFER -> { + // Poison the cell to ensure correctness. + if (segment.casState(index, state, POISONED)) { + // When the cell becomes poisoned, it is essentially + // the same as storing an already cancelled receiver. + // Thus, the `expandBuffer()` procedure should be invoked. + expandBuffer() + return false + } + } + // The cell stores a buffered element. + state === BUFFERED -> return true + // The cell stores an interrupted sender. + state === INTERRUPTED_SEND -> return false + // This channel is already closed. + state === CHANNEL_CLOSED -> return false + // The cell is already processed + // by a concurrent receiver. + state === DONE_RCV -> return false + // The cell is already poisoned + // by a concurrent receiver. + state === POISONED -> return false + // A concurrent `expandBuffer()` is resuming + // a suspended sender. This function is eligible + // to linearize before the buffer expansion procedure. + state === RESUMING_BY_EB -> return true + // A concurrent receiver is resuming + // a suspended sender. The element + // is no longer available for retrieval. + state === RESUMING_BY_RCV -> return false + // The cell stores a suspended request. + // However, it is possible that this request + // is receiver if the cell is covered by both + // send and receive operations. + // In case the cell is already covered by + // a receiver, the element is no longer + // available for retrieval, and this function + // return `false`. Otherwise, it is guaranteed + // that the suspended request is sender, so + // this function returns `true`. + else -> return globalIndex == receiversCounter + } + } + } + + // ####################### + // # Segments Management # + // ####################### + + /** + * Finds the segment with the specified [id] starting by the [startFrom] + * segment and following the [ChannelSegment.next] references. In case + * the required segment has not been created yet, this function attempts + * to add it to the underlying linked list. Finally, it updates [sendSegment] + * to the found segment if its [ChannelSegment.id] is greater than the one + * of the already stored segment. + * + * In case the requested segment is already removed, or if it should be allocated + * but the linked list structure is closed for new segments addition, this function + * returns `null`. The implementation also efficiently skips a sequence of removed + * segments, updating the counter value in [sendersAndCloseStatus] correspondingly. + */ + private fun findSegmentSend(id: Long, startFrom: ChannelSegment): ChannelSegment? { + return sendSegment.findSegmentAndMoveForward(id, startFrom, createSegmentFunction()).let { + if (it.isClosed) { + // The required segment has not been found and new segments + // cannot be added, as the linked listed in already added. + // This channel is already closed or cancelled; help to complete + // the closing or cancellation procedure. + completeCloseOrCancel() + // Clean the `prev` reference of the provided segment + // if all the previous cells are already covered by senders. + // It is important to clean the `prev` reference only in + // this case, as the closing/cancellation procedure may + // need correct value to traverse the linked list from right to left. + if (startFrom.id * SEGMENT_SIZE < receiversCounter) startFrom.cleanPrev() + // As the required segment is not found and cannot be allocated, return `null`. + null + } else { + // Get the found segment. + val segment = it.segment + // Is the required segment removed? + if (segment.id > id) { + // The required segment has been removed; `segment` is the first + // segment with `id` not lower than the required one. + // Skip the sequence of removed cells in O(1). + updateSendersCounterIfLower(segment.id * SEGMENT_SIZE) + // Clean the `prev` reference of the provided segment + // if all the previous cells are already covered by senders. + // It is important to clean the `prev` reference only in + // this case, as the closing/cancellation procedure may + // need correct value to traverse the linked list from right to left. + if (segment.id * SEGMENT_SIZE < receiversCounter) segment.cleanPrev() + // As the required segment is not found and cannot be allocated, return `null`. + null + } else { + assert { segment.id == id } + // The required segment has been found; return it! + segment + } + } + } + } + + /** + * Finds the segment with the specified [id] starting by the [startFrom] + * segment and following the [ChannelSegment.next] references. In case + * the required segment has not been created yet, this function attempts + * to add it to the underlying linked list. Finally, it updates [receiveSegment] + * to the found segment if its [ChannelSegment.id] is greater than the one + * of the already stored segment. + * + * In case the requested segment is already removed, or if it should be allocated + * but the linked list structure is closed for new segments addition, this function + * returns `null`. The implementation also efficiently skips a sequence of removed + * segments, updating the [receivers] counter correspondingly. + */ + private fun findSegmentReceive(id: Long, startFrom: ChannelSegment): ChannelSegment? = + receiveSegment.findSegmentAndMoveForward(id, startFrom, createSegmentFunction()).let { + if (it.isClosed) { + // The required segment has not been found and new segments + // cannot be added, as the linked listed in already added. + // This channel is already closed or cancelled; help to complete + // the closing or cancellation procedure. + completeCloseOrCancel() + // Clean the `prev` reference of the provided segment + // if all the previous cells are already covered by senders. + // It is important to clean the `prev` reference only in + // this case, as the closing/cancellation procedure may + // need correct value to traverse the linked list from right to left. + if (startFrom.id * SEGMENT_SIZE < sendersCounter) startFrom.cleanPrev() + // As the required segment is not found and cannot be allocated, return `null`. + null + } else { + // Get the found segment. + val segment = it.segment + // Advance the `bufferEnd` segment if required. + if (!isRendezvousOrUnlimited && id <= bufferEndCounter / SEGMENT_SIZE) { + bufferEndSegment.moveForward(segment) + } + // Is the required segment removed? + if (segment.id > id) { + // The required segment has been removed; `segment` is the first + // segment with `id` not lower than the required one. + // Skip the sequence of removed cells in O(1). + updateReceiversCounterIfLower(segment.id * SEGMENT_SIZE) + // Clean the `prev` reference of the provided segment + // if all the previous cells are already covered by senders. + // It is important to clean the `prev` reference only in + // this case, as the closing/cancellation procedure may + // need correct value to traverse the linked list from right to left. + if (segment.id * SEGMENT_SIZE < sendersCounter) segment.cleanPrev() + // As the required segment is already removed, return `null`. + null + } else { + assert { segment.id == id } + // The required segment has been found; return it! + segment + } + } + } + + /** + * Importantly, when this function does not find the requested segment, + * it always updates the number of completed `expandBuffer()` attempts. + */ + private fun findSegmentBufferEnd(id: Long, startFrom: ChannelSegment, currentBufferEndCounter: Long): ChannelSegment? = + bufferEndSegment.findSegmentAndMoveForward(id, startFrom, createSegmentFunction()).let { + if (it.isClosed) { + // The required segment has not been found and new segments + // cannot be added, as the linked listed in already added. + // This channel is already closed or cancelled; help to complete + // the closing or cancellation procedure. + completeCloseOrCancel() + // Update `bufferEndSegment` to the last segment + // in the linked list to avoid memory leaks. + moveSegmentBufferEndToSpecifiedOrLast(id, startFrom) + // When this function does not find the requested segment, + // it should update the number of completed `expandBuffer()` attempts. + incCompletedExpandBufferAttempts() + null + } else { + // Get the found segment. + val segment = it.segment + // Is the required segment removed? + if (segment.id > id) { + // The required segment has been removed; `segment` is the first segment + // with `id` not lower than the required one. + // Try to skip the sequence of removed cells in O(1) by increasing the `bufferEnd` counter. + // Importantly, when this function does not find the requested segment, + // it should update the number of completed `expandBuffer()` attempts. + if (bufferEnd.compareAndSet(currentBufferEndCounter + 1, segment.id * SEGMENT_SIZE)) { + incCompletedExpandBufferAttempts(segment.id * SEGMENT_SIZE - currentBufferEndCounter) + } else { + incCompletedExpandBufferAttempts() + } + // As the required segment is already removed, return `null`. + null + } else { + assert { segment.id == id } + // The required segment has been found; return it! + segment + } + } + } + + /** + * Updates [bufferEndSegment] to the one with the specified [id] or + * to the last existing segment, if the required segment is not yet created. + * + * Unlike [findSegmentBufferEnd], this function does not allocate new segments. + */ + private fun moveSegmentBufferEndToSpecifiedOrLast(id: Long, startFrom: ChannelSegment) { + // Start searching the required segment from the specified one. + var segment: ChannelSegment = startFrom + while (segment.id < id) { + segment = segment.next ?: break + } + // Skip all removed segments and try to update `bufferEndSegment` + // to the first non-removed one. This part should succeed eventually, + // as the tail segment is never removed. + while (true) { + while (segment.isRemoved) { + segment = segment.next ?: break + } + // Try to update `bufferEndSegment`. On failure, + // the found segment is already removed, so it + // should be skipped. + if (bufferEndSegment.moveForward(segment)) return + } + } + + /** + * Updates the `senders` counter if its value + * is lower that the specified one. + * + * Senders use this function to efficiently skip + * a sequence of cancelled receivers. + */ + private fun updateSendersCounterIfLower(value: Long): Unit = + sendersAndCloseStatus.loop { cur -> + val curCounter = cur.sendersCounter + if (curCounter >= value) return + val update = constructSendersAndCloseStatus(curCounter, cur.sendersCloseStatus) + if (sendersAndCloseStatus.compareAndSet(cur, update)) return + } + + /** + * Updates the `receivers` counter if its value + * is lower that the specified one. + * + * Receivers use this function to efficiently skip + * a sequence of cancelled senders. + */ + private fun updateReceiversCounterIfLower(value: Long): Unit = + receivers.loop { cur -> + if (cur >= value) return + if (receivers.compareAndSet(cur, value)) return + } + + // ################### + // # Debug Functions # + // ################### + + @Suppress("ConvertTwoComparisonsToRangeCheck") + override fun toString(): String { + val sb = StringBuilder() + // Append the close status + when (sendersAndCloseStatus.value.sendersCloseStatus) { + CLOSE_STATUS_CLOSED -> sb.append("closed,") + CLOSE_STATUS_CANCELLED -> sb.append("cancelled,") + } + // Append the buffer capacity + sb.append("capacity=$capacity,") + // Append the data + sb.append("data=[") + val firstSegment = listOf(receiveSegment.value, sendSegment.value, bufferEndSegment.value) + .filter { it !== NULL_SEGMENT } + .minBy { it.id } + val r = receiversCounter + val s = sendersCounter + var segment = firstSegment + append_elements@ while (true) { + process_cell@ for (i in 0 until SEGMENT_SIZE) { + val globalCellIndex = segment.id * SEGMENT_SIZE + i + if (globalCellIndex >= s && globalCellIndex >= r) break@append_elements + val cellState = segment.getState(i) + val element = segment.getElement(i) + val cellStateString = when (cellState) { + is CancellableContinuation<*> -> { + when { + globalCellIndex < r && globalCellIndex >= s -> "receive" + globalCellIndex < s && globalCellIndex >= r -> "send" + else -> "cont" + } + } + is SelectInstance<*> -> { + when { + globalCellIndex < r && globalCellIndex >= s -> "onReceive" + globalCellIndex < s && globalCellIndex >= r -> "onSend" + else -> "select" + } + } + is ReceiveCatching<*> -> "receiveCatching" + is SendBroadcast -> "sendBroadcast" + is WaiterEB -> "EB($cellState)" + RESUMING_BY_RCV, RESUMING_BY_EB -> "resuming_sender" + null, IN_BUFFER, DONE_RCV, POISONED, INTERRUPTED_RCV, INTERRUPTED_SEND, CHANNEL_CLOSED -> continue@process_cell + else -> cellState.toString() // leave it just in case something is missed. + } + if (element != null) { + sb.append("($cellStateString,$element),") + } else { + sb.append("$cellStateString,") + } + } + // Process the next segment if exists. + segment = segment.next ?: break + } + if (sb.last() == ',') sb.deleteAt(sb.length - 1) + sb.append("]") + // The string representation is constructed. + return sb.toString() + } + + // Returns a debug representation of this channel, + // which is actively used in Lincheck tests. + internal fun toStringDebug(): String { + val sb = StringBuilder() + // Append the counter values and the close status + sb.append("S=${sendersCounter},R=${receiversCounter},B=${bufferEndCounter},B'=${completedExpandBuffersAndPauseFlag.value},C=${sendersAndCloseStatus.value.sendersCloseStatus},") + when (sendersAndCloseStatus.value.sendersCloseStatus) { + CLOSE_STATUS_CANCELLATION_STARTED -> sb.append("CANCELLATION_STARTED,") + CLOSE_STATUS_CLOSED -> sb.append("CLOSED,") + CLOSE_STATUS_CANCELLED -> sb.append("CANCELLED,") + } + // Append the segment references + sb.append("SEND_SEGM=${sendSegment.value.hexAddress},RCV_SEGM=${receiveSegment.value.hexAddress}") + if (!isRendezvousOrUnlimited) sb.append(",EB_SEGM=${bufferEndSegment.value.hexAddress}") + sb.append(" ") // add some space + // Append the linked list of segments. + val firstSegment = listOf(receiveSegment.value, sendSegment.value, bufferEndSegment.value) + .filter { it !== NULL_SEGMENT } + .minBy { it.id } + var segment = firstSegment + while (true) { + sb.append("${segment.hexAddress}=[${if (segment.isRemoved) "*" else ""}${segment.id},prev=${segment.prev?.hexAddress},") + repeat(SEGMENT_SIZE) { i -> + val cellState = segment.getState(i) + val element = segment.getElement(i) + val cellStateString = when (cellState) { + is CancellableContinuation<*> -> "cont" + is SelectInstance<*> -> "select" + is ReceiveCatching<*> -> "receiveCatching" + is SendBroadcast -> "send(broadcast)" + is WaiterEB -> "EB($cellState)" + else -> cellState.toString() + } + sb.append("[$i]=($cellStateString,$element),") + } + sb.append("next=${segment.next?.hexAddress}] ") + // Process the next segment if exists. + segment = segment.next ?: break + } + // The string representation of this channel is now constructed! + return sb.toString() + } + + + // This is an internal methods for tests. + fun checkSegmentStructureInvariants() { + if (isRendezvousOrUnlimited) { + check(bufferEndSegment.value === NULL_SEGMENT) { + "bufferEndSegment must be NULL_SEGMENT for rendezvous and unlimited channels; they do not manipulate it.\n" + + "Channel state: $this" + } + } else { + check(receiveSegment.value.id <= bufferEndSegment.value.id) { + "bufferEndSegment should not have lower id than receiveSegment.\n" + + "Channel state: $this" + } + } + val firstSegment = listOf(receiveSegment.value, sendSegment.value, bufferEndSegment.value) + .filter { it !== NULL_SEGMENT } + .minBy { it.id } + check(firstSegment.prev == null) { + "All processed segments should be unreachable from the data structure, but the `prev` link of the leftmost segment is non-null.\n" + + "Channel state: $this" + } + // Check that the doubly-linked list of segments does not + // contain full-of-cancelled-cells segments. + var segment = firstSegment + while (segment.next != null) { + // Note that the `prev` reference can be `null` if this channel is closed. + check(segment.next!!.prev == null || segment.next!!.prev === segment) { + "The `segment.next.prev === segment` invariant is violated.\n" + + "Channel state: $this" + } + // Count the number of closed/interrupted cells + // and check that all cells are in expected states. + var interruptedOrClosedCells = 0 + for (i in 0 until SEGMENT_SIZE) { + when (val state = segment.getState(i)) { + BUFFERED -> {} // The cell stores a buffered element. + is Waiter -> {} // The cell stores a suspended request. + INTERRUPTED_RCV, INTERRUPTED_SEND, CHANNEL_CLOSED -> { + // The cell stored an interrupted request or indicates + // that this channel is already closed. + // Check that the element slot is cleaned and increment + // the number of cells in closed/interrupted state. + check(segment.getElement(i) == null) + interruptedOrClosedCells++ + } + POISONED, DONE_RCV -> { + // The cell is successfully processed or poisoned. + // Check that the element slot is cleaned. + check(segment.getElement(i) == null) + } + // Other states are illegal after all running operations finish. + else -> error("Unexpected segment cell state: $state.\nChannel state: $this") + } + } + // Is this segment full of cancelled/closed cells? + // If so, this segment should be removed from the + // linked list if nether `receiveSegment`, nor + // `sendSegment`, nor `bufferEndSegment` reference it. + if (interruptedOrClosedCells == SEGMENT_SIZE) { + check(segment === receiveSegment.value || segment === sendSegment.value || segment === bufferEndSegment.value) { + "Logically removed segment is reachable.\nChannel state: $this" + } + } + // Process the next segment. + segment = segment.next!! + } + } + + private fun OnUndeliveredElement.bindCancellationFunResult() = ::onCancellationChannelResultImplDoNotCall + + /** + * Do not call directly. Go through [bindCancellationFunResult] to ensure the callback isn't null. + * [bindCancellationFunResult] could have just returned a lambda as well, but there would be a risk of that + * lambda capturing the environment. + */ + private fun onCancellationChannelResultImplDoNotCall( + cause: Throwable, element: ChannelResult, context: CoroutineContext + ) { + onUndeliveredElement!!.callUndeliveredElement(element.getOrNull()!!, context) + } + + private fun OnUndeliveredElement.bindCancellationFun(element: E): + (Throwable, Any?, CoroutineContext) -> Unit = + { _: Throwable, _, context: CoroutineContext -> callUndeliveredElement(element, context) } + + private fun OnUndeliveredElement.bindCancellationFun() = ::onCancellationImplDoNotCall + + /** + * Do not call directly. Go through [bindCancellationFun] to ensure the callback isn't null. + * [bindCancellationFun] could have just returned a lambda as well, but there would be a risk of that + * lambda capturing the environment. + */ + private fun onCancellationImplDoNotCall(cause: Throwable, element: E, context: CoroutineContext) { + onUndeliveredElement!!.callUndeliveredElement(element, context) + } +} + +/** + * The channel is represented as a list of segments, which simulates an infinite array. + * Each segment has its own [id], which increase from the beginning. These [id]s help + * to update [BufferedChannel.sendSegment], [BufferedChannel.receiveSegment], + * and [BufferedChannel.bufferEndSegment] correctly. + */ +internal class ChannelSegment(id: Long, prev: ChannelSegment?, channel: BufferedChannel?, pointers: Int) : Segment>(id, prev, pointers) { + private val _channel: BufferedChannel? = channel + val channel get() = _channel!! // always non-null except for `NULL_SEGMENT` + + private val data = atomicArrayOfNulls(SEGMENT_SIZE * 2) // 2 registers per slot: state + element + override val numberOfSlots: Int get() = SEGMENT_SIZE + + // ######################################## + // # Manipulation with the Element Fields # + // ######################################## + + internal fun storeElement(index: Int, element: E) { + setElementLazy(index, element) + } + + @Suppress("UNCHECKED_CAST") + internal fun getElement(index: Int) = data[index * 2].value as E + + internal fun retrieveElement(index: Int): E = getElement(index).also { cleanElement(index) } + + internal fun cleanElement(index: Int) { + setElementLazy(index, null) + } + + private fun setElementLazy(index: Int, value: Any?) { + data[index * 2].lazySet(value) + } + + // ###################################### + // # Manipulation with the State Fields # + // ###################################### + + internal fun getState(index: Int): Any? = data[index * 2 + 1].value + + internal fun setState(index: Int, value: Any?) { + data[index * 2 + 1].value = value + } + + internal fun casState(index: Int, from: Any?, to: Any?) = data[index * 2 + 1].compareAndSet(from, to) + + internal fun getAndSetState(index: Int, update: Any?) = data[index * 2 + 1].getAndSet(update) + + + // ######################## + // # Cancellation Support # + // ######################## + + override fun onCancellation(index: Int, cause: Throwable?, context: CoroutineContext) { + // To distinguish cancelled senders and receivers, senders equip the index value with + // an additional marker, adding `SEGMENT_SIZE` to the value. + val isSender = index >= SEGMENT_SIZE + // Unwrap the index. + @Suppress("NAME_SHADOWING") val index = if (isSender) index - SEGMENT_SIZE else index + // Read the element, which may be needed further to call `onUndeliveredElement`. + val element = getElement(index) + // Update the cell state. + while (true) { + // CAS-loop + // Read the current state of the cell. + val cur = getState(index) + when { + // The cell stores a waiter. + cur is Waiter || cur is WaiterEB -> { + // The cancelled request is either send or receive. + // Update the cell state correspondingly. + val update = if (isSender) INTERRUPTED_SEND else INTERRUPTED_RCV + if (casState(index, cur, update)) { + // The waiter has been successfully cancelled. + // Clean the element slot and invoke `onSlotCleaned()`, + // which may cause deleting the whole segment from the linked list. + // In case the cancelled request is receiver, it is critical to ensure + // that the `expandBuffer()` attempt that processes this cell is completed, + // so `onCancelledRequest(..)` waits for its completion before invoking `onSlotCleaned()`. + cleanElement(index) + onCancelledRequest(index, !isSender) + // Call `onUndeliveredElement` if needed. + if (isSender) { + channel.onUndeliveredElement?.callUndeliveredElement(element, context) + } + return + } + } + // The cell already indicates that the operation is cancelled. + cur === INTERRUPTED_SEND || cur === INTERRUPTED_RCV -> { + // Clean the element slot to avoid memory leaks, + // invoke `onUndeliveredElement` if needed, and finish + cleanElement(index) + // Call `onUndeliveredElement` if needed. + if (isSender) { + channel.onUndeliveredElement?.callUndeliveredElement(element, context) + } + return + } + // An opposite operation is resuming this request; + // wait until the cell state updates. + // It is possible that an opposite operation has already + // resumed this request, which will result in updating + // the cell state to `DONE_RCV` or `BUFFERED`, while the + // current cancellation is caused by prompt cancellation. + cur === RESUMING_BY_EB || cur === RESUMING_BY_RCV -> continue + // This request was successfully resumed, so this cancellation + // is caused by the prompt cancellation feature and should be ignored. + cur === DONE_RCV || cur === BUFFERED -> return + // The cell state indicates that the channel is closed; + // this cancellation should be ignored. + cur === CHANNEL_CLOSED -> return + else -> error("unexpected state: $cur") + } + } + } + + /** + * Invokes `onSlotCleaned()` preceded by a `waitExpandBufferCompletion(..)` call + * in case the cancelled request is receiver. + */ + fun onCancelledRequest(index: Int, receiver: Boolean) { + if (receiver) channel.waitExpandBufferCompletion(id * SEGMENT_SIZE + index) + onSlotCleaned() + } +} + +// WA for atomicfu + JVM_IR compiler bug that lead to SMAP-related compiler crashes: KT-55983 +internal fun createSegmentFunction(): KFunction2, ChannelSegment> = ::createSegment + +private fun createSegment(id: Long, prev: ChannelSegment) = ChannelSegment( + id = id, + prev = prev, + channel = prev.channel, + pointers = 0 +) +private val NULL_SEGMENT = ChannelSegment(id = -1, prev = null, channel = null, pointers = 0) + +/** + * Number of cells in each segment. + */ +@JvmField +internal val SEGMENT_SIZE = systemProp("kotlinx.coroutines.bufferedChannel.segmentSize", 32) + +/** + * Number of iterations to wait in [BufferedChannel.waitExpandBufferCompletion] until the numbers of started and completed + * [BufferedChannel.expandBuffer] calls coincide. When the limit is reached, [BufferedChannel.waitExpandBufferCompletion] + * blocks further [BufferedChannel.expandBuffer]-s to avoid starvation. + */ +private val EXPAND_BUFFER_COMPLETION_WAIT_ITERATIONS = systemProp("kotlinx.coroutines.bufferedChannel.expandBufferCompletionWaitIterations", 10_000) + +/** + * Tries to resume this continuation with the specified + * value. Returns `true` on success and `false` on failure. + */ +private fun CancellableContinuation.tryResume0( + value: T, + onCancellation: ((cause: Throwable, value: T, context: CoroutineContext) -> Unit)? = null +): Boolean = + tryResume(value, null, onCancellation).let { token -> + if (token != null) { + completeResume(token) + true + } else false + } + +/* + If the channel is rendezvous or unlimited, the `bufferEnd` counter + should be initialized with the corresponding value below and never change. + In this case, the `expandBuffer(..)` operation does nothing. + */ +private const val BUFFER_END_RENDEZVOUS = 0L // no buffer +private const val BUFFER_END_UNLIMITED = Long.MAX_VALUE // infinite buffer +private fun initialBufferEnd(capacity: Int): Long = when (capacity) { + Channel.RENDEZVOUS -> BUFFER_END_RENDEZVOUS + Channel.UNLIMITED -> BUFFER_END_UNLIMITED + else -> capacity.toLong() +} + +/* + Cell states. The initial "empty" state is represented with `null`, + and suspended operations are represented with [Waiter] instances. + */ + +// The cell stores a buffered element. +@JvmField +internal val BUFFERED = Symbol("BUFFERED") +// Concurrent `expandBuffer(..)` can inform the +// upcoming sender that it should buffer the element. +private val IN_BUFFER = Symbol("SHOULD_BUFFER") +// Indicates that a receiver (RCV suffix) is resuming +// the suspended sender; after that, it should update +// the state to either `DONE_RCV` (on success) or +// `INTERRUPTED_SEND` (on failure). +private val RESUMING_BY_RCV = Symbol("S_RESUMING_BY_RCV") +// Indicates that `expandBuffer(..)` (RCV suffix) is resuming +// the suspended sender; after that, it should update +// the state to either `BUFFERED` (on success) or +// `INTERRUPTED_SEND` (on failure). +private val RESUMING_BY_EB = Symbol("RESUMING_BY_EB") +// When a receiver comes to the cell already covered by +// a sender (according to the counters), but the cell +// is still in `EMPTY` or `IN_BUFFER` state, it breaks +// the cell by changing its state to `POISONED`. +private val POISONED = Symbol("POISONED") +// When the element is successfully transferred +// to a receiver, the cell changes to `DONE_RCV`. +private val DONE_RCV = Symbol("DONE_RCV") +// Cancelled sender. +private val INTERRUPTED_SEND = Symbol("INTERRUPTED_SEND") +// Cancelled receiver. +private val INTERRUPTED_RCV = Symbol("INTERRUPTED_RCV") +// Indicates that the channel is closed. +internal val CHANNEL_CLOSED = Symbol("CHANNEL_CLOSED") +// When the cell is already covered by both sender and +// receiver (`sender` and `receivers` counters are greater +// than the cell number), the `expandBuffer(..)` procedure +// cannot distinguish which kind of operation is stored +// in the cell. Thus, it wraps the waiter with this descriptor, +// informing the possibly upcoming receiver that it should +// complete the `expandBuffer(..)` procedure if the waiter stored +// in the cell is sender. In turn, senders ignore this information. +private class WaiterEB(@JvmField val waiter: Waiter) { + override fun toString() = "WaiterEB($waiter)" +} + + + +/** + * To distinguish suspended [BufferedChannel.receive] and + * [BufferedChannel.receiveCatching] operations, the latter + * uses this wrapper for its continuation. + */ +private class ReceiveCatching( + @JvmField val cont: CancellableContinuationImpl> +) : Waiter by cont + +/* + Internal results for [BufferedChannel.updateCellReceive]. + On successful rendezvous with waiting sender or + buffered element retrieval, the corresponding element + is returned as result of [BufferedChannel.updateCellReceive]. + */ +private val SUSPEND = Symbol("SUSPEND") +private val SUSPEND_NO_WAITER = Symbol("SUSPEND_NO_WAITER") +private val FAILED = Symbol("FAILED") + +/* + Internal results for [BufferedChannel.updateCellSend] + */ +private const val RESULT_RENDEZVOUS = 0 +private const val RESULT_BUFFERED = 1 +private const val RESULT_SUSPEND = 2 +private const val RESULT_SUSPEND_NO_WAITER = 3 +private const val RESULT_CLOSED = 4 +private const val RESULT_FAILED = 5 + +/** + * Special value for [BufferedChannel.BufferedChannelIterator.receiveResult] + * that indicates the absence of pre-received result. + */ +private val NO_RECEIVE_RESULT = Symbol("NO_RECEIVE_RESULT") + +/* + As [BufferedChannel.invokeOnClose] can be invoked concurrently + with channel closing, we have to synchronize them. These two + markers help with the synchronization. + */ +private val CLOSE_HANDLER_CLOSED = Symbol("CLOSE_HANDLER_CLOSED") +private val CLOSE_HANDLER_INVOKED = Symbol("CLOSE_HANDLER_INVOKED") + +/** + * Specifies the absence of closing cause, stored in [BufferedChannel._closeCause]. + * When the channel is closed or cancelled without exception, this [NO_CLOSE_CAUSE] + * marker should be replaced with `null`. + */ +private val NO_CLOSE_CAUSE = Symbol("NO_CLOSE_CAUSE") + +/* + The channel close statuses. The transition scheme is the following: + +--------+ +----------------------+ +-----------+ + | ACTIVE |-->| CANCELLATION_STARTED |-->| CANCELLED | + +--------+ +----------------------+ +-----------+ + | ^ + | +--------+ | + +------------>| CLOSED |------------------+ + +--------+ + We need `CANCELLATION_STARTED` to synchronize + concurrent closing and cancellation. + */ +private const val CLOSE_STATUS_ACTIVE = 0 +private const val CLOSE_STATUS_CANCELLATION_STARTED = 1 +private const val CLOSE_STATUS_CLOSED = 2 +private const val CLOSE_STATUS_CANCELLED = 3 + +/* + The `senders` counter and the channel close status + are stored in a single 64-bit register to save the space + and reduce the number of reads in sending operations. + The code below encapsulates the required bit arithmetics. + */ +private const val SENDERS_CLOSE_STATUS_SHIFT = 60 +private const val SENDERS_COUNTER_MASK = (1L shl SENDERS_CLOSE_STATUS_SHIFT) - 1 +private inline val Long.sendersCounter get() = this and SENDERS_COUNTER_MASK +private inline val Long.sendersCloseStatus: Int get() = (this shr SENDERS_CLOSE_STATUS_SHIFT).toInt() +private fun constructSendersAndCloseStatus(counter: Long, closeStatus: Int): Long = + (closeStatus.toLong() shl SENDERS_CLOSE_STATUS_SHIFT) + counter + +/* + The `completedExpandBuffersAndPauseFlag` 64-bit counter contains + the number of completed `expandBuffer()` attempts along with a special + flag that pauses progress to avoid starvation in `waitExpandBufferCompletion(..)`. + The code below encapsulates the required bit arithmetics. + */ +private const val EB_COMPLETED_PAUSE_EXPAND_BUFFERS_BIT = 1L shl 62 +private const val EB_COMPLETED_COUNTER_MASK = EB_COMPLETED_PAUSE_EXPAND_BUFFERS_BIT - 1 +private inline val Long.ebCompletedCounter get() = this and EB_COMPLETED_COUNTER_MASK +private inline val Long.ebPauseExpandBuffers: Boolean get() = (this and EB_COMPLETED_PAUSE_EXPAND_BUFFERS_BIT) != 0L +private fun constructEBCompletedAndPauseFlag(counter: Long, pauseEB: Boolean): Long = + (if (pauseEB) EB_COMPLETED_PAUSE_EXPAND_BUFFERS_BIT else 0) + counter diff --git a/kotlinx-coroutines-core/common/src/channels/Channel.kt b/kotlinx-coroutines-core/common/src/channels/Channel.kt new file mode 100644 index 0000000000..3e3c0f5fae --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/Channel.kt @@ -0,0 +1,1484 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.Channel.Factory.BUFFERED +import kotlinx.coroutines.channels.Channel.Factory.CHANNEL_DEFAULT_CAPACITY +import kotlinx.coroutines.channels.Channel.Factory.CONFLATED +import kotlinx.coroutines.channels.Channel.Factory.RENDEZVOUS +import kotlinx.coroutines.channels.Channel.Factory.UNLIMITED +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlin.contracts.* +import kotlin.internal.* +import kotlin.jvm.* + +/** + * Sender's interface to a [Channel]. + * + * Combined, [SendChannel] and [ReceiveChannel] define the complete [Channel] interface. + * + * It is not expected that this interface will be implemented directly. + * Instead, the existing [Channel] implementations can be used or delegated to. + */ +public interface SendChannel { + /** + * Returns `true` if this channel was closed by an invocation of [close] or its receiving side was [cancelled][ReceiveChannel.cancel]. + * This means that calling [send] will result in an exception. + * + * Note that if this property returns `false`, it does not guarantee that a subsequent call to [send] will succeed, + * as the channel can be concurrently closed right after the check. + * For such scenarios, [trySend] is the more robust solution: it attempts to send the element and returns + * a result that says whether the channel was closed, and if not, whether sending a value was successful. + * + * ``` + * // DANGER! THIS CHECK IS NOT RELIABLE! + * if (!channel.isClosedForSend) { + * channel.send(element) // can still fail! + * } else { + * println("Can not send: the channel is closed") + * } + * // DO THIS INSTEAD: + * channel.trySend(element).onClosed { + * println("Can not send: the channel is closed") + * } + * ``` + * + * The primary intended usage of this property is skipping some portions of code that should not be executed if the + * channel is already known to be closed. + * For example: + * + * ``` + * if (channel.isClosedForSend) { + * // fast path + * return + * } else { + * // slow path: actually computing the value + * val nextElement = run { + * // some heavy computation + * } + * channel.send(nextElement) // can fail anyway, + * // but at least we tried to avoid the computation + * } + * ``` + * + * However, in many cases, even that can be achieved more idiomatically by cancelling the coroutine producing the + * elements to send. + * See [produce] for a way to launch a coroutine that produces elements and cancels itself when the channel is + * closed. + * + * [isClosedForSend] can also be used for assertions and diagnostics to verify the expected state of the channel. + * + * @see SendChannel.trySend + * @see SendChannel.close + * @see ReceiveChannel.cancel + */ + @DelicateCoroutinesApi + public val isClosedForSend: Boolean + + /** + * Sends the specified [element] to this channel. + * + * This function suspends if it does not manage to pass the element to the channel's buffer + * (or directly the receiving side if there's no buffer), + * and it can be cancelled with or without having successfully passed the element. + * See the "Suspending and cancellation" section below for details. + * If the channel is [closed][close], an exception is thrown (see below). + * + * ``` + * val channel = Channel() + * launch { + * check(channel.receive() == 5) + * } + * channel.send(5) // suspends until 5 is received + * ``` + * + * ## Suspending and cancellation + * + * If the [BufferOverflow] strategy of this channel is [BufferOverflow.SUSPEND], + * this function may suspend. + * The exact scenarios differ depending on the channel's capacity: + * - If the channel is [rendezvous][RENDEZVOUS], + * the sender will be suspended until the receiver calls [ReceiveChannel.receive]. + * - If the channel is [unlimited][UNLIMITED] or [conflated][CONFLATED], + * the sender will never be suspended even with the [BufferOverflow.SUSPEND] strategy. + * - If the channel is buffered (either [BUFFERED] or uses a non-default buffer capacity), + * the sender will be suspended until the buffer has free space. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if [send] managed to send the element, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + * + * Because of the prompt cancellation guarantee, an exception does not always mean a failure to deliver the element. + * See the "Undelivered elements" section in the [Channel] documentation + * for details on handling undelivered elements. + * + * Note that this function does not check for cancellation when it is not suspended. + * Use [ensureActive] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed: + * + * ``` + * // because of UNLIMITED, sending to this channel never suspends + * val channel = Channel(Channel.UNLIMITED) + * val job = launch { + * while (isActive) { + * channel.send(42) + * } + * // the loop exits when the job is cancelled + * } + * ``` + * + * This isn't needed if other cancellable functions are called inside the loop, like [delay]. + * + * ## Sending to a closed channel + * + * If a channel was [closed][close] before [send] was called and no cause was specified, + * an [ClosedSendChannelException] will be thrown from [send]. + * If a channel was [closed][close] with a cause before [send] was called, + * then [send] will rethrow the same (in the `===` sense) exception that was passed to [close]. + * + * In both cases, it is guaranteed that the element was not delivered to the consumer, + * and the `onUndeliveredElement` callback will be called. + * See the "Undelivered elements" section in the [Channel] documentation + * for details on handling undelivered elements. + * + * [Closing][close] a channel _after_ this function suspends does not cause this suspended [send] invocation + * to abort: although subsequent invocations of [send] fail, the existing ones will continue to completion, + * unless the sending coroutine is cancelled. + * + * ## Related + * + * This function can be used in [select] invocations with the [onSend] clause. + * Use [trySend] to try sending to this channel without waiting and throwing. + */ + public suspend fun send(element: E) + + /** + * Clause for the [select] expression of the [send] suspending function that selects when the element that is + * specified as the parameter is sent to the channel. + * When the clause is selected, the reference to this channel is passed into the corresponding block. + * + * The [select] invocation fails with an exception if the channel [is closed for `send`][isClosedForSend] before + * the [select] suspends (see the "Sending to a closed channel" section of [send]). + * + * Example: + * ``` + * val sendChannels = List(4) { index -> + * Channel(onUndeliveredElement = { + * println("Undelivered element $it for $index") + * }).also { channel -> + * // launch a consumer for this channel + * launch { + * withTimeout(1.seconds) { + * println("Consumer $index receives: ${channel.receive()}") + * } + * } + * } + * } + * val element = 42 + * select { + * for (channel in sendChannels) { + * channel.onSend(element) { + * println("Sent to channel $it") + * } + * } + * } + * ``` + * Here, we start a [select] expression that waits for exactly one of the four [onSend] invocations + * to successfully send the element to the receiver, + * and the other three will instead invoke the `onUndeliveredElement` callback. + * See the "Undelivered elements" section in the [Channel] documentation + * for details on handling undelivered elements. + * + * Like [send], [onSend] obeys the rules of prompt cancellation: + * [select] may finish with a [CancellationException] even if the element was successfully sent. + */ + public val onSend: SelectClause2> + + /** + * Attempts to add the specified [element] to this channel without waiting. + * + * [trySend] never suspends and never throws exceptions. + * Instead, it returns a [ChannelResult] that encapsulates the result of the operation. + * This makes it different from [send], which can suspend and throw exceptions. + * + * If this channel is currently full and cannot receive new elements at the time or is [closed][close], + * this function returns a result that indicates [a failure][ChannelResult.isFailure]. + * In this case, it is guaranteed that the element was not delivered to the consumer and the + * `onUndeliveredElement` callback, if one is provided during the [Channel]'s construction, does *not* get called. + * + * [trySend] can be used as a non-`suspend` alternative to [send] in cases where it's known beforehand + * that the channel's buffer can not overflow. + * ``` + * class Coordinates(val x: Int, val y: Int) + * // A channel for a single subscriber that stores the latest mouse position update. + * // If more than one subscriber is expected, consider using a `StateFlow` instead. + * val mousePositionUpdates = Channel(Channel.CONFLATED) + * // Notifies the subscriber about the new mouse position. + * // If the subscriber is slow, the intermediate updates are dropped. + * fun moveMouse(coordinates: Coordinates) { + * val result = mousePositionUpdates.trySend(coordinates) + * if (result.isClosed) { + * error("Mouse position is no longer being processed") + * } + * } + * ``` + */ + public fun trySend(element: E): ChannelResult + + /** + * Closes this channel so that subsequent attempts to [send] to it fail. + * + * Returns `true` if the channel was not closed previously and the call to this function closed it. + * If the channel was already closed, this function does nothing and returns `false`. + * + * The existing elements in the channel remain there, and likewise, + * the calls to [send] an [onSend] that have suspended before [close] was called will not be affected. + * Only the subsequent calls to [send], [trySend], or [onSend] will fail. + * [isClosedForSend] will start returning `true` immediately after this function is called. + * + * Once all the existing elements are received, the channel will be considered closed for `receive` as well. + * This means that [receive][ReceiveChannel.receive] will also start throwing exceptions. + * At that point, [isClosedForReceive][ReceiveChannel.isClosedForReceive] will start returning `true`. + * + * If the [cause] is non-null, it will be thrown from all the subsequent attempts to [send] to this channel, + * as well as from all the attempts to [receive][ReceiveChannel.receive] from the channel after no elements remain. + * + * If the [cause] is null, the channel is considered to have completed normally. + * All subsequent calls to [send] will throw a [ClosedSendChannelException], + * whereas calling [receive][ReceiveChannel.receive] will throw a [ClosedReceiveChannelException] + * after there are no more elements. + * + * ``` + * val channel = Channel() + * channel.send(1) + * channel.close() + * try { + * channel.send(2) + * error("The channel is closed, so this line is never reached") + * } catch (e: ClosedSendChannelException) { + * // expected + * } + * ``` + */ + public fun close(cause: Throwable? = null): Boolean + + /** + * Registers a [handler] that is synchronously invoked once the channel is [closed][close] + * or the receiving side of this channel is [cancelled][ReceiveChannel.cancel]. + * Only one handler can be attached to a channel during its lifetime. + * The `handler` is invoked when [isClosedForSend] starts to return `true`. + * If the channel is closed already, the handler is invoked immediately. + * + * The meaning of `cause` that is passed to the handler: + * - `null` if the channel was [closed][close] normally with `cause = null`. + * - Instance of [CancellationException] if the channel was [cancelled][ReceiveChannel.cancel] normally + * without the corresponding argument. + * - The cause of `close` or `cancel` otherwise. + * + * ### Execution context and exception safety + * + * The [handler] is executed as part of the closing or cancelling operation, + * and only after the channel reaches its final state. + * This means that if the handler throws an exception or hangs, + * the channel will still be successfully closed or cancelled. + * Unhandled exceptions from [handler] are propagated to the closing or cancelling operation's caller. + * + * Example of usage: + * ``` + * val events = Channel(Channel.UNLIMITED) + * callbackBasedApi.registerCallback { event -> + * events.trySend(event) + * .onClosed { /* channel is already closed, but the callback hasn't stopped yet */ } + * } + * + * val uiUpdater = uiScope.launch(Dispatchers.Main) { + * events.consume { /* handle events */ } + * } + * // Stop the callback after the channel is closed or cancelled + * events.invokeOnClose { callbackBasedApi.stop() } + * ``` + * + * **Stability note.** This function constitutes a stable API surface, with the only exception being + * that an [IllegalStateException] is thrown when multiple handlers are registered. + * This restriction could be lifted in the future. + * + * @throws UnsupportedOperationException if the underlying channel does not support [invokeOnClose]. + * Implementation note: currently, [invokeOnClose] is unsupported only by Rx-like integrations. + * + * @throws IllegalStateException if another handler was already registered + */ + public fun invokeOnClose(handler: (cause: Throwable?) -> Unit) + + /** + * **Deprecated** offer method. + * + * This method was deprecated in the favour of [trySend]. + * It has proven itself as the most error-prone method in Channel API: + * + * - `Boolean` return type creates the false sense of security, implying that `false` + * is returned instead of throwing an exception. + * - It was used mostly from non-suspending APIs where CancellationException triggered + * internal failures in the application (the most common source of bugs). + * - Due to signature and explicit `if (ch.offer(...))` checks it was easy to + * oversee such error during code review. + * - Its name was not aligned with the rest of the API and tried to mimic Java's queue instead. + * + * **NB** Automatic migration provides best-effort for the user experience, but requires removal + * or adjusting of the code that relied on the exception handling. + * The complete replacement has a more verbose form: + * ``` + * channel.trySend(element) + * .onClosed { throw it ?: ClosedSendChannelException("Channel was closed normally") } + * .isSuccess + * ``` + * + * See https://github.com/Kotlin/kotlinx.coroutines/issues/974 for more context. + * + * @suppress **Deprecated**. + */ + @Deprecated( + level = DeprecationLevel.ERROR, + message = "Deprecated in the favour of 'trySend' method", + replaceWith = ReplaceWith("trySend(element).isSuccess") + ) // Warning since 1.5.0, error since 1.6.0, not hidden until 1.8+ because API is quite widespread + public fun offer(element: E): Boolean { + val result = trySend(element) + if (result.isSuccess) return true + throw recoverStackTrace(result.exceptionOrNull() ?: return false) + } +} + +/** + * Receiver's interface to a [Channel]. + * + * Combined, [SendChannel] and [ReceiveChannel] define the complete [Channel] interface. + */ +public interface ReceiveChannel { + /** + * Returns `true` if the sending side of this channel was [closed][SendChannel.close] + * and all previously sent items were already received (which also happens for [cancelled][cancel] channels). + * + * Note that if this property returns `false`, + * it does not guarantee that a subsequent call to [receive] will succeed, + * as the channel can be concurrently cancelled or closed right after the check. + * For such scenarios, [receiveCatching] is the more robust solution: + * if the channel is closed, instead of throwing an exception, [receiveCatching] returns a result that allows + * querying it. + * + * ``` + * // DANGER! THIS CHECK IS NOT RELIABLE! + * if (!channel.isClosedForReceive) { + * channel.receive() // can still fail! + * } else { + * println("Can not receive: the channel is closed") + * null + * } + * // DO THIS INSTEAD: + * channel.receiveCatching().onClosed { + * println("Can not receive: the channel is closed") + * }.getOrNull() + * ``` + * + * The primary intended usage of this property is for assertions and diagnostics to verify the expected state of + * the channel. + * Using it in production code is discouraged. + * + * @see ReceiveChannel.receiveCatching + * @see ReceiveChannel.cancel + * @see SendChannel.close + */ + @DelicateCoroutinesApi + public val isClosedForReceive: Boolean + + /** + * Returns `true` if the channel contains no elements and isn't [closed for `receive`][isClosedForReceive]. + * + * If [isEmpty] returns `true`, it means that calling [receive] at exactly the same moment would suspend. + * However, calling [receive] immediately after checking [isEmpty] may or may not suspend, as new elements + * could have been added or removed or the channel could have been closed for `receive` between the two invocations. + * Consider using [tryReceive] in cases when suspensions are undesirable: + * + * ``` + * // DANGER! THIS CHECK IS NOT RELIABLE! + * while (!channel.isEmpty) { + * // can still suspend if other `receive` happens in parallel! + * val element = channel.receive() + * println(element) + * } + * // DO THIS INSTEAD: + * while (true) { + * val element = channel.tryReceive().getOrNull() ?: break + * println(element) + * } + * ``` + */ + @ExperimentalCoroutinesApi + public val isEmpty: Boolean + + /** + * Retrieves an element, removing it from the channel. + * + * This function suspends if the channel is empty, waiting until an element is available. + * If the channel is [closed for `receive`][isClosedForReceive], an exception is thrown (see below). + * ``` + * val channel = Channel() + * launch { + * val element = channel.receive() // suspends until 5 is available + * check(element == 5) + * } + * channel.send(5) + * ``` + * + * ## Suspending and cancellation + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if [receive] managed to retrieve the element from the channel, + * but was cancelled while suspended, [CancellationException] will be thrown, and, if + * the channel has an `onUndeliveredElement` callback installed, the retrieved element will be passed to it. + * See the "Undelivered elements" section in the [Channel] documentation + * for details on handling undelivered elements. + * See [suspendCancellableCoroutine] for the low-level details of prompt cancellation. + * + * Note that this function does not check for cancellation when it manages to immediately receive an element without + * suspending. + * Use [ensureActive] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed: + * + * ``` + * val channel = Channel() + * launch { // a very fast producer + * while (true) { + * channel.send(42) + * } + * } + * val consumer = launch { // a slow consumer + * while (isActive) { + * val element = channel.receive() + * // some slow computation involving `element` + * } + * } + * delay(100.milliseconds) + * consumer.cancelAndJoin() + * ``` + * + * ## Receiving from a closed channel + * + * - Attempting to [receive] from a [closed][SendChannel.close] channel while there are still some elements + * will successfully retrieve an element from the channel. + * - When a channel is [closed][SendChannel.close] and there are no elements remaining, + * the channel becomes [closed for `receive`][isClosedForReceive]. + * After that, + * [receive] will rethrow the same (in the `===` sense) exception that was passed to [SendChannel.close], + * or [ClosedReceiveChannelException] if none was given. + * + * ## Related + * + * This function can be used in [select] invocations with the [onReceive] clause. + * Use [tryReceive] to try receiving from this channel without waiting and throwing. + * Use [receiveCatching] to receive from this channel without throwing. + */ + public suspend fun receive(): E + + /** + * Clause for the [select] expression of the [receive] suspending function that selects with the element + * received from the channel. + * + * The [select] invocation fails with an exception if the channel [is closed for `receive`][isClosedForReceive] + * at any point, even if other [select] clauses could still work. + * + * Example: + * ``` + * class ScreenSize(val width: Int, val height: Int) + * class MouseClick(val x: Int, val y: Int) + * val screenResizes = Channel(Channel.CONFLATED) + * val mouseClicks = Channel(Channel.CONFLATED) + * + * launch(Dispatchers.Main) { + * while (true) { + * select { + * screenResizes.onReceive { newSize -> + * // update the UI to the new screen size + * } + * mouseClicks.onReceive { click -> + * // react to a mouse click + * } + * } + * } + * } + * ``` + * + * Like [receive], [onReceive] obeys the rules of prompt cancellation: + * [select] may finish with a [CancellationException] even if an element was successfully retrieved, + * in which case the `onUndeliveredElement` callback will be called. + */ + public val onReceive: SelectClause1 + + /** + * Retrieves an element, removing it from the channel. + * + * A difference from [receive] is that this function encapsulates a failure in its return value instead of throwing + * an exception. + * However, it will still throw [CancellationException] if the coroutine calling [receiveCatching] is cancelled. + * + * It is guaranteed that the only way this function can return a [failed][ChannelResult.isFailure] result is when + * the channel is [closed for `receive`][isClosedForReceive], so [ChannelResult.isClosed] is also true. + * + * This function suspends if the channel is empty, waiting until an element is available or the channel becomes + * closed. + * ``` + * val channel = Channel() + * launch { + * while (true) { + * val result = channel.receiveCatching() // suspends + * when (val element = result.getOrNull()) { + * null -> break // the channel is closed + * else -> check(element == 5) + * } + * } + * } + * channel.send(5) + * ``` + * + * ## Suspending and cancellation + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if [receiveCatching] managed to retrieve the element from the + * channel, but was cancelled while suspended, [CancellationException] will be thrown, and, if + * the channel has an `onUndeliveredElement` callback installed, the retrieved element will be passed to it. + * See the "Undelivered elements" section in the [Channel] documentation + * for details on handling undelivered elements. + * See [suspendCancellableCoroutine] for the low-level details of prompt cancellation. + * + * Note that this function does not check for cancellation when it manages to immediately receive an element without + * suspending. + * Use [ensureActive] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed: + * + * ``` + * val channel = Channel() + * launch { // a very fast producer + * while (true) { + * channel.send(42) + * } + * } + * val consumer = launch { // a slow consumer + * while (isActive) { + * val element = channel.receiveCatching().getOrNull() ?: break + * // some slow computation involving `element` + * } + * } + * delay(100.milliseconds) + * consumer.cancelAndJoin() + * ``` + * + * ## Receiving from a closed channel + * + * - Attempting to [receiveCatching] from a [closed][SendChannel.close] channel while there are still some elements + * will successfully retrieve an element from the channel. + * - When a channel is [closed][SendChannel.close] and there are no elements remaining, + * the channel becomes [closed for `receive`][isClosedForReceive]. + * After that, [receiveCatching] will return a result with [ChannelResult.isClosed] set. + * [ChannelResult.exceptionOrNull] will be the exact (in the `===` sense) exception + * that was passed to [SendChannel.close], + * or `null` if none was given. + * + * ## Related + * + * This function can be used in [select] invocations with the [onReceiveCatching] clause. + * Use [tryReceive] to try receiving from this channel without waiting and throwing. + * Use [receive] to receive from this channel and throw exceptions on error. + */ + public suspend fun receiveCatching(): ChannelResult + + /** + * Clause for the [select] expression of the [receiveCatching] suspending function that selects + * with a [ChannelResult] when an element is retrieved or the channel gets closed. + * + * Like [receiveCatching], [onReceiveCatching] obeys the rules of prompt cancellation: + * [select] may finish with a [CancellationException] even if an element was successfully retrieved, + * in which case the `onUndeliveredElement` callback will be called. + */ + // TODO: think of an example of when this could be useful + public val onReceiveCatching: SelectClause1> + + /** + * Attempts to retrieve an element without waiting, removing it from the channel. + * + * - When the channel is non-empty, a [successful][ChannelResult.isSuccess] result is returned, + * and [ChannelResult.getOrNull] returns the retrieved element. + * - When the channel is empty, a [failed][ChannelResult.isFailure] result is returned. + * - When the channel is already [closed for `receive`][isClosedForReceive], + * returns the ["channel is closed"][ChannelResult.isClosed] result. + * If the channel was [closed][SendChannel.close] with a cause (for example, [cancelled][cancel]), + * [ChannelResult.exceptionOrNull] contains the cause. + * + * This function is useful when implementing on-demand allocation of resources to be stored in the channel: + * + * ``` + * val resourcePool = Channel(maxResources) + * + * suspend fun withResource(block: (Resource) -> Unit) { + * val result = resourcePool.tryReceive() + * val resource = result.getOrNull() + * ?: tryCreateNewResource() // try to create a new resource + * ?: resourcePool.receive() // could not create: actually wait for the resource + * try { + * block(resource) + * } finally { + * resourcePool.trySend(resource) + * } + * } + * ``` + */ + public fun tryReceive(): ChannelResult + + /** + * Returns a new iterator to receive elements from this channel using a `for` loop. + * Iteration completes normally when the channel [is closed for `receive`][isClosedForReceive] without a cause and + * throws the exception passed to [close][SendChannel.close] if there was one. + * + * Instances of [ChannelIterator] are not thread-safe and shall not be used from concurrent coroutines. + * + * Example: + * + * ``` + * val channel = produce { + * repeat(1000) { + * send(it) + * } + * } + * for (v in channel) { + * println(v) + * } + * ``` + * + * Note that if an early return happens from the `for` loop, the channel does not get cancelled. + * To forbid sending new elements after the iteration is completed, use [consumeEach] or + * call [cancel] manually. + */ + public operator fun iterator(): ChannelIterator + + /** + * [Closes][SendChannel.close] the channel for new elements and removes all existing ones. + * + * A [cause] can be used to specify an error message or to provide other details on + * the cancellation reason for debugging purposes. + * If the cause is not specified, then an instance of [CancellationException] with a + * default message is created to [close][SendChannel.close] the channel. + * + * If the channel was already [closed][SendChannel.close], + * [cancel] only has the effect of removing all elements from the channel. + * + * Immediately after the invocation of this function, + * [isClosedForReceive] and, on the [SendChannel] side, [isClosedForSend][SendChannel.isClosedForSend] + * start returning `true`. + * Any attempt to send to or receive from this channel will lead to a [CancellationException]. + * This also applies to the existing senders and receivers that are suspended at the time of the call: + * they will be resumed with a [CancellationException] immediately after [cancel] is called. + * + * If the channel has an `onUndeliveredElement` callback installed, this function will invoke it for each of the + * elements still in the channel, since these elements will be inaccessible otherwise. + * If the callback is not installed, these elements will simply be removed from the channel for garbage collection. + * + * ``` + * val channel = Channel() + * channel.send(1) + * channel.send(2) + * channel.cancel() + * channel.trySend(3) // returns ChannelResult.isClosed + * for (element in channel) { println(element) } // prints nothing + * ``` + * + * [consume] and [consumeEach] are convenient shorthands for cancelling the channel after the single consumer + * has finished processing. + */ + public fun cancel(cause: CancellationException? = null) + + /** + * @suppress This method implements old version of JVM ABI. Use [cancel]. + */ + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") + public fun cancel(): Unit = cancel(null) + + /** + * @suppress This method has bad semantics when cause is not a [CancellationException]. Use [cancel]. + */ + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") + public fun cancel(cause: Throwable? = null): Boolean + + /** + * **Deprecated** poll method. + * + * This method was deprecated in the favour of [tryReceive]. + * It has proven itself as error-prone method in Channel API: + * + * - Nullable return type creates the false sense of security, implying that `null` + * is returned instead of throwing an exception. + * - It was used mostly from non-suspending APIs where CancellationException triggered + * internal failures in the application (the most common source of bugs). + * - Its name was not aligned with the rest of the API and tried to mimic Java's queue instead. + * + * See https://github.com/Kotlin/kotlinx.coroutines/issues/974 for more context. + * + * ### Replacement note + * + * The replacement `tryReceive().getOrNull()` is a default that ignores all close exceptions and + * proceeds with `null`, while `poll` throws an exception if the channel was closed with an exception. + * Replacement with the very same 'poll' semantics is `tryReceive().onClosed { if (it != null) throw it }.getOrNull()` + * + * @suppress **Deprecated**. + */ + @Deprecated( + level = DeprecationLevel.ERROR, + message = "Deprecated in the favour of 'tryReceive'. " + + "Please note that the provided replacement does not rethrow channel's close cause as 'poll' did, " + + "for the precise replacement please refer to the 'poll' documentation", + replaceWith = ReplaceWith("tryReceive().getOrNull()") + ) // Warning since 1.5.0, error since 1.6.0, not hidden until 1.8+ because API is quite widespread + public fun poll(): E? { + val result = tryReceive() + if (result.isSuccess) return result.getOrThrow() + throw recoverStackTrace(result.exceptionOrNull() ?: return null) + } + + /** + * This function was deprecated since 1.3.0 and is no longer recommended to use + * or to implement in subclasses. + * + * It had the following pitfalls: + * - Didn't allow to distinguish 'null' as "closed channel" from "null as a value" + * - Was throwing if the channel has failed even though its signature may suggest it returns 'null' + * - It didn't really belong to core channel API and can be exposed as an extension instead. + * + * ### Replacement note + * + * The replacement `receiveCatching().getOrNull()` is a safe default that ignores all close exceptions and + * proceeds with `null`, while `receiveOrNull` throws an exception if the channel was closed with an exception. + * Replacement with the very same `receiveOrNull` semantics is `receiveCatching().onClosed { if (it != null) throw it }.getOrNull()`. + * + * @suppress **Deprecated** + */ + @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + @LowPriorityInOverloadResolution + @Deprecated( + message = "Deprecated in favor of 'receiveCatching'. " + + "Please note that the provided replacement does not rethrow channel's close cause as 'receiveOrNull' did, " + + "for the detailed replacement please refer to the 'receiveOrNull' documentation", + level = DeprecationLevel.ERROR, + replaceWith = ReplaceWith("receiveCatching().getOrNull()") + ) // Warning since 1.3.0, error in 1.5.0, cannot be hidden due to deprecated extensions + public suspend fun receiveOrNull(): E? = receiveCatching().getOrNull() + + /** + * This function was deprecated since 1.3.0 and is no longer recommended to use + * or to implement in subclasses. + * See [receiveOrNull] documentation. + * + * @suppress **Deprecated**: in favor of onReceiveCatching extension. + */ + @Suppress("DEPRECATION_ERROR") + @Deprecated( + message = "Deprecated in favor of onReceiveCatching extension", + level = DeprecationLevel.ERROR, + replaceWith = ReplaceWith("onReceiveCatching") + ) // Warning since 1.3.0, error in 1.5.0, will be hidden or removed in 1.7.0 + public val onReceiveOrNull: SelectClause1 get() = (this as BufferedChannel).onReceiveOrNull +} + +/** + * A discriminated union representing a channel operation result. + * It encapsulates the knowledge of whether the operation succeeded, failed with an option to retry, + * or failed because the channel was closed. + * + * If the operation was [successful][isSuccess], [T] is the result of the operation: + * for example, for [ReceiveChannel.receiveCatching] and [ReceiveChannel.tryReceive], + * it is the element received from the channel, and for [Channel.trySend], it is [Unit], + * as the channel does not receive anything in return for sending a channel. + * This value can be retrieved with [getOrNull] or [getOrThrow]. + * + * If the operation [failed][isFailure], it does not necessarily mean that the channel itself is closed. + * For example, [ReceiveChannel.receiveCatching] and [ReceiveChannel.tryReceive] can fail because the channel is empty, + * and [Channel.trySend] can fail because the channel is full. + * + * If the operation [failed][isFailure] because the channel was closed for that operation, [isClosed] returns `true`. + * The opposite is also true: if [isClosed] returns `true`, then the channel is closed for that operation + * ([ReceiveChannel.isClosedForReceive] or [SendChannel.isClosedForSend]). + * In this case, retrying the operation is meaningless: once closed, the channel will remain closed. + * The [exceptionOrNull] function returns the reason the channel was closed, if any was given. + * + * Manually obtaining a [ChannelResult] instance is not supported. + * See the documentation for [ChannelResult]-returning functions for usage examples. + */ +@JvmInline +public value class ChannelResult +@PublishedApi internal constructor(@PublishedApi internal val holder: Any?) { + /** + * Whether the operation succeeded. + * + * If this returns `true`, the operation was successful. + * In this case, [getOrNull] and [getOrThrow] can be used to retrieve the value. + * + * If this returns `false`, the operation failed. + * [isClosed] can be used to determine whether the operation failed because the channel was closed + * (and therefore retrying the operation is meaningless). + * + * ``` + * val result = channel.tryReceive() + * if (result.isSuccess) { + * println("Successfully received the value ${result.getOrThrow()}") + * } else { + * println("Failed to receive the value.") + * if (result.isClosed) { + * println("The channel is closed.") + * if (result.exceptionOrNull() != null) { + * println("The reason: ${result.exceptionOrNull()}") + * } + * } + * } + * ``` + * + * [isFailure] is a shorthand for `!isSuccess`. + * [getOrNull] can simplify [isSuccess] followed by [getOrThrow] into just one check if [T] is known + * to be non-nullable. + */ + public val isSuccess: Boolean get() = holder !is Failed + + /** + * Whether the operation failed. + * + * A shorthand for `!isSuccess`. See [isSuccess] for more details. + */ + public val isFailure: Boolean get() = holder is Failed + + /** + * Whether the operation failed because the channel was closed. + * + * If this returns `true`, the channel was closed for the operation that returned this result. + * In this case, retrying the operation is meaningless: once closed, the channel will remain closed. + * [isSuccess] will return `false`. + * [exceptionOrNull] can be used to determine the reason the channel was [closed][SendChannel.close] + * if one was given. + * + * If this returns `false`, subsequent attempts to perform the same operation may succeed. + * + * ``` + * val result = channel.trySend(42) + * if (result.isClosed) { + * println("The channel is closed.") + * if (result.exceptionOrNull() != null) { + * println("The reason: ${result.exceptionOrNull()}") + * } + * } + */ + public val isClosed: Boolean get() = holder is Closed + + /** + * Returns the encapsulated [T] if the operation succeeded, or `null` if it failed. + * + * For non-nullable [T], the following code can be used to handle the result: + * ``` + * val result = channel.tryReceive() + * val value = result.getOrNull() + * if (value == null) { + * if (result.isClosed) { + * println("The channel is closed.") + * if (result.exceptionOrNull() != null) { + * println("The reason: ${result.exceptionOrNull()}") + * } + * } + * return + * } + * println("Successfully received the value $value") + * ``` + * + * If [T] is nullable, [getOrThrow] together with [isSuccess] is a more reliable way to handle the result. + */ + @Suppress("UNCHECKED_CAST") + public fun getOrNull(): T? = if (holder !is Failed) holder as T else null + + /** + * Returns the encapsulated [T] if the operation succeeded, or throws the encapsulated exception if it failed. + * + * Example: + * ``` + * val result = channel.tryReceive() + * if (result.isSuccess) { + * println("Successfully received the value ${result.getOrThrow()}") + * } + * ``` + * + * @throws IllegalStateException if the operation failed, but the channel was not closed with a cause. + */ + public fun getOrThrow(): T { + @Suppress("UNCHECKED_CAST") + if (holder !is Failed) return holder as T + if (holder is Closed) { + check(holder.cause != null) { "Trying to call 'getOrThrow' on a channel closed without a cause" } + throw holder.cause + } + error("Trying to call 'getOrThrow' on a failed result of a non-closed channel") + } + + /** + * Returns the exception with which the channel was closed, or `null` if the channel was not closed or was closed + * without a cause. + * + * [exceptionOrNull] can only return a non-`null` value if [isClosed] is `true`, + * but even if [isClosed] is `true`, + * [exceptionOrNull] can still return `null` if the channel was closed without a cause. + * + * ``` + * val result = channel.tryReceive() + * if (result.isClosed) { + * // Now we know not to retry the operation later. + * // Check if the channel was closed with a cause and rethrow the exception: + * result.exceptionOrNull()?.let { throw it } + * // Otherwise, the channel was closed without a cause. + * } + * ``` + */ + public fun exceptionOrNull(): Throwable? = (holder as? Closed)?.cause + + internal open class Failed { + override fun toString(): String = "Failed" + } + + internal class Closed(@JvmField val cause: Throwable?): Failed() { + override fun equals(other: Any?): Boolean = other is Closed && cause == other.cause + override fun hashCode(): Int = cause.hashCode() + override fun toString(): String = "Closed($cause)" + } + + /** + * @suppress **This is internal API and it is subject to change.** + */ + @InternalCoroutinesApi + public companion object { + private val failed = Failed() + + @InternalCoroutinesApi + public fun success(value: E): ChannelResult = + ChannelResult(value) + + @InternalCoroutinesApi + public fun failure(): ChannelResult = + ChannelResult(failed) + + @InternalCoroutinesApi + public fun closed(cause: Throwable?): ChannelResult = + ChannelResult(Closed(cause)) + } + + public override fun toString(): String = + when (holder) { + is Closed -> holder.toString() + else -> "Value($holder)" + } +} + +/** + * Returns the encapsulated value if the operation [succeeded][ChannelResult.isSuccess], or the + * result of [onFailure] function for [ChannelResult.exceptionOrNull] otherwise. + * + * A shorthand for `if (isSuccess) getOrNull() else onFailure(exceptionOrNull())`. + * + * @see ChannelResult.getOrNull + * @see ChannelResult.exceptionOrNull + */ +@OptIn(ExperimentalContracts::class) +public inline fun ChannelResult.getOrElse(onFailure: (exception: Throwable?) -> T): T { + contract { + callsInPlace(onFailure, InvocationKind.AT_MOST_ONCE) + } + @Suppress("UNCHECKED_CAST") + return if (holder is ChannelResult.Failed) onFailure(exceptionOrNull()) else holder as T +} + +/** + * Performs the given [action] on the encapsulated value if the operation [succeeded][ChannelResult.isSuccess]. + * Returns the original `ChannelResult` unchanged. + * + * A shorthand for `this.also { if (isSuccess) action(getOrThrow()) }`. + */ +@OptIn(ExperimentalContracts::class) +public inline fun ChannelResult.onSuccess(action: (value: T) -> Unit): ChannelResult { + contract { + callsInPlace(action, InvocationKind.AT_MOST_ONCE) + } + @Suppress("UNCHECKED_CAST") + if (holder !is ChannelResult.Failed) action(holder as T) + return this +} + +/** + * Performs the given [action] if the operation [failed][ChannelResult.isFailure]. + * The result of [ChannelResult.exceptionOrNull] is passed to the [action] parameter. + * + * Returns the original `ChannelResult` unchanged. + * + * A shorthand for `this.also { if (isFailure) action(exceptionOrNull()) }`. + */ +@OptIn(ExperimentalContracts::class) +public inline fun ChannelResult.onFailure(action: (exception: Throwable?) -> Unit): ChannelResult { + contract { + callsInPlace(action, InvocationKind.AT_MOST_ONCE) + } + if (holder is ChannelResult.Failed) action(exceptionOrNull()) + return this +} + +/** + * Performs the given [action] if the operation failed because the channel was [closed][ChannelResult.isClosed] for + * that operation. + * The result of [ChannelResult.exceptionOrNull] is passed to the [action] parameter. + * + * It is guaranteed that if action is invoked, then the channel is either [closed for send][Channel.isClosedForSend] + * or is [closed for receive][Channel.isClosedForReceive] depending on the failed operation. + * + * Returns the original `ChannelResult` unchanged. + * + * A shorthand for `this.also { if (isClosed) action(exceptionOrNull()) }`. + */ +@OptIn(ExperimentalContracts::class) +public inline fun ChannelResult.onClosed(action: (exception: Throwable?) -> Unit): ChannelResult { + contract { + callsInPlace(action, InvocationKind.AT_MOST_ONCE) + } + if (holder is ChannelResult.Closed) action(exceptionOrNull()) + return this +} + +/** + * Iterator for a [ReceiveChannel]. + * Instances of this interface are *not thread-safe* and shall not be used from concurrent coroutines. + */ +public interface ChannelIterator { + /** + * Prepare an element for retrieval by the invocation of [next]. + * + * - If the element that was retrieved by an earlier [hasNext] call was not yet consumed by [next], returns `true`. + * - If the channel has an element available, returns `true` and removes it from the channel. + * This element will be returned by the subsequent invocation of [next]. + * - If the channel is [closed for receiving][ReceiveChannel.isClosedForReceive] without a cause, returns `false`. + * - If the channel is closed with a cause, throws the original [close][SendChannel.close] cause exception. + * - If the channel is not closed but does not contain an element, + * suspends until either an element is sent to the channel or the channel gets closed. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if [hasNext] retrieves the element from the channel during + * its operation, but was cancelled while suspended, [CancellationException] will be thrown. + * See [suspendCancellableCoroutine] for low-level details. + * + * Because of the prompt cancellation guarantee, some values retrieved from the channel can become lost. + * See the "Undelivered elements" section in the [Channel] documentation + * for details on handling undelivered elements. + * + * Note that this function does not check for cancellation when it is not suspended, that is, + * if the next element is immediately available. + * Use [ensureActive] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. + */ + public suspend operator fun hasNext(): Boolean + + @Deprecated(message = "Since 1.3.0, binary compatibility with versions <= 1.2.x", level = DeprecationLevel.HIDDEN) + @Suppress("INAPPLICABLE_JVM_NAME") + @JvmName("next") + public suspend fun next0(): E { + /* + * Before 1.3.0 the "next()" could have been used without invoking "hasNext" first and there were code samples + * demonstrating this behavior, so we preserve this logic for full binary backwards compatibility with previously + * compiled code. + */ + if (!hasNext()) throw ClosedReceiveChannelException(DEFAULT_CLOSE_MESSAGE) + return next() + } + + /** + * Retrieves the element removed from the channel by the preceding call to [hasNext], or + * throws an [IllegalStateException] if [hasNext] was not invoked. + * + * This method can only be used together with [hasNext]: + * ``` + * while (iterator.hasNext()) { + * val element = iterator.next() + * // ... handle the element ... + * } + * ``` + * + * A more idiomatic way to iterate over a channel is to use a `for` loop: + * ``` + * for (element in channel) { + * // ... handle the element ... + * } + * ``` + * + * This method never throws if [hasNext] returned `true`. + * If [hasNext] threw the cause with which the channel was closed, this method will rethrow the same exception. + * If [hasNext] returned `false` because the channel was closed without a cause, this method throws + * a [ClosedReceiveChannelException]. + */ + public operator fun next(): E +} + +/** + * Channel is a non-blocking primitive for communication between a sender (via [SendChannel]) and a receiver (via [ReceiveChannel]). + * Conceptually, a channel is similar to `java.util.concurrent.BlockingQueue`, + * but it has suspending operations instead of blocking ones and can be [closed][SendChannel.close]. + * + * ### Channel capacity + * + * Most ways to create a [Channel] (in particular, the `Channel()` factory function) allow specifying a capacity, + * which determines how elements are buffered in the channel. + * There are several predefined constants for the capacity that have special behavior: + * + * - [Channel.RENDEZVOUS] (or 0) creates a _rendezvous_ channel, which does not have a buffer at all. + * Instead, the sender and the receiver must rendezvous (meet): + * [SendChannel.send] suspends until another coroutine invokes [ReceiveChannel.receive], and vice versa. + * - [Channel.CONFLATED] creates a buffer for a single element and automatically changes the + * [buffer overflow strategy][BufferOverflow] to [BufferOverflow.DROP_OLDEST]. + * - [Channel.UNLIMITED] creates a channel with an unlimited buffer, which never suspends the sender. + * - [Channel.BUFFERED] creates a channel with a buffer whose size depends on + * the [buffer overflow strategy][BufferOverflow]. + * + * See each constant's documentation for more details. + * + * If the capacity is positive but less than [Channel.UNLIMITED], the channel has a buffer with the specified capacity. + * It is safe to construct a channel with a large buffer, as memory is only allocated gradually as elements are added. + * + * Constructing a channel with a negative capacity not equal to a predefined constant is not allowed + * and throws an [IllegalArgumentException]. + * + * ### Buffer overflow + * + * Some ways to create a [Channel] also expose a [BufferOverflow] parameter (by convention, `onBufferOverflow`), + * which does not affect the receiver but determines the behavior of the sender when the buffer is full. + * The options include [suspending][BufferOverflow.SUSPEND] until there is space in the buffer, + * [dropping the oldest element][BufferOverflow.DROP_OLDEST] to make room for the new one, or + * [dropping the element to be sent][BufferOverflow.DROP_LATEST]. See the [BufferOverflow] documentation. + * + * By convention, the default value for [BufferOverflow] whenever it can not be configured is [BufferOverflow.SUSPEND]. + * + * See the [Channel.RENDEZVOUS], [Channel.CONFLATED], and [Channel.UNLIMITED] documentation for a description of how + * they interact with the [BufferOverflow] parameter. + * + * ### Prompt cancellation guarantee + * + * All suspending functions with channels provide **prompt cancellation guarantee**. + * If the job was cancelled while send or receive function was suspended, it will not resume successfully, even if it + * already changed the channel's state, but throws a [CancellationException]. + * With a single-threaded [dispatcher][CoroutineDispatcher] like [Dispatchers.Main], this gives a + * guarantee that the coroutine promptly reacts to the cancellation of its [Job] and does not resume its execution. + * + * > **Prompt cancellation guarantee** for channel operations was added in `kotlinx.coroutines` version `1.4.0` + * > and has replaced the channel-specific atomic cancellation that was not consistent with other suspending functions. + * > The low-level mechanics of prompt cancellation are explained in the [suspendCancellableCoroutine] documentation. + * + * ### Undelivered elements + * + * As a result of the prompt cancellation guarantee, when a closeable resource + * (like an open file or a handle to another native resource) is transferred via a channel, + * it can be successfully extracted from the channel, + * but still be lost if the receiving operation is cancelled in parallel. + * + * The `Channel()` factory function has the optional parameter `onUndeliveredElement`. + * When that parameter is set, the corresponding function is called once for each element + * that was sent to the channel with the call to the [send][SendChannel.send] function but failed to be delivered, + * which can happen in the following cases: + * + * - When an element is dropped due to the limited buffer capacity. + * This can happen when the overflow strategy is [BufferOverflow.DROP_LATEST] or [BufferOverflow.DROP_OLDEST]. + * - When the sending operations like [send][SendChannel.send] or [onSend][SendChannel.onSend] + * throw an exception because it was cancelled + * before it had a chance to actually send the element + * or because the channel was [closed][SendChannel.close] or [cancelled][ReceiveChannel.cancel]. + * - When the receiving operations like [receive][ReceiveChannel.receive], + * [onReceive][ReceiveChannel.onReceive], or [hasNext][ChannelIterator.hasNext] + * throw an exception after retrieving the element from the channel + * because of being cancelled before the code following them had a chance to resume. + * - When the channel was [cancelled][ReceiveChannel.cancel], in which case `onUndeliveredElement` is called on every + * remaining element in the channel's buffer. + * + * Note that `onUndeliveredElement` is called synchronously in an arbitrary context. + * It should be fast, non-blocking, and should not throw exceptions. + * Any exception thrown by `onUndeliveredElement` is wrapped into an internal runtime exception + * which is either rethrown from the caller method or handed off to the exception handler in the current context + * (see [CoroutineExceptionHandler]) when one is available. + * + * A typical usage for `onUndeliveredElement` is to close a resource that is being transferred via the channel. The + * following code pattern guarantees that opened resources are closed even if the producer, the consumer, + * and/or the channel are cancelled. Resources are never lost. + * + * ``` + * // Create a channel with an onUndeliveredElement block that closes a resource + * val channel = Channel(onUndeliveredElement = { resource -> resource.close() }) + * + * // Producer code + * val resourceToSend = openResource() + * channel.send(resourceToSend) + * + * // Consumer code + * val resourceReceived = channel.receive() + * try { + * // work with received resource + * } finally { + * resourceReceived.close() + * } + * ``` + * + * > Note that if any work happens between `openResource()` and `channel.send(...)`, + * > it is your responsibility to ensure that resource gets closed in case this additional code fails. + */ +public interface Channel : SendChannel, ReceiveChannel { + /** + * Constants for the channel factory function `Channel()`. + */ + public companion object Factory { + /** + * An unlimited buffer capacity. + * + * `Channel(UNLIMITED)` creates a channel with an unlimited buffer, which never suspends the sender. + * The total amount of elements that can be sent to the channel is limited only by the available memory. + * + * If [BufferOverflow] is specified for the channel, it is completely ignored, + * as the channel never suspends the sender. + * + * ``` + * val channel = Channel(Channel.UNLIMITED) + * repeat(1000) { + * channel.trySend(it) + * } + * repeat(1000) { + * check(channel.tryReceive().getOrNull() == it) + * } + * ``` + */ + public const val UNLIMITED: Int = Int.MAX_VALUE + + /** + * The zero buffer capacity. + * + * For the default [BufferOverflow] value of [BufferOverflow.SUSPEND], + * `Channel(RENDEZVOUS)` creates a channel without a buffer. + * An element is transferred from the sender to the receiver only when [send] and [receive] invocations meet + * in time (that is, they _rendezvous_), + * so [send] suspends until another coroutine invokes [receive], + * and [receive] suspends until another coroutine invokes [send]. + * + * ``` + * val channel = Channel(Channel.RENDEZVOUS) + * check(channel.trySend(5).isFailure) // sending fails: no receiver is waiting + * launch(start = CoroutineStart.UNDISPATCHED) { + * val element = channel.receive() // suspends + * check(element == 3) + * } + * check(channel.trySend(3).isSuccess) // sending succeeds: receiver is waiting + * ``` + * + * If a different [BufferOverflow] is specified, + * `Channel(RENDEZVOUS)` creates a channel with a buffer of size 1: + * + * ``` + * val channel = Channel(0, onBufferOverflow = BufferOverflow.DROP_OLDEST) + * // None of the calls suspend, since the buffer overflow strategy is not SUSPEND + * channel.send(1) + * channel.send(2) + * channel.send(3) + * check(channel.receive() == 3) + * ``` + */ + public const val RENDEZVOUS: Int = 0 + + /** + * A single-element buffer with conflating behavior. + * + * Specifying [CONFLATED] as the capacity in the `Channel(...)` factory function is equivalent to + * creating a channel with a buffer of size 1 and a [BufferOverflow] strategy of [BufferOverflow.DROP_OLDEST]: + * `Channel(1, onBufferOverflow = BufferOverflow.DROP_OLDEST)`. + * Such a channel buffers at most one element and conflates all subsequent `send` and `trySend` invocations + * so that the receiver always gets the last element sent, **losing** the previously sent elements: + * see the "Undelivered elements" section in the [Channel] documentation. + * [Sending][send] to this channel never suspends, and [trySend] always succeeds. + * + * ``` + * val channel = Channel(Channel.CONFLATED) + * channel.send(1) + * channel.send(2) + * channel.send(3) + * check(channel.receive() == 3) + * ``` + * + * Specifying a [BufferOverflow] other than [BufferOverflow.SUSPEND] is not allowed with [CONFLATED], and + * an [IllegalArgumentException] is thrown if such a combination is used. + * For creating a conflated channel that instead keeps the existing element in the channel and throws out + * the new one, use `Channel(1, onBufferOverflow = BufferOverflow.DROP_LATEST)`. + */ + public const val CONFLATED: Int = -1 + + /** + * A channel capacity marker that is substituted by the default buffer capacity. + * + * When passed as a parameter to the `Channel(...)` factory function, the default buffer capacity is used. + * For [BufferOverflow.SUSPEND] (the default buffer overflow strategy), the default capacity is 64, + * but on the JVM it can be overridden by setting the [DEFAULT_BUFFER_PROPERTY_NAME] system property. + * The overridden value is used for all channels created with a default buffer capacity, + * including those created in third-party libraries. + * + * ``` + * val channel = Channel(Channel.BUFFERED) + * repeat(100) { + * channel.trySend(it) + * } + * channel.close() + * // The check can fail if the default buffer capacity is changed + * check(channel.toList() == (0..<64).toList()) + * ``` + * + * If a different [BufferOverflow] is specified, `Channel(BUFFERED)` creates a channel with a buffer of size 1: + * + * ``` + * val channel = Channel(Channel.BUFFERED, onBufferOverflow = BufferOverflow.DROP_OLDEST) + * channel.send(1) + * channel.send(2) + * channel.send(3) + * channel.close() + * check(channel.toList() == listOf(3)) + * ``` + */ + public const val BUFFERED: Int = -2 + + // only for internal use, cannot be used with Channel(...) + internal const val OPTIONAL_CHANNEL = -3 + + /** + * Name of the JVM system property for the default channel capacity (64 by default). + * + * See [BUFFERED] for details on how this property is used. + * + * Setting this property affects the default channel capacity for channel constructors, + * channel-backed coroutines and flow operators that imply channel usage, + * including ones defined in 3rd-party libraries. + * + * Usage of this property is highly discouraged and is intended to be used as a last-ditch effort + * as an immediate measure for hot fixes and duct-taping. + */ + @DelicateCoroutinesApi + public const val DEFAULT_BUFFER_PROPERTY_NAME: String = "kotlinx.coroutines.channels.defaultBuffer" + + internal val CHANNEL_DEFAULT_CAPACITY = systemProp(DEFAULT_BUFFER_PROPERTY_NAME, + 64, 1, UNLIMITED - 1 + ) + } +} + +/** + * Creates a channel. See the [Channel] interface documentation for details. + * + * This function is the most flexible way to create a channel. + * It allows specifying the channel's capacity, buffer overflow strategy, and an optional function to call + * to handle undelivered elements. + * + * ``` + * val allocatedResources = HashSet() + * // An autocloseable resource that must be closed when it is no longer needed + * class Resource(val id: Int): AutoCloseable { + * init { + * allocatedResources.add(id) + * } + * override fun close() { + * allocatedResources.remove(id) + * } + * } + * // A channel with a 15-element buffer that drops the oldest element on buffer overflow + * // and closes the elements that were not delivered to the consumer + * val channel = Channel( + * capacity = 15, + * onBufferOverflow = BufferOverflow.DROP_OLDEST, + * onUndeliveredElement = { element -> element.close() } + * ) + * // A sender's view of the channel + * val sendChannel: SendChannel = channel + * repeat(100) { + * sendChannel.send(Resource(it)) + * } + * sendChannel.close() + * // A receiver's view of the channel + * val receiveChannel: ReceiveChannel = channel + * val receivedResources = receiveChannel.toList() + * // Check that the last 15 sent resources were received + * check(receivedResources.map { it.id } == (85 until 100).toList()) + * // Close the resources that were successfully received + * receivedResources.forEach { it.close() } + * // The dropped resources were closed by the channel itself + * check(allocatedResources.isEmpty()) + * ``` + * + * For a full explanation of every parameter and their interaction, see the [Channel] interface documentation. + * + * @param capacity either a positive channel capacity or one of the constants defined in [Channel.Factory]. + * See the "Channel capacity" section in the [Channel] documentation. + * @param onBufferOverflow configures an action on buffer overflow. + * See the "Buffer overflow" section in the [Channel] documentation. + * @param onUndeliveredElement a function that is called when element was sent but was not delivered to the consumer. + * See the "Undelivered elements" section in the [Channel] documentation. + * @throws IllegalArgumentException when [capacity] < -2 + */ +public fun Channel( + capacity: Int = RENDEZVOUS, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND, + onUndeliveredElement: ((E) -> Unit)? = null +): Channel = + when (capacity) { + RENDEZVOUS -> { + if (onBufferOverflow == BufferOverflow.SUSPEND) + BufferedChannel(RENDEZVOUS, onUndeliveredElement) // an efficient implementation of rendezvous channel + else + ConflatedBufferedChannel(1, onBufferOverflow, onUndeliveredElement) // support buffer overflow with buffered channel + } + CONFLATED -> { + require(onBufferOverflow == BufferOverflow.SUSPEND) { + "CONFLATED capacity cannot be used with non-default onBufferOverflow" + } + ConflatedBufferedChannel(1, BufferOverflow.DROP_OLDEST, onUndeliveredElement) + } + UNLIMITED -> BufferedChannel(UNLIMITED, onUndeliveredElement) // ignores onBufferOverflow: it has buffer, but it never overflows + BUFFERED -> { // uses default capacity with SUSPEND + if (onBufferOverflow == BufferOverflow.SUSPEND) BufferedChannel(CHANNEL_DEFAULT_CAPACITY, onUndeliveredElement) + else ConflatedBufferedChannel(1, onBufferOverflow, onUndeliveredElement) + } + else -> { + if (onBufferOverflow === BufferOverflow.SUSPEND) BufferedChannel(capacity, onUndeliveredElement) + else ConflatedBufferedChannel(capacity, onBufferOverflow, onUndeliveredElement) + } + } + +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.4.0, binary compatibility with earlier versions") +public fun Channel(capacity: Int = RENDEZVOUS): Channel = Channel(capacity) + +/** + * Indicates an attempt to [send][SendChannel.send] to a [closed-for-sending][SendChannel.isClosedForSend] channel + * that was [closed][SendChannel.close] without a cause. + * + * If a cause was provided, that cause is thrown from [send][SendChannel.send] instead of this exception. + * In particular, if the channel was closed because it was [cancelled][ReceiveChannel.cancel], + * this exception will never be thrown: either the `cause` of the cancellation is thrown, + * or a new [CancellationException] gets constructed to be thrown from [SendChannel.send]. + * + * This exception is a subclass of [IllegalStateException], because the sender should not attempt to send to a closed + * channel after it itself has [closed][SendChannel.close] it, and indicates an error on the part of the programmer. + * Usually, this exception can be avoided altogether by restructuring the code. + */ +public class ClosedSendChannelException(message: String?) : IllegalStateException(message) + +/** + * Indicates an attempt to [receive][ReceiveChannel.receive] from a + * [closed-for-receiving][ReceiveChannel.isClosedForReceive] channel + * that was [closed][SendChannel.close] without a cause. + * + * If a clause was provided, that clause is thrown from [receive][ReceiveChannel.receive] instead of this exception. + * In particular, if the channel was closed because it was [cancelled][ReceiveChannel.cancel], + * this exception will never be thrown: either the `cause` of the cancellation is thrown, + * or a new [CancellationException] gets constructed to be thrown from [ReceiveChannel.receive]. + * + * This exception is a subclass of [NoSuchElementException] to be consistent with plain collections. + */ +public class ClosedReceiveChannelException(message: String?) : NoSuchElementException(message) diff --git a/kotlinx-coroutines-core/common/src/channels/ChannelCoroutine.kt b/kotlinx-coroutines-core/common/src/channels/ChannelCoroutine.kt new file mode 100644 index 0000000000..44bbf5353b --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/ChannelCoroutine.kt @@ -0,0 +1,38 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +internal open class ChannelCoroutine( + parentContext: CoroutineContext, + protected val _channel: Channel, + initParentJob: Boolean, + active: Boolean +) : AbstractCoroutine(parentContext, initParentJob, active), Channel by _channel { + + val channel: Channel get() = this + + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") + override fun cancel() { + cancelInternal(defaultCancellationException()) + } + + @Suppress("MULTIPLE_DEFAULTS_INHERITED_FROM_SUPERTYPES_DEPRECATION_WARNING") // do not remove the MULTIPLE_DEFAULTS suppression: required in K2 + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") + final override fun cancel(cause: Throwable?): Boolean { + cancelInternal(defaultCancellationException()) + return true + } + + @Suppress("MULTIPLE_DEFAULTS_INHERITED_FROM_SUPERTYPES_DEPRECATION_WARNING") // do not remove the MULTIPLE_DEFAULTS suppression: required in K2 + final override fun cancel(cause: CancellationException?) { + if (isCancelled) return // Do not create an exception if the coroutine (-> the channel) is already cancelled + cancelInternal(cause ?: defaultCancellationException()) + } + + override fun cancelInternal(cause: Throwable) { + val exception = cause.toCancellationException() + _channel.cancel(exception) // cancel the channel + cancelCoroutine(exception) // cancel the job + } +} diff --git a/kotlinx-coroutines-core/common/src/channels/Channels.common.kt b/kotlinx-coroutines-core/common/src/channels/Channels.common.kt new file mode 100644 index 0000000000..15534b08fe --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/Channels.common.kt @@ -0,0 +1,203 @@ +@file:JvmMultifileClass +@file:JvmName("ChannelsKt") +@file:OptIn(ExperimentalContracts::class) + +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import kotlin.contracts.* +import kotlin.jvm.* + +internal const val DEFAULT_CLOSE_MESSAGE = "Channel was closed" + + +// -------- Operations on BroadcastChannel -------- + +/** + * This function is deprecated in the favour of [ReceiveChannel.receiveCatching]. + * + * This function is considered error-prone for the following reasons; + * - Is throwing if the channel has failed even though its signature may suggest it returns 'null' + * - It is easy to forget that exception handling still have to be explicit + * - During code reviews and code reading, intentions of the code are frequently unclear: + * are potential exceptions ignored deliberately or not? + * + * @suppress doc + */ +@Deprecated( + "Deprecated in the favour of 'receiveCatching'", + ReplaceWith("receiveCatching().getOrNull()"), + DeprecationLevel.HIDDEN +) // Warning since 1.5.0, ERROR in 1.6.0, HIDDEN in 1.7.0 +@Suppress("EXTENSION_SHADOWED_BY_MEMBER", "DEPRECATION_ERROR") +public suspend fun ReceiveChannel.receiveOrNull(): E? { + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") + return (this as ReceiveChannel).receiveOrNull() +} + +/** + * This function is deprecated in the favour of [ReceiveChannel.onReceiveCatching] + */ +@Deprecated( + "Deprecated in the favour of 'onReceiveCatching'", + level = DeprecationLevel.HIDDEN +) // Warning since 1.5.0, ERROR in 1.6.0, HIDDEN in 1.7.0 +@Suppress("DEPRECATION_ERROR") +public fun ReceiveChannel.onReceiveOrNull(): SelectClause1 { + return (this as ReceiveChannel).onReceiveOrNull +} + +/** + * Executes the [block] and then [cancels][ReceiveChannel.cancel] the channel. + * + * It is guaranteed that, after invoking this operation, the channel will be [cancelled][ReceiveChannel.cancel], so + * the operation is _terminal_. + * If the [block] finishes with an exception, that exception will be used for cancelling the channel and rethrown. + * + * This function is useful for building more complex terminal operators while ensuring that the producers stop sending + * new elements to the channel. + * + * Example: + * ``` + * suspend fun ReceiveChannel.consumeFirst(): E = + * consume { return receive() } + * // Launch a coroutine that constantly sends new values + * val channel = produce(Dispatchers.Default) { + * var i = 0 + * while (true) { + * // Will fail with a `CancellationException` + * // after `consumeFirst` finishes. + * send(i++) + * } + * } + * // Grab the first value and discard everything else + * val firstElement = channel.consumeFirst() + * check(firstElement == 0) + * // *Note*: some elements could be lost in the channel! + * ``` + * + * In this example, the channel will get closed, and the producer coroutine will finish its work after the first + * element is obtained. + * If `consumeFirst` was implemented as `for (e in this) { return e }` instead, the producer coroutine would be active + * until it was cancelled some other way. + * + * [consume] does not guarantee that new elements will not enter the channel after [block] finishes executing, so + * some channel elements may be lost. + * Use the `onUndeliveredElement` parameter of a manually created [Channel] to define what should happen with these + * elements during [ReceiveChannel.cancel]. + */ +public inline fun ReceiveChannel.consume(block: ReceiveChannel.() -> R): R { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + var cause: Throwable? = null + try { + return block() + } catch (e: Throwable) { + cause = e + throw e + } finally { + cancelConsumed(cause) + } +} + +/** + * Performs the given [action] for each received element and [cancels][ReceiveChannel.cancel] the channel afterward. + * + * This function stops processing elements when either the channel is [closed][SendChannel.close], + * the coroutine in which the collection is performed gets cancelled and there are no readily available elements in the + * channel's buffer, + * [action] fails with an exception, + * or an early return from [action] happens. + * If the [action] finishes with an exception, that exception will be used for cancelling the channel and rethrown. + * If the channel is [closed][SendChannel.close] with a cause, this cause will be rethrown from [consumeEach]. + * + * When the channel does not need to be closed after iterating over its elements, + * a regular `for` loop (`for (element in channel)`) should be used instead. + * + * The operation is _terminal_. + * This function [consumes][ReceiveChannel.consume] the elements of the original [ReceiveChannel]. + * + * This function is useful in cases when this channel is only expected to have a single consumer that decides when + * the producer may stop. + * Example: + * + * ``` + * val channel = Channel(1) + * // Launch several procedures that create values + * repeat(5) { + * launch(Dispatchers.Default) { + * while (true) { + * channel.send(Random.nextInt(40, 50)) + * } + * } + * } + * // Launch the exclusive consumer + * val result = run { + * channel.consumeEach { + * if (it == 42) { + * println("Found the answer") + * return@run it // forcibly stop collection + * } + * } + * // *Note*: some elements could be lost in the channel! + * } + * check(result == 42) + * ``` + * + * In this example, several coroutines put elements into a single channel, and a single consumer processes the elements. + * Once it finds the elements it's looking for, it stops [consumeEach] by making an early return. + * + * **Pitfall**: even though the name says "each", some elements could be left unprocessed if they are added after + * this function decided to close the channel. + * In this case, the elements will simply be lost. + * If the elements of the channel are resources that must be closed (like file handles, sockets, etc.), + * an `onUndeliveredElement` must be passed to the [Channel] on construction. + * It will be called for each element left in the channel at the point of cancellation. + */ +public suspend inline fun ReceiveChannel.consumeEach(action: (E) -> Unit): Unit = + consume { + for (e in this) action(e) + } + +/** + * Returns a [List] containing all the elements sent to this channel, preserving their order. + * + * This function will attempt to receive elements and put them into the list until the channel is + * [closed][SendChannel.close]. + * Calling [toList] on channels that are not eventually closed is always incorrect: + * - It will suspend indefinitely if the channel is not closed, but no new elements arrive. + * - If new elements do arrive and the channel is not eventually closed, [toList] will use more and more memory + * until exhausting it. + * + * If the channel is [closed][SendChannel.close] with a cause, [toList] will rethrow that cause. + * + * The operation is _terminal_. + * This function [consumes][ReceiveChannel.consume] all elements of the original [ReceiveChannel]. + * + * Example: + * ``` + * val values = listOf(1, 5, 2, 9, 3, 3, 1) + * // start a new coroutine that creates a channel, + * // sends elements to it, and closes it + * // once the coroutine's body finishes + * val channel = produce { + * values.forEach { send(it) } + * } + * check(channel.toList() == values) + * ``` + */ +public suspend fun ReceiveChannel.toList(): List = buildList { + consumeEach { + add(it) + } +} + +@PublishedApi +internal fun ReceiveChannel<*>.cancelConsumed(cause: Throwable?) { + cancel(cause?.let { + it as? CancellationException ?: CancellationException("Channel was consumed, consumer had failed", it) + }) +} + diff --git a/kotlinx-coroutines-core/common/src/channels/ConflatedBufferedChannel.kt b/kotlinx-coroutines-core/common/src/channels/ConflatedBufferedChannel.kt new file mode 100644 index 0000000000..0805c7faa1 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/ConflatedBufferedChannel.kt @@ -0,0 +1,89 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.channels.BufferOverflow.* +import kotlinx.coroutines.channels.ChannelResult.Companion.success +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* + +/** + * This is a special [BufferedChannel] extension that supports [DROP_OLDEST] and [DROP_LATEST] + * strategies for buffer overflowing. This implementation ensures that `send(e)` never suspends, + * either extracting the first element ([DROP_OLDEST]) or dropping the sending one ([DROP_LATEST]) + * when the channel capacity exceeds. + */ +internal open class ConflatedBufferedChannel( + private val capacity: Int, + private val onBufferOverflow: BufferOverflow, + onUndeliveredElement: OnUndeliveredElement? = null +) : BufferedChannel(capacity = capacity, onUndeliveredElement = onUndeliveredElement) { + init { + require(onBufferOverflow !== SUSPEND) { + "This implementation does not support suspension for senders, use ${BufferedChannel::class.simpleName} instead" + } + require(capacity >= 1) { + "Buffered channel capacity must be at least 1, but $capacity was specified" + } + } + + override val isConflatedDropOldest: Boolean + get() = onBufferOverflow == DROP_OLDEST + + override suspend fun send(element: E) { + // Should never suspend, implement via `trySend(..)`. + trySendImpl(element, isSendOp = true).onClosed { // fails only when this channel is closed. + onUndeliveredElement?.callUndeliveredElementCatchingException(element)?.let { + it.addSuppressed(sendException) + throw it + } + throw sendException + } + } + + override suspend fun sendBroadcast(element: E): Boolean { + // Should never suspend, implement via `trySend(..)`. + trySendImpl(element, isSendOp = true) // fails only when this channel is closed. + .onSuccess { return true } + return false + } + + override fun trySend(element: E): ChannelResult = trySendImpl(element, isSendOp = false) + + private fun trySendImpl(element: E, isSendOp: Boolean) = + if (onBufferOverflow === DROP_LATEST) trySendDropLatest(element, isSendOp) + else trySendDropOldest(element) + + private fun trySendDropLatest(element: E, isSendOp: Boolean): ChannelResult { + // Try to send the element without suspension. + val result = super.trySend(element) + // Complete on success or if this channel is closed. + if (result.isSuccess || result.isClosed) return result + // This channel is full. Drop the sending element. + // Call the `onUndeliveredElement` lambda ONLY for 'send()' invocations, + // for 'trySend()' it is responsibility of the caller + if (isSendOp) { + onUndeliveredElement?.callUndeliveredElementCatchingException(element)?.let { + throw it + } + } + return success(Unit) + } + + @Suppress("UNCHECKED_CAST") + override fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // The plain `send(..)` operation never suspends. Thus, either this + // attempt to send the element succeeds or the channel is closed. + // In any case, complete this `select` in the registration phase. + trySend(element as E).let { + it.onSuccess { + select.selectInRegistrationPhase(Unit) + return + }.onClosed { + select.selectInRegistrationPhase(CHANNEL_CLOSED) + return + } + } + error("unreachable") + } + + override fun shouldSendSuspend() = false // never suspends. +} diff --git a/kotlinx-coroutines-core/common/src/channels/Deprecated.kt b/kotlinx-coroutines-core/common/src/channels/Deprecated.kt new file mode 100644 index 0000000000..463adcb598 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/Deprecated.kt @@ -0,0 +1,508 @@ +@file:JvmMultifileClass +@file:JvmName("ChannelsKt") +@file:Suppress("unused") + +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.jvm.* + +/** + * Opens subscription to this [BroadcastChannel] and makes sure that the given [block] consumes all elements + * from it by always invoking [cancel][ReceiveChannel.cancel] after the execution of the block. + * + * **Note: This API is obsolete since 1.5.0 and deprecated for removal since 1.7.0** + * It is replaced with [SharedFlow][kotlinx.coroutines.flow.SharedFlow]. + * + * Safe to remove in 1.9.0 as was inline before. + */ +@ObsoleteCoroutinesApi +@Suppress("DEPRECATION_ERROR") +@Deprecated(level = DeprecationLevel.ERROR, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") +public inline fun BroadcastChannel.consume(block: ReceiveChannel.() -> R): R { + val channel = openSubscription() + try { + return channel.block() + } finally { + channel.cancel() + } +} + +/** + * Subscribes to this [BroadcastChannel] and performs the specified action for each received element. + * + * **Note: This API is obsolete since 1.5.0 and deprecated for removal since 1.7.0** + */ +@Deprecated(level = DeprecationLevel.ERROR, message = "BroadcastChannel is deprecated in the favour of SharedFlow and is no longer supported") +@Suppress("DEPRECATION", "DEPRECATION_ERROR") +public suspend inline fun BroadcastChannel.consumeEach(action: (E) -> Unit): Unit = + consume { + for (element in this) action(element) + } + +/** @suppress **/ +@PublishedApi // Binary compatibility +internal fun consumesAll(vararg channels: ReceiveChannel<*>): CompletionHandler = + { cause: Throwable? -> + var exception: Throwable? = null + for (channel in channels) + try { + channel.cancelConsumed(cause) + } catch (e: Throwable) { + if (exception == null) { + exception = e + } else { + exception.addSuppressed(e) + } + } + exception?.let { throw it } + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.elementAt(index: Int): E = consume { + if (index < 0) + throw IndexOutOfBoundsException("ReceiveChannel doesn't contain element at index $index.") + var count = 0 + for (element in this) { + @Suppress("UNUSED_CHANGED_VALUE") // KT-47628 + if (index == count++) + return element + } + throw IndexOutOfBoundsException("ReceiveChannel doesn't contain element at index $index.") +} + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.elementAtOrNull(index: Int): E? = + consume { + if (index < 0) + return null + var count = 0 + for (element in this) { + if (index == count++) + return element + } + return null + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.first(): E = + consume { + val iterator = iterator() + if (!iterator.hasNext()) + throw NoSuchElementException("ReceiveChannel is empty.") + return iterator.next() + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.firstOrNull(): E? = + consume { + val iterator = iterator() + if (!iterator.hasNext()) + return null + return iterator.next() + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.indexOf(element: E): Int { + var index = 0 + consumeEach { + if (element == it) + return index + index++ + } + return -1 +} + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.last(): E = + consume { + val iterator = iterator() + if (!iterator.hasNext()) + throw NoSuchElementException("ReceiveChannel is empty.") + var last = iterator.next() + while (iterator.hasNext()) + last = iterator.next() + return last + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.lastIndexOf(element: E): Int { + var lastIndex = -1 + var index = 0 + consumeEach { + if (element == it) + lastIndex = index + index++ + } + return lastIndex +} + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.lastOrNull(): E? = + consume { + val iterator = iterator() + if (!iterator.hasNext()) + return null + var last = iterator.next() + while (iterator.hasNext()) + last = iterator.next() + return last + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.single(): E = + consume { + val iterator = iterator() + if (!iterator.hasNext()) + throw NoSuchElementException("ReceiveChannel is empty.") + val single = iterator.next() + if (iterator.hasNext()) + throw IllegalArgumentException("ReceiveChannel has more than one element.") + return single + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.singleOrNull(): E? = + consume { + val iterator = iterator() + if (!iterator.hasNext()) + return null + val single = iterator.next() + if (iterator.hasNext()) + return null + return single + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.drop(n: Int, context: CoroutineContext = Dispatchers.Unconfined): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + require(n >= 0) { "Requested element count $n is less than zero." } + var remaining: Int = n + if (remaining > 0) + for (e in this@drop) { + remaining-- + if (remaining == 0) + break + } + for (e in this@drop) { + send(e) + } + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.dropWhile( + context: CoroutineContext = Dispatchers.Unconfined, + predicate: suspend (E) -> Boolean +): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + for (e in this@dropWhile) { + if (!predicate(e)) { + send(e) + break + } + } + for (e in this@dropWhile) { + send(e) + } + } + +@PublishedApi +internal fun ReceiveChannel.filter( + context: CoroutineContext = Dispatchers.Unconfined, + predicate: suspend (E) -> Boolean +): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + for (e in this@filter) { + if (predicate(e)) send(e) + } + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.filterIndexed( + context: CoroutineContext = Dispatchers.Unconfined, + predicate: suspend (index: Int, E) -> Boolean +): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + var index = 0 + for (e in this@filterIndexed) { + if (predicate(index++, e)) send(e) + } + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.filterNot( + context: CoroutineContext = Dispatchers.Unconfined, + predicate: suspend (E) -> Boolean +): ReceiveChannel = + filter(context) { !predicate(it) } + +@PublishedApi +@Suppress("UNCHECKED_CAST") +internal fun ReceiveChannel.filterNotNull(): ReceiveChannel = + filter { it != null } as ReceiveChannel + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun > ReceiveChannel.filterNotNullTo(destination: C): C { + consumeEach { + if (it != null) destination.add(it) + } + return destination +} + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun > ReceiveChannel.filterNotNullTo(destination: C): C { + consumeEach { + if (it != null) destination.send(it) + } + return destination +} + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.take(n: Int, context: CoroutineContext = Dispatchers.Unconfined): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + if (n == 0) return@produce + require(n >= 0) { "Requested element count $n is less than zero." } + var remaining: Int = n + for (e in this@take) { + send(e) + remaining-- + if (remaining == 0) + return@produce + } + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.takeWhile( + context: CoroutineContext = Dispatchers.Unconfined, + predicate: suspend (E) -> Boolean +): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + for (e in this@takeWhile) { + if (!predicate(e)) return@produce + send(e) + } + } + +@PublishedApi +internal suspend fun > ReceiveChannel.toChannel(destination: C): C { + consumeEach { + destination.send(it) + } + return destination +} + +@PublishedApi +internal suspend fun > ReceiveChannel.toCollection(destination: C): C { + consumeEach { + destination.add(it) + } + return destination +} + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel>.toMap(): Map = + toMap(LinkedHashMap()) + +@PublishedApi +internal suspend fun > ReceiveChannel>.toMap(destination: M): M { + consumeEach { + destination += it + } + return destination +} + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.toMutableList(): MutableList = + toCollection(ArrayList()) + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.toSet(): Set = + this.toMutableSet() + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.flatMap( + context: CoroutineContext = Dispatchers.Unconfined, + transform: suspend (E) -> ReceiveChannel +): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + for (e in this@flatMap) { + transform(e).toChannel(this) + } + } + +@PublishedApi +internal fun ReceiveChannel.map( + context: CoroutineContext = Dispatchers.Unconfined, + transform: suspend (E) -> R +): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + consumeEach { + send(transform(it)) + } + } + +@PublishedApi +internal fun ReceiveChannel.mapIndexed( + context: CoroutineContext = Dispatchers.Unconfined, + transform: suspend (index: Int, E) -> R +): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + var index = 0 + for (e in this@mapIndexed) { + send(transform(index++, e)) + } + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.mapIndexedNotNull( + context: CoroutineContext = Dispatchers.Unconfined, + transform: suspend (index: Int, E) -> R? +): ReceiveChannel = + mapIndexed(context, transform).filterNotNull() + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.mapNotNull( + context: CoroutineContext = Dispatchers.Unconfined, + transform: suspend (E) -> R? +): ReceiveChannel = + map(context, transform).filterNotNull() + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.withIndex(context: CoroutineContext = Dispatchers.Unconfined): ReceiveChannel> = + GlobalScope.produce(context, onCompletion = consumes()) { + var index = 0 + for (e in this@withIndex) { + send(IndexedValue(index++, e)) + } + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.distinct(): ReceiveChannel = + this.distinctBy { it } + +@PublishedApi +internal fun ReceiveChannel.distinctBy( + context: CoroutineContext = Dispatchers.Unconfined, + selector: suspend (E) -> K +): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumes()) { + val keys = HashSet() + for (e in this@distinctBy) { + val k = selector(e) + if (k !in keys) { + send(e) + keys += k + } + } + } + +@PublishedApi +internal suspend fun ReceiveChannel.toMutableSet(): MutableSet = + toCollection(LinkedHashSet()) + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.any(): Boolean = + consume { + return iterator().hasNext() + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.count(): Int { + var count = 0 + consumeEach { count++ } + return count +} + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.maxWith(comparator: Comparator): E? = + consume { + val iterator = iterator() + if (!iterator.hasNext()) return null + var max = iterator.next() + while (iterator.hasNext()) { + val e = iterator.next() + if (comparator.compare(max, e) < 0) max = e + } + return max + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.minWith(comparator: Comparator): E? = + consume { + val iterator = iterator() + if (!iterator.hasNext()) return null + var min = iterator.next() + while (iterator.hasNext()) { + val e = iterator.next() + if (comparator.compare(min, e) > 0) min = e + } + return min + } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public suspend fun ReceiveChannel.none(): Boolean = + consume { + return !iterator().hasNext() + } + +/** @suppress **/ +@Deprecated(message = "Left for binary compatibility", level = DeprecationLevel.HIDDEN) +public fun ReceiveChannel.requireNoNulls(): ReceiveChannel = + map { it ?: throw IllegalArgumentException("null element found in $this.") } + +/** @suppress **/ +@Deprecated(message = "Binary compatibility", level = DeprecationLevel.HIDDEN) +public infix fun ReceiveChannel.zip(other: ReceiveChannel): ReceiveChannel> = + zip(other) { t1, t2 -> t1 to t2 } + +@PublishedApi // Binary compatibility +internal fun ReceiveChannel.zip( + other: ReceiveChannel, + context: CoroutineContext = Dispatchers.Unconfined, + transform: (a: E, b: R) -> V +): ReceiveChannel = + GlobalScope.produce(context, onCompletion = consumesAll(this, other)) { + val otherIterator = other.iterator() + this@zip.consumeEach { element1 -> + if (!otherIterator.hasNext()) return@consumeEach + val element2 = otherIterator.next() + send(transform(element1, element2)) + } + } + +@PublishedApi // Binary compatibility +internal fun ReceiveChannel<*>.consumes(): CompletionHandler = { cause: Throwable? -> + cancelConsumed(cause) +} diff --git a/kotlinx-coroutines-core/common/src/channels/Produce.kt b/kotlinx-coroutines-core/common/src/channels/Produce.kt new file mode 100644 index 0000000000..e746c37d13 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/channels/Produce.kt @@ -0,0 +1,299 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlinx.coroutines.flow.* + +/** + * Scope for the [produce][CoroutineScope.produce], [callbackFlow] and [channelFlow] builders. + */ +public interface ProducerScope : CoroutineScope, SendChannel { + /** + * A reference to the channel this coroutine [sends][send] elements to. + * It is provided for convenience, so that the code in the coroutine can refer + * to the channel as `channel` as opposed to `this`. + * All the [SendChannel] functions on this interface delegate to + * the channel instance returned by this property. + */ + public val channel: SendChannel +} + +/** + * Suspends the current coroutine until the channel is either + * [closed][SendChannel.close] or [cancelled][ReceiveChannel.cancel]. + * + * The given [block] will be executed unconditionally before this function returns. + * `awaitClose { cleanup() }` is a convenient shorthand for the often useful form + * `try { awaitClose() } finally { cleanup() }`. + * + * This function can only be invoked directly inside the same coroutine that is its receiver. + * Specifying the receiver of [awaitClose] explicitly is most probably a mistake. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is [cancelled][CoroutineScope.cancel] + * while this suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + * + * Example of usage: + * ``` + * val callbackEventsStream = produce { + * val disposable = registerChannelInCallback(channel) + * awaitClose { disposable.dispose() } + * } + * ``` + * + * Internally, [awaitClose] is implemented using [SendChannel.invokeOnClose]. + * Currently, every channel can have at most one [SendChannel.invokeOnClose] handler. + * This means that calling [awaitClose] several times in a row or combining it with other [SendChannel.invokeOnClose] + * invocations is prohibited. + * An [IllegalStateException] will be thrown if this rule is broken. + * + * **Pitfall**: when used in [produce], if the channel is [cancelled][ReceiveChannel.cancel], [awaitClose] can either + * return normally or throw a [CancellationException] due to a race condition. + * The reason is that, for [produce], cancelling the channel and cancelling the coroutine of the [ProducerScope] is + * done simultaneously. + * + * @throws IllegalStateException if invoked from outside the [ProducerScope] (by leaking `this` outside the producer + * coroutine). + * @throws IllegalStateException if this channel already has a [SendChannel.invokeOnClose] handler registered. + */ +public suspend fun ProducerScope<*>.awaitClose(block: () -> Unit = {}) { + check(kotlin.coroutines.coroutineContext[Job] === this) { "awaitClose() can only be invoked from the producer context" } + try { + suspendCancellableCoroutine { cont -> + invokeOnClose { + cont.resume(Unit) + } + } + } finally { + block() + } +} + +/** + * Launches a new coroutine to produce a stream of values by sending them to a channel + * and returns a reference to the coroutine as a [ReceiveChannel]. This resulting + * object can be used to [receive][ReceiveChannel.receive] elements produced by this coroutine. + * + * The scope of the coroutine contains the [ProducerScope] interface, which implements + * both [CoroutineScope] and [SendChannel], so that the coroutine can invoke [send][SendChannel.send] directly. + * + * The kind of the resulting channel depends on the specified [capacity] parameter. + * See the [Channel] interface documentation for details. + * By default, an unbuffered channel is created. + * If an invalid [capacity] value is specified, an [IllegalArgumentException] is thrown. + * + * ### Behavior on termination + * + * The channel is [closed][SendChannel.close] when the coroutine completes. + * + * ``` + * val values = listOf(1, 2, 3, 4) + * val channel = produce { + * for (value in values) { + * send(value) + * } + * } + * check(channel.toList() == values) + * ``` + * + * The running coroutine is cancelled when the channel is [cancelled][ReceiveChannel.cancel]. + * + * ``` + * val channel = produce { + * send(1) + * send(2) + * try { + * send(3) // will throw CancellationException + * } catch (e: CancellationException) { + * println("The channel was cancelled!) + * throw e // always rethrow CancellationException + * } + * } + * check(channel.receive() == 1) + * check(channel.receive() == 2) + * channel.cancel() + * ``` + * + * If this coroutine finishes with an exception, it will close the channel with that exception as the cause, + * so after receiving all the existing elements, + * all further attempts to receive from it will throw the exception with which the coroutine finished. + * + * ``` + * val produceJob = Job() + * // create and populate a channel with a buffer + * val channel = produce(produceJob, capacity = Channel.UNLIMITED) { + * repeat(5) { send(it) } + * throw TestException() + * } + * produceJob.join() // wait for `produce` to fail + * check(produceJob.isCancelled == true) + * // prints 0, 1, 2, 3, 4, then throws `TestException` + * for (value in channel) { println(value) } + * ``` + * + * When the coroutine is cancelled via structured concurrency and not the `cancel` function, + * the channel does not automatically close until the coroutine completes, + * so it is possible that some elements will be sent even after the coroutine is cancelled: + * + * ``` + * val parentScope = CoroutineScope(Dispatchers.Default) + * val channel = parentScope.produce(capacity = Channel.UNLIMITED) { + * repeat(5) { + * send(it) + * } + * parentScope.cancel() + * // suspending after this point would fail, but sending succeeds + * send(-1) + * } + * for (c in channel) { + * println(c) // 0, 1, 2, 3, 4, -1 + * } // throws a `CancellationException` exception after reaching -1 + * ``` + * + * Note that cancelling `produce` via structured concurrency closes the channel with a cause. + * + * The behavior around coroutine cancellation and error handling is experimental and may change in a future release. + * + * ### Coroutine context + * + * The coroutine context is inherited from this [CoroutineScope]. Additional context elements can be specified with the [context] argument. + * If the context does not have any dispatcher or other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * The parent job is inherited from the [CoroutineScope] as well, but it can also be overridden + * with a corresponding [context] element. + * + * See [newCoroutineContext] for a description of debugging facilities available for newly created coroutines. + * + * ### Undelivered elements + * + * Some values that [produce] creates may be lost: + * + * ``` + * val channel = produce(Dispatchers.Default, capacity = 5) { + * repeat(100) { + * send(it) + * println("Sent $it") + * } + * } + * channel.cancel() // no elements can be received after this! + * ``` + * + * There is no way to recover these lost elements. + * If this is unsuitable, please create a [Channel] manually and pass the `onUndeliveredElement` callback to the + * constructor: [Channel(onUndeliveredElement = ...)][Channel]. + * + * ### Usage example + * + * ``` + * /* Generate random integers until we find the square root of 9801. + * To calculate whether the given number is that square root, + * use several coroutines that separately process these integers. + * Alternatively, we may randomly give up during value generation. + * `produce` is used to generate the integers and put them into a + * channel, from which the square-computing coroutines take them. */ + * val parentScope = CoroutineScope(SupervisorJob()) + * val channel = parentScope.produce( + * Dispatchers.IO, + * capacity = 16 // buffer of size 16 + * ) { + * // this code will run on Dispatchers.IO + * while (true) { + * val request = run { + * // simulate waiting for the next request + * delay(5.milliseconds) + * val randomInt = Random.nextInt(-1, 100) + * if (randomInt == -1) { + * // external termination request received + * println("Producer: no longer accepting requests") + * return@produce + * } + * println("Producer: sending a request ($randomInt)") + * randomInt + * } + * send(request) + * } + * } + * // Launch consumers + * repeat(4) { + * launch(Dispatchers.Default) { + * for (request in channel) { + * // simulate processing a request + * delay(25.milliseconds) + * println("Consumer $it: received a request ($request)") + * if (request * request == 9801) { + * println("Consumer $it found the square root of 9801!") + * /* the work is done, the producer may finish. + * the internal termination request will cancel + * the producer on the next suspension point. */ + * channel.cancel() + * } + * } + * } + * } + * ``` + * + * **Note: This is an experimental api.** Behaviour of producers that work as children in a parent scope with respect + * to cancellation and error handling may change in the future. + */ +@ExperimentalCoroutinesApi +public fun CoroutineScope.produce( + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = Channel.RENDEZVOUS, + @BuilderInference block: suspend ProducerScope.() -> Unit +): ReceiveChannel = + produce(context, capacity, BufferOverflow.SUSPEND, CoroutineStart.DEFAULT, onCompletion = null, block = block) + +/** + * **This is an internal API and should not be used from general code.** + * The `onCompletion` parameter will be redesigned. + * If you have to use the `onCompletion` operator, please report to https://github.com/Kotlin/kotlinx.coroutines/issues/. + * As a temporary solution, [invokeOnCompletion][Job.invokeOnCompletion] can be used instead: + * ``` + * fun ReceiveChannel.myOperator(): ReceiveChannel = GlobalScope.produce(Dispatchers.Unconfined) { + * coroutineContext[Job]?.invokeOnCompletion { consumes() } + * } + * ``` + * @suppress + */ +@InternalCoroutinesApi +public fun CoroutineScope.produce( + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = 0, + start: CoroutineStart = CoroutineStart.DEFAULT, + onCompletion: CompletionHandler? = null, + @BuilderInference block: suspend ProducerScope.() -> Unit +): ReceiveChannel = + produce(context, capacity, BufferOverflow.SUSPEND, start, onCompletion, block) + +// Internal version of produce that is maximally flexible, but is not exposed through public API (too many params) +internal fun CoroutineScope.produce( + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = 0, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND, + start: CoroutineStart = CoroutineStart.DEFAULT, + onCompletion: CompletionHandler? = null, + @BuilderInference block: suspend ProducerScope.() -> Unit +): ReceiveChannel { + val channel = Channel(capacity, onBufferOverflow) + val newContext = newCoroutineContext(context) + val coroutine = ProducerCoroutine(newContext, channel) + if (onCompletion != null) coroutine.invokeOnCompletion(handler = onCompletion) + coroutine.start(start, coroutine, block) + return coroutine +} + +private class ProducerCoroutine( + parentContext: CoroutineContext, channel: Channel +) : ChannelCoroutine(parentContext, channel, true, active = true), ProducerScope { + override val isActive: Boolean + get() = super.isActive + + override fun onCompleted(value: Unit) { + _channel.close() + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + val processed = _channel.close(cause) + if (!processed && !handled) handleCoroutineException(context, cause) + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/Builders.kt b/kotlinx-coroutines-core/common/src/flow/Builders.kt new file mode 100644 index 0000000000..d56666f37f --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/Builders.kt @@ -0,0 +1,349 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.channels.Channel.Factory.BUFFERED +import kotlinx.coroutines.flow.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* +import kotlinx.coroutines.flow.internal.unsafeFlow as flow + +/** + * Creates a _cold_ flow from the given suspendable [block]. + * The flow being _cold_ means that the [block] is called every time a terminal operator is applied to the resulting flow. + * + * Example of usage: + * + * ``` + * fun fibonacci(): Flow = flow { + * var x = BigInteger.ZERO + * var y = BigInteger.ONE + * while (true) { + * emit(x) + * x = y.also { + * y += x + * } + * } + * } + * + * fibonacci().take(100).collect { println(it) } + * ``` + * + * Emissions from [flow] builder are [cancellable] by default — each call to [emit][FlowCollector.emit] + * also calls [ensureActive][CoroutineContext.ensureActive]. + * + * `emit` should happen strictly in the dispatchers of the [block] in order to preserve the flow context. + * For example, the following code will result in an [IllegalStateException]: + * + * ``` + * flow { + * emit(1) // Ok + * withContext(Dispatcher.IO) { + * emit(2) // Will fail with ISE + * } + * } + * ``` + * + * If you want to switch the context of execution of a flow, use the [flowOn] operator. + */ +public fun flow(@BuilderInference block: suspend FlowCollector.() -> Unit): Flow = SafeFlow(block) + +// Named anonymous object +private class SafeFlow(private val block: suspend FlowCollector.() -> Unit) : AbstractFlow() { + override suspend fun collectSafely(collector: FlowCollector) { + collector.block() + } +} + +/** + * Creates a _cold_ flow that produces a single value from the given functional type. + */ +public fun (() -> T).asFlow(): Flow = flow { + emit(invoke()) +} + +/** + * Creates a _cold_ flow that produces a single value from the given functional type. + * + * Example of usage: + * + * ``` + * suspend fun remoteCall(): R = ... + * fun remoteCallFlow(): Flow = ::remoteCall.asFlow() + * ``` + */ +public fun (suspend () -> T).asFlow(): Flow = flow { + emit(invoke()) +} + +/** + * Creates a _cold_ flow that produces values from the given iterable. + */ +public fun Iterable.asFlow(): Flow = flow { + forEach { value -> + emit(value) + } +} + +/** + * Creates a _cold_ flow that produces values from the given iterator. + */ +public fun Iterator.asFlow(): Flow = flow { + forEach { value -> + emit(value) + } +} + +/** + * Creates a _cold_ flow that produces values from the given sequence. + */ +public fun Sequence.asFlow(): Flow = flow { + forEach { value -> + emit(value) + } +} + +/** + * Creates a flow that produces values from the specified `vararg`-arguments. + * + * Example of usage: + * + * ``` + * flowOf(1, 2, 3) + * ``` + */ +public fun flowOf(vararg elements: T): Flow = flow { + for (element in elements) { + emit(element) + } +} + +/** + * Creates a flow that produces the given [value]. + */ +public fun flowOf(value: T): Flow = flow { + /* + * Implementation note: this is just an "optimized" overload of flowOf(vararg) + * which significantly reduces the footprint of widespread single-value flows. + */ + emit(value) +} + +/** + * Returns an empty flow. + */ +public fun emptyFlow(): Flow = EmptyFlow + +private object EmptyFlow : Flow { + override suspend fun collect(collector: FlowCollector) = Unit +} + +/** + * Creates a _cold_ flow that produces values from the given array. + * The flow being _cold_ means that the array components are read every time a terminal operator is applied + * to the resulting flow. + */ +public fun Array.asFlow(): Flow = flow { + forEach { value -> + emit(value) + } +} + +/** + * Creates a _cold_ flow that produces values from the array. + * The flow being _cold_ means that the array components are read every time a terminal operator is applied + * to the resulting flow. + */ +public fun IntArray.asFlow(): Flow = flow { + forEach { value -> + emit(value) + } +} + +/** + * Creates a _cold_ flow that produces values from the given array. + * The flow being _cold_ means that the array components are read every time a terminal operator is applied + * to the resulting flow. + */ +public fun LongArray.asFlow(): Flow = flow { + forEach { value -> + emit(value) + } +} + +/** + * Creates a flow that produces values from the range. + */ +public fun IntRange.asFlow(): Flow = flow { + forEach { value -> + emit(value) + } +} + +/** + * Creates a flow that produces values from the range. + */ +public fun LongRange.asFlow(): Flow = flow { + forEach { value -> + emit(value) + } +} + +/** + * Creates an instance of a _cold_ [Flow] with elements that are sent to a [SendChannel] + * provided to the builder's [block] of code via [ProducerScope]. It allows elements to be + * produced by code that is running in a different context or concurrently. + * The resulting flow is _cold_, which means that [block] is called every time a terminal operator + * is applied to the resulting flow. + * + * This builder ensures thread-safety and context preservation, thus the provided [ProducerScope] can be used + * concurrently from different contexts. + * The resulting flow completes as soon as the code in the [block] and all its children completes. + * Use [awaitClose] as the last statement to keep it running. + * A more detailed example is provided in the documentation of [callbackFlow]. + * + * A channel with the [default][Channel.BUFFERED] buffer size is used. Use the [buffer] operator on the + * resulting flow to specify a user-defined value and to control what happens when data is produced faster + * than consumed, i.e. to control the back-pressure behavior. + * + * Adjacent applications of [channelFlow], [flowOn], [buffer], and [produceIn] are + * always fused so that only one properly configured channel is used for execution. + * + * Examples of usage: + * + * ``` + * fun Flow.merge(other: Flow): Flow = channelFlow { + * // collect from one coroutine and send it + * launch { + * collect { send(it) } + * } + * // collect and send from this coroutine, too, concurrently + * other.collect { send(it) } + * } + * + * fun contextualFlow(): Flow = channelFlow { + * // send from one coroutine + * launch(Dispatchers.IO) { + * send(computeIoValue()) + * } + * // send from another coroutine, concurrently + * launch(Dispatchers.Default) { + * send(computeCpuValue()) + * } + * } + * ``` + */ +public fun channelFlow(@BuilderInference block: suspend ProducerScope.() -> Unit): Flow = + ChannelFlowBuilder(block) + +/** + * Creates an instance of a _cold_ [Flow] with elements that are sent to a [SendChannel] + * provided to the builder's [block] of code via [ProducerScope]. It allows elements to be + * produced by code that is running in a different context or concurrently. + * + * The resulting flow is _cold_, which means that [block] is called every time a terminal operator + * is applied to the resulting flow. + * + * This builder ensures thread-safety and context preservation, thus the provided [ProducerScope] can be used + * from any context, e.g. from a callback-based API. + * The resulting flow completes as soon as the code in the [block] completes. + * [awaitClose] should be used to keep the flow running, otherwise the channel will be closed immediately + * when block completes. + * [awaitClose] argument is called either when a flow consumer cancels the flow collection + * or when a callback-based API invokes [SendChannel.close] manually and is typically used + * to cleanup the resources after the completion, e.g. unregister a callback. + * Using [awaitClose] is mandatory in order to prevent memory leaks when the flow collection is cancelled, + * otherwise the callback may keep running even when the flow collector is already completed. + * To avoid such leaks, this method throws [IllegalStateException] if block returns, but the channel + * is not closed yet. + * + * A channel with the [default][Channel.BUFFERED] buffer size is used. Use the [buffer] operator on the + * resulting flow to specify a user-defined value and to control what happens when data is produced faster + * than consumed, i.e. to control the back-pressure behavior. + * + * Adjacent applications of [callbackFlow], [flowOn], [buffer], and [produceIn] are + * always fused so that only one properly configured channel is used for execution. + * + * Example of usage that converts a multi-shot callback API to a flow. + * For single-shot callbacks use [suspendCancellableCoroutine]. + * + * ``` + * fun flowFrom(api: CallbackBasedApi): Flow = callbackFlow { + * val callback = object : Callback { // Implementation of some callback interface + * override fun onNextValue(value: T) { + * // To avoid blocking you can configure channel capacity using + * // either buffer(Channel.CONFLATED) or buffer(Channel.UNLIMITED) to avoid overfill + * trySendBlocking(value) + * .onFailure { throwable -> + * // Downstream has been cancelled or failed, can log here + * } + * } + * override fun onApiError(cause: Throwable) { + * cancel(CancellationException("API Error", cause)) + * } + * override fun onCompleted() = channel.close() + * } + * api.register(callback) + * /* + * * Suspends until either 'onCompleted'/'onApiError' from the callback is invoked + * * or flow collector is cancelled (e.g. by 'take(1)' or because a collector's coroutine was cancelled). + * * In both cases, callback will be properly unregistered. + * */ + * awaitClose { api.unregister(callback) } + * } + * ``` + * + * > The callback `register`/`unregister` methods provided by an external API must be thread-safe, because + * > `awaitClose` block can be called at any time due to asynchronous nature of cancellation, even + * > concurrently with the call of the callback. + */ +public fun callbackFlow(@BuilderInference block: suspend ProducerScope.() -> Unit): Flow = CallbackFlowBuilder(block) + +// ChannelFlow implementation that is the first in the chain of flow operations and introduces (builds) a flow +private open class ChannelFlowBuilder( + private val block: suspend ProducerScope.() -> Unit, + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = BUFFERED, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND +) : ChannelFlow(context, capacity, onBufferOverflow) { + override fun create(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): ChannelFlow = + ChannelFlowBuilder(block, context, capacity, onBufferOverflow) + + override suspend fun collectTo(scope: ProducerScope) = + block(scope) + + override fun toString(): String = + "block[$block] -> ${super.toString()}" +} + +private class CallbackFlowBuilder( + private val block: suspend ProducerScope.() -> Unit, + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = BUFFERED, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND +) : ChannelFlowBuilder(block, context, capacity, onBufferOverflow) { + + override suspend fun collectTo(scope: ProducerScope) { + super.collectTo(scope) + /* + * We expect user either call `awaitClose` from within a block (then the channel is closed at this moment) + * or being closed/cancelled externally/manually. Otherwise "user forgot to call + * awaitClose and receives unhelpful ClosedSendChannelException exceptions" situation is detected. + */ + if (!scope.isClosedForSend) { + throw IllegalStateException( + """ + 'awaitClose { yourCallbackOrListener.cancel() }' should be used in the end of callbackFlow block. + Otherwise, a callback/listener may leak in case of external cancellation. + See callbackFlow API documentation for the details. + """.trimIndent() + ) + } + } + + override fun create(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): ChannelFlow = + CallbackFlowBuilder(block, context, capacity, onBufferOverflow) +} diff --git a/kotlinx-coroutines-core/common/src/flow/Channels.kt b/kotlinx-coroutines-core/common/src/flow/Channels.kt new file mode 100644 index 0000000000..4c14028da3 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/Channels.kt @@ -0,0 +1,157 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* +import kotlinx.coroutines.flow.internal.unsafeFlow as flow + +/** + * Emits all elements from the given [channel] to this flow collector and [cancels][cancel] (consumes) + * the channel afterwards. If you need to iterate over the channel without consuming it, + * a regular `for` loop should be used instead. + * + * Note, that emitting values from a channel into a flow is not atomic. A value that was received from the + * channel many not reach the flow collector if it was cancelled and will be lost. + * + * This function provides a more efficient shorthand for `channel.consumeEach { value -> emit(value) }`. + * See [consumeEach][ReceiveChannel.consumeEach]. + */ +public suspend fun FlowCollector.emitAll(channel: ReceiveChannel): Unit = + emitAllImpl(channel, consume = true) + +private suspend fun FlowCollector.emitAllImpl(channel: ReceiveChannel, consume: Boolean) { + ensureActive() + var cause: Throwable? = null + try { + for (element in channel) { + emit(element) + } + } catch (e: Throwable) { + cause = e + throw e + } finally { + if (consume) channel.cancelConsumed(cause) + } +} + +/** + * Represents the given receive channel as a hot flow and [receives][ReceiveChannel.receive] from the channel + * in fan-out fashion every time this flow is collected. One element will be emitted to one collector only. + * + * See also [consumeAsFlow] which ensures that the resulting flow is collected just once. + * + * ### Cancellation semantics + * + * - Flow collectors are cancelled when the original channel is [closed][SendChannel.close] with an exception. + * - Flow collectors complete normally when the original channel is [closed][SendChannel.close] normally. + * - Failure or cancellation of the flow collector does not affect the channel. + * However, if a flow collector gets cancelled after receiving an element from the channel but before starting + * to process it, the element will be lost, and the `onUndeliveredElement` callback of the [Channel], + * if provided on channel construction, will be invoked. + * See [Channel.receive] for details of the effect of the prompt cancellation guarantee on element delivery. + * + * ### Operator fusion + * + * Adjacent applications of [flowOn], [buffer], [conflate], and [produceIn] to the result of `receiveAsFlow` are fused. + * In particular, [produceIn] returns the original channel. + * Calls to [flowOn] have generally no effect, unless [buffer] is used to explicitly request buffering. + */ +public fun ReceiveChannel.receiveAsFlow(): Flow = ChannelAsFlow(this, consume = false) + +/** + * Represents the given receive channel as a hot flow and [consumes][ReceiveChannel.consume] the channel + * on the first collection from this flow. The resulting flow can be collected just once and throws + * [IllegalStateException] when trying to collect it more than once. + * + * See also [receiveAsFlow] which supports multiple collectors of the resulting flow. + * + * ### Cancellation semantics + * + * - Flow collector is cancelled when the original channel is [closed][SendChannel.close] with an exception. + * - Flow collector completes normally when the original channel is [closed][SendChannel.close] normally. + * - If the flow collector fails with an exception (for example, by getting cancelled), + * the source channel is [cancelled][ReceiveChannel.cancel]. + * + * ### Operator fusion + * + * Adjacent applications of [flowOn], [buffer], [conflate], and [produceIn] to the result of `consumeAsFlow` are fused. + * In particular, [produceIn] returns the original channel (but throws [IllegalStateException] on repeated calls). + * Calls to [flowOn] have generally no effect, unless [buffer] is used to explicitly request buffering. + */ +public fun ReceiveChannel.consumeAsFlow(): Flow = ChannelAsFlow(this, consume = true) + +/** + * Represents an existing [channel] as [ChannelFlow] implementation. + * It fuses with subsequent [flowOn] operators, but for the most part ignores the specified context. + * However, additional [buffer] calls cause a separate buffering channel to be created and that is where + * the context might play a role, because it is used by the producing coroutine. + */ +private class ChannelAsFlow( + private val channel: ReceiveChannel, + private val consume: Boolean, + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = Channel.OPTIONAL_CHANNEL, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND +) : ChannelFlow(context, capacity, onBufferOverflow) { + private val consumed = atomic(false) + + private fun markConsumed() { + if (consume) { + check(!consumed.getAndSet(true)) { "ReceiveChannel.consumeAsFlow can be collected just once" } + } + } + + override fun create(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): ChannelFlow = + ChannelAsFlow(channel, consume, context, capacity, onBufferOverflow) + + override fun dropChannelOperators(): Flow = + ChannelAsFlow(channel, consume) + + override suspend fun collectTo(scope: ProducerScope) = + SendingCollector(scope).emitAllImpl(channel, consume) // use efficient channel receiving code from emitAll + + override fun produceImpl(scope: CoroutineScope): ReceiveChannel { + markConsumed() // fail fast on repeated attempt to collect it + return if (capacity == Channel.OPTIONAL_CHANNEL) { + channel // direct + } else + super.produceImpl(scope) // extra buffering channel + } + + override suspend fun collect(collector: FlowCollector) { + if (capacity == Channel.OPTIONAL_CHANNEL) { + markConsumed() + collector.emitAllImpl(channel, consume) // direct + } else { + super.collect(collector) // extra buffering channel, produceImpl will mark it as consumed + } + } + + override fun additionalToStringProps(): String = "channel=$channel" +} + +/** + * Creates a [produce] coroutine that collects the given flow. + * + * This transformation is **stateful**, it launches a [produce] coroutine + * that collects the given flow, and has the same behavior: + * + * - if collecting the flow throws, the channel will be closed with that exception + * - if the [ReceiveChannel] is cancelled, the collection of the flow will be cancelled + * - if collecting the flow completes normally, the [ReceiveChannel] will be closed normally + * + * A channel with [default][Channel.Factory.BUFFERED] buffer size is created. + * Use [buffer] operator on the flow before calling `produceIn` to specify a value other than + * default and to control what happens when data is produced faster than it is consumed, + * that is to control backpressure behavior. + */ +public fun Flow.produceIn( + scope: CoroutineScope +): ReceiveChannel = + asChannelFlow().produceImpl(scope) diff --git a/kotlinx-coroutines-core/common/src/flow/Flow.kt b/kotlinx-coroutines-core/common/src/flow/Flow.kt new file mode 100644 index 0000000000..17b7c53ca6 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/Flow.kt @@ -0,0 +1,246 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.internal.* +import kotlin.coroutines.* + +/** + * An asynchronous data stream that sequentially emits values and completes normally or with an exception. + * + * _Intermediate operators_ on the flow such as [map], [filter], [take], [zip], etc are functions that are + * applied to the _upstream_ flow or flows and return a _downstream_ flow where further operators can be applied to. + * Intermediate operations do not execute any code in the flow and are not suspending functions themselves. + * They only set up a chain of operations for future execution and quickly return. + * This is known as a _cold flow_ property. + * + * _Terminal operators_ on the flow are either suspending functions such as [collect], [single], [reduce], [toList], etc. + * or [launchIn] operator that starts collection of the flow in the given scope. + * They are applied to the upstream flow and trigger execution of all operations. + * Execution of the flow is also called _collecting the flow_ and is always performed in a suspending manner + * without actual blocking. Terminal operators complete normally or exceptionally depending on successful or failed + * execution of all the flow operations in the upstream. The most basic terminal operator is [collect], for example: + * + * ``` + * try { + * flow.collect { value -> + * println("Received $value") + * } + * } catch (e: Exception) { + * println("The flow has thrown an exception: $e") + * } + * ``` + * + * By default, flows are _sequential_ and all flow operations are executed sequentially in the same coroutine, + * with an exception for a few operations specifically designed to introduce concurrency into flow + * execution such as [buffer] and [flatMapMerge]. See their documentation for details. + * + * The `Flow` interface does not carry information whether a flow is a _cold_ stream that can be collected repeatedly and + * triggers execution of the same code every time it is collected, or if it is a _hot_ stream that emits different + * values from the same running source on each collection. Usually flows represent _cold_ streams, but + * there is a [SharedFlow] subtype that represents _hot_ streams. In addition to that, any flow can be turned + * into a _hot_ one by the [stateIn] and [shareIn] operators, or by converting the flow into a hot channel + * via the [produceIn] operator. + * + * ### Flow builders + * + * There are the following basic ways to create a flow: + * + * - [flowOf(...)][flowOf] functions to create a flow from a fixed set of values. + * - [asFlow()][asFlow] extension functions on various types to convert them into flows. + * - [flow { ... }][flow] builder function to construct arbitrary flows from + * sequential calls to [emit][FlowCollector.emit] function. + * - [channelFlow { ... }][channelFlow] builder function to construct arbitrary flows from + * potentially concurrent calls to the [send][kotlinx.coroutines.channels.SendChannel.send] function. + * - [MutableStateFlow] and [MutableSharedFlow] define the corresponding constructor functions to create + * a _hot_ flow that can be directly updated. + * + * ### Flow constraints + * + * All implementations of the `Flow` interface must adhere to two key properties described in detail below: + * + * - Context preservation. + * - Exception transparency. + * + * These properties ensure the ability to perform local reasoning about the code with flows and modularize the code + * in such a way that upstream flow emitters can be developed separately from downstream flow collectors. + * A user of a flow does not need to be aware of implementation details of the upstream flows it uses. + * + * ### Context preservation + * + * The flow has a context preservation property: it encapsulates its own execution context and never propagates or leaks + * it downstream, thus making reasoning about the execution context of particular transformations or terminal + * operations trivial. + * + * There is only one way to change the context of a flow: the [flowOn][Flow.flowOn] operator + * that changes the upstream context ("everything above the `flowOn` operator"). + * For additional information refer to its documentation. + * + * This reasoning can be demonstrated in practice: + * + * ``` + * val flowA = flowOf(1, 2, 3) + * .map { it + 1 } // Will be executed in ctxA + * .flowOn(ctxA) // Changes the upstream context: flowOf and map + * + * // Now we have a context-preserving flow: it is executed somewhere but this information is encapsulated in the flow itself + * + * val filtered = flowA // ctxA is encapsulated in flowA + * .filter { it == 3 } // Pure operator without a context yet + * + * withContext(Dispatchers.Main) { + * // All non-encapsulated operators will be executed in Main: filter and single + * val result = filtered.single() + * myUi.text = result + * } + * ``` + * + * From the implementation point of view, it means that all flow implementations should + * only emit from the same coroutine context. + * This constraint is efficiently enforced by the default [flow] builder. + * The [flow] builder should be used if the flow implementation does not start any coroutines. + * Its implementation prevents most of the development mistakes: + * + * ``` + * val myFlow = flow { + * // GlobalScope.launch { // is prohibited + * // launch(Dispatchers.IO) { // is prohibited + * // withContext(CoroutineName("myFlow")) { // is prohibited + * emit(1) // OK + * coroutineScope { + * emit(2) // OK -- still the same coroutine + * } + * } + * ``` + * + * Use [channelFlow] if the collection and emission of a flow are to be separated into multiple coroutines. + * It encapsulates all the context preservation work and allows you to focus on your + * domain-specific problem, rather than invariant implementation details. + * It is possible to use any combination of coroutine builders from within [channelFlow]. + * + * If you are looking for performance and are sure that no concurrent emits and context jumps will happen, + * the [flow] builder can be used alongside a [coroutineScope] or [supervisorScope] instead: + * - Scoped primitive should be used to provide a [CoroutineScope]. + * - Changing the context of emission is prohibited, no matter whether it is `withContext(ctx)` or + * a builder argument (e.g. `launch(ctx)`). + * - Collecting another flow from a separate context is allowed, but it has the same effect as + * applying the [flowOn] operator to that flow, which is more efficient. + * + * ### Exception transparency + * + * When `emit` or `emitAll` throws, the Flow implementations must immediately stop emitting new values and finish with an exception. + * For diagnostics or application-specific purposes, the exception may be different from the one thrown by the emit operation, + * suppressing the original exception as discussed below. + * If there is a need to emit values after the downstream failed, please use the [catch][Flow.catch] operator. + * + * The [catch][Flow.catch] operator only catches upstream exceptions, but passes + * all downstream exceptions. Similarly, terminal operators like [collect][Flow.collect] + * throw any unhandled exceptions that occur in their code or in upstream flows, for example: + * + * ``` + * flow { emitData() } + * .map { computeOne(it) } + * .catch { ... } // catches exceptions in emitData and computeOne + * .map { computeTwo(it) } + * .collect { process(it) } // throws exceptions from process and computeTwo + * ``` + * The same reasoning can be applied to the [onCompletion] operator that is a declarative replacement for the `finally` block. + * + * All exception-handling Flow operators follow the principle of exception suppression: + * + * If the upstream flow throws an exception during its completion when the downstream exception has been thrown, + * the downstream exception becomes superseded and suppressed by the upstream exception, being a semantic + * equivalent of throwing from `finally` block. However, this doesn't affect the operation of the exception-handling operators, + * which consider the downstream exception to be the root cause and behave as if the upstream didn't throw anything. + * + * Failure to adhere to the exception transparency requirement can lead to strange behaviors which make + * it hard to reason about the code because an exception in the `collect { ... }` could be somehow "caught" + * by an upstream flow, limiting the ability of local reasoning about the code. + * + * Flow machinery enforces exception transparency at runtime and throws [IllegalStateException] on any attempt to emit a value, + * if an exception has been thrown on previous attempt. + * + * ### Reactive streams + * + * Flow is [Reactive Streams](http://www.reactive-streams.org/) compliant, you can safely interop it with + * reactive streams using [Flow.asPublisher] and [Publisher.asFlow] from `kotlinx-coroutines-reactive` module. + * + * ### Not stable for inheritance + * + * **The `Flow` interface is not stable for inheritance in 3rd party libraries**, as new methods + * might be added to this interface in the future, but is stable for use. + * + * Use the `flow { ... }` builder function to create an implementation, or extend [AbstractFlow]. + * These implementations ensure that the context preservation property is not violated, and prevent most + * of the developer mistakes related to concurrency, inconsistent flow dispatchers, and cancellation. + */ +public interface Flow { + + /** + * Accepts the given [collector] and [emits][FlowCollector.emit] values into it. + * + * This method can be used along with SAM-conversion of [FlowCollector]: + * ``` + * myFlow.collect { value -> println("Collected $value") } + * ``` + * + * ### Method inheritance + * + * To ensure the context preservation property, it is not recommended implementing this method directly. + * Instead, [AbstractFlow] can be used as the base type to properly ensure flow's properties. + * + * All default flow implementations ensure context preservation and exception transparency properties on a best-effort basis + * and throw [IllegalStateException] if a violation was detected. + */ + public suspend fun collect(collector: FlowCollector) +} + +/** + * Base class for stateful implementations of `Flow`. + * It tracks all the properties required for context preservation and throws an [IllegalStateException] + * if any of the properties are violated. + * + * Example of the implementation: + * + * ``` + * // list.asFlow() + collect counter + * class CountingListFlow(private val values: List) : AbstractFlow() { + * private val collectedCounter = AtomicInteger(0) + * + * override suspend fun collectSafely(collector: FlowCollector) { + * collectedCounter.incrementAndGet() // Increment collected counter + * values.forEach { // Emit all the values + * collector.emit(it) + * } + * } + * + * fun toDiagnosticString(): String = "Flow with values $values was collected ${collectedCounter.value} times" + * } + * ``` + */ +@ExperimentalCoroutinesApi +public abstract class AbstractFlow : Flow, CancellableFlow { + + public final override suspend fun collect(collector: FlowCollector) { + val safeCollector = SafeCollector(collector, coroutineContext) + try { + collectSafely(safeCollector) + } finally { + safeCollector.releaseIntercepted() + } + } + + /** + * Accepts the given [collector] and [emits][FlowCollector.emit] values into it. + * + * A valid implementation of this method has the following constraints: + * 1) It should not change the coroutine context (e.g. with `withContext(Dispatchers.IO)`) when emitting values. + * The emission should happen in the context of the [collect] call. + * Please refer to the top-level [Flow] documentation for more details. + * 2) It should serialize calls to [emit][FlowCollector.emit] as [FlowCollector] implementations are not + * thread-safe by default. + * To automatically serialize emissions [channelFlow] builder can be used instead of [flow] + * + * @throws IllegalStateException if any of the invariants are violated. + */ + public abstract suspend fun collectSafely(collector: FlowCollector) +} diff --git a/kotlinx-coroutines-core/common/src/flow/FlowCollector.kt b/kotlinx-coroutines-core/common/src/flow/FlowCollector.kt new file mode 100644 index 0000000000..2110a82800 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/FlowCollector.kt @@ -0,0 +1,32 @@ +package kotlinx.coroutines.flow + +/** + * [FlowCollector] is used as an intermediate or a terminal collector of the flow and represents + * an entity that accepts values emitted by the [Flow]. + * + * This interface should usually not be implemented directly, but rather used as a receiver in a [flow] builder when implementing a custom operator, + * or with SAM-conversion. + * Implementations of this interface are not thread-safe. + * + * Example of usage: + * + * ``` + * val flow = getMyEvents() + * try { + * flow.collect { value -> + * println("Received $value") + * } + * println("My events are consumed successfully") + * } catch (e: Throwable) { + * println("Exception from the flow: $e") + * } + * ``` + */ +public fun interface FlowCollector { + + /** + * Collects the value emitted by the upstream. + * This method is not thread-safe and should not be invoked concurrently. + */ + public suspend fun emit(value: T) +} diff --git a/kotlinx-coroutines-core/common/src/flow/Migration.kt b/kotlinx-coroutines-core/common/src/flow/Migration.kt new file mode 100644 index 0000000000..eef5e9eb5e --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/Migration.kt @@ -0,0 +1,491 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") +@file:Suppress("unused", "DeprecatedCallableAddReplaceWith", "UNUSED_PARAMETER") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.jvm.* + +/** + * **GENERAL NOTE** + * + * These deprecations are added to improve user experience when they will start to + * search for their favourite operators and/or patterns that are missing or renamed in Flow. + * Deprecated functions also are moved here when they renamed. The difference is that they have + * a body with their implementation while pure stubs have [noImpl]. + */ +internal fun noImpl(): Nothing = + throw UnsupportedOperationException("Not implemented, should not be called") + +/** + * `observeOn` has no direct match in [Flow] API because all terminal flow operators are suspending and + * thus use the context of the caller. + * + * For example, the following code: + * ``` + * flowable + * .observeOn(Schedulers.io()) + * .doOnEach { value -> println("Received $value") } + * .subscribe() + * ``` + * + * has the following Flow equivalent: + * ``` + * withContext(Dispatchers.IO) { + * flow.collect { value -> println("Received $value") } + * } + * + * ``` + * @suppress + */ +@Deprecated(message = "Collect flow in the desired context instead", level = DeprecationLevel.ERROR) +public fun Flow.observeOn(context: CoroutineContext): Flow = noImpl() + +/** + * `publishOn` has no direct match in [Flow] API because all terminal flow operators are suspending and + * thus use the context of the caller. + * + * For example, the following code: + * ``` + * flux + * .publishOn(Schedulers.io()) + * .doOnEach { value -> println("Received $value") } + * .subscribe() + * ``` + * + * has the following Flow equivalent: + * ``` + * withContext(Dispatchers.IO) { + * flow.collect { value -> println("Received $value") } + * } + * + * ``` + * @suppress + */ +@Deprecated(message = "Collect flow in the desired context instead", level = DeprecationLevel.ERROR) +public fun Flow.publishOn(context: CoroutineContext): Flow = noImpl() + +/** + * `subscribeOn` has no direct match in [Flow] API because [Flow] preserves its context and does not leak it. + * + * For example, the following code: + * ``` + * flowable + * .map { value -> println("Doing map in IO"); value } + * .subscribeOn(Schedulers.io()) + * .observeOn(Schedulers.computation()) + * .doOnEach { value -> println("Processing $value in computation") + * .subscribe() + * ``` + * has the following Flow equivalent: + * ``` + * withContext(Dispatchers.Default) { + * flow + * .map { value -> println("Doing map in IO"); value } + * .flowOn(Dispatchers.IO) // Works upstream, doesn't change downstream + * .collect { value -> + * println("Processing $value in computation") + * } + * } + * ``` + * Opposed to subscribeOn, it is **possible** to use multiple `flowOn` operators in the one flow + * @suppress + */ +@Deprecated(message = "Use 'flowOn' instead", level = DeprecationLevel.ERROR) +public fun Flow.subscribeOn(context: CoroutineContext): Flow = noImpl() + +/** + * Flow analogue of `onErrorXxx` is [catch]. + * Use `catch { emitAll(fallback) }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'onErrorXxx' is 'catch'. Use 'catch { emitAll(fallback) }'", + replaceWith = ReplaceWith("catch { emitAll(fallback) }") +) +public fun Flow.onErrorResume(fallback: Flow): Flow = noImpl() + +/** + * Flow analogue of `onErrorXxx` is [catch]. + * Use `catch { emitAll(fallback) }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'onErrorXxx' is 'catch'. Use 'catch { emitAll(fallback) }'", + replaceWith = ReplaceWith("catch { emitAll(fallback) }") +) +public fun Flow.onErrorResumeNext(fallback: Flow): Flow = noImpl() + +/** + * `subscribe` is Rx-specific API that has no direct match in flows. + * One can use [launchIn] instead, for example the following: + * ``` + * flowable + * .observeOn(Schedulers.io()) + * .subscribe({ println("Received $it") }, { println("Exception $it happened") }, { println("Flowable is completed successfully") } + * ``` + * + * has the following Flow equivalent: + * ``` + * flow + * .onEach { value -> println("Received $value") } + * .onCompletion { cause -> if (cause == null) println("Flow is completed successfully") } + * .catch { cause -> println("Exception $cause happened") } + * .flowOn(Dispatchers.IO) + * .launchIn(myScope) + * ``` + * + * Note that resulting value of [launchIn] is not used because the provided scope takes care of cancellation. + * + * Or terminal operators like [single] can be used from suspend functions. + * @suppress + */ +@Deprecated( + message = "Use 'launchIn' with 'onEach', 'onCompletion' and 'catch' instead", + level = DeprecationLevel.ERROR +) +public fun Flow.subscribe(): Unit = noImpl() + +/** + * Use [launchIn] with [onEach], [onCompletion] and [catch] operators instead. + * @suppress + */ +@Deprecated( + message = "Use 'launchIn' with 'onEach', 'onCompletion' and 'catch' instead", + level = DeprecationLevel.ERROR +)public fun Flow.subscribe(onEach: suspend (T) -> Unit): Unit = noImpl() + +/** + * Use [launchIn] with [onEach], [onCompletion] and [catch] operators instead. + * @suppress + */ +@Deprecated( + message = "Use 'launchIn' with 'onEach', 'onCompletion' and 'catch' instead", + level = DeprecationLevel.ERROR +)public fun Flow.subscribe(onEach: suspend (T) -> Unit, onError: suspend (Throwable) -> Unit): Unit = noImpl() + +/** + * Note that this replacement is sequential (`concat`) by default. + * For concurrent flatMap [flatMapMerge] can be used instead. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue is 'flatMapConcat'", + replaceWith = ReplaceWith("flatMapConcat(mapper)") +) +public fun Flow.flatMap(mapper: suspend (T) -> Flow): Flow = noImpl() + +/** + * Flow analogue of `concatMap` is [flatMapConcat]. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'concatMap' is 'flatMapConcat'", + replaceWith = ReplaceWith("flatMapConcat(mapper)") +) +public fun Flow.concatMap(mapper: (T) -> Flow): Flow = noImpl() + +/** + * Note that this replacement is sequential (`concat`) by default. + * For concurrent flatMap [flattenMerge] can be used instead. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'merge' is 'flattenConcat'", + replaceWith = ReplaceWith("flattenConcat()") +) +public fun Flow>.merge(): Flow = noImpl() + +/** + * Flow analogue of `flatten` is [flattenConcat]. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'flatten' is 'flattenConcat'", + replaceWith = ReplaceWith("flattenConcat()") +) +public fun Flow>.flatten(): Flow = noImpl() + +/** + * Kotlin has a built-in generic mechanism for making chained calls. + * If you wish to write something like + * ``` + * myFlow.compose(MyFlowExtensions.ignoreErrors()).collect { ... } + * ``` + * you can replace it with + * + * ``` + * myFlow.let(MyFlowExtensions.ignoreErrors()).collect { ... } + * ``` + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'compose' is 'let'", + replaceWith = ReplaceWith("let(transformer)") +) +public fun Flow.compose(transformer: Flow.() -> Flow): Flow = noImpl() + +/** + * Flow analogue of `skip` is [drop]. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'skip' is 'drop'", + replaceWith = ReplaceWith("drop(count)") +) +public fun Flow.skip(count: Int): Flow = noImpl() + +/** + * Flow extension to iterate over elements is [collect]. + * Foreach wasn't introduced deliberately to avoid confusion. + * Flow is not a collection, iteration over it may be not idempotent + * and can *launch* computations with side-effects. + * This behaviour is not reflected in [forEach] name. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'forEach' is 'collect'", + replaceWith = ReplaceWith("collect(action)") +) +public fun Flow.forEach(action: suspend (value: T) -> Unit): Unit = noImpl() + +/** + * Flow has less verbose [scan] shortcut. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow has less verbose 'scan' shortcut", + replaceWith = ReplaceWith("scan(initial, operation)") +) +public fun Flow.scanFold(initial: R, @BuilderInference operation: suspend (accumulator: R, value: T) -> R): Flow = + noImpl() + +/** + * Flow analogue of `onErrorXxx` is [catch]. + * Use `catch { emit(fallback) }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'onErrorXxx' is 'catch'. Use 'catch { emit(fallback) }'", + replaceWith = ReplaceWith("catch { emit(fallback) }") +) +// Note: this version without predicate gives better "replaceWith" action +public fun Flow.onErrorReturn(fallback: T): Flow = noImpl() + +/** + * Flow analogue of `onErrorXxx` is [catch]. + * Use `catch { e -> if (predicate(e)) emit(fallback) else throw e }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'onErrorXxx' is 'catch'. Use 'catch { e -> if (predicate(e)) emit(fallback) else throw e }'", + replaceWith = ReplaceWith("catch { e -> if (predicate(e)) emit(fallback) else throw e }") +) +public fun Flow.onErrorReturn(fallback: T, predicate: (Throwable) -> Boolean = { true }): Flow = + catch { e -> + // Note: default value is for binary compatibility with preview version, that is why it has body + if (!predicate(e)) throw e + emit(fallback) + } + +/** + * Flow analogue of `startWith` is [onStart]. + * Use `onStart { emit(value) }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'startWith' is 'onStart'. Use 'onStart { emit(value) }'", + replaceWith = ReplaceWith("onStart { emit(value) }") +) +public fun Flow.startWith(value: T): Flow = noImpl() + +/** + * Flow analogue of `startWith` is [onStart]. + * Use `onStart { emitAll(other) }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'startWith' is 'onStart'. Use 'onStart { emitAll(other) }'", + replaceWith = ReplaceWith("onStart { emitAll(other) }") +) +public fun Flow.startWith(other: Flow): Flow = noImpl() + +/** + * Flow analogue of `concatWith` is [onCompletion]. + * Use `onCompletion { emit(value) }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'concatWith' is 'onCompletion'. Use 'onCompletion { emit(value) }'", + replaceWith = ReplaceWith("onCompletion { emit(value) }") +) +public fun Flow.concatWith(value: T): Flow = noImpl() + +/** + * Flow analogue of `concatWith` is [onCompletion]. + * Use `onCompletion { if (it == null) emitAll(other) }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'concatWith' is 'onCompletion'. Use 'onCompletion { if (it == null) emitAll(other) }'", + replaceWith = ReplaceWith("onCompletion { if (it == null) emitAll(other) }") +) +public fun Flow.concatWith(other: Flow): Flow = noImpl() + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'combineLatest' is 'combine'", + replaceWith = ReplaceWith("this.combine(other, transform)") +) +public fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow = + combine(this, other, transform) + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'combineLatest' is 'combine'", + replaceWith = ReplaceWith("combine(this, other, other2, transform)") +) +public fun Flow.combineLatest( + other: Flow, + other2: Flow, + transform: suspend (T1, T2, T3) -> R +): Flow = combine(this, other, other2, transform) + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'combineLatest' is 'combine'", + replaceWith = ReplaceWith("combine(this, other, other2, other3, transform)") +) +public fun Flow.combineLatest( + other: Flow, + other2: Flow, + other3: Flow, + transform: suspend (T1, T2, T3, T4) -> R +): Flow = combine(this, other, other2, other3, transform) + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'combineLatest' is 'combine'", + replaceWith = ReplaceWith("combine(this, other, other2, other3, transform)") +) +public fun Flow.combineLatest( + other: Flow, + other2: Flow, + other3: Flow, + other4: Flow, + transform: suspend (T1, T2, T3, T4, T5) -> R +): Flow = combine(this, other, other2, other3, other4, transform) + +/** + * Delays the emission of values from this flow for the given [timeMillis]. + * Use `onStart { delay(timeMillis) }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, // since 1.3.0, error in 1.5.0 + message = "Use 'onStart { delay(timeMillis) }'", + replaceWith = ReplaceWith("onStart { delay(timeMillis) }") +) +public fun Flow.delayFlow(timeMillis: Long): Flow = onStart { delay(timeMillis) } + +/** + * Delays each element emitted by the given flow for the given [timeMillis]. + * Use `onEach { delay(timeMillis) }`. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, // since 1.3.0, error in 1.5.0 + message = "Use 'onEach { delay(timeMillis) }'", + replaceWith = ReplaceWith("onEach { delay(timeMillis) }") +) +public fun Flow.delayEach(timeMillis: Long): Flow = onEach { delay(timeMillis) } + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogues of 'switchMap' are 'transformLatest', 'flatMapLatest' and 'mapLatest'", + replaceWith = ReplaceWith("this.flatMapLatest(transform)") +) +public fun Flow.switchMap(transform: suspend (value: T) -> Flow): Flow = flatMapLatest(transform) + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, // Warning since 1.3.8, was experimental when deprecated, ERROR since 1.5.0 + message = "'scanReduce' was renamed to 'runningReduce' to be consistent with Kotlin standard library", + replaceWith = ReplaceWith("runningReduce(operation)") +) +public fun Flow.scanReduce(operation: suspend (accumulator: T, value: T) -> T): Flow = runningReduce(operation) + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'publish()' is 'shareIn'. \n" + + "publish().connect() is the default strategy (no extra call is needed), \n" + + "publish().autoConnect() translates to 'started = SharingStarted.Lazily' argument, \n" + + "publish().refCount() translates to 'started = SharingStarted.WhileSubscribed()' argument.", + replaceWith = ReplaceWith("this.shareIn(scope, 0)") +) +public fun Flow.publish(): Flow = noImpl() + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'publish(bufferSize)' is 'buffer' followed by 'shareIn'. \n" + + "publish().connect() is the default strategy (no extra call is needed), \n" + + "publish().autoConnect() translates to 'started = SharingStarted.Lazily' argument, \n" + + "publish().refCount() translates to 'started = SharingStarted.WhileSubscribed()' argument.", + replaceWith = ReplaceWith("this.buffer(bufferSize).shareIn(scope, 0)") +) +public fun Flow.publish(bufferSize: Int): Flow = noImpl() + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'replay()' is 'shareIn' with unlimited replay. \n" + + "replay().connect() is the default strategy (no extra call is needed), \n" + + "replay().autoConnect() translates to 'started = SharingStarted.Lazily' argument, \n" + + "replay().refCount() translates to 'started = SharingStarted.WhileSubscribed()' argument.", + replaceWith = ReplaceWith("this.shareIn(scope, Int.MAX_VALUE)") +) +public fun Flow.replay(): Flow = noImpl() + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'replay(bufferSize)' is 'shareIn' with the specified replay parameter. \n" + + "replay().connect() is the default strategy (no extra call is needed), \n" + + "replay().autoConnect() translates to 'started = SharingStarted.Lazily' argument, \n" + + "replay().refCount() translates to 'started = SharingStarted.WhileSubscribed()' argument.", + replaceWith = ReplaceWith("this.shareIn(scope, bufferSize)") +) +public fun Flow.replay(bufferSize: Int): Flow = noImpl() + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Flow analogue of 'cache()' is 'shareIn' with unlimited replay and 'started = SharingStarted.Lazily' argument'", + replaceWith = ReplaceWith("this.shareIn(scope, started = SharingStarted.Lazily, replay = Int.MAX_VALUE)") +) +public fun Flow.cache(): Flow = noImpl() diff --git a/kotlinx-coroutines-core/common/src/flow/SharedFlow.kt b/kotlinx-coroutines-core/common/src/flow/SharedFlow.kt new file mode 100644 index 0000000000..4f19641e48 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/SharedFlow.kt @@ -0,0 +1,745 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.internal.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* + +/** + * A _hot_ [Flow] that shares emitted values among all its collectors in a broadcast fashion, so that all collectors + * get all emitted values. A shared flow is called _hot_ because its active instance exists independently of the + * presence of collectors. This is opposed to a regular [Flow], such as defined by the [`flow { ... }`][flow] function, + * which is _cold_ and is started separately for each collector. + * + * **Shared flow never completes**. A call to [Flow.collect] on a shared flow never completes normally, and + * neither does a coroutine started by the [Flow.launchIn] function. An active collector of a shared flow is called a _subscriber_. + * + * A subscriber of a shared flow can be cancelled. This usually happens when the scope in which the coroutine is running + * is cancelled. A subscriber to a shared flow is always [cancellable][Flow.cancellable], and checks for + * cancellation before each emission. Note that most terminal operators like [Flow.toList] would also not complete, + * when applied to a shared flow, but flow-truncating operators like [Flow.take] and [Flow.takeWhile] can be used on a + * shared flow to turn it into a completing one. + * + * A [mutable shared flow][MutableSharedFlow] is created using the [MutableSharedFlow(...)] constructor function. + * Its state can be updated by [emitting][MutableSharedFlow.emit] values to it and performing other operations. + * See the [MutableSharedFlow] documentation for details. + * + * [SharedFlow] is useful for broadcasting events that happen inside an application to subscribers that can come and go. + * For example, the following class encapsulates an event bus that distributes events to all subscribers + * in a _rendezvous_ manner, suspending until all subscribers receive emitted event: + * + * ``` + * class EventBus { + * private val _events = MutableSharedFlow() // private mutable shared flow + * val events = _events.asSharedFlow() // publicly exposed as read-only shared flow + * + * suspend fun produceEvent(event: Event) { + * _events.emit(event) // suspends until all subscribers receive it + * } + * } + * ``` + * + * As an alternative to the above usage with the `MutableSharedFlow(...)` constructor function, + * any _cold_ [Flow] can be converted to a shared flow using the [shareIn] operator. + * + * There is a specialized implementation of shared flow for the case where the most recent state value needs + * to be shared. See [StateFlow] for details. + * + * ### Replay cache and buffer + * + * A shared flow keeps a specific number of the most recent values in its _replay cache_. Every new subscriber first + * gets the values from the replay cache and then gets new emitted values. The maximum size of the replay cache is + * specified when the shared flow is created by the `replay` parameter. A snapshot of the current replay cache + * is available via the [replayCache] property and it can be reset with the [MutableSharedFlow.resetReplayCache] function. + * + * A replay cache also provides buffer for emissions to the shared flow, allowing slow subscribers to + * get values from the buffer without suspending emitters. The buffer space determines how much slow subscribers + * can lag from the fast ones. When creating a shared flow, additional buffer capacity beyond replay can be reserved + * using the `extraBufferCapacity` parameter. + * + * A shared flow with a buffer can be configured to avoid suspension of emitters on buffer overflow using + * the `onBufferOverflow` parameter, which is equal to one of the entries of the [BufferOverflow] enum. When a strategy other + * than [SUSPENDED][BufferOverflow.SUSPEND] is configured, emissions to the shared flow never suspend. + * + * **Buffer overflow condition can happen only when there is at least one subscriber that is not ready to accept + * the new value.** In the absence of subscribers only the most recent `replay` values are stored and the buffer + * overflow behavior is never triggered and has no effect. In particular, in the absence of subscribers emitter never + * suspends despite [BufferOverflow.SUSPEND] option and [BufferOverflow.DROP_LATEST] option does not have effect either. + * Essentially, the behavior in the absence of subscribers is always similar to [BufferOverflow.DROP_OLDEST], + * but the buffer is just of `replay` size (without any `extraBufferCapacity`). + * + * ### Unbuffered shared flow + * + * A default implementation of a shared flow that is created with `MutableSharedFlow()` constructor function + * without parameters has no replay cache nor additional buffer. + * [emit][MutableSharedFlow.emit] call to such a shared flow suspends until all subscribers receive the emitted value + * and returns immediately if there are no subscribers. + * Thus, [tryEmit][MutableSharedFlow.tryEmit] call succeeds and returns `true` only if + * there are no subscribers (in which case the emitted value is immediately lost). + * + * ### SharedFlow vs BroadcastChannel + * + * Conceptually shared flow is similar to [BroadcastChannel][BroadcastChannel] + * and is designed to completely replace it. + * It has the following important differences: + * + * - `SharedFlow` is simpler, because it does not have to implement all the [Channel] APIs, which allows + * for faster and simpler implementation. + * - `SharedFlow` supports configurable replay and buffer overflow strategy. + * - `SharedFlow` has a clear separation into a read-only `SharedFlow` interface and a [MutableSharedFlow]. + * - `SharedFlow` cannot be closed like `BroadcastChannel` and can never represent a failure. + * All errors and completion signals should be explicitly _materialized_ if needed. + * + * To migrate [BroadcastChannel] usage to [SharedFlow], start by replacing usages of the `BroadcastChannel(capacity)` + * constructor with `MutableSharedFlow(0, extraBufferCapacity=capacity)` (broadcast channel does not replay + * values to new subscribers). Replace [send][BroadcastChannel.send] and [trySend][BroadcastChannel.trySend] calls + * with [emit][MutableStateFlow.emit] and [tryEmit][MutableStateFlow.tryEmit], and convert subscribers' code to flow operators. + * + * ### Concurrency + * + * All methods of shared flow are **thread-safe** and can be safely invoked from concurrent coroutines without + * external synchronization. + * + * ### Operator fusion + * + * Application of [flowOn][Flow.flowOn], [buffer] with [RENDEZVOUS][Channel.RENDEZVOUS] capacity, + * or [cancellable] operators to a shared flow has no effect. + * + * ### Implementation notes + * + * Shared flow implementation uses a lock to ensure thread-safety, but suspending collector and emitter coroutines are + * resumed outside of this lock to avoid deadlocks when using unconfined coroutines. Adding new subscribers + * has `O(1)` amortized cost, but emitting has `O(N)` cost, where `N` is the number of subscribers. + * + * ### Not stable for inheritance + * + * **The `SharedFlow` interface is not stable for inheritance in 3rd party libraries**, as new methods + * might be added to this interface in the future, but is stable for use. + * Use the `MutableSharedFlow(replay, ...)` constructor function to create an implementation. + */ +@OptIn(ExperimentalSubclassOptIn::class) +@SubclassOptInRequired(ExperimentalForInheritanceCoroutinesApi::class) +public interface SharedFlow : Flow { + /** + * A snapshot of the replay cache. + */ + public val replayCache: List + + /** + * Accepts the given [collector] and [emits][FlowCollector.emit] values into it. + * To emit values from a shared flow into a specific collector, either `collector.emitAll(flow)` or `collect { ... }` + * SAM-conversion can be used. + * + * **A shared flow never completes**. A call to [Flow.collect] or any other terminal operator + * on a shared flow never completes normally. + * + * It is guaranteed that, by the time the first suspension happens, [collect] has already subscribed to the + * [SharedFlow] and is eligible for receiving emissions. In particular, the following code will always print `1`: + * ``` + * val flow = MutableSharedFlow() + * launch(start = CoroutineStart.UNDISPATCHED) { + * flow.collect { println(1) } + * } + * flow.emit(1) + * ``` + * + * @see [Flow.collect] for implementation and inheritance details. + */ + override suspend fun collect(collector: FlowCollector): Nothing +} + +/** + * A mutable [SharedFlow] that provides functions to [emit] values to the flow. + * An instance of `MutableSharedFlow` with the given configuration parameters can be created using `MutableSharedFlow(...)` + * constructor function. + * + * See the [SharedFlow] documentation for details on shared flows. + * + * `MutableSharedFlow` is a [SharedFlow] that also provides the abilities to [emit] a value, + * to [tryEmit] without suspension if possible, to track the [subscriptionCount], + * and to [resetReplayCache]. + * + * ### Concurrency + * + * All methods of shared flow are **thread-safe** and can be safely invoked from concurrent coroutines without + * external synchronization. + * + * ### Not stable for inheritance + * + * **The `MutableSharedFlow` interface is not stable for inheritance in 3rd party libraries**, as new methods + * might be added to this interface in the future, but is stable for use. + * Use the `MutableSharedFlow(...)` constructor function to create an implementation. + */ +@OptIn(ExperimentalSubclassOptIn::class) +@SubclassOptInRequired(ExperimentalForInheritanceCoroutinesApi::class) +public interface MutableSharedFlow : SharedFlow, FlowCollector { + /** + * Emits a [value] to this shared flow, suspending on buffer overflow. + * + * This call can suspend only when the [BufferOverflow] strategy is + * [SUSPEND][BufferOverflow.SUSPEND] **and** there are subscribers collecting this shared flow. + * + * If there are no subscribers, the buffer is not used. + * Instead, the most recently emitted value is simply stored into + * the replay cache if one was configured, displacing the older elements there, + * or dropped if no replay cache was configured. + * + * See [tryEmit] for a non-suspending variant of this function. + * + * This method is **thread-safe** and can be safely invoked from concurrent coroutines without + * external synchronization. + */ + override suspend fun emit(value: T) + + /** + * Tries to emit a [value] to this shared flow without suspending. It returns `true` if the value was + * emitted successfully (see below). When this function returns `false`, it means that a call to a plain [emit] + * function would suspend until there is buffer space available. + * + * This call can return `false` only when the [BufferOverflow] strategy is + * [SUSPEND][BufferOverflow.SUSPEND] **and** there are subscribers collecting this shared flow. + * + * If there are no subscribers, the buffer is not used. + * Instead, the most recently emitted value is simply stored into + * the replay cache if one was configured, displacing the older elements there, + * or dropped if no replay cache was configured. In any case, `tryEmit` returns `true`. + * + * This method is **thread-safe** and can be safely invoked from concurrent coroutines without + * external synchronization. + */ + public fun tryEmit(value: T): Boolean + + /** + * The number of subscribers (active collectors) to this shared flow. + * + * The integer in the resulting [StateFlow] is not negative and starts with zero for a freshly created + * shared flow. + * + * This state can be used to react to changes in the number of subscriptions to this shared flow. + * For example, if you need to call `onActive` when the first subscriber appears and `onInactive` + * when the last one disappears, you can set it up like this: + * + * ``` + * sharedFlow.subscriptionCount + * .map { count -> count > 0 } // map count into active/inactive flag + * .distinctUntilChanged() // only react to true<->false changes + * .onEach { isActive -> // configure an action + * if (isActive) onActive() else onInactive() + * } + * .launchIn(scope) // launch it + * ``` + * + * Usually, [StateFlow] conflates values, but [subscriptionCount] is not conflated. + * This is done so that any subscribers that need to be notified when subscribers appear do + * reliably observe it. With conflation, if a single subscriber appeared and immediately left, those + * collecting [subscriptionCount] could fail to notice it due to `0` immediately conflating the + * subscription count. + */ + public val subscriptionCount: StateFlow + + /** + * Resets the [replayCache] of this shared flow to an empty state. + * New subscribers will be receiving only the values that were emitted after this call, + * while old subscribers will still be receiving previously buffered values. + * To reset a shared flow to an initial value, emit the value after this call. + * + * On a [MutableStateFlow], which always contains a single value, this function is not + * supported, and throws an [UnsupportedOperationException]. To reset a [MutableStateFlow] + * to an initial value, just update its [value][MutableStateFlow.value]. + * + * This method is **thread-safe** and can be safely invoked from concurrent coroutines without + * external synchronization. + * + * **Note: This is an experimental api.** This function may be removed or renamed in the future. + */ + @ExperimentalCoroutinesApi + public fun resetReplayCache() +} + +/** + * Creates a [MutableSharedFlow] with the given configuration parameters. + * + * This function throws [IllegalArgumentException] on unsupported values of parameters or combinations thereof. + * + * @param replay the number of values replayed to new subscribers (cannot be negative, defaults to zero). + * @param extraBufferCapacity the number of values buffered in addition to `replay`. + * [emit][MutableSharedFlow.emit] does not suspend while there is a buffer space remaining (optional, cannot be negative, defaults to zero). + * @param onBufferOverflow configures an [emit][MutableSharedFlow.emit] action on buffer overflow. Optional, defaults to + * [suspending][BufferOverflow.SUSPEND] attempts to emit a value. + * Values other than [BufferOverflow.SUSPEND] are supported only when `replay > 0` or `extraBufferCapacity > 0`. + * **Buffer overflow can happen only when there is at least one subscriber that is not ready to accept + * the new value.** In the absence of subscribers only the most recent [replay] values are stored and + * the buffer overflow behavior is never triggered and has no effect. + */ +@Suppress("FunctionName", "UNCHECKED_CAST") +public fun MutableSharedFlow( + replay: Int = 0, + extraBufferCapacity: Int = 0, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND +): MutableSharedFlow { + require(replay >= 0) { "replay cannot be negative, but was $replay" } + require(extraBufferCapacity >= 0) { "extraBufferCapacity cannot be negative, but was $extraBufferCapacity" } + require(replay > 0 || extraBufferCapacity > 0 || onBufferOverflow == BufferOverflow.SUSPEND) { + "replay or extraBufferCapacity must be positive with non-default onBufferOverflow strategy $onBufferOverflow" + } + val bufferCapacity0 = replay + extraBufferCapacity + val bufferCapacity = if (bufferCapacity0 < 0) Int.MAX_VALUE else bufferCapacity0 // coerce to MAX_VALUE on overflow + return SharedFlowImpl(replay, bufferCapacity, onBufferOverflow) +} + +// ------------------------------------ Implementation ------------------------------------ + +internal class SharedFlowSlot : AbstractSharedFlowSlot>() { + @JvmField + var index = -1L // current "to-be-emitted" index, -1 means the slot is free now + + @JvmField + var cont: Continuation? = null // collector waiting for new value + + override fun allocateLocked(flow: SharedFlowImpl<*>): Boolean { + if (index >= 0) return false // not free + index = flow.updateNewCollectorIndexLocked() + return true + } + + override fun freeLocked(flow: SharedFlowImpl<*>): Array?> { + assert { index >= 0 } + val oldIndex = index + index = -1L + cont = null // cleanup continuation reference + return flow.updateCollectorIndexLocked(oldIndex) + } +} + +@OptIn(ExperimentalForInheritanceCoroutinesApi::class) +internal open class SharedFlowImpl( + private val replay: Int, + private val bufferCapacity: Int, + private val onBufferOverflow: BufferOverflow +) : AbstractSharedFlow(), MutableSharedFlow, CancellableFlow, FusibleFlow { + /* + Logical structure of the buffer + + buffered values + /-----------------------\ + replayCache queued emitters + /----------\/----------------------\ + +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ + | | 1 | 2 | 3 | 4 | 5 | 6 | E | E | E | E | E | E | | | | + +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ + ^ ^ ^ ^ + | | | | + head | head + bufferSize head + totalSize + | | | + index of the slowest | index of the fastest + possible collector | possible collector + | | + | replayIndex == new collector's index + \---------------------- / + range of possible minCollectorIndex + + head == minOf(minCollectorIndex, replayIndex) // by definition + totalSize == bufferSize + queueSize // by definition + + INVARIANTS: + minCollectorIndex = activeSlots.minOf { it.index } ?: (head + bufferSize) + replayIndex <= head + bufferSize + */ + + // Stored state + private var buffer: Array? = null // allocated when needed, allocated size always power of two + private var replayIndex = 0L // minimal index from which new collector gets values + private var minCollectorIndex = 0L // minimal index of active collectors, equal to replayIndex if there are none + private var bufferSize = 0 // number of buffered values + private var queueSize = 0 // number of queued emitters + + // Computed state + private val head: Long get() = minOf(minCollectorIndex, replayIndex) + private val replaySize: Int get() = (head + bufferSize - replayIndex).toInt() + private val totalSize: Int get() = bufferSize + queueSize + private val bufferEndIndex: Long get() = head + bufferSize + private val queueEndIndex: Long get() = head + bufferSize + queueSize + + override val replayCache: List + get() = synchronized(this) { + val replaySize = this.replaySize + if (replaySize == 0) return emptyList() + val result = ArrayList(replaySize) + val buffer = buffer!! // must be allocated, because replaySize > 0 + @Suppress("UNCHECKED_CAST") + for (i in 0 until replaySize) result += buffer.getBufferAt(replayIndex + i) as T + result + } + + /* + * A tweak for SubscriptionCountStateFlow to get the latest value. + */ + @Suppress("UNCHECKED_CAST") + protected val lastReplayedLocked: T + get() = buffer!!.getBufferAt(replayIndex + replaySize - 1) as T + + @Suppress("UNCHECKED_CAST") + override suspend fun collect(collector: FlowCollector): Nothing { + val slot = allocateSlot() + try { + if (collector is SubscribedFlowCollector) collector.onSubscription() + val collectorJob = currentCoroutineContext()[Job] + while (true) { + var newValue: Any? + while (true) { + newValue = tryTakeValue(slot) // attempt no-suspend fast path first + if (newValue !== NO_VALUE) break + awaitValue(slot) // await signal that the new value is available + } + collectorJob?.ensureActive() + collector.emit(newValue as T) + } + } finally { + freeSlot(slot) + } + } + + override fun tryEmit(value: T): Boolean { + var resumes: Array?> = EMPTY_RESUMES + val emitted = synchronized(this) { + if (tryEmitLocked(value)) { + resumes = findSlotsToResumeLocked(resumes) + true + } else { + false + } + } + for (cont in resumes) cont?.resume(Unit) + return emitted + } + + override suspend fun emit(value: T) { + if (tryEmit(value)) return // fast-path + emitSuspend(value) + } + + @Suppress("UNCHECKED_CAST") + private fun tryEmitLocked(value: T): Boolean { + // Fast path without collectors -> no buffering + if (nCollectors == 0) return tryEmitNoCollectorsLocked(value) // always returns true + // With collectors we'll have to buffer + // cannot emit now if buffer is full & blocked by slow collectors + if (bufferSize >= bufferCapacity && minCollectorIndex <= replayIndex) { + when (onBufferOverflow) { + BufferOverflow.SUSPEND -> return false // will suspend + BufferOverflow.DROP_LATEST -> return true // just drop incoming + BufferOverflow.DROP_OLDEST -> {} // force enqueue & drop oldest instead + } + } + enqueueLocked(value) + bufferSize++ // value was added to buffer + // drop oldest from the buffer if it became more than bufferCapacity + if (bufferSize > bufferCapacity) dropOldestLocked() + // keep replaySize not larger that needed + if (replaySize > replay) { // increment replayIndex by one + updateBufferLocked(replayIndex + 1, minCollectorIndex, bufferEndIndex, queueEndIndex) + } + return true + } + + private fun tryEmitNoCollectorsLocked(value: T): Boolean { + assert { nCollectors == 0 } + if (replay == 0) return true // no need to replay, just forget it now + enqueueLocked(value) // enqueue to replayCache + bufferSize++ // value was added to buffer + // drop oldest from the buffer if it became more than replay + if (bufferSize > replay) dropOldestLocked() + minCollectorIndex = head + bufferSize // a default value (max allowed) + return true + } + + private fun dropOldestLocked() { + buffer!!.setBufferAt(head, null) + bufferSize-- + val newHead = head + 1 + if (replayIndex < newHead) replayIndex = newHead + if (minCollectorIndex < newHead) correctCollectorIndexesOnDropOldest(newHead) + assert { head == newHead } // since head = minOf(minCollectorIndex, replayIndex) it should have updated + } + + private fun correctCollectorIndexesOnDropOldest(newHead: Long) { + forEachSlotLocked { slot -> + @Suppress("ConvertTwoComparisonsToRangeCheck") // Bug in JS backend + if (slot.index >= 0 && slot.index < newHead) { + slot.index = newHead // force move it up (this collector was too slow and missed the value at its index) + } + } + minCollectorIndex = newHead + } + + // enqueues item to buffer array, caller shall increment either bufferSize or queueSize + private fun enqueueLocked(item: Any?) { + val curSize = totalSize + val buffer = when (val curBuffer = buffer) { + null -> growBuffer(null, 0, 2) + else -> if (curSize >= curBuffer.size) growBuffer(curBuffer, curSize,curBuffer.size * 2) else curBuffer + } + buffer.setBufferAt(head + curSize, item) + } + + private fun growBuffer(curBuffer: Array?, curSize: Int, newSize: Int): Array { + check(newSize > 0) { "Buffer size overflow" } + val newBuffer = arrayOfNulls(newSize).also { buffer = it } + if (curBuffer == null) return newBuffer + val head = head + for (i in 0 until curSize) { + newBuffer.setBufferAt(head + i, curBuffer.getBufferAt(head + i)) + } + return newBuffer + } + + private suspend fun emitSuspend(value: T) = suspendCancellableCoroutine sc@{ cont -> + var resumes: Array?> = EMPTY_RESUMES + val emitter = synchronized(this) lock@{ + // recheck buffer under lock again (make sure it is really full) + if (tryEmitLocked(value)) { + cont.resume(Unit) + resumes = findSlotsToResumeLocked(resumes) + return@lock null + } + // add suspended emitter to the buffer + Emitter(this, head + totalSize, value, cont).also { + enqueueLocked(it) + queueSize++ // added to queue of waiting emitters + // synchronous shared flow might rendezvous with waiting emitter + if (bufferCapacity == 0) resumes = findSlotsToResumeLocked(resumes) + } + } + // outside of the lock: register dispose on cancellation + emitter?.let { cont.disposeOnCancellation(it) } + // outside of the lock: resume slots if needed + for (r in resumes) r?.resume(Unit) + } + + private fun cancelEmitter(emitter: Emitter) = synchronized(this) { + if (emitter.index < head) return // already skipped past this index + val buffer = buffer!! + if (buffer.getBufferAt(emitter.index) !== emitter) return // already resumed + buffer.setBufferAt(emitter.index, NO_VALUE) + cleanupTailLocked() + } + + internal fun updateNewCollectorIndexLocked(): Long { + val index = replayIndex + if (index < minCollectorIndex) minCollectorIndex = index + return index + } + + // Is called when a collector disappears or changes index, returns a list of continuations to resume after lock + internal fun updateCollectorIndexLocked(oldIndex: Long): Array?> { + assert { oldIndex >= minCollectorIndex } + if (oldIndex > minCollectorIndex) return EMPTY_RESUMES // nothing changes, it was not min + // start computing new minimal index of active collectors + val head = head + var newMinCollectorIndex = head + bufferSize + // take into account a special case of sync shared flow that can go past 1st queued emitter + if (bufferCapacity == 0 && queueSize > 0) newMinCollectorIndex++ + forEachSlotLocked { slot -> + @Suppress("ConvertTwoComparisonsToRangeCheck") // Bug in JS backend + if (slot.index >= 0 && slot.index < newMinCollectorIndex) newMinCollectorIndex = slot.index + } + assert { newMinCollectorIndex >= minCollectorIndex } // can only grow + if (newMinCollectorIndex <= minCollectorIndex) return EMPTY_RESUMES // nothing changes + // Compute new buffer size if we drop items we no longer need and no emitter is resumed: + // We must keep all the items from newMinIndex to the end of buffer + var newBufferEndIndex = bufferEndIndex // var to grow when waiters are resumed + val maxResumeCount = if (nCollectors > 0) { + // If we have collectors we can resume up to maxResumeCount waiting emitters + // a) queueSize -> that's how many waiting emitters we have + // b) bufferCapacity - newBufferSize0 -> that's how many we can afford to resume to add w/o exceeding bufferCapacity + val newBufferSize0 = (newBufferEndIndex - newMinCollectorIndex).toInt() + minOf(queueSize, bufferCapacity - newBufferSize0) + } else { + // If we don't have collectors anymore we must resume all waiting emitters + queueSize // that's how many waiting emitters we have (at most) + } + var resumes: Array?> = EMPTY_RESUMES + val newQueueEndIndex = newBufferEndIndex + queueSize + if (maxResumeCount > 0) { // collect emitters to resume if we have them + resumes = arrayOfNulls(maxResumeCount) + var resumeCount = 0 + val buffer = buffer!! + for (curEmitterIndex in newBufferEndIndex until newQueueEndIndex) { + val emitter = buffer.getBufferAt(curEmitterIndex) + if (emitter !== NO_VALUE) { + emitter as Emitter // must have Emitter class + resumes[resumeCount++] = emitter.cont + buffer.setBufferAt(curEmitterIndex, NO_VALUE) // make as canceled if we moved ahead + buffer.setBufferAt(newBufferEndIndex, emitter.value) + newBufferEndIndex++ + if (resumeCount >= maxResumeCount) break // enough resumed, done + } + } + } + // Compute new buffer size -> how many values we now actually have after resume + val newBufferSize1 = (newBufferEndIndex - head).toInt() + // Note: When nCollectors == 0 we resume ALL queued emitters and we might have resumed more than bufferCapacity, + // and newMinCollectorIndex might pointing the wrong place because of that. The easiest way to fix it is by + // forcing newMinCollectorIndex = newBufferEndIndex. We do not needed to update newBufferSize1 (which could be + // too big), because the only use of newBufferSize1 in the below code is in the minOf(replay, newBufferSize1) + // expression, which coerces values that are too big anyway. + if (nCollectors == 0) newMinCollectorIndex = newBufferEndIndex + // Compute new replay size -> limit to replay the number of items we need, take into account that it can only grow + var newReplayIndex = maxOf(replayIndex, newBufferEndIndex - minOf(replay, newBufferSize1)) + // adjustment for synchronous case with cancelled emitter (NO_VALUE) + if (bufferCapacity == 0 && newReplayIndex < newQueueEndIndex && buffer!!.getBufferAt(newReplayIndex) == NO_VALUE) { + newBufferEndIndex++ + newReplayIndex++ + } + // Update buffer state + updateBufferLocked(newReplayIndex, newMinCollectorIndex, newBufferEndIndex, newQueueEndIndex) + // just in case we've moved all buffered emitters and have NO_VALUE's at the tail now + cleanupTailLocked() + // We need to waken up suspended collectors if any emitters were resumed here + if (resumes.isNotEmpty()) resumes = findSlotsToResumeLocked(resumes) + return resumes + } + + private fun updateBufferLocked( + newReplayIndex: Long, + newMinCollectorIndex: Long, + newBufferEndIndex: Long, + newQueueEndIndex: Long + ) { + // Compute new head value + val newHead = minOf(newMinCollectorIndex, newReplayIndex) + assert { newHead >= head } + // cleanup items we don't have to buffer anymore (because head is about to move) + for (index in head until newHead) buffer!!.setBufferAt(index, null) + // update all state variables to newly computed values + replayIndex = newReplayIndex + minCollectorIndex = newMinCollectorIndex + bufferSize = (newBufferEndIndex - newHead).toInt() + queueSize = (newQueueEndIndex - newBufferEndIndex).toInt() + // check our key invariants (just in case) + assert { bufferSize >= 0 } + assert { queueSize >= 0 } + assert { replayIndex <= this.head + bufferSize } + } + + // Removes all the NO_VALUE items from the end of the queue and reduces its size + private fun cleanupTailLocked() { + // If we have synchronous case, then keep one emitter queued + if (bufferCapacity == 0 && queueSize <= 1) return // return, don't clear it + val buffer = buffer!! + while (queueSize > 0 && buffer.getBufferAt(head + totalSize - 1) === NO_VALUE) { + queueSize-- + buffer.setBufferAt(head + totalSize, null) + } + } + + // returns NO_VALUE if cannot take value without suspension + private fun tryTakeValue(slot: SharedFlowSlot): Any? { + var resumes: Array?> = EMPTY_RESUMES + val value = synchronized(this) { + val index = tryPeekLocked(slot) + if (index < 0) { + NO_VALUE + } else { + val oldIndex = slot.index + val newValue = getPeekedValueLockedAt(index) + slot.index = index + 1 // points to the next index after peeked one + resumes = updateCollectorIndexLocked(oldIndex) + newValue + } + } + for (resume in resumes) resume?.resume(Unit) + return value + } + + // returns -1 if cannot peek value without suspension + private fun tryPeekLocked(slot: SharedFlowSlot): Long { + // return buffered value if possible + val index = slot.index + if (index < bufferEndIndex) return index + if (bufferCapacity > 0) return -1L // if there's a buffer, never try to rendezvous with emitters + // Synchronous shared flow (bufferCapacity == 0) tries to rendezvous + if (index > head) return -1L // ... but only with the first emitter (never look forward) + if (queueSize == 0) return -1L // nothing there to rendezvous with + return index // rendezvous with the first emitter + } + + private fun getPeekedValueLockedAt(index: Long): Any? = + when (val item = buffer!!.getBufferAt(index)) { + is Emitter -> item.value + else -> item + } + + private suspend fun awaitValue(slot: SharedFlowSlot): Unit = suspendCancellableCoroutine { cont -> + synchronized(this) lock@{ + val index = tryPeekLocked(slot) // recheck under this lock + if (index < 0) { + slot.cont = cont // Ok -- suspending + } else { + cont.resume(Unit) // has value, no need to suspend + return@lock + } + slot.cont = cont // suspend, waiting + } + } + + private fun findSlotsToResumeLocked(resumesIn: Array?>): Array?> { + var resumes: Array?> = resumesIn + var resumeCount = resumesIn.size + forEachSlotLocked loop@{ slot -> + val cont = slot.cont ?: return@loop // only waiting slots + if (tryPeekLocked(slot) < 0) return@loop // only slots that can peek a value + if (resumeCount >= resumes.size) resumes = resumes.copyOf(maxOf(2, 2 * resumes.size)) + resumes[resumeCount++] = cont + slot.cont = null // not waiting anymore + } + return resumes + } + + override fun createSlot() = SharedFlowSlot() + override fun createSlotArray(size: Int): Array = arrayOfNulls(size) + + override fun resetReplayCache() = synchronized(this) { + // Update buffer state + updateBufferLocked( + newReplayIndex = bufferEndIndex, + newMinCollectorIndex = minCollectorIndex, + newBufferEndIndex = bufferEndIndex, + newQueueEndIndex = queueEndIndex + ) + } + + override fun fuse(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow) = + fuseSharedFlow(context, capacity, onBufferOverflow) + + private class Emitter( + @JvmField val flow: SharedFlowImpl<*>, + @JvmField var index: Long, + @JvmField val value: Any?, + @JvmField val cont: Continuation + ) : DisposableHandle { + override fun dispose() = flow.cancelEmitter(this) + } +} + +@JvmField +internal val NO_VALUE = Symbol("NO_VALUE") + +private fun Array.getBufferAt(index: Long) = get(index.toInt() and (size - 1)) +private fun Array.setBufferAt(index: Long, item: Any?) = set(index.toInt() and (size - 1), item) + +internal fun SharedFlow.fuseSharedFlow( + context: CoroutineContext, + capacity: Int, + onBufferOverflow: BufferOverflow +): Flow { + // context is irrelevant for shared flow and making additional rendezvous is meaningless + // however, additional non-trivial buffering after shared flow could make sense for very slow subscribers + if ((capacity == Channel.RENDEZVOUS || capacity == Channel.OPTIONAL_CHANNEL) && onBufferOverflow == BufferOverflow.SUSPEND) { + return this + } + // Apply channel flow operator as usual + return ChannelFlowOperatorImpl(this, context, capacity, onBufferOverflow) +} diff --git a/kotlinx-coroutines-core/common/src/flow/SharingStarted.kt b/kotlinx-coroutines-core/common/src/flow/SharingStarted.kt new file mode 100644 index 0000000000..b9b73603c4 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/SharingStarted.kt @@ -0,0 +1,204 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.IgnoreJreRequirement +import kotlin.time.* + +/** + * A command emitted by [SharingStarted] implementations to control the sharing coroutine in + * the [shareIn] and [stateIn] operators. + */ +public enum class SharingCommand { + /** + * Starts sharing, launching collection of the upstream flow. + * + * Emitting this command again does not do anything. Emit [STOP] and then [START] to restart an + * upstream flow. + */ + START, + + /** + * Stops sharing, cancelling collection of the upstream flow. + */ + STOP, + + /** + * Stops sharing, cancelling collection of the upstream flow, and resets the [SharedFlow.replayCache] + * to its initial state. + * The [shareIn] operator calls [MutableSharedFlow.resetReplayCache]; + * the [stateIn] operator resets the value to its original `initialValue`. + */ + STOP_AND_RESET_REPLAY_CACHE +} + +/** + * A strategy for starting and stopping the sharing coroutine in [shareIn] and [stateIn] operators. + * + * This functional interface provides a set of built-in strategies: [Eagerly], [Lazily], [WhileSubscribed], and + * supports custom strategies by implementing this interface's [command] function. + * + * For example, it is possible to define a custom strategy that starts the upstream only when the number + * of subscribers exceeds the given `threshold` and make it an extension on [SharingStarted.Companion] so + * that it looks like a built-in strategy on the use-site: + * + * ``` + * fun SharingStarted.Companion.WhileSubscribedAtLeast(threshold: Int) = + * SharingStarted { subscriptionCount: StateFlow -> + * subscriptionCount.map { if (it >= threshold) SharingCommand.START else SharingCommand.STOP } + * } + * ``` + * + * ### Commands + * + * The `SharingStarted` strategy works by emitting [commands][SharingCommand] that control upstream flow from its + * [`command`][command] flow implementation function. Back-to-back emissions of the same command have no effect. + * Only emission of a different command has effect: + * + * - [START][SharingCommand.START] — the upstream flow is started. + * - [STOP][SharingCommand.STOP] — the upstream flow is stopped. + * - [STOP_AND_RESET_REPLAY_CACHE][SharingCommand.STOP_AND_RESET_REPLAY_CACHE] — + * the upstream flow is stopped and the [SharedFlow.replayCache] is reset to its initial state. + * The [shareIn] operator calls [MutableSharedFlow.resetReplayCache]; + * the [stateIn] operator resets the value to its original `initialValue`. + * + * Initially, the upstream flow is stopped and is in the initial state, so the emission of additional + * [STOP][SharingCommand.STOP] and [STOP_AND_RESET_REPLAY_CACHE][SharingCommand.STOP_AND_RESET_REPLAY_CACHE] commands will + * have no effect. + * + * The completion of the `command` flow normally has no effect (the upstream flow keeps running if it was running). + * The failure of the `command` flow cancels the sharing coroutine and the upstream flow. + */ +public fun interface SharingStarted { + public companion object { + /** + * Sharing is started immediately and never stops. + */ + public val Eagerly: SharingStarted = StartedEagerly() + + /** + * Sharing is started when the first subscriber appears and never stops. + */ + public val Lazily: SharingStarted = StartedLazily() + + /** + * Sharing is started when the first subscriber appears, immediately stops when the last + * subscriber disappears (by default), keeping the replay cache forever (by default). + * + * It has the following optional parameters: + * + * - [stopTimeoutMillis] — configures a delay (in milliseconds) between the disappearance of the last + * subscriber and the stopping of the sharing coroutine. It defaults to zero (stop immediately). + * - [replayExpirationMillis] — configures a delay (in milliseconds) between the stopping of + * the sharing coroutine and the resetting of the replay cache (which makes the cache empty for the [shareIn] operator + * and resets the cached value to the original `initialValue` for the [stateIn] operator). + * It defaults to `Long.MAX_VALUE` (keep replay cache forever, never reset buffer). + * Use zero value to expire the cache immediately. + * + * This function throws [IllegalArgumentException] when either [stopTimeoutMillis] or [replayExpirationMillis] + * are negative. + */ + @Suppress("FunctionName") + public fun WhileSubscribed( + stopTimeoutMillis: Long = 0, + replayExpirationMillis: Long = Long.MAX_VALUE + ): SharingStarted = + StartedWhileSubscribed(stopTimeoutMillis, replayExpirationMillis) + } + + /** + * Transforms the [subscriptionCount][MutableSharedFlow.subscriptionCount] state of the shared flow into the + * flow of [commands][SharingCommand] that control the sharing coroutine. See the [SharingStarted] interface + * documentation for details. + */ + public fun command(subscriptionCount: StateFlow): Flow +} + +/** + * Sharing is started when the first subscriber appears, immediately stops when the last + * subscriber disappears (by default), keeping the replay cache forever (by default). + * + * It has the following optional parameters: + * + * - [stopTimeout] — configures a delay between the disappearance of the last + * subscriber and the stopping of the sharing coroutine. It defaults to zero (stop immediately). + * - [replayExpiration] — configures a delay between the stopping of + * the sharing coroutine and the resetting of the replay cache (which makes the cache empty for the [shareIn] operator + * and resets the cached value to the original `initialValue` for the [stateIn] operator). + * It defaults to [Duration.INFINITE] (keep replay cache forever, never reset buffer). + * Use [Duration.ZERO] value to expire the cache immediately. + * + * This function throws [IllegalArgumentException] when either [stopTimeout] or [replayExpiration] + * are negative. + */ +@Suppress("FunctionName") +public fun SharingStarted.Companion.WhileSubscribed( + stopTimeout: Duration = Duration.ZERO, + replayExpiration: Duration = Duration.INFINITE +): SharingStarted = + StartedWhileSubscribed(stopTimeout.inWholeMilliseconds, replayExpiration.inWholeMilliseconds) + +// -------------------------------- implementation -------------------------------- + +private class StartedEagerly : SharingStarted { + override fun command(subscriptionCount: StateFlow): Flow = + flowOf(SharingCommand.START) + override fun toString(): String = "SharingStarted.Eagerly" +} + +private class StartedLazily : SharingStarted { + override fun command(subscriptionCount: StateFlow): Flow = flow { + var started = false + subscriptionCount.collect { count -> + if (count > 0 && !started) { + started = true + emit(SharingCommand.START) + } + } + } + + override fun toString(): String = "SharingStarted.Lazily" +} + +private class StartedWhileSubscribed( + private val stopTimeout: Long, + private val replayExpiration: Long +) : SharingStarted { + init { + require(stopTimeout >= 0) { "stopTimeout($stopTimeout ms) cannot be negative" } + require(replayExpiration >= 0) { "replayExpiration($replayExpiration ms) cannot be negative" } + } + + override fun command(subscriptionCount: StateFlow): Flow = subscriptionCount + .transformLatest { count -> + if (count > 0) { + emit(SharingCommand.START) + } else { + delay(stopTimeout) + if (replayExpiration > 0) { + emit(SharingCommand.STOP) + delay(replayExpiration) + } + emit(SharingCommand.STOP_AND_RESET_REPLAY_CACHE) + } + } + .dropWhile { it != SharingCommand.START } // don't emit any STOP/RESET_BUFFER to start with, only START + .distinctUntilChanged() // just in case somebody forgets it, don't leak our multiple sending of START + + @OptIn(ExperimentalStdlibApi::class) + override fun toString(): String { + val params = buildList(2) { + if (stopTimeout > 0) add("stopTimeout=${stopTimeout}ms") + if (replayExpiration < Long.MAX_VALUE) add("replayExpiration=${replayExpiration}ms") + } + return "SharingStarted.WhileSubscribed(${params.joinToString()})" + } + + // equals & hashcode to facilitate testing, not documented in public contract + override fun equals(other: Any?): Boolean = + other is StartedWhileSubscribed && + stopTimeout == other.stopTimeout && + replayExpiration == other.replayExpiration + + @IgnoreJreRequirement // desugared hashcode implementation + override fun hashCode(): Int = stopTimeout.hashCode() * 31 + replayExpiration.hashCode() +} diff --git a/kotlinx-coroutines-core/common/src/flow/StateFlow.kt b/kotlinx-coroutines-core/common/src/flow/StateFlow.kt new file mode 100644 index 0000000000..ab48dbc77b --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/StateFlow.kt @@ -0,0 +1,432 @@ +package kotlinx.coroutines.flow + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.internal.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +/** + * A [SharedFlow] that represents a read-only state with a single updatable data [value] that emits updates + * to the value to its collectors. A state flow is a _hot_ flow because its active instance exists independently + * of the presence of collectors. Its current value can be retrieved via the [value] property. + * + * **State flow never completes**. A call to [Flow.collect] on a state flow never completes normally, and + * neither does a coroutine started by the [Flow.launchIn] function. An active collector of a state flow is called a _subscriber_. + * + * A [mutable state flow][MutableStateFlow] is created using `MutableStateFlow(value)` constructor function with + * the initial value. The value of mutable state flow can be updated by setting its [value] property. + * Updates to the [value] are always [conflated][Flow.conflate]. So a slow collector skips fast updates, + * but always collects the most recently emitted value. + * + * [StateFlow] is useful as a data-model class to represent any kind of state. + * Derived values can be defined using various operators on the flows, with [combine] operator being especially + * useful to combine values from multiple state flows using arbitrary functions. + * + * For example, the following class encapsulates an integer state and increments its value on each call to `inc`: + * + * ``` + * class CounterModel { + * private val _counter = MutableStateFlow(0) // private mutable state flow + * val counter = _counter.asStateFlow() // publicly exposed as read-only state flow + * + * fun inc() { + * _counter.update { count -> count + 1 } // atomic, safe for concurrent use + * } + * } + * ``` + * + * Having two instances of the above `CounterModel` class one can define the sum of their counters like this: + * + * ``` + * val aModel = CounterModel() + * val bModel = CounterModel() + * val sumFlow: Flow = aModel.counter.combine(bModel.counter) { a, b -> a + b } + * ``` + * + * As an alternative to the above usage with the `MutableStateFlow(...)` constructor function, + * any _cold_ [Flow] can be converted to a state flow using the [stateIn] operator. + * + * ### Strong equality-based conflation + * + * Values in state flow are conflated using [Any.equals] comparison in a similar way to + * [distinctUntilChanged] operator. It is used to conflate incoming updates + * to [value][MutableStateFlow.value] in [MutableStateFlow] and to suppress emission of the values to collectors + * when new value is equal to the previously emitted one. State flow behavior with classes that violate + * the contract for [Any.equals] is unspecified. + * + * ### State flow is a shared flow + * + * State flow is a special-purpose, high-performance, and efficient implementation of [SharedFlow] for the narrow, + * but widely used case of sharing a state. See the [SharedFlow] documentation for the basic rules, + * constraints, and operators that are applicable to all shared flows. + * + * State flow always has an initial value, replays one most recent value to new subscribers, does not buffer any + * more values, but keeps the last emitted one, and does not support [resetReplayCache][MutableSharedFlow.resetReplayCache]. + * A state flow behaves identically to a shared flow when it is created + * with the following parameters and the [distinctUntilChanged] operator is applied to it: + * + * ``` + * // MutableStateFlow(initialValue) is a shared flow with the following parameters: + * val shared = MutableSharedFlow( + * replay = 1, + * onBufferOverflow = BufferOverflow.DROP_OLDEST + * ) + * shared.tryEmit(initialValue) // emit the initial value + * val state = shared.distinctUntilChanged() // get StateFlow-like behavior + * ``` + * + * Use [SharedFlow] when you need a [StateFlow] with tweaks in its behavior such as extra buffering, replaying more + * values, or omitting the initial value. + * + * ### StateFlow vs ConflatedBroadcastChannel + * + * Conceptually, state flow is similar to [ConflatedBroadcastChannel] + * and is designed to completely replace it. + * It has the following important differences: + * + * - `StateFlow` is simpler, because it does not have to implement all the [Channel] APIs, which allows + * for faster, garbage-free implementation, unlike `ConflatedBroadcastChannel` implementation that + * allocates objects on each emitted value. + * - `StateFlow` always has a value which can be safely read at any time via [value] property. + * Unlike `ConflatedBroadcastChannel`, there is no way to create a state flow without a value. + * - `StateFlow` has a clear separation into a read-only `StateFlow` interface and a [MutableStateFlow]. + * - `StateFlow` conflation is based on equality like [distinctUntilChanged] operator, + * unlike conflation in `ConflatedBroadcastChannel` that is based on reference identity. + * - `StateFlow` cannot be closed like `ConflatedBroadcastChannel` and can never represent a failure. + * All errors and completion signals should be explicitly _materialized_ if needed. + * + * `StateFlow` is designed to better cover typical use-cases of keeping track of state changes in time, taking + * more pragmatic design choices for the sake of convenience. + * + * To migrate [ConflatedBroadcastChannel] usage to [StateFlow], start by replacing usages of the `ConflatedBroadcastChannel()` + * constructor with `MutableStateFlow(initialValue)`, using `null` as an initial value if you don't have one. + * Replace [send][ConflatedBroadcastChannel.send] and [trySend][ConflatedBroadcastChannel.trySend] calls + * with updates to the state flow's [MutableStateFlow.value], and convert subscribers' code to flow operators. + * You can use the [filterNotNull] operator to mimic behavior of a `ConflatedBroadcastChannel` without initial value. + * + * ### Concurrency + * + * All methods of state flow are **thread-safe** and can be safely invoked from concurrent coroutines without + * external synchronization. + * + * ### Operator fusion + * + * Application of [flowOn][Flow.flowOn], [conflate][Flow.conflate], + * [buffer] with [CONFLATED][Channel.CONFLATED] or [RENDEZVOUS][Channel.RENDEZVOUS] capacity, + * [distinctUntilChanged][Flow.distinctUntilChanged], or [cancellable] operators to a state flow has no effect. + * + * ### Implementation notes + * + * State flow implementation is optimized for memory consumption and allocation-freedom. It uses a lock to ensure + * thread-safety, but suspending collector coroutines are resumed outside of this lock to avoid dead-locks when + * using unconfined coroutines. Adding new subscribers has `O(1)` amortized cost, but updating a [value] has `O(N)` + * cost, where `N` is the number of active subscribers. + * + * ### Not stable for inheritance + * + * **`The StateFlow` interface is not stable for inheritance in 3rd party libraries**, as new methods + * might be added to this interface in the future, but is stable for use. + * Use the `MutableStateFlow(value)` constructor function to create an implementation. + */ +@OptIn(ExperimentalSubclassOptIn::class) +@SubclassOptInRequired(ExperimentalForInheritanceCoroutinesApi::class) +public interface StateFlow : SharedFlow { + /** + * The current value of this state flow. + */ + public val value: T +} + +/** + * A mutable [StateFlow] that provides a setter for [value]. + * An instance of `MutableStateFlow` with the given initial `value` can be created using + * `MutableStateFlow(value)` constructor function. + + * See the [StateFlow] documentation for details on state flows. + * Note that all emission-related operators, such as [value]'s setter, [emit], and [tryEmit], are conflated using [Any.equals]. + * + * ### Not stable for inheritance + * + * **The `MutableStateFlow` interface is not stable for inheritance in 3rd party libraries**, as new methods + * might be added to this interface in the future, but is stable for use. + * Use the `MutableStateFlow()` constructor function to create an implementation. + */ +@OptIn(ExperimentalSubclassOptIn::class) +@SubclassOptInRequired(ExperimentalForInheritanceCoroutinesApi::class) +public interface MutableStateFlow : StateFlow, MutableSharedFlow { + /** + * The current value of this state flow. + * + * Setting a value that is [equal][Any.equals] to the previous one does nothing. + * + * This property is **thread-safe** and can be safely updated from concurrent coroutines without + * external synchronization. + */ + public override var value: T + + /** + * Atomically compares the current [value] with [expect] and sets it to [update] if it is equal to [expect]. + * The result is `true` if the [value] was set to [update] and `false` otherwise. + * + * This function use a regular comparison using [Any.equals]. If both [expect] and [update] are equal to the + * current [value], this function returns `true`, but it does not actually change the reference that is + * stored in the [value]. + * + * This method is **thread-safe** and can be safely invoked from concurrent coroutines without + * external synchronization. + */ + public fun compareAndSet(expect: T, update: T): Boolean +} + +/** + * Creates a [MutableStateFlow] with the given initial [value]. + */ +@Suppress("FunctionName") +public fun MutableStateFlow(value: T): MutableStateFlow = StateFlowImpl(value ?: NULL) + +// ------------------------------------ Update methods ------------------------------------ + +/** + * Updates the [MutableStateFlow.value] atomically using the specified [function] of its value, and returns the new + * value. + * + * [function] may be evaluated multiple times, if [value] is being concurrently updated. + */ +public inline fun MutableStateFlow.updateAndGet(function: (T) -> T): T { + while (true) { + val prevValue = value + val nextValue = function(prevValue) + if (compareAndSet(prevValue, nextValue)) { + return nextValue + } + } +} + +/** + * Updates the [MutableStateFlow.value] atomically using the specified [function] of its value, and returns its + * prior value. + * + * [function] may be evaluated multiple times, if [value] is being concurrently updated. + */ +public inline fun MutableStateFlow.getAndUpdate(function: (T) -> T): T { + while (true) { + val prevValue = value + val nextValue = function(prevValue) + if (compareAndSet(prevValue, nextValue)) { + return prevValue + } + } +} + + +/** + * Updates the [MutableStateFlow.value] atomically using the specified [function] of its value. + * + * [function] may be evaluated multiple times, if [value] is being concurrently updated. + */ +public inline fun MutableStateFlow.update(function: (T) -> T) { + while (true) { + val prevValue = value + val nextValue = function(prevValue) + if (compareAndSet(prevValue, nextValue)) { + return + } + } +} + +// ------------------------------------ Implementation ------------------------------------ + +private val NONE = Symbol("NONE") + +private val PENDING = Symbol("PENDING") + +// StateFlow slots are allocated for its collectors +private class StateFlowSlot : AbstractSharedFlowSlot>() { + /** + * Each slot can have one of the following states: + * + * - `null` -- it is not used right now. Can [allocateLocked] to new collector. + * - `NONE` -- used by a collector, but neither suspended nor has pending value. + * - `PENDING` -- pending to process new value. + * - `CancellableContinuationImpl` -- suspended waiting for new value. + * + * It is important that default `null` value is used, because there can be a race between allocation + * of a new slot and trying to do [makePending] on this slot. + * + * === + * This should be `atomic(null)` instead of the atomic reference, but because of #3820 + * it is used as a **temporary** solution starting from 1.8.1 version. + * Depending on the fix rollout on Android, it will be removed in 1.9.0 or 2.0.0. + * See https://issuetracker.google.com/issues/325123736 + */ + private val _state = WorkaroundAtomicReference(null) + + override fun allocateLocked(flow: StateFlowImpl<*>): Boolean { + // No need for atomic check & update here, since allocated happens under StateFlow lock + if (_state.value != null) return false // not free + _state.value = NONE // allocated + return true + } + + override fun freeLocked(flow: StateFlowImpl<*>): Array?> { + _state.value = null // free now + return EMPTY_RESUMES // nothing more to do + } + + @Suppress("UNCHECKED_CAST") + fun makePending() { + _state.loop { state -> + when { + state == null -> return // this slot is free - skip it + state === PENDING -> return // already pending, nothing to do + state === NONE -> { // mark as pending + if (_state.compareAndSet(state, PENDING)) return + } + else -> { // must be a suspend continuation state + // we must still use CAS here since continuation may get cancelled and free the slot at any time + if (_state.compareAndSet(state, NONE)) { + (state as CancellableContinuationImpl).resume(Unit) + return + } + } + } + } + } + + fun takePending(): Boolean = _state.getAndSet(NONE)!!.let { state -> + assert { state !is CancellableContinuationImpl<*> } + return state === PENDING + } + + suspend fun awaitPending(): Unit = suspendCancellableCoroutine sc@ { cont -> + assert { _state.value !is CancellableContinuationImpl<*> } // can be NONE or PENDING + if (_state.compareAndSet(NONE, cont)) return@sc // installed continuation, waiting for pending + // CAS failed -- the only possible reason is that it is already in pending state now + assert { _state.value === PENDING } + cont.resume(Unit) + } +} + +@OptIn(ExperimentalForInheritanceCoroutinesApi::class) +private class StateFlowImpl( + initialState: Any // T | NULL +) : AbstractSharedFlow(), MutableStateFlow, CancellableFlow, FusibleFlow { + private val _state = atomic(initialState) // T | NULL + private var sequence = 0 // serializes updates, value update is in process when sequence is odd + + public override var value: T + get() = NULL.unbox(_state.value) + set(value) { updateState(null, value ?: NULL) } + + override fun compareAndSet(expect: T, update: T): Boolean = + updateState(expect ?: NULL, update ?: NULL) + + private fun updateState(expectedState: Any?, newState: Any): Boolean { + var curSequence: Int + var curSlots: Array? // benign race, we will not use it + synchronized(this) { + val oldState = _state.value + if (expectedState != null && oldState != expectedState) return false // CAS support + if (oldState == newState) return true // Don't do anything if value is not changing, but CAS -> true + _state.value = newState + curSequence = sequence + if (curSequence and 1 == 0) { // even sequence means quiescent state flow (no ongoing update) + curSequence++ // make it odd + sequence = curSequence + } else { + // update is already in process, notify it, and return + sequence = curSequence + 2 // change sequence to notify, keep it odd + return true // updated + } + curSlots = slots // read current reference to collectors under lock + } + /* + Fire value updates outside of the lock to avoid deadlocks with unconfined coroutines. + Loop until we're done firing all the changes. This is a sort of simple flat combining that + ensures sequential firing of concurrent updates and avoids the storm of collector resumes + when updates happen concurrently from many threads. + */ + while (true) { + // Benign race on element read from array + curSlots?.forEach { + it?.makePending() + } + // check if the value was updated again while we were updating the old one + synchronized(this) { + if (sequence == curSequence) { // nothing changed, we are done + sequence = curSequence + 1 // make sequence even again + return true // done, updated + } + // reread everything for the next loop under the lock + curSequence = sequence + curSlots = slots + } + } + } + + override val replayCache: List + get() = listOf(value) + + override fun tryEmit(value: T): Boolean { + this.value = value + return true + } + + override suspend fun emit(value: T) { + this.value = value + } + + @Suppress("UNCHECKED_CAST") + override fun resetReplayCache() { + throw UnsupportedOperationException("MutableStateFlow.resetReplayCache is not supported") + } + + override suspend fun collect(collector: FlowCollector): Nothing { + val slot = allocateSlot() + try { + if (collector is SubscribedFlowCollector) collector.onSubscription() + val collectorJob = currentCoroutineContext()[Job] + var oldState: Any? = null // previously emitted T!! | NULL (null -- nothing emitted yet) + // The loop is arranged so that it starts delivering current value without waiting first + while (true) { + // Here the coroutine could have waited for a while to be dispatched, + // so we use the most recent state here to ensure the best possible conflation of stale values + val newState = _state.value + // always check for cancellation + collectorJob?.ensureActive() + // Conflate value emissions using equality + if (oldState == null || oldState != newState) { + collector.emit(NULL.unbox(newState)) + oldState = newState + } + // Note: if awaitPending is cancelled, then it bails out of this loop and calls freeSlot + if (!slot.takePending()) { // try fast-path without suspending first + slot.awaitPending() // only suspend for new values when needed + } + } + } finally { + freeSlot(slot) + } + } + + override fun createSlot() = StateFlowSlot() + override fun createSlotArray(size: Int): Array = arrayOfNulls(size) + + override fun fuse(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow) = + fuseStateFlow(context, capacity, onBufferOverflow) +} + +internal fun StateFlow.fuseStateFlow( + context: CoroutineContext, + capacity: Int, + onBufferOverflow: BufferOverflow +): Flow { + // state flow is always conflated so additional conflation does not have any effect + assert { capacity != Channel.CONFLATED } // should be desugared by callers + if ((capacity in 0..1 || capacity == Channel.BUFFERED) && onBufferOverflow == BufferOverflow.DROP_OLDEST) { + return this + } + return fuseSharedFlow(context, capacity, onBufferOverflow) +} diff --git a/kotlinx-coroutines-core/common/src/flow/internal/AbstractSharedFlow.kt b/kotlinx-coroutines-core/common/src/flow/internal/AbstractSharedFlow.kt new file mode 100644 index 0000000000..6831ad7d72 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/AbstractSharedFlow.kt @@ -0,0 +1,129 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* + +@JvmField +internal val EMPTY_RESUMES = arrayOfNulls?>(0) + +internal abstract class AbstractSharedFlowSlot { + abstract fun allocateLocked(flow: F): Boolean + abstract fun freeLocked(flow: F): Array?> // returns continuations to resume after lock +} + +internal abstract class AbstractSharedFlow> : SynchronizedObject() { + protected var slots: Array? = null // allocated when needed + private set + protected var nCollectors = 0 // number of allocated (!free) slots + private set + private var nextIndex = 0 // oracle for the next free slot index + private var _subscriptionCount: SubscriptionCountStateFlow? = null // init on first need + + val subscriptionCount: StateFlow + get() = synchronized(this) { + // allocate under lock in sync with nCollectors variable + _subscriptionCount ?: SubscriptionCountStateFlow(nCollectors).also { + _subscriptionCount = it + } + } + + protected abstract fun createSlot(): S + + protected abstract fun createSlotArray(size: Int): Array + + @Suppress("UNCHECKED_CAST") + protected fun allocateSlot(): S { + // Actually create slot under lock + val subscriptionCount: SubscriptionCountStateFlow? + val slot = synchronized(this) { + val slots = when (val curSlots = slots) { + null -> createSlotArray(2).also { slots = it } + else -> if (nCollectors >= curSlots.size) { + curSlots.copyOf(2 * curSlots.size).also { slots = it } + } else { + curSlots + } + } + var index = nextIndex + var slot: S + while (true) { + slot = slots[index] ?: createSlot().also { slots[index] = it } + index++ + if (index >= slots.size) index = 0 + if ((slot as AbstractSharedFlowSlot).allocateLocked(this)) break // break when found and allocated free slot + } + nextIndex = index + nCollectors++ + subscriptionCount = _subscriptionCount // retrieve under lock if initialized + slot + } + // increments subscription count + subscriptionCount?.increment(1) + return slot + } + + @Suppress("UNCHECKED_CAST") + protected fun freeSlot(slot: S) { + // Release slot under lock + val subscriptionCount: SubscriptionCountStateFlow? + val resumes = synchronized(this) { + nCollectors-- + subscriptionCount = _subscriptionCount // retrieve under lock if initialized + // Reset next index oracle if we have no more active collectors for more predictable behavior next time + if (nCollectors == 0) nextIndex = 0 + (slot as AbstractSharedFlowSlot).freeLocked(this) + } + /* + * Resume suspended coroutines. + * This can happen when the subscriber that was freed was a slow one and was holding up buffer. + * When this subscriber was freed, previously queued emitted can now wake up and are resumed here. + */ + for (cont in resumes) cont?.resume(Unit) + // decrement subscription count + subscriptionCount?.increment(-1) + } + + protected inline fun forEachSlotLocked(block: (S) -> Unit) { + if (nCollectors == 0) return + slots?.forEach { slot -> + if (slot != null) block(slot) + } + } +} + +/** + * [StateFlow] that represents the number of subscriptions. + * + * It is exposed as a regular [StateFlow] in our public API, but it is implemented as [SharedFlow] undercover to + * avoid conflations of consecutive updates because the subscription count is very sensitive to it. + * + * The importance of non-conflating can be demonstrated with the following example: + * ``` + * val shared = flowOf(239).stateIn(this, SharingStarted.Lazily, 42) // stateIn for the sake of the initial value + * println(shared.first()) + * yield() + * println(shared.first()) + * ``` + * If the flow is shared within the same dispatcher (e.g. Main) or with a slow/throttled one, + * the `SharingStarted.Lazily` will never be able to start the source: `first` sees the initial value and immediately + * unsubscribes, leaving the asynchronous `SharingStarted` with conflated zero. + * + * To avoid that (especially in a more complex scenarios), we do not conflate subscription updates. + */ +@OptIn(ExperimentalForInheritanceCoroutinesApi::class) +private class SubscriptionCountStateFlow(initialValue: Int) : StateFlow, + SharedFlowImpl(1, Int.MAX_VALUE, BufferOverflow.DROP_OLDEST) +{ + init { tryEmit(initialValue) } + + override val value: Int + get() = synchronized(this) { lastReplayedLocked } + + fun increment(delta: Int) = synchronized(this) { + tryEmit(lastReplayedLocked + delta) + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/internal/ChannelFlow.kt b/kotlinx-coroutines-core/common/src/flow/internal/ChannelFlow.kt new file mode 100644 index 0000000000..e00c1fdd61 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/ChannelFlow.kt @@ -0,0 +1,240 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlin.jvm.* + +internal fun Flow.asChannelFlow(): ChannelFlow = + this as? ChannelFlow ?: ChannelFlowOperatorImpl(this) + +/** + * Operators that can fuse with **downstream** [buffer] and [flowOn] operators implement this interface. + * + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public interface FusibleFlow : Flow { + /** + * This function is called by [flowOn] (with context) and [buffer] (with capacity) operators + * that are applied to this flow. Should not be used with [capacity] of [Channel.CONFLATED] + * (it shall be desugared to `capacity = 0, onBufferOverflow = DROP_OLDEST`). + */ + public fun fuse( + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = Channel.OPTIONAL_CHANNEL, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND + ): Flow +} + +/** + * Operators that use channels as their "output" extend this `ChannelFlow` and are always fused with each other. + * This class servers as a skeleton implementation of [FusibleFlow] and provides other cross-cutting + * methods like ability to [produceIn] the corresponding flow, thus making it + * possible to directly use the backing channel if it exists (hence the `ChannelFlow` name). + * + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public abstract class ChannelFlow( + // upstream context + @JvmField public val context: CoroutineContext, + // buffer capacity between upstream and downstream context + @JvmField public val capacity: Int, + // buffer overflow strategy + @JvmField public val onBufferOverflow: BufferOverflow +) : FusibleFlow { + init { + assert { capacity != Channel.CONFLATED } // CONFLATED must be desugared to 0, DROP_OLDEST by callers + } + + // shared code to create a suspend lambda from collectTo function in one place + internal val collectToFun: suspend (ProducerScope) -> Unit + get() = { collectTo(it) } + + internal val produceCapacity: Int + get() = if (capacity == Channel.OPTIONAL_CHANNEL) Channel.BUFFERED else capacity + + /** + * When this [ChannelFlow] implementation can work without a channel (supports [Channel.OPTIONAL_CHANNEL]), + * then it should return a non-null value from this function, so that a caller can use it without the effect of + * additional [flowOn] and [buffer] operators, by incorporating its + * [context], [capacity], and [onBufferOverflow] into its own implementation. + */ + public open fun dropChannelOperators(): Flow? = null + + public override fun fuse(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): Flow { + assert { capacity != Channel.CONFLATED } // CONFLATED must be desugared to (0, DROP_OLDEST) by callers + // note: previous upstream context (specified before) takes precedence + val newContext = context + this.context + val newCapacity: Int + val newOverflow: BufferOverflow + if (onBufferOverflow != BufferOverflow.SUSPEND) { + // this additional buffer never suspends => overwrite preceding buffering configuration + newCapacity = capacity + newOverflow = onBufferOverflow + } else { + // combine capacities, keep previous overflow strategy + newCapacity = when { + this.capacity == Channel.OPTIONAL_CHANNEL -> capacity + capacity == Channel.OPTIONAL_CHANNEL -> this.capacity + this.capacity == Channel.BUFFERED -> capacity + capacity == Channel.BUFFERED -> this.capacity + else -> { + // sanity checks + assert { this.capacity >= 0 } + assert { capacity >= 0 } + // combine capacities clamping to UNLIMITED on overflow + val sum = this.capacity + capacity + if (sum >= 0) sum else Channel.UNLIMITED // unlimited on int overflow + } + } + newOverflow = this.onBufferOverflow + } + if (newContext == this.context && newCapacity == this.capacity && newOverflow == this.onBufferOverflow) + return this + return create(newContext, newCapacity, newOverflow) + } + + protected abstract fun create(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): ChannelFlow + + protected abstract suspend fun collectTo(scope: ProducerScope) + + /** + * Here we use ATOMIC start for a reason (#1825). + * NB: [produceImpl] is used for [flowOn]. + * For non-atomic start it is possible to observe the situation, + * where the pipeline after the [flowOn] call successfully executes (mostly, its `onCompletion`) + * handlers, while the pipeline before does not, because it was cancelled during its dispatch. + * Thus `onCompletion` and `finally` blocks won't be executed and it may lead to a different kinds of memory leaks. + */ + public open fun produceImpl(scope: CoroutineScope): ReceiveChannel = + scope.produce(context, produceCapacity, onBufferOverflow, start = CoroutineStart.ATOMIC, block = collectToFun) + + override suspend fun collect(collector: FlowCollector): Unit = + coroutineScope { + collector.emitAll(produceImpl(this)) + } + + protected open fun additionalToStringProps(): String? = null + + // debug toString + override fun toString(): String { + val props = ArrayList(4) + additionalToStringProps()?.let { props.add(it) } + if (context !== EmptyCoroutineContext) props.add("context=$context") + if (capacity != Channel.OPTIONAL_CHANNEL) props.add("capacity=$capacity") + if (onBufferOverflow != BufferOverflow.SUSPEND) props.add("onBufferOverflow=$onBufferOverflow") + return "$classSimpleName[${props.joinToString(", ")}]" + } +} + +// ChannelFlow implementation that operates on another flow before it +internal abstract class ChannelFlowOperator( + @JvmField protected val flow: Flow, + context: CoroutineContext, + capacity: Int, + onBufferOverflow: BufferOverflow +) : ChannelFlow(context, capacity, onBufferOverflow) { + protected abstract suspend fun flowCollect(collector: FlowCollector) + + // Changes collecting context upstream to the specified newContext, while collecting in the original context + private suspend fun collectWithContextUndispatched(collector: FlowCollector, newContext: CoroutineContext) { + val originalContextCollector = collector.withUndispatchedContextCollector(coroutineContext) + // invoke flowCollect(originalContextCollector) in the newContext + return withContextUndispatched(newContext, block = { flowCollect(it) }, value = originalContextCollector) + } + + // Slow path when output channel is required + protected override suspend fun collectTo(scope: ProducerScope) = + flowCollect(SendingCollector(scope)) + + // Optimizations for fast-path when channel creation is optional + override suspend fun collect(collector: FlowCollector) { + // Fast-path: When channel creation is optional (flowOn/flowWith operators without buffer) + if (capacity == Channel.OPTIONAL_CHANNEL) { + val collectContext = coroutineContext + val newContext = collectContext.newCoroutineContext(context) // compute resulting collect context + // #1: If the resulting context happens to be the same as it was -- fallback to plain collect + if (newContext == collectContext) + return flowCollect(collector) + // #2: If we don't need to change the dispatcher we can go without channels + if (newContext[ContinuationInterceptor] == collectContext[ContinuationInterceptor]) + return collectWithContextUndispatched(collector, newContext) + } + // Slow-path: create the actual channel + super.collect(collector) + } + + // debug toString + override fun toString(): String = "$flow -> ${super.toString()}" +} + +/** + * Simple channel flow operator: [flowOn], [buffer], or their fused combination. + */ +internal class ChannelFlowOperatorImpl( + flow: Flow, + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = Channel.OPTIONAL_CHANNEL, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND +) : ChannelFlowOperator(flow, context, capacity, onBufferOverflow) { + override fun create(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): ChannelFlow = + ChannelFlowOperatorImpl(flow, context, capacity, onBufferOverflow) + + override fun dropChannelOperators(): Flow = flow + + override suspend fun flowCollect(collector: FlowCollector) = + flow.collect(collector) +} + +// Now if the underlying collector was accepting concurrent emits, then this one is too +// todo: we might need to generalize this pattern for "thread-safe" operators that can fuse with channels +private fun FlowCollector.withUndispatchedContextCollector(emitContext: CoroutineContext): FlowCollector = when (this) { + // SendingCollector & NopCollector do not care about the context at all and can be used as is + is SendingCollector, is NopCollector -> this + // Otherwise just wrap into UndispatchedContextCollector interface implementation + else -> UndispatchedContextCollector(this, emitContext) +} + +private class UndispatchedContextCollector( + downstream: FlowCollector, + private val emitContext: CoroutineContext +) : FlowCollector { + private val countOrElement = threadContextElements(emitContext) // precompute for fast withContextUndispatched + private val emitRef: suspend (T) -> Unit = { downstream.emit(it) } // allocate suspend function ref once on creation + + override suspend fun emit(value: T): Unit = + withContextUndispatched(emitContext, value, countOrElement, emitRef) +} + +// Efficiently computes block(value) in the newContext +internal suspend fun withContextUndispatched( + newContext: CoroutineContext, + value: V, + countOrElement: Any = threadContextElements(newContext), // can be precomputed for speed + block: suspend (V) -> T +): T = + suspendCoroutineUninterceptedOrReturn { uCont -> + withCoroutineContext(newContext, countOrElement) { + block.startCoroutineUninterceptedOrReturn(value, StackFrameContinuation(uCont, newContext)) + } + } + +// Continuation that links the caller with uCont with walkable CoroutineStackFrame +private class StackFrameContinuation( + private val uCont: Continuation, override val context: CoroutineContext +) : Continuation, CoroutineStackFrame { + + override val callerFrame: CoroutineStackFrame? + get() = uCont as? CoroutineStackFrame + + override fun resumeWith(result: Result) { + uCont.resumeWith(result) + } + + override fun getStackTraceElement(): StackTraceElement? = null +} diff --git a/kotlinx-coroutines-core/common/src/flow/internal/Combine.kt b/kotlinx-coroutines-core/common/src/flow/internal/Combine.kt new file mode 100644 index 0000000000..60fcc8e04c --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/Combine.kt @@ -0,0 +1,139 @@ +@file:Suppress("UNCHECKED_CAST") // KT-32203 + +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.internal.* +private typealias Update = IndexedValue + +@PublishedApi +internal suspend fun FlowCollector.combineInternal( + flows: Array>, + arrayFactory: () -> Array?, // Array factory is required to workaround array typing on JVM + transform: suspend FlowCollector.(Array) -> Unit +): Unit = flowScope { // flow scope so any cancellation within the source flow will cancel the whole scope + val size = flows.size + if (size == 0) return@flowScope // bail-out for empty input + val latestValues = arrayOfNulls(size) + latestValues.fill(UNINITIALIZED) // Smaller bytecode & faster than Array(size) { UNINITIALIZED } + val resultChannel = Channel(size) + val nonClosed = LocalAtomicInt(size) + var remainingAbsentValues = size + for (i in 0 until size) { + // Coroutine per flow that keeps track of its value and sends result to downstream + launch { + try { + flows[i].collect { value -> + resultChannel.send(Update(i, value)) + yield() // Emulate fairness, giving each flow chance to emit + } + } finally { + // Close the channel when there is no more flows + if (nonClosed.decrementAndGet() == 0) { + resultChannel.close() + } + } + } + } + + /* + * Batch-receive optimization: read updates in batches, but bail-out + * as soon as we encountered two values from the same source + */ + val lastReceivedEpoch = ByteArray(size) + var currentEpoch: Byte = 0 + while (true) { + ++currentEpoch + // Start batch + // The very first receive in epoch should be suspending + var element = resultChannel.receiveCatching().getOrNull() ?: break // Channel is closed, nothing to do here + while (true) { + val index = element.index + // Update values + val previous = latestValues[index] + latestValues[index] = element.value + if (previous === UNINITIALIZED) --remainingAbsentValues + // Check epoch + // Received the second value from the same flow in the same epoch -- bail out + if (lastReceivedEpoch[index] == currentEpoch) break + lastReceivedEpoch[index] = currentEpoch + element = resultChannel.tryReceive().getOrNull() ?: break + } + + // Process batch result if there is enough data + if (remainingAbsentValues == 0) { + /* + * If arrayFactory returns null, then we can avoid array copy because + * it's our own safe transformer that immediately deconstructs the array + */ + val results = arrayFactory() + if (results == null) { + transform(latestValues as Array) + } else { + (latestValues as Array).copyInto(results) + transform(results as Array) + } + } + } +} + +internal fun zipImpl(flow: Flow, flow2: Flow, transform: suspend (T1, T2) -> R): Flow = + unsafeFlow { + coroutineScope { + val second = produce { + flow2.collect { value -> + return@collect channel.send(value ?: NULL) + } + } + + /* + * This approach only works with rendezvous channel and is required to enforce correctness + * in the following scenario: + * ``` + * val f1 = flow { emit(1); delay(Long.MAX_VALUE) } + * val f2 = flowOf(1) + * f1.zip(f2) { ... } + * ``` + * + * Invariant: this clause is invoked only when all elements from the channel were processed (=> rendezvous restriction). + */ + val collectJob = Job() + (second as SendChannel<*>).invokeOnClose { + // Optimization to avoid AFE allocation when the other flow is done + if (collectJob.isActive) collectJob.cancel(AbortFlowException(collectJob)) + } + + try { + /* + * Non-trivial undispatched (because we are in the right context and there is no structured concurrency) + * hierarchy: + * -Outer coroutineScope that owns the whole zip process + * - First flow is collected by the child of coroutineScope, collectJob. + * So it can be safely cancelled as soon as the second flow is done + * - **But** the downstream MUST NOT be cancelled when the second flow is done, + * so we emit to downstream from coroutineScope job. + * Typically, such hierarchy requires coroutine for collector that communicates + * with coroutines scope via a channel, but it's way too expensive, so + * we are using this trick instead. + */ + val scopeContext = coroutineContext + val cnt = threadContextElements(scopeContext) + withContextUndispatched(coroutineContext + collectJob, Unit) { + flow.collect { value -> + withContextUndispatched(scopeContext, Unit, cnt) { + val otherValue = second.receiveCatching().getOrElse { + throw it ?: AbortFlowException(collectJob) + } + emit(transform(value, NULL.unbox(otherValue))) + } + } + } + } catch (e: AbortFlowException) { + e.checkOwnership(owner = collectJob) + } finally { + second.cancel() + } + } + } diff --git a/kotlinx-coroutines-core/common/src/flow/internal/FlowCoroutine.kt b/kotlinx-coroutines-core/common/src/flow/internal/FlowCoroutine.kt new file mode 100644 index 0000000000..9510ee8d23 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/FlowCoroutine.kt @@ -0,0 +1,58 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.intrinsics.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlinx.coroutines.flow.internal.unsafeFlow as flow + +/** + * Creates a [CoroutineScope] and calls the specified suspend block with this scope. + * This builder is similar to [coroutineScope] with the only exception that it *ties* lifecycle of children + * and itself regarding the cancellation, thus being cancelled when one of the children becomes cancelled. + * + * For example: + * ``` + * flowScope { + * launch { + * throw CancellationException() + * } + * } // <- CE will be rethrown here + * ``` + */ +internal suspend fun flowScope(@BuilderInference block: suspend CoroutineScope.() -> R): R = + suspendCoroutineUninterceptedOrReturn { uCont -> + val coroutine = FlowCoroutine(uCont.context, uCont) + coroutine.startUndispatchedOrReturn(coroutine, block) + } + +/** + * Creates a flow that also provides a [CoroutineScope] for each collector + * Shorthand for: + * ``` + * flow { + * flowScope { + * ... + * } + * } + * ``` + * with additional constraint on cancellation. + * To cancel child without cancelling itself, `cancel(ChildCancelledException())` should be used. + */ +internal fun scopedFlow(@BuilderInference block: suspend CoroutineScope.(FlowCollector) -> Unit): Flow = + flow { + flowScope { block(this@flow) } + } + +private class FlowCoroutine( + context: CoroutineContext, + uCont: Continuation +) : ScopeCoroutine(context, uCont) { + override fun childCancelled(cause: Throwable): Boolean { + if (cause is ChildCancelledException) return true + return cancelImpl(cause) + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/internal/FlowExceptions.common.kt b/kotlinx-coroutines-core/common/src/flow/internal/FlowExceptions.common.kt new file mode 100644 index 0000000000..628296c3e2 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/FlowExceptions.common.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +/** + * This exception is thrown when an operator needs no more elements from the flow. + * The operator should never allow this exception to be thrown past its own boundary. + * This exception can be safely ignored by non-terminal flow operator if and only if it was caught by its owner + * (see usages of [checkOwnership]). + * Therefore, the [owner] parameter must be unique for every invocation of every operator. + */ +internal expect class AbortFlowException(owner: Any) : CancellationException { + val owner: Any +} + +internal fun AbortFlowException.checkOwnership(owner: Any) { + if (this.owner !== owner) throw this +} + +/** + * Exception used to cancel child of [scopedFlow] without cancelling the whole scope. + */ +internal expect class ChildCancelledException() : CancellationException + +@Suppress("NOTHING_TO_INLINE") +@PublishedApi +internal inline fun checkIndexOverflow(index: Int): Int { + if (index < 0) { + throw ArithmeticException("Index overflow has happened") + } + return index +} diff --git a/kotlinx-coroutines-core/common/src/flow/internal/Merge.kt b/kotlinx-coroutines-core/common/src/flow/internal/Merge.kt new file mode 100644 index 0000000000..986a41bd06 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/Merge.kt @@ -0,0 +1,95 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.sync.* +import kotlin.coroutines.* + +internal class ChannelFlowTransformLatest( + private val transform: suspend FlowCollector.(value: T) -> Unit, + flow: Flow, + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = Channel.BUFFERED, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND +) : ChannelFlowOperator(flow, context, capacity, onBufferOverflow) { + override fun create(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): ChannelFlow = + ChannelFlowTransformLatest(transform, flow, context, capacity, onBufferOverflow) + + override suspend fun flowCollect(collector: FlowCollector) { + assert { collector is SendingCollector } // So cancellation behaviour is not leaking into the downstream + coroutineScope { + var previousFlow: Job? = null + flow.collect { value -> + previousFlow?.apply { + cancel(ChildCancelledException()) + join() + } + // Do not pay for dispatch here, it's never necessary + previousFlow = launch(start = CoroutineStart.UNDISPATCHED) { + collector.transform(value) + } + } + } + } +} + +internal class ChannelFlowMerge( + private val flow: Flow>, + private val concurrency: Int, + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = Channel.BUFFERED, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND +) : ChannelFlow(context, capacity, onBufferOverflow) { + override fun create(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): ChannelFlow = + ChannelFlowMerge(flow, concurrency, context, capacity, onBufferOverflow) + + override fun produceImpl(scope: CoroutineScope): ReceiveChannel { + return scope.produce(context, capacity, block = collectToFun) + } + + override suspend fun collectTo(scope: ProducerScope) { + val semaphore = Semaphore(concurrency) + val collector = SendingCollector(scope) + val job: Job? = coroutineContext[Job] + flow.collect { inner -> + /* + * We launch a coroutine on each emitted element and the only potential + * suspension point in this collector is `semaphore.acquire` that rarely suspends, + * so we manually check for cancellation to propagate it to the upstream in time. + */ + job?.ensureActive() + semaphore.acquire() + scope.launch { + try { + inner.collect(collector) + } finally { + semaphore.release() // Release concurrency permit + } + } + } + } + + override fun additionalToStringProps(): String = "concurrency=$concurrency" +} + +internal class ChannelLimitedFlowMerge( + private val flows: Iterable>, + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = Channel.BUFFERED, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND +) : ChannelFlow(context, capacity, onBufferOverflow) { + override fun create(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): ChannelFlow = + ChannelLimitedFlowMerge(flows, context, capacity, onBufferOverflow) + + override fun produceImpl(scope: CoroutineScope): ReceiveChannel { + return scope.produce(context, capacity, block = collectToFun) + } + + override suspend fun collectTo(scope: ProducerScope) { + val collector = SendingCollector(scope) + flows.forEach { flow -> + scope.launch { flow.collect(collector) } + } + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/internal/NopCollector.kt b/kotlinx-coroutines-core/common/src/flow/internal/NopCollector.kt new file mode 100644 index 0000000000..c4e68abe6d --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/NopCollector.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.flow.* + +internal object NopCollector : FlowCollector { + override suspend fun emit(value: Any?) { + // does nothing + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/internal/NullSurrogate.kt b/kotlinx-coroutines-core/common/src/flow/internal/NullSurrogate.kt new file mode 100644 index 0000000000..230f466e2e --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/NullSurrogate.kt @@ -0,0 +1,26 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.internal.* +import kotlin.jvm.* + +/** + * This value is used a a surrogate `null` value when needed. + * It should never leak to the outside world. + * Its usage typically are paired with [Symbol.unbox] usages. + */ +@JvmField +internal val NULL = Symbol("NULL") + +/** + * Symbol to indicate that the value is not yet initialized. + * It should never leak to the outside world. + */ +@JvmField +internal val UNINITIALIZED = Symbol("UNINITIALIZED") + +/* + * Symbol used to indicate that the flow is complete. + * It should never leak to the outside world. + */ +@JvmField +internal val DONE = Symbol("DONE") diff --git a/kotlinx-coroutines-core/common/src/flow/internal/SafeCollector.common.kt b/kotlinx-coroutines-core/common/src/flow/internal/SafeCollector.common.kt new file mode 100644 index 0000000000..4e82ca65a7 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/SafeCollector.common.kt @@ -0,0 +1,110 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.internal.ScopeCoroutine +import kotlin.coroutines.* +import kotlin.jvm.* + +// Collector that ensures exception transparency and context preservation on a best-effort basis. +// See an explanation in SafeCollector JVM actualization. +internal expect class SafeCollector( + collector: FlowCollector, + collectContext: CoroutineContext +) : FlowCollector { + internal val collector: FlowCollector + internal val collectContext: CoroutineContext + internal val collectContextSize: Int + public fun releaseIntercepted() + public override suspend fun emit(value: T) +} + +@JvmName("checkContext") // For prettier stack traces +internal fun SafeCollector<*>.checkContext(currentContext: CoroutineContext) { + val result = currentContext.fold(0) fold@{ count, element -> + val key = element.key + val collectElement = collectContext[key] + if (key !== Job) { + return@fold if (element !== collectElement) Int.MIN_VALUE + else count + 1 + } + + val collectJob = collectElement as Job? + val emissionParentJob = (element as Job).transitiveCoroutineParent(collectJob) + /* + * Code like + * ``` + * coroutineScope { + * launch { + * emit(1) + * } + * + * launch { + * emit(2) + * } + * } + * ``` + * is prohibited because 'emit' is not thread-safe by default. Use 'channelFlow' instead if you need concurrent emission + * or want to switch context dynamically (e.g. with `withContext`). + * + * Note that collecting from another coroutine is allowed, e.g.: + * ``` + * coroutineScope { + * val channel = produce { + * collect { value -> + * send(value) + * } + * } + * channel.consumeEach { value -> + * emit(value) + * } + * } + * ``` + * is a completely valid. + */ + if (emissionParentJob !== collectJob) { + error( + "Flow invariant is violated:\n" + + "\t\tEmission from another coroutine is detected.\n" + + "\t\tChild of $emissionParentJob, expected child of $collectJob.\n" + + "\t\tFlowCollector is not thread-safe and concurrent emissions are prohibited.\n" + + "\t\tTo mitigate this restriction please use 'channelFlow' builder instead of 'flow'" + ) + } + + /* + * If collect job is null (-> EmptyCoroutineContext, probably run from `suspend fun main`), then invariant is maintained + * (common transitive parent is "null"), but count check will fail, so just do not count job context element when + * flow is collected from EmptyCoroutineContext + */ + if (collectJob == null) count else count + 1 + } + if (result != collectContextSize) { + error( + "Flow invariant is violated:\n" + + "\t\tFlow was collected in $collectContext,\n" + + "\t\tbut emission happened in $currentContext.\n" + + "\t\tPlease refer to 'flow' documentation or use 'flowOn' instead" + ) + } +} + +internal tailrec fun Job?.transitiveCoroutineParent(collectJob: Job?): Job? { + if (this === null) return null + if (this === collectJob) return this + if (this !is ScopeCoroutine<*>) return this + return parent.transitiveCoroutineParent(collectJob) +} + +/** + * An analogue of the [flow] builder that does not check the context of execution of the resulting flow. + * Used in our own operators where we trust the context of invocations. + */ +@PublishedApi +internal inline fun unsafeFlow(@BuilderInference crossinline block: suspend FlowCollector.() -> Unit): Flow { + return object : Flow { + override suspend fun collect(collector: FlowCollector) { + collector.block() + } + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/internal/SendingCollector.kt b/kotlinx-coroutines-core/common/src/flow/internal/SendingCollector.kt new file mode 100644 index 0000000000..a6a1557151 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/internal/SendingCollector.kt @@ -0,0 +1,16 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* + +/** + * Collection that sends to channel + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public class SendingCollector( + private val channel: SendChannel +) : FlowCollector { + override suspend fun emit(value: T): Unit = channel.send(value) +} diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Context.kt b/kotlinx-coroutines-core/common/src/flow/operators/Context.kt new file mode 100644 index 0000000000..01e5be232f --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Context.kt @@ -0,0 +1,287 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.channels.Channel.Factory.BUFFERED +import kotlinx.coroutines.channels.Channel.Factory.CONFLATED +import kotlinx.coroutines.flow.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* + +/** + * Buffers flow emissions via channel of a specified capacity and runs collector in a separate coroutine. + * + * Normally, [flows][Flow] are _sequential_. It means that the code of all operators is executed in the + * same coroutine. For example, consider the following code using [onEach] and [collect] operators: + * + * ``` + * flowOf("A", "B", "C") + * .onEach { println("1$it") } + * .collect { println("2$it") } + * ``` + * + * It is going to be executed in the following order by the coroutine `Q` that calls this code: + * + * ``` + * Q : -->-- [1A] -- [2A] -- [1B] -- [2B] -- [1C] -- [2C] -->-- + * ``` + * + * So if the operator's code takes considerable time to execute, then the total execution time is going to be + * the sum of execution times for all operators. + * + * The `buffer` operator creates a separate coroutine during execution for the flow it applies to. + * Consider the following code: + * + * ``` + * flowOf("A", "B", "C") + * .onEach { println("1$it") } + * .buffer() // <--------------- buffer between onEach and collect + * .collect { println("2$it") } + * ``` + * + * It will use two coroutines for execution of the code. A coroutine `Q` that calls this code is + * going to execute `collect`, and the code before `buffer` will be executed in a separate + * new coroutine `P` concurrently with `Q`: + * + * ``` + * P : -->-- [1A] -- [1B] -- [1C] ---------->-- // flowOf(...).onEach { ... } + * + * | + * | channel // buffer() + * V + * + * Q : -->---------- [2A] -- [2B] -- [2C] -->-- // collect + * ``` + * + * When the operator's code takes some time to execute, this decreases the total execution time of the flow. + * A [channel][Channel] is used between the coroutines to send elements emitted by the coroutine `P` to + * the coroutine `Q`. If the code before `buffer` operator (in the coroutine `P`) is faster than the code after + * `buffer` operator (in the coroutine `Q`), then this channel will become full at some point and will suspend + * the producer coroutine `P` until the consumer coroutine `Q` catches up. + * The [capacity] parameter defines the size of this buffer. + * + * ### Buffer overflow + * + * By default, the emitter is suspended when the buffer overflows, to let collector catch up. This strategy can be + * overridden with an optional [onBufferOverflow] parameter so that the emitter is never suspended. In this + * case, on buffer overflow either the oldest value in the buffer is dropped with the [DROP_OLDEST][BufferOverflow.DROP_OLDEST] + * strategy and the latest emitted value is added to the buffer, + * or the latest value that is being emitted is dropped with the [DROP_LATEST][BufferOverflow.DROP_LATEST] strategy, + * keeping the buffer intact. + * To implement either of the custom strategies, a buffer of at least one element is used. + * + * ### Operator fusion + * + * Adjacent applications of [channelFlow], [flowOn], [buffer], and [produceIn] are + * always fused so that only one properly configured channel is used for execution. + * + * Explicitly specified buffer capacity takes precedence over `buffer()` or `buffer(Channel.BUFFERED)` calls, + * which effectively requests a buffer of any size. Multiple requests with a specified buffer + * size produce a buffer with the sum of the requested buffer sizes. + * + * A `buffer` call with a non-[SUSPEND] value of the [onBufferOverflow] parameter overrides all immediately preceding + * buffering operators, because it never suspends its upstream, and thus no upstream buffer would ever be used. + * + * ### Conceptual implementation + * + * The actual implementation of `buffer` is not trivial due to the fusing, but conceptually its basic + * implementation is equivalent to the following code that can be written using [produce] + * coroutine builder to produce a channel and [consumeEach][ReceiveChannel.consumeEach] extension to consume it: + * + * ``` + * fun Flow.buffer(capacity: Int = DEFAULT): Flow = flow { + * coroutineScope { // limit the scope of concurrent producer coroutine + * val channel = produce(capacity = capacity) { + * collect { send(it) } // send all to channel + * } + * // emit all received values + * channel.consumeEach { emit(it) } + * } + * } + * ``` + * + * ### Conflation + * + * Usage of this function with [capacity] of [Channel.CONFLATED][Channel.CONFLATED] is a shortcut to + * `buffer(capacity = 0, onBufferOverflow = `[`BufferOverflow.DROP_OLDEST`][BufferOverflow.DROP_OLDEST]`)`, + * and is available via a separate [conflate] operator. + * + * @param capacity type/capacity of the buffer between coroutines. Allowed values are the same as in `Channel(...)` + * factory function: [BUFFERED][Channel.BUFFERED] (by default), [CONFLATED][Channel.CONFLATED], + * [RENDEZVOUS][Channel.RENDEZVOUS], [UNLIMITED][Channel.UNLIMITED] or a non-negative value indicating + * an explicitly requested size. + * @param onBufferOverflow configures an action on buffer overflow (optional, defaults to + * [SUSPEND][BufferOverflow.SUSPEND], supported only when `capacity >= 0` or `capacity == Channel.BUFFERED`, + * implicitly creates a channel with at least one buffered element). + */ +@Suppress("NAME_SHADOWING") +public fun Flow.buffer(capacity: Int = BUFFERED, onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND): Flow { + require(capacity >= 0 || capacity == BUFFERED || capacity == CONFLATED) { + "Buffer size should be non-negative, BUFFERED, or CONFLATED, but was $capacity" + } + require(capacity != CONFLATED || onBufferOverflow == BufferOverflow.SUSPEND) { + "CONFLATED capacity cannot be used with non-default onBufferOverflow" + } + // desugar CONFLATED capacity to (0, DROP_OLDEST) + var capacity = capacity + var onBufferOverflow = onBufferOverflow + if (capacity == CONFLATED) { + capacity = 0 + onBufferOverflow = BufferOverflow.DROP_OLDEST + } + // create a flow + return when (this) { + is FusibleFlow -> fuse(capacity = capacity, onBufferOverflow = onBufferOverflow) + else -> ChannelFlowOperatorImpl(this, capacity = capacity, onBufferOverflow = onBufferOverflow) + } +} + +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.4.0, binary compatibility with earlier versions") +public fun Flow.buffer(capacity: Int = BUFFERED): Flow = buffer(capacity) + +/** + * Conflates flow emissions via conflated channel and runs collector in a separate coroutine. + * The effect of this is that emitter is never suspended due to a slow collector, but collector + * always gets the most recent value emitted. + * + * This is a shortcut for `buffer(capacity = 0, onBufferOverflow = BufferOverflow.DROP_OLDEST)`. + * See the [buffer] operator for other configuration options. + * + * For example, consider the flow that emits integers from 1 to 30 with 100 ms delay between them: + * + * ``` + * val flow = flow { + * for (i in 1..30) { + * delay(100) + * emit(i) + * } + * } + * ``` + * + * Applying `conflate()` operator to it allows a collector that delays 1 second on each element to get + * integers 1, 10, 20, 30: + * + * ``` + * val result = flow.conflate().onEach { delay(1000) }.toList() + * assertEquals(listOf(1, 10, 20, 30), result) + * ``` + * + * Note that `conflate` operator is a shortcut for [buffer] with `capacity` of [Channel.CONFLATED][Channel.CONFLATED], + * which is, in turn, a shortcut to a buffer that only keeps the latest element as + * created by `buffer(onBufferOverflow = `[`BufferOverflow.DROP_OLDEST`][BufferOverflow.DROP_OLDEST]`)`. + * + * ### Operator fusion + * + * Adjacent applications of `conflate`/[buffer], [channelFlow], [flowOn] and [produceIn] are + * always fused so that only one properly configured channel is used for execution. + * + * If there was no explicit buffer size specified, then the buffer size is `0`. + * Otherwise, the buffer size is unchanged. + * The strategy for buffer overflow becomes [BufferOverflow.DROP_OLDEST] after the application of this operator, + * but can be overridden later. + * + * Note that any instance of [StateFlow] already behaves as if `conflate` operator is + * applied to it, so applying `conflate` to a `StateFlow` has no effect. + * See [StateFlow] documentation on Operator Fusion. + */ +public fun Flow.conflate(): Flow = buffer(CONFLATED) + +/** + * Changes the context where this flow is executed to the given [context]. + * This operator is composable and affects only preceding operators that do not have its own context. + * This operator is context preserving: [context] **does not** leak into the downstream flow. + * + * For example: + * + * ``` + * withContext(Dispatchers.Main) { + * val singleValue = intFlow // will be executed on IO if context wasn't specified before + * .map { ... } // Will be executed in IO + * .flowOn(Dispatchers.IO) + * .filter { ... } // Will be executed in Default + * .flowOn(Dispatchers.Default) + * .single() // Will be executed in the Main + * } + * ``` + * + * For more explanation of context preservation please refer to [Flow] documentation. + * + * This operator retains a _sequential_ nature of flow if changing the context does not call for changing + * the [dispatcher][CoroutineDispatcher]. Otherwise, if changing dispatcher is required, it collects + * flow emissions in one coroutine that is run using a specified [context] and emits them from another coroutines + * with the original collector's context using a channel with a [default][Channel.BUFFERED] buffer size + * between two coroutines similarly to [buffer] operator, unless [buffer] operator is explicitly called + * before or after `flowOn`, which requests buffering behavior and specifies channel size. + * + * Note, that flows operating across different dispatchers might lose some in-flight elements when cancelled. + * In particular, this operator ensures that downstream flow does not resume on cancellation even if the element + * was already emitted by the upstream flow. + * + * ### Operator fusion + * + * Adjacent applications of [channelFlow], [flowOn], [buffer], and [produceIn] are + * always fused so that only one properly configured channel is used for execution. + * + * Multiple `flowOn` operators fuse to a single `flowOn` with a combined context. The elements of the context of + * the first `flowOn` operator naturally take precedence over the elements of the second `flowOn` operator + * when they have the same context keys, for example: + * + * ``` + * flow.map { ... } // Will be executed in IO + * .flowOn(Dispatchers.IO) // This one takes precedence + * .flowOn(Dispatchers.Default) + * ``` + * + * Note that an instance of [SharedFlow] does not have an execution context by itself, + * so applying `flowOn` to a `SharedFlow` has not effect. See the [SharedFlow] documentation on Operator Fusion. + * + * @throws [IllegalArgumentException] if provided context contains [Job] instance. + */ +public fun Flow.flowOn(context: CoroutineContext): Flow { + checkFlowContext(context) + return when { + context == EmptyCoroutineContext -> this + this is FusibleFlow -> fuse(context = context) + else -> ChannelFlowOperatorImpl(this, context = context) + } +} + +/** + * Returns a flow which checks cancellation status on each emission and throws + * the corresponding cancellation cause if flow collector was cancelled. + * Note that [flow] builder and all implementations of [SharedFlow] are [cancellable] by default. + * + * This operator provides a shortcut for `.onEach { currentCoroutineContext().ensureActive() }`. + * See [ensureActive][CoroutineContext.ensureActive] for details. + */ +public fun Flow.cancellable(): Flow = + when (this) { + is CancellableFlow<*> -> this // Fast-path, already cancellable + else -> CancellableFlowImpl(this) + } + +/** + * Internal marker for flows that are [cancellable]. + */ +internal interface CancellableFlow : Flow + +/** + * Named implementation class for a flow that is defined by the [cancellable] function. + */ +private class CancellableFlowImpl(private val flow: Flow) : CancellableFlow { + override suspend fun collect(collector: FlowCollector) { + flow.collect { + currentCoroutineContext().ensureActive() + collector.emit(it) + } + } +} + +private fun checkFlowContext(context: CoroutineContext) { + require(context[Job] == null) { + "Flow context cannot contain job in it. Had $context" + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Delay.kt b/kotlinx-coroutines-core/common/src/flow/operators/Delay.kt new file mode 100644 index 0000000000..2a701c0c12 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Delay.kt @@ -0,0 +1,406 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.internal.* +import kotlinx.coroutines.selects.* +import kotlin.jvm.* +import kotlin.time.* + +/* Scaffolding for Knit code examples + + +*/ + +/** + * Returns a flow that mirrors the original flow, but filters out values + * that are followed by the newer values within the given [timeout][timeoutMillis]. + * The latest value is always emitted. + * + * Example: + * + * ```kotlin + * flow { + * emit(1) + * delay(90) + * emit(2) + * delay(90) + * emit(3) + * delay(1010) + * emit(4) + * delay(1010) + * emit(5) + * }.debounce(1000) + * ``` + * + * + * produces the following emissions + * + * ```text + * 3, 4, 5 + * ``` + * + * + * Note that the resulting flow does not emit anything as long as the original flow emits + * items faster than every [timeoutMillis] milliseconds. + */ +@FlowPreview +public fun Flow.debounce(timeoutMillis: Long): Flow { + require(timeoutMillis >= 0L) { "Debounce timeout should not be negative" } + if (timeoutMillis == 0L) return this + return debounceInternal { timeoutMillis } +} + +/** + * Returns a flow that mirrors the original flow, but filters out values + * that are followed by the newer values within the given [timeout][timeoutMillis]. + * The latest value is always emitted. + * + * A variation of [debounce] that allows specifying the timeout value dynamically. + * + * Example: + * + * ```kotlin + * flow { + * emit(1) + * delay(90) + * emit(2) + * delay(90) + * emit(3) + * delay(1010) + * emit(4) + * delay(1010) + * emit(5) + * }.debounce { + * if (it == 1) { + * 0L + * } else { + * 1000L + * } + * } + * ``` + * + * + * produces the following emissions + * + * ```text + * 1, 3, 4, 5 + * ``` + * + * + * Note that the resulting flow does not emit anything as long as the original flow emits + * items faster than every [timeoutMillis] milliseconds. + * + * @param timeoutMillis [T] is the emitted value and the return value is timeout in milliseconds. + */ +@FlowPreview +@OverloadResolutionByLambdaReturnType +public fun Flow.debounce(timeoutMillis: (T) -> Long): Flow = + debounceInternal(timeoutMillis) + +/** + * Returns a flow that mirrors the original flow, but filters out values + * that are followed by the newer values within the given [timeout]. + * The latest value is always emitted. + * + * Example: + * + * ```kotlin + * flow { + * emit(1) + * delay(90.milliseconds) + * emit(2) + * delay(90.milliseconds) + * emit(3) + * delay(1010.milliseconds) + * emit(4) + * delay(1010.milliseconds) + * emit(5) + * }.debounce(1000.milliseconds) + * ``` + * + * + * produces the following emissions + * + * ```text + * 3, 4, 5 + * ``` + * + * + * Note that the resulting flow does not emit anything as long as the original flow emits + * items faster than every [timeout] milliseconds. + */ +@FlowPreview +public fun Flow.debounce(timeout: Duration): Flow = + debounce(timeout.toDelayMillis()) + +/** + * Returns a flow that mirrors the original flow, but filters out values + * that are followed by the newer values within the given [timeout]. + * The latest value is always emitted. + * + * A variation of [debounce] that allows specifying the timeout value dynamically. + * + * Example: + * + * ```kotlin + * flow { + * emit(1) + * delay(90.milliseconds) + * emit(2) + * delay(90.milliseconds) + * emit(3) + * delay(1010.milliseconds) + * emit(4) + * delay(1010.milliseconds) + * emit(5) + * }.debounce { + * if (it == 1) { + * 0.milliseconds + * } else { + * 1000.milliseconds + * } + * } + * ``` + * + * + * produces the following emissions + * + * ```text + * 1, 3, 4, 5 + * ``` + * + * + * Note that the resulting flow does not emit anything as long as the original flow emits + * items faster than every [timeout] unit. + * + * @param timeout [T] is the emitted value and the return value is timeout in [Duration]. + */ +@FlowPreview +@JvmName("debounceDuration") +@OverloadResolutionByLambdaReturnType +public fun Flow.debounce(timeout: (T) -> Duration): Flow = + debounceInternal { emittedItem -> + timeout(emittedItem).toDelayMillis() + } + +private fun Flow.debounceInternal(timeoutMillisSelector: (T) -> Long): Flow = + scopedFlow { downstream -> + // Produce the values using the default (rendezvous) channel + val values = produce { + collect { value -> send(value ?: NULL) } + } + // Now consume the values + var lastValue: Any? = null + while (lastValue !== DONE) { + var timeoutMillis = 0L // will be always computed when lastValue != null + // Compute timeout for this value + if (lastValue != null) { + timeoutMillis = timeoutMillisSelector(NULL.unbox(lastValue)) + require(timeoutMillis >= 0L) { "Debounce timeout should not be negative" } + if (timeoutMillis == 0L) { + downstream.emit(NULL.unbox(lastValue)) + lastValue = null // Consume the value + } + } + // assert invariant: lastValue != null implies timeoutMillis > 0 + assert { lastValue == null || timeoutMillis > 0 } + // wait for the next value with timeout + select { + // Set timeout when lastValue exists and is not consumed yet + if (lastValue != null) { + onTimeout(timeoutMillis) { + downstream.emit(NULL.unbox(lastValue)) + lastValue = null // Consume the value + } + } + values.onReceiveCatching { value -> + value + .onSuccess { lastValue = it } + .onFailure { + it?.let { throw it } + // If closed normally, emit the latest value + if (lastValue != null) downstream.emit(NULL.unbox(lastValue)) + lastValue = DONE + } + } + } + } + } + +/** + * Returns a flow that emits only the latest value emitted by the original flow during the given sampling [period][periodMillis]. + * + * Example: + * + * ```kotlin + * flow { + * repeat(10) { + * emit(it) + * delay(110) + * } + * }.sample(200) + * ``` + * + * + * produces the following emissions + * + * ```text + * 1, 3, 5, 7, 9 + * ``` + * + * + * Note that the latest element is not emitted if it does not fit into the sampling window. + */ +@FlowPreview +public fun Flow.sample(periodMillis: Long): Flow { + require(periodMillis > 0) { "Sample period should be positive" } + return scopedFlow { downstream -> + val values = produce(capacity = Channel.CONFLATED) { + collect { value -> send(value ?: NULL) } + } + var lastValue: Any? = null + val ticker = fixedPeriodTicker(periodMillis) + while (lastValue !== DONE) { + select { + values.onReceiveCatching { result -> + result + .onSuccess { lastValue = it } + .onFailure { + it?.let { throw it } + ticker.cancel(ChildCancelledException()) + lastValue = DONE + } + } + + // todo: shall be start sampling only when an element arrives or sample aways as here? + ticker.onReceive { + val value = lastValue ?: return@onReceive + lastValue = null // Consume the value + downstream.emit(NULL.unbox(value)) + } + } + } + } +} + +/* + * TODO this design (and design of the corresponding operator) depends on #540 + */ +internal fun CoroutineScope.fixedPeriodTicker( + delayMillis: Long, +): ReceiveChannel { + return produce(capacity = 0) { + delay(delayMillis) + while (true) { + channel.send(Unit) + delay(delayMillis) + } + } +} + +/** + * Returns a flow that emits only the latest value emitted by the original flow during the given sampling [period]. + * + * Example: + * + * ```kotlin + * flow { + * repeat(10) { + * emit(it) + * delay(110.milliseconds) + * } + * }.sample(200.milliseconds) + * ``` + * + * + * produces the following emissions + * + * ```text + * 1, 3, 5, 7, 9 + * ``` + * + * + * Note that the latest element is not emitted if it does not fit into the sampling window. + */ +@FlowPreview +public fun Flow.sample(period: Duration): Flow = sample(period.toDelayMillis()) + +/** + * Returns a flow that will emit a [TimeoutCancellationException] if the upstream doesn't emit an item within the given time. + * + * Example: + * + * ```kotlin + * flow { + * emit(1) + * delay(100) + * emit(2) + * delay(100) + * emit(3) + * delay(1000) + * emit(4) + * }.timeout(100.milliseconds).catch { exception -> + * if (exception is TimeoutCancellationException) { + * // Catch the TimeoutCancellationException emitted above. + * // Emit desired item on timeout. + * emit(-1) + * } else { + * // Throw other exceptions. + * throw exception + * } + * }.onEach { + * delay(300) // This will not cause a timeout + * } + * ``` + * + * + * produces the following emissions + * + * ```text + * 1, 2, 3, -1 + * ``` + * + * + * Note that delaying on the downstream doesn't trigger the timeout. + * + * @param timeout Timeout duration. If non-positive, the flow is timed out immediately + */ +@FlowPreview +public fun Flow.timeout( + timeout: Duration +): Flow = timeoutInternal(timeout) + +private fun Flow.timeoutInternal( + timeout: Duration +): Flow = scopedFlow { downStream -> + if (timeout <= Duration.ZERO) throw TimeoutCancellationException("Timed out immediately") + val values = buffer(Channel.RENDEZVOUS).produceIn(this) + whileSelect { + values.onReceiveCatching { value -> + value.onSuccess { + downStream.emit(it) + }.onClosed { + it?.let { throw it } + return@onReceiveCatching false + } + return@onReceiveCatching true + } + onTimeout(timeout) { + throw TimeoutCancellationException("Timed out waiting for $timeout") + } + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Distinct.kt b/kotlinx-coroutines-core/common/src/flow/operators/Distinct.kt new file mode 100644 index 0000000000..78e4d71a89 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Distinct.kt @@ -0,0 +1,77 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.flow.internal.* +import kotlin.jvm.* + +/** + * Returns flow where all subsequent repetitions of the same value are filtered out. + * + * Note that any instance of [StateFlow] already behaves as if `distinctUntilChanged` operator is + * applied to it, so applying `distinctUntilChanged` to a `StateFlow` has no effect. + * See [StateFlow] documentation on Operator Fusion. + * Also, repeated application of `distinctUntilChanged` operator on any flow has no effect. + */ +public fun Flow.distinctUntilChanged(): Flow = + when (this) { + is StateFlow<*> -> this // state flows are always distinct + else -> distinctUntilChangedBy(keySelector = defaultKeySelector, areEquivalent = defaultAreEquivalent) + } + +/** + * Returns flow where all subsequent repetitions of the same value are filtered out, when compared + * with each other via the provided [areEquivalent] function. + * + * Note that repeated application of `distinctUntilChanged` operator with the same parameter has no effect. + */ +@Suppress("UNCHECKED_CAST") +public fun Flow.distinctUntilChanged(areEquivalent: (old: T, new: T) -> Boolean): Flow = + distinctUntilChangedBy(keySelector = defaultKeySelector, areEquivalent = areEquivalent as (Any?, Any?) -> Boolean) + +/** + * Returns flow where all subsequent repetitions of the same key are filtered out, where + * key is extracted with [keySelector] function. + * + * Note that repeated application of `distinctUntilChanged` operator with the same parameter has no effect. + */ +public fun Flow.distinctUntilChangedBy(keySelector: (T) -> K): Flow = + distinctUntilChangedBy(keySelector = keySelector, areEquivalent = defaultAreEquivalent) + +private val defaultKeySelector: (Any?) -> Any? = { it } + +private val defaultAreEquivalent: (Any?, Any?) -> Boolean = { old, new -> old == new } + +/** + * Returns flow where all subsequent repetitions of the same key are filtered out, where + * keys are extracted with [keySelector] function and compared with each other via the + * provided [areEquivalent] function. + * + * NOTE: It is non-inline to share a single implementing class. + */ +private fun Flow.distinctUntilChangedBy( + keySelector: (T) -> Any?, + areEquivalent: (old: Any?, new: Any?) -> Boolean +): Flow = when { + this is DistinctFlowImpl<*> && this.keySelector === keySelector && this.areEquivalent === areEquivalent -> this // same + else -> DistinctFlowImpl(this, keySelector, areEquivalent) +} + +private class DistinctFlowImpl( + private val upstream: Flow, + @JvmField val keySelector: (T) -> Any?, + @JvmField val areEquivalent: (old: Any?, new: Any?) -> Boolean +): Flow { + override suspend fun collect(collector: FlowCollector) { + var previousKey: Any? = NULL + upstream.collect { value -> + val key = keySelector(value) + @Suppress("UNCHECKED_CAST") + if (previousKey === NULL || !areEquivalent(previousKey, key)) { + previousKey = key + collector.emit(value) + } + } + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Emitters.kt b/kotlinx-coroutines-core/common/src/flow/operators/Emitters.kt new file mode 100644 index 0000000000..ceeb336392 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Emitters.kt @@ -0,0 +1,217 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") +@file:Suppress("UNCHECKED_CAST") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.internal.* +import kotlin.jvm.* + +// ------------------ WARNING ------------------ +// These emitting operators must use safe flow builder, because they allow +// user code to directly emit to the underlying FlowCollector. + +/** + * Applies [transform] function to each value of the given flow. + * + * The receiver of the `transform` is [FlowCollector] and thus `transform` is a + * flexible function that may transform emitted element, skip it or emit it multiple times. + * + * This operator generalizes [filter] and [map] operators and + * can be used as a building block for other operators, for example: + * + * ``` + * fun Flow.skipOddAndDuplicateEven(): Flow = transform { value -> + * if (value % 2 == 0) { // Emit only even values, but twice + * emit(value) + * emit(value) + * } // Do nothing if odd + * } + * ``` + */ +public inline fun Flow.transform( + @BuilderInference crossinline transform: suspend FlowCollector.(value: T) -> Unit +): Flow = flow { // Note: safe flow is used here, because collector is exposed to transform on each operation + collect { value -> + // kludge, without it Unit will be returned and TCE won't kick in, KT-28938 + return@collect transform(value) + } +} + +// For internal operator implementation +@PublishedApi +internal inline fun Flow.unsafeTransform( + @BuilderInference crossinline transform: suspend FlowCollector.(value: T) -> Unit +): Flow = unsafeFlow { // Note: unsafe flow is used here, because unsafeTransform is only for internal use + collect { value -> + // kludge, without it Unit will be returned and TCE won't kick in, KT-28938 + return@collect transform(value) + } +} + +/** + * Returns a flow that invokes the given [action] **before** this flow starts to be collected. + * + * The [action] is called before the upstream flow is started, so if it is used with a [SharedFlow] + * there is **no guarantee** that emissions from the upstream flow that happen inside or immediately + * after this `onStart` action will be collected + * (see [onSubscription] for an alternative operator on shared flows). + * + * The receiver of the [action] is [FlowCollector], so `onStart` can emit additional elements. + * For example: + * + * ``` + * flowOf("a", "b", "c") + * .onStart { emit("Begin") } + * .collect { println(it) } // prints Begin, a, b, c + * ``` + */ +public fun Flow.onStart( + action: suspend FlowCollector.() -> Unit +): Flow = unsafeFlow { // Note: unsafe flow is used here, but safe collector is used to invoke start action + val safeCollector = SafeCollector(this, currentCoroutineContext()) + try { + safeCollector.action() + } finally { + safeCollector.releaseIntercepted() + } + collect(this) // directly delegate +} + +/** + * Returns a flow that invokes the given [action] **after** the flow is completed or cancelled, passing + * the cancellation exception or failure as cause parameter of [action]. + * + * Conceptually, `onCompletion` is similar to wrapping the flow collection into a `finally` block, + * for example the following imperative snippet: + * + * ``` + * try { + * myFlow.collect { value -> + * println(value) + * } + * } finally { + * println("Done") + * } + * ``` + * + * can be replaced with a declarative one using `onCompletion`: + * + * ``` + * myFlow + * .onEach { println(it) } + * .onCompletion { println("Done") } + * .collect() + * ``` + * + * Unlike [catch], this operator reports exception that occur both upstream and downstream + * and observe exceptions that are thrown to cancel the flow. Exception is empty if and only if + * the flow had fully completed successfully. Conceptually, the following code: + * + * ``` + * myFlow.collect { value -> + * println(value) + * } + * println("Completed successfully") + * ``` + * + * can be replaced with: + * + * ``` + * myFlow + * .onEach { println(it) } + * .onCompletion { if (it == null) println("Completed successfully") } + * .collect() + * ``` + * + * The receiver of the [action] is [FlowCollector] and this operator can be used to emit additional + * elements at the end **if it completed successfully**. For example: + * + * ``` + * flowOf("a", "b", "c") + * .onCompletion { emit("Done") } + * .collect { println(it) } // prints a, b, c, Done + * ``` + * + * In case of failure or cancellation, any attempt to emit additional elements throws the corresponding exception. + * Use [catch] if you need to suppress failure and replace it with emission of elements. + */ +public fun Flow.onCompletion( + action: suspend FlowCollector.(cause: Throwable?) -> Unit +): Flow = unsafeFlow { // Note: unsafe flow is used here, but safe collector is used to invoke completion action + try { + collect(this) + } catch (e: Throwable) { + /* + * Use throwing collector to prevent any emissions from the + * completion sequence when downstream has failed, otherwise it may + * lead to a non-sequential behaviour impossible with `finally` + */ + ThrowingCollector(e).invokeSafely(action, e) + throw e + } + // Normal completion + val sc = SafeCollector(this, currentCoroutineContext()) + try { + sc.action(null) + } finally { + sc.releaseIntercepted() + } +} + +/** + * Invokes the given [action] when this flow completes without emitting any elements. + * The receiver of the [action] is [FlowCollector], so `onEmpty` can emit additional elements. + * For example: + * + * ``` + * emptyFlow().onEmpty { + * emit(1) + * emit(2) + * }.collect { println(it) } // prints 1, 2 + * ``` + */ +public fun Flow.onEmpty( + action: suspend FlowCollector.() -> Unit +): Flow = unsafeFlow { + var isEmpty = true + collect { + isEmpty = false + emit(it) + } + if (isEmpty) { + val collector = SafeCollector(this, currentCoroutineContext()) + try { + collector.action() + } finally { + collector.releaseIntercepted() + } + } +} + +/* + * 'emitAll' methods call this to fail-fast before starting to collect + * their sources (that may not have any elements for a long time). + */ +internal fun FlowCollector<*>.ensureActive() { + if (this is ThrowingCollector) throw e +} + +internal class ThrowingCollector(@JvmField val e: Throwable) : FlowCollector { + override suspend fun emit(value: Any?) { + throw e + } +} + +private suspend fun FlowCollector.invokeSafely( + action: suspend FlowCollector.(cause: Throwable?) -> Unit, + cause: Throwable? +) { + try { + action(cause) + } catch (e: Throwable) { + if (cause !== null && cause !== e) e.addSuppressed(cause) + throw e + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Errors.kt b/kotlinx-coroutines-core/common/src/flow/operators/Errors.kt new file mode 100644 index 0000000000..7da73bf2cf --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Errors.kt @@ -0,0 +1,219 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* +import kotlinx.coroutines.flow.internal.unsafeFlow as flow + +/** + * Catches exceptions in the flow completion and calls a specified [action] with + * the caught exception. This operator is *transparent* to exceptions that occur + * in downstream flow and does not catch exceptions that are thrown to cancel the flow. + * + * For example: + * + * ``` + * flow { emitData() } + * .map { computeOne(it) } + * .catch { ... } // catches exceptions in emitData and computeOne + * .map { computeTwo(it) } + * .collect { process(it) } // throws exceptions from process and computeTwo + * ``` + * + * Conceptually, the action of `catch` operator is similar to wrapping the code of upstream flows with + * `try { ... } catch (e: Throwable) { action(e) }`. + * + * Any exception in the [action] code itself proceeds downstream where it can be + * caught by further `catch` operators if needed. If a particular exception does not need to be + * caught it can be rethrown from the action of `catch` operator. For example: + * + * ``` + * flow.catch { e -> + * if (e !is IOException) throw e // rethrow all but IOException + * // e is IOException here + * ... + * } + * ``` + * + * The [action] code has [FlowCollector] as a receiver and can [emit][FlowCollector.emit] values downstream. + * For example, caught exception can be replaced with some wrapper value for errors: + * + * ``` + * flow.catch { e -> emit(ErrorWrapperValue(e)) } + * ``` + * + * The [action] can also use [emitAll] to fallback on some other flow in case of an error. However, to + * retry an original flow use [retryWhen] operator that can retry the flow multiple times without + * introducing ever-growing stack of suspending calls. + */ +public fun Flow.catch(action: suspend FlowCollector.(cause: Throwable) -> Unit): Flow = + flow { + val exception = catchImpl(this) + if (exception != null) action(exception) + } + +/** + * Retries collection of the given flow up to [retries] times when an exception that matches the + * given [predicate] occurs in the upstream flow. This operator is *transparent* to exceptions that occur + * in downstream flow and does not retry on exceptions that are thrown to cancel the flow. + * + * See [catch] for details on how exceptions are caught in flows. + * + * The default value of [retries] parameter is [Long.MAX_VALUE]. This value effectively means to retry forever. + * This operator is a shorthand for the following code (see [retryWhen]). Note that `attempt` is checked first + * and [predicate] is not called when it reaches the given number of [retries]: + * + * ``` + * retryWhen { cause, attempt -> attempt < retries && predicate(cause) } + * ``` + * + * The [predicate] parameter is always true by default. The [predicate] is a suspending function, + * so it can be also used to introduce delay before retry, for example: + * + * ``` + * flow.retry(3) { e -> + * // retry on any IOException but also introduce delay if retrying + * (e is IOException).also { if (it) delay(1000) } + * } + * ``` + * + * @throws IllegalArgumentException when [retries] is not positive. + */ +public fun Flow.retry( + retries: Long = Long.MAX_VALUE, + predicate: suspend (cause: Throwable) -> Boolean = { true } +): Flow { + require(retries > 0) { "Expected positive amount of retries, but had $retries" } + return retryWhen { cause, attempt -> attempt < retries && predicate(cause) } +} + +/** + * Retries collection of the given flow when an exception occurs in the upstream flow and the + * [predicate] returns true. The predicate also receives an `attempt` number as parameter, + * starting from zero on the initial call. This operator is *transparent* to exceptions that occur + * in downstream flow and does not retry on exceptions that are thrown to cancel the flow. + * + * For example, the following call retries the flow forever if the error is caused by `IOException`, but + * stops after 3 retries on any other exception: + * + * ``` + * flow.retryWhen { cause, attempt -> cause is IOException || attempt < 3 } + * ``` + * + * To implement a simple retry logic with a limit on the number of retries use [retry] operator. + * + * Similarly to [catch] operator, the [predicate] code has [FlowCollector] as a receiver and can + * [emit][FlowCollector.emit] values downstream. + * The [predicate] is a suspending function, so it can be used to introduce delay before retry, for example: + * + * ``` + * flow.retryWhen { cause, attempt -> + * if (cause is IOException) { // retry on IOException + * emit(RetryWrapperValue(e)) + * delay(1000) // delay for one second before retry + * true + * } else { // do not retry otherwise + * false + * } + * } + * ``` + * + * See [catch] for more details. + */ +public fun Flow.retryWhen(predicate: suspend FlowCollector.(cause: Throwable, attempt: Long) -> Boolean): Flow = + flow { + var attempt = 0L + var shallRetry: Boolean + do { + shallRetry = false + val cause = catchImpl(this) + if (cause != null) { + if (predicate(cause, attempt)) { + shallRetry = true + attempt++ + } else { + throw cause + } + } + } while (shallRetry) + } + +// Return exception from upstream or null +@Suppress("NAME_SHADOWING") +internal suspend fun Flow.catchImpl( + collector: FlowCollector +): Throwable? { + var fromDownstream: Throwable? = null + try { + collect { + try { + collector.emit(it) + } catch (e: Throwable) { + fromDownstream = e + throw e + } + } + } catch (e: Throwable) { + // Otherwise, smartcast is impossible + val fromDownstream = fromDownstream + /* + * First check ensures that we catch an original exception, not one rethrown by an operator. + * Seconds check ignores cancellation causes, they cannot be caught. + */ + if (e.isSameExceptionAs(fromDownstream) || e.isCancellationCause(coroutineContext)) { + throw e // Rethrow exceptions from downstream and cancellation causes + } else { + /* + * The exception came from the upstream [semi-] independently. + * For pure failures, when the downstream functions normally, we handle the exception as intended. + * But if the downstream has failed prior to or concurrently + * with the upstream, we forcefully rethrow it, preserving the contextual information and ensuring that it's not lost. + */ + if (fromDownstream == null) { + return e + } + /* + * We consider the upstream exception as the superseding one when both upstream and downstream + * fail, suppressing the downstream exception, and operating similarly to `finally` block with + * the useful addition of adding the original downstream exception to suppressed ones. + * + * That's important for the following scenarios: + * ``` + * flow { + * val resource = ... + * try { + * ... emit as well ... + * } finally { + * resource.close() // Throws in the shutdown sequence when 'collect' already has thrown an exception + * } + * }.catch { } // or retry + * .collect { ... } + * ``` + * when *the downstream* throws. + */ + if (e is CancellationException) { + fromDownstream.addSuppressed(e) + throw fromDownstream + } else { + e.addSuppressed(fromDownstream) + throw e + } + } + } + return null +} + +private fun Throwable.isCancellationCause(coroutineContext: CoroutineContext): Boolean { + val job = coroutineContext[Job] + if (job == null || !job.isCancelled) return false + return isSameExceptionAs(job.getCancellationException()) +} + +private fun Throwable.isSameExceptionAs(other: Throwable?): Boolean = + other != null && unwrap(other) == unwrap(this) + + diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Limit.kt b/kotlinx-coroutines-core/common/src/flow/operators/Limit.kt new file mode 100644 index 0000000000..2c37e24162 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Limit.kt @@ -0,0 +1,140 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* +import kotlinx.coroutines.flow.flow as safeFlow +import kotlinx.coroutines.flow.internal.unsafeFlow as flow + +/** + * Returns a flow that ignores first [count] elements. + * Throws [IllegalArgumentException] if [count] is negative. + */ +public fun Flow.drop(count: Int): Flow { + require(count >= 0) { "Drop count should be non-negative, but had $count" } + return flow { + var skipped = 0 + collect { value -> + if (skipped >= count) emit(value) else ++skipped + } + } +} + +/** + * Returns a flow containing all elements except first elements that satisfy the given predicate. + */ +public fun Flow.dropWhile(predicate: suspend (T) -> Boolean): Flow = flow { + var matched = false + collect { value -> + if (matched) { + emit(value) + } else if (!predicate(value)) { + matched = true + emit(value) + } + } +} + +/** + * Returns a flow that contains first [count] elements. + * When [count] elements are consumed, the original flow is cancelled. + * Throws [IllegalArgumentException] if [count] is not positive. + */ +public fun Flow.take(count: Int): Flow { + require(count > 0) { "Requested element count $count should be positive" } + return flow { + val ownershipMarker = Any() + var consumed = 0 + try { + collect { value -> + // Note: this for take is not written via collectWhile on purpose. + // It checks condition first and then makes a tail-call to either emit or emitAbort. + // This way normal execution does not require a state machine, only a termination (emitAbort). + // See "TakeBenchmark" for comparision of different approaches. + if (++consumed < count) { + return@collect emit(value) + } else { + return@collect emitAbort(value, ownershipMarker) + } + } + } catch (e: AbortFlowException) { + e.checkOwnership(owner = ownershipMarker) + } + } +} + +private suspend fun FlowCollector.emitAbort(value: T, ownershipMarker: Any) { + emit(value) + throw AbortFlowException(ownershipMarker) +} + +/** + * Returns a flow that contains first elements satisfying the given [predicate]. + * + * Note, that the resulting flow does not contain the element on which the [predicate] returned `false`. + * See [transformWhile] for a more flexible operator. + */ +public fun Flow.takeWhile(predicate: suspend (T) -> Boolean): Flow = flow { + // This return is needed to work around a bug in JS BE: KT-39227 + return@flow collectWhile { value -> + if (predicate(value)) { + emit(value) + true + } else { + false + } + } +} + +/** + * Applies [transform] function to each value of the given flow while this + * function returns `true`. + * + * The receiver of the `transformWhile` is [FlowCollector] and thus `transformWhile` is a + * flexible function that may transform emitted element, skip it or emit it multiple times. + * + * This operator generalizes [takeWhile] and can be used as a building block for other operators. + * For example, a flow of download progress messages can be completed when the + * download is done but emit this last message (unlike `takeWhile`): + * + * ``` + * fun Flow.completeWhenDone(): Flow = + * transformWhile { progress -> + * emit(progress) // always emit progress + * !progress.isDone() // continue while download is not done + * } + * ``` + */ +public fun Flow.transformWhile( + @BuilderInference transform: suspend FlowCollector.(value: T) -> Boolean +): Flow = + safeFlow { // Note: safe flow is used here, because collector is exposed to transform on each operation + // This return is needed to work around a bug in JS BE: KT-39227 + return@safeFlow collectWhile { value -> + transform(value) + } + } + +// Internal building block for non-tailcalling flow-truncating operators +internal suspend inline fun Flow.collectWhile(crossinline predicate: suspend (value: T) -> Boolean) { + val collector = object : FlowCollector { + override suspend fun emit(value: T) { + // Note: we are checking predicate first, then throw. If the predicate does suspend (calls emit, for example) + // the resulting code is never tail-suspending and produces a state-machine + if (!predicate(value)) { + throw AbortFlowException(this) + } + } + } + try { + collect(collector) + } catch (e: AbortFlowException) { + e.checkOwnership(collector) + // The task might have been cancelled before AbortFlowException was thrown. + coroutineContext.ensureActive() + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Lint.kt b/kotlinx-coroutines-core/common/src/flow/operators/Lint.kt new file mode 100644 index 0000000000..4247a72346 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Lint.kt @@ -0,0 +1,188 @@ +@file:Suppress("unused", "INVISIBLE_REFERENCE", "INVISIBLE_MEMBER", "UNUSED_PARAMETER") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.internal.InlineOnly + +/** + * Applying [cancellable][Flow.cancellable] to a [SharedFlow] has no effect. + * See the [SharedFlow] documentation on Operator Fusion. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Applying 'cancellable' to a SharedFlow has no effect. See the SharedFlow documentation on Operator Fusion.", + replaceWith = ReplaceWith("this") +) +public fun SharedFlow.cancellable(): Flow = noImpl() + +/** + * Applying [flowOn][Flow.flowOn] to [SharedFlow] has no effect. + * See the [SharedFlow] documentation on Operator Fusion. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Applying 'flowOn' to SharedFlow has no effect. See the SharedFlow documentation on Operator Fusion.", + replaceWith = ReplaceWith("this") +) +public fun SharedFlow.flowOn(context: CoroutineContext): Flow = noImpl() + +/** + * Applying [conflate][Flow.conflate] to [StateFlow] has no effect. + * See the [StateFlow] documentation on Operator Fusion. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Applying 'conflate' to StateFlow has no effect. See the StateFlow documentation on Operator Fusion.", + replaceWith = ReplaceWith("this") +) +public fun StateFlow.conflate(): Flow = noImpl() + +/** + * Applying [distinctUntilChanged][Flow.distinctUntilChanged] to [StateFlow] has no effect. + * See the [StateFlow] documentation on Operator Fusion. + * @suppress + */ +@Deprecated( + level = DeprecationLevel.ERROR, + message = "Applying 'distinctUntilChanged' to StateFlow has no effect. See the StateFlow documentation on Operator Fusion.", + replaceWith = ReplaceWith("this") +) +public fun StateFlow.distinctUntilChanged(): Flow = noImpl() + +/** + * @suppress + */ +@Deprecated( + message = "isActive is resolved into the extension of outer CoroutineScope which is likely to be an error. " + + "Use currentCoroutineContext().isActive or cancellable() operator instead " + + "or specify the receiver of isActive explicitly. " + + "Additionally, flow {} builder emissions are cancellable by default.", + level = DeprecationLevel.ERROR, + replaceWith = ReplaceWith("currentCoroutineContext().isActive") +) +public val FlowCollector<*>.isActive: Boolean + get() = noImpl() + +/** + * @suppress + */ +@Deprecated( + message = "cancel() is resolved into the extension of outer CoroutineScope which is likely to be an error. " + + "Use currentCoroutineContext().cancel() instead or specify the receiver of cancel() explicitly", + level = DeprecationLevel.ERROR, + replaceWith = ReplaceWith("currentCoroutineContext().cancel(cause)") +) +public fun FlowCollector<*>.cancel(cause: CancellationException? = null): Unit = noImpl() + +/** + * @suppress + */ +@Deprecated( + message = "coroutineContext is resolved into the property of outer CoroutineScope which is likely to be an error. " + + "Use currentCoroutineContext() instead or specify the receiver of coroutineContext explicitly", + level = DeprecationLevel.ERROR, + replaceWith = ReplaceWith("currentCoroutineContext()") +) +public val FlowCollector<*>.coroutineContext: CoroutineContext + get() = noImpl() + +/** + * @suppress + */ +@Deprecated( + message = "SharedFlow never completes, so this operator typically has not effect, it can only " + + "catch exceptions from 'onSubscribe' operator", + level = DeprecationLevel.WARNING, + replaceWith = ReplaceWith("this") +) +@InlineOnly +public inline fun SharedFlow.catch(noinline action: suspend FlowCollector.(cause: Throwable) -> Unit): Flow = + (this as Flow).catch(action) + +/** + * @suppress + */ +@Deprecated( + message = "SharedFlow never completes, so this operator has no effect.", + level = DeprecationLevel.WARNING, + replaceWith = ReplaceWith("this") +) +@InlineOnly +public inline fun SharedFlow.retry( + retries: Long = Long.MAX_VALUE, + noinline predicate: suspend (cause: Throwable) -> Boolean = { true } +): Flow = + (this as Flow).retry(retries, predicate) + +/** + * @suppress + */ +@Deprecated( + message = "SharedFlow never completes, so this operator has no effect.", + level = DeprecationLevel.WARNING, + replaceWith = ReplaceWith("this") +) +@InlineOnly +public inline fun SharedFlow.retryWhen(noinline predicate: suspend FlowCollector.(cause: Throwable, attempt: Long) -> Boolean): Flow = + (this as Flow).retryWhen(predicate) + +/** + * @suppress + */ +@Suppress("DeprecatedCallableAddReplaceWith") +@Deprecated( + message = "SharedFlow never completes, so this terminal operation never completes.", + level = DeprecationLevel.WARNING +) +@InlineOnly +public suspend inline fun SharedFlow.toList(): List = + (this as Flow).toList() + +/** + * A specialized version of [Flow.toList] that returns [Nothing] + * to indicate that [SharedFlow] collection never completes. + */ +@InlineOnly +public suspend inline fun SharedFlow.toList(destination: MutableList): Nothing { + (this as Flow).toList(destination) + throw IllegalStateException("this code is supposed to be unreachable") +} + +/** + * @suppress + */ +@Suppress("DeprecatedCallableAddReplaceWith") +@Deprecated( + message = "SharedFlow never completes, so this terminal operation never completes.", + level = DeprecationLevel.WARNING +) +@InlineOnly +public suspend inline fun SharedFlow.toSet(): Set = + (this as Flow).toSet() + +/** + * A specialized version of [Flow.toSet] that returns [Nothing] + * to indicate that [SharedFlow] collection never completes. + */ +@InlineOnly +public suspend inline fun SharedFlow.toSet(destination: MutableSet): Nothing { + (this as Flow).toSet(destination) + throw IllegalStateException("this code is supposed to be unreachable") +} + +/** + * @suppress + */ +@Suppress("DeprecatedCallableAddReplaceWith") +@Deprecated( + message = "SharedFlow never completes, so this terminal operation never completes.", + level = DeprecationLevel.WARNING +) +@InlineOnly +public suspend inline fun SharedFlow.count(): Int = + (this as Flow).count() diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Merge.kt b/kotlinx-coroutines-core/common/src/flow/operators/Merge.kt new file mode 100644 index 0000000000..92ebeabbaa --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Merge.kt @@ -0,0 +1,213 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") +@file:Suppress("unused") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.internal.* +import kotlinx.coroutines.internal.* +import kotlin.jvm.* +import kotlinx.coroutines.flow.internal.unsafeFlow as flow + +/** + * Name of the property that defines the value of [DEFAULT_CONCURRENCY]. + * This is a preview API and can be changed in a backwards-incompatible manner within a single release. + */ +@FlowPreview +public const val DEFAULT_CONCURRENCY_PROPERTY_NAME: String = "kotlinx.coroutines.flow.defaultConcurrency" + +/** + * Default concurrency limit that is used by [flattenMerge] and [flatMapMerge] operators. + * It is 16 by default and can be changed on JVM using [DEFAULT_CONCURRENCY_PROPERTY_NAME] property. + * This is a preview API and can be changed in a backwards-incompatible manner within a single release. + */ +@FlowPreview +public val DEFAULT_CONCURRENCY: Int = systemProp( + DEFAULT_CONCURRENCY_PROPERTY_NAME, + 16, 1, Int.MAX_VALUE +) + +/** + * Transforms elements emitted by the original flow by applying [transform], that returns another flow, + * and then concatenating and flattening these flows. + * + * This method is a shortcut for `map(transform).flattenConcat()`. See [flattenConcat]. + * + * Note that even though this operator looks very familiar, we discourage its usage in a regular application-specific flows. + * Most likely, suspending operation in [map] operator will be sufficient and linear transformations are much easier to reason about. + */ +@ExperimentalCoroutinesApi +public fun Flow.flatMapConcat(transform: suspend (value: T) -> Flow): Flow = + map(transform).flattenConcat() + +/** + * Transforms elements emitted by the original flow by applying [transform], that returns another flow, + * and then merging and flattening these flows. + * + * This operator calls [transform] *sequentially* and then merges the resulting flows with a [concurrency] + * limit on the number of concurrently collected flows. + * It is a shortcut for `map(transform).flattenMerge(concurrency)`. + * See [flattenMerge] for details. + * + * Note that even though this operator looks very familiar, we discourage its usage in a regular application-specific flows. + * Most likely, suspending operation in [map] operator will be sufficient and linear transformations are much easier to reason about. + * + * ### Operator fusion + * + * Applications of [flowOn], [buffer], and [produceIn] _after_ this operator are fused with + * its concurrent merging so that only one properly configured channel is used for execution of merging logic. + * + * @param concurrency controls the number of in-flight flows, at most [concurrency] flows are collected + * at the same time. By default, it is equal to [DEFAULT_CONCURRENCY]. + */ +@ExperimentalCoroutinesApi +public fun Flow.flatMapMerge( + concurrency: Int = DEFAULT_CONCURRENCY, + transform: suspend (value: T) -> Flow +): Flow = + map(transform).flattenMerge(concurrency) + +/** + * Flattens the given flow of flows into a single flow in a sequential manner, without interleaving nested flows. + * + * Inner flows are collected by this operator *sequentially*. + */ +@ExperimentalCoroutinesApi +public fun Flow>.flattenConcat(): Flow = flow { + collect { value -> emitAll(value) } +} + +/** + * Merges the given flows into a single flow without preserving an order of elements. + * All flows are merged concurrently, without limit on the number of simultaneously collected flows. + * + * ### Operator fusion + * + * Applications of [flowOn], [buffer], and [produceIn] _after_ this operator are fused with + * its concurrent merging so that only one properly configured channel is used for execution of merging logic. + */ +public fun Iterable>.merge(): Flow { + /* + * This is a fuseable implementation of the following operator: + * channelFlow { + * forEach { flow -> + * launch { + * flow.collect { send(it) } + * } + * } + * } + */ + return ChannelLimitedFlowMerge(this) +} + +/** + * Merges the given flows into a single flow without preserving an order of elements. + * All flows are merged concurrently, without limit on the number of simultaneously collected flows. + * + * ### Operator fusion + * + * Applications of [flowOn], [buffer], and [produceIn] _after_ this operator are fused with + * its concurrent merging so that only one properly configured channel is used for execution of merging logic. + */ +public fun merge(vararg flows: Flow): Flow = flows.asIterable().merge() + +/** + * Flattens the given flow of flows into a single flow with a [concurrency] limit on the number of + * concurrently collected flows. + * + * If [concurrency] is more than 1, then inner flows are collected by this operator *concurrently*. + * With `concurrency == 1` this operator is identical to [flattenConcat]. + * + * ### Operator fusion + * + * Applications of [flowOn], [buffer], and [produceIn] _after_ this operator are fused with + * its concurrent merging so that only one properly configured channel is used for execution of merging logic. + * + * When [concurrency] is greater than 1, this operator is [buffered][buffer] by default + * and size of its output buffer can be changed by applying subsequent [buffer] operator. + * + * @param concurrency controls the number of in-flight flows, at most [concurrency] flows are collected + * at the same time. By default, it is equal to [DEFAULT_CONCURRENCY]. + */ +@ExperimentalCoroutinesApi +public fun Flow>.flattenMerge(concurrency: Int = DEFAULT_CONCURRENCY): Flow { + require(concurrency > 0) { "Expected positive concurrency level, but had $concurrency" } + return if (concurrency == 1) flattenConcat() else ChannelFlowMerge(this, concurrency) +} + +/** + * Returns a flow that produces element by [transform] function every time the original flow emits a value. + * When the original flow emits a new value, the previous `transform` block is cancelled, thus the name `transformLatest`. + * + * For example, the following flow: + * ``` + * flow { + * emit("a") + * delay(100) + * emit("b") + * }.transformLatest { value -> + * emit(value) + * delay(200) + * emit(value + "_last") + * } + * ``` + * produces `a b b_last`. + * + * This operator is [buffered][buffer] by default + * and size of its output buffer can be changed by applying subsequent [buffer] operator. + */ +@ExperimentalCoroutinesApi +public fun Flow.transformLatest(@BuilderInference transform: suspend FlowCollector.(value: T) -> Unit): Flow = + ChannelFlowTransformLatest(transform, this) + +/** + * Returns a flow that switches to a new flow produced by [transform] function every time the original flow emits a value. + * When the original flow emits a new value, the previous flow produced by `transform` block is cancelled. + * + * For example, the following flow: + * ``` + * flow { + * emit("a") + * delay(100) + * emit("b") + * }.flatMapLatest { value -> + * flow { + * emit(value) + * delay(200) + * emit(value + "_last") + * } + * } + * ``` + * produces `a b b_last` + * + * This operator is [buffered][buffer] by default and size of its output buffer can be changed by applying subsequent [buffer] operator. + */ +@ExperimentalCoroutinesApi +public inline fun Flow.flatMapLatest(@BuilderInference crossinline transform: suspend (value: T) -> Flow): Flow = + transformLatest { emitAll(transform(it)) } + +/** + * Returns a flow that emits elements from the original flow transformed by [transform] function. + * When the original flow emits a new value, computation of the [transform] block for previous value is cancelled. + * + * For example, the following flow: + * ``` + * flow { + * emit("a") + * delay(100) + * emit("b") + * }.mapLatest { value -> + * println("Started computing $value") + * delay(200) + * "Computed $value" + * } + * ``` + * will print "Started computing a" and "Started computing b", but the resulting flow will contain only "Computed b" value. + * + * This operator is [buffered][buffer] by default and size of its output buffer can be changed by applying subsequent [buffer] operator. + */ +@ExperimentalCoroutinesApi +public fun Flow.mapLatest(@BuilderInference transform: suspend (value: T) -> R): Flow = + transformLatest { emit(transform(it)) } diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Share.kt b/kotlinx-coroutines-core/common/src/flow/operators/Share.kt new file mode 100644 index 0000000000..da656f8307 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Share.kt @@ -0,0 +1,428 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* + +// -------------------------------- shareIn -------------------------------- + +/** + * Converts a _cold_ [Flow] into a _hot_ [SharedFlow] that is started in the given coroutine [scope], + * sharing emissions from a single running instance of the upstream flow with multiple downstream subscribers, + * and replaying a specified number of [replay] values to new subscribers. See the [SharedFlow] documentation + * for the general concepts of shared flows. + * + * The starting of the sharing coroutine is controlled by the [started] parameter. The following options + * are supported. + * + * - [Eagerly][SharingStarted.Eagerly] — the upstream flow is started even before the first subscriber appears. Note + * that in this case all values emitted by the upstream beyond the most recent values as specified by + * [replay] parameter **will be immediately discarded**. + * - [Lazily][SharingStarted.Lazily] — starts the upstream flow after the first subscriber appears, which guarantees + * that this first subscriber gets all the emitted values, while subsequent subscribers are only guaranteed to + * get the most recent [replay] values. The upstream flow continues to be active even when all subscribers + * disappear, but only the most recent [replay] values are cached without subscribers. + * - [WhileSubscribed()][SharingStarted.WhileSubscribed] — starts the upstream flow when the first subscriber + * appears, immediately stops when the last subscriber disappears, keeping the replay cache forever. + * It has additional optional configuration parameters as explained in its documentation. + * - A custom strategy can be supplied by implementing the [SharingStarted] interface. + * + * The `shareIn` operator is useful in situations when there is a _cold_ flow that is expensive to create and/or + * to maintain, but there are multiple subscribers that need to collect its values. For example, consider a + * flow of messages coming from a backend over the expensive network connection, taking a lot of + * time to establish. Conceptually, it might be implemented like this: + * + * ``` + * val backendMessages: Flow = flow { + * connectToBackend() // takes a lot of time + * try { + * while (true) { + * emit(receiveMessageFromBackend()) + * } + * } finally { + * disconnectFromBackend() + * } + * } + * ``` + * + * If this flow is directly used in the application, then every time it is collected a fresh connection is + * established, and it will take a while before messages start flowing. However, we can share a single connection + * and establish it eagerly like this: + * + * ``` + * val messages: SharedFlow = backendMessages.shareIn(scope, SharingStarted.Eagerly) + * ``` + * + * Now a single connection is shared between all collectors from `messages`, and there is a chance that the connection + * is already established by the time it is needed. + * + * ### Upstream completion and error handling + * + * **Normal completion of the upstream flow has no effect on subscribers**, and the sharing coroutine continues to run. If a + * strategy like [SharingStarted.WhileSubscribed] is used, then the upstream can get restarted again. If a special + * action on upstream completion is needed, then an [onCompletion] operator can be used before the + * `shareIn` operator to emit a special value in this case, like this: + * + * ``` + * backendMessages + * .onCompletion { cause -> if (cause == null) emit(UpstreamHasCompletedMessage) } + * .shareIn(scope, SharingStarted.Eagerly) + * ``` + * + * Any exception in the upstream flow terminates the sharing coroutine without affecting any of the subscribers, + * and will be handled by the [scope] in which the sharing coroutine is launched. Custom exception handling + * can be configured by using the [catch] or [retry] operators before the `shareIn` operator. + * For example, to retry connection on any `IOException` with 1 second delay between attempts, use: + * + * ``` + * val messages = backendMessages + * .retry { e -> + * val shallRetry = e is IOException // other exception are bugs - handle them + * if (shallRetry) delay(1000) + * shallRetry + * } + * .shareIn(scope, SharingStarted.Eagerly) + * ``` + * + * ### Initial value + * + * When a special initial value is needed to signal to subscribers that the upstream is still loading the data, + * use the [onStart] operator on the upstream flow. For example: + * + * ``` + * backendMessages + * .onStart { emit(UpstreamIsStartingMessage) } + * .shareIn(scope, SharingStarted.Eagerly, 1) // replay one most recent message + * ``` + * + * ### Buffering and conflation + * + * The `shareIn` operator runs the upstream flow in a separate coroutine, and buffers emissions from upstream as explained + * in the [buffer] operator's description, using a buffer of [replay] size or the default (whichever is larger). + * This default buffering can be overridden with an explicit buffer configuration by preceding the `shareIn` call + * with [buffer] or [conflate], for example: + * + * - `buffer(0).shareIn(scope, started, 0)` — overrides the default buffer size and creates a [SharedFlow] without a buffer. + * Effectively, it configures sequential processing between the upstream emitter and subscribers, + * as the emitter is suspended until all subscribers process the value. Note, that the value is still immediately + * discarded when there are no subscribers. + * - `buffer(b).shareIn(scope, started, r)` — creates a [SharedFlow] with `replay = r` and `extraBufferCapacity = b`. + * - `conflate().shareIn(scope, started, r)` — creates a [SharedFlow] with `replay = r`, `onBufferOverflow = DROP_OLDEST`, + * and `extraBufferCapacity = 1` when `replay == 0` to support this strategy. + * + * ### Operator fusion + * + * Application of [flowOn][Flow.flowOn], [buffer] with [RENDEZVOUS][Channel.RENDEZVOUS] capacity, + * or [cancellable] operators to the resulting shared flow has no effect. + * + * ### Exceptions + * + * This function throws [IllegalArgumentException] on unsupported values of parameters or combinations thereof. + * + * @param scope the coroutine scope in which sharing is started. + * @param started the strategy that controls when sharing is started and stopped. + * @param replay the number of values replayed to new subscribers (cannot be negative, defaults to zero). + */ +public fun Flow.shareIn( + scope: CoroutineScope, + started: SharingStarted, + replay: Int = 0 +): SharedFlow { + val config = configureSharing(replay) + val shared = MutableSharedFlow( + replay = replay, + extraBufferCapacity = config.extraBufferCapacity, + onBufferOverflow = config.onBufferOverflow + ) + @Suppress("UNCHECKED_CAST") + val job = scope.launchSharing(config.context, config.upstream, shared, started, NO_VALUE as T) + return ReadonlySharedFlow(shared, job) +} + +private class SharingConfig( + @JvmField val upstream: Flow, + @JvmField val extraBufferCapacity: Int, + @JvmField val onBufferOverflow: BufferOverflow, + @JvmField val context: CoroutineContext +) + +// Decomposes upstream flow to fuse with it when possible +private fun Flow.configureSharing(replay: Int): SharingConfig { + assert { replay >= 0 } + val defaultExtraCapacity = replay.coerceAtLeast(Channel.CHANNEL_DEFAULT_CAPACITY) - replay + // Combine with preceding buffer/flowOn and channel-using operators + if (this is ChannelFlow) { + // Check if this ChannelFlow can operate without a channel + val upstream = dropChannelOperators() + if (upstream != null) { // Yes, it can => eliminate the intermediate channel + return SharingConfig( + upstream = upstream, + extraBufferCapacity = when (capacity) { + Channel.OPTIONAL_CHANNEL, Channel.BUFFERED, 0 -> // handle special capacities + when { + onBufferOverflow == BufferOverflow.SUSPEND -> // buffer was configured with suspension + if (capacity == 0) 0 else defaultExtraCapacity // keep explicitly configured 0 or use default + replay == 0 -> 1 // no suspension => need at least buffer of one + else -> 0 // replay > 0 => no need for extra buffer beyond replay because we don't suspend + } + else -> capacity // otherwise just use the specified capacity as extra capacity + }, + onBufferOverflow = onBufferOverflow, + context = context + ) + } + } + // Add sharing operator on top with a default buffer + return SharingConfig( + upstream = this, + extraBufferCapacity = defaultExtraCapacity, + onBufferOverflow = BufferOverflow.SUSPEND, + context = EmptyCoroutineContext + ) +} + +// Launches sharing coroutine +private fun CoroutineScope.launchSharing( + context: CoroutineContext, + upstream: Flow, + shared: MutableSharedFlow, + started: SharingStarted, + initialValue: T +): Job { + /* + * Conditional start: in the case when sharing and subscribing happens in the same dispatcher, we want to + * have the following invariants preserved: + * - Delayed sharing strategies have a chance to immediately observe consecutive subscriptions. + * E.g. in the cases like `flow.shareIn(...); flow.take(1)` we want sharing strategy to see the initial subscription + * - Eager sharing does not start immediately, so the subscribers have actual chance to subscribe _prior_ to sharing. + */ + val start = if (started == SharingStarted.Eagerly) CoroutineStart.DEFAULT else CoroutineStart.UNDISPATCHED + return launch(context, start = start) { // the single coroutine to rule the sharing + // Optimize common built-in started strategies + when { + started === SharingStarted.Eagerly -> { + // collect immediately & forever + upstream.collect(shared) + } + started === SharingStarted.Lazily -> { + // start collecting on the first subscriber - wait for it first + shared.subscriptionCount.first { it > 0 } + upstream.collect(shared) + } + else -> { + // other & custom strategies + started.command(shared.subscriptionCount) + .distinctUntilChanged() // only changes in command have effect + .collectLatest { // cancels block on new emission + when (it) { + SharingCommand.START -> upstream.collect(shared) // can be cancelled + SharingCommand.STOP -> { /* just cancel and do nothing else */ } + SharingCommand.STOP_AND_RESET_REPLAY_CACHE -> { + if (initialValue === NO_VALUE) { + shared.resetReplayCache() // regular shared flow -> reset cache + } else { + shared.tryEmit(initialValue) // state flow -> reset to initial value + } + } + } + } + } + } + } +} + +// -------------------------------- stateIn -------------------------------- + +/** + * Converts a _cold_ [Flow] into a _hot_ [StateFlow] that is started in the given coroutine [scope], + * sharing the most recently emitted value from a single running instance of the upstream flow with multiple + * downstream subscribers. See the [StateFlow] documentation for the general concepts of state flows. + * + * The starting of the sharing coroutine is controlled by the [started] parameter, as explained in the + * documentation for [shareIn] operator. + * + * The `stateIn` operator is useful in situations when there is a _cold_ flow that provides updates to the + * value of some state and is expensive to create and/or to maintain, but there are multiple subscribers + * that need to collect the most recent state value. For example, consider a + * flow of state updates coming from a backend over the expensive network connection, taking a lot of + * time to establish. Conceptually it might be implemented like this: + * + * ``` + * val backendState: Flow = flow { + * connectToBackend() // takes a lot of time + * try { + * while (true) { + * emit(receiveStateUpdateFromBackend()) + * } + * } finally { + * disconnectFromBackend() + * } + * } + * ``` + * + * If this flow is directly used in the application, then every time it is collected a fresh connection is + * established, and it will take a while before state updates start flowing. However, we can share a single connection + * and establish it eagerly like this: + * + * ``` + * val state: StateFlow = backendMessages.stateIn(scope, SharingStarted.Eagerly, State.LOADING) + * ``` + * + * Now, a single connection is shared between all collectors from `state`, and there is a chance that the connection + * is already established by the time it is needed. + * + * ### Upstream completion and error handling + * + * **Normal completion of the upstream flow has no effect on subscribers**, and the sharing coroutine continues to run. If a + * a strategy like [SharingStarted.WhileSubscribed] is used, then the upstream can get restarted again. If a special + * action on upstream completion is needed, then an [onCompletion] operator can be used before + * the `stateIn` operator to emit a special value in this case. See the [shareIn] operator's documentation for an example. + * + * Any exception in the upstream flow terminates the sharing coroutine without affecting any of the subscribers, + * and will be handled by the [scope] in which the sharing coroutine is launched. Custom exception handling + * can be configured by using the [catch] or [retry] operators before the `stateIn` operator, similarly to + * the [shareIn] operator. + * + * ### Operator fusion + * + * Application of [flowOn][Flow.flowOn], [conflate][Flow.conflate], + * [buffer] with [CONFLATED][Channel.CONFLATED] or [RENDEZVOUS][Channel.RENDEZVOUS] capacity, + * [distinctUntilChanged][Flow.distinctUntilChanged], or [cancellable] operators to a state flow has no effect. + * + * @param scope the coroutine scope in which sharing is started. + * @param started the strategy that controls when sharing is started and stopped. + * @param initialValue the initial value of the state flow. + * This value is also used when the state flow is reset using the [SharingStarted.WhileSubscribed] strategy + * with the `replayExpirationMillis` parameter. + */ +public fun Flow.stateIn( + scope: CoroutineScope, + started: SharingStarted, + initialValue: T +): StateFlow { + val config = configureSharing(1) + val state = MutableStateFlow(initialValue) + val job = scope.launchSharing(config.context, config.upstream, state, started, initialValue) + return ReadonlyStateFlow(state, job) +} + +/** + * Starts the upstream flow in a given [scope], suspends until the first value is emitted, and returns a _hot_ + * [StateFlow] of future emissions, sharing the most recently emitted value from this running instance of the upstream flow + * with multiple downstream subscribers. See the [StateFlow] documentation for the general concepts of state flows. + * + * @param scope the coroutine scope in which sharing is started. + * @throws NoSuchElementException if the upstream flow does not emit any value. + */ +public suspend fun Flow.stateIn(scope: CoroutineScope): StateFlow { + val config = configureSharing(1) + val result = CompletableDeferred>>(scope.coroutineContext[Job]) + scope.launchSharingDeferred(config.context, config.upstream, result) + return result.await().getOrThrow() +} + +private fun CoroutineScope.launchSharingDeferred( + context: CoroutineContext, + upstream: Flow, + result: CompletableDeferred>>, +) { + launch(context) { + try { + var state: MutableStateFlow? = null + upstream.collect { value -> + state?.let { it.value = value } ?: run { + state = MutableStateFlow(value).also { + result.complete(Result.success(ReadonlyStateFlow(it, coroutineContext.job))) + } + } + } + if (state == null) { + result.complete(Result.failure(NoSuchElementException("Flow is empty"))) + } + } catch (e: Throwable) { + // Notify the waiter that the flow has failed + result.completeExceptionally(e) + // But still cancel the scope where state was (not) produced + throw e + } + } +} + +// -------------------------------- asSharedFlow/asStateFlow -------------------------------- + +/** + * Represents this mutable shared flow as a read-only shared flow. + */ +public fun MutableSharedFlow.asSharedFlow(): SharedFlow = + ReadonlySharedFlow(this, null) + +/** + * Represents this mutable state flow as a read-only state flow. + */ +public fun MutableStateFlow.asStateFlow(): StateFlow = + ReadonlyStateFlow(this, null) + +@OptIn(ExperimentalForInheritanceCoroutinesApi::class) +private class ReadonlySharedFlow( + flow: SharedFlow, + @Suppress("unused") + private val job: Job? // keeps a strong reference to the job (if present) +) : SharedFlow by flow, CancellableFlow, FusibleFlow { + override fun fuse(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow) = + fuseSharedFlow(context, capacity, onBufferOverflow) +} + +@OptIn(ExperimentalForInheritanceCoroutinesApi::class) +private class ReadonlyStateFlow( + flow: StateFlow, + @Suppress("unused") + private val job: Job? // keeps a strong reference to the job (if present) +) : StateFlow by flow, CancellableFlow, FusibleFlow { + override fun fuse(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow) = + fuseStateFlow(context, capacity, onBufferOverflow) +} + +// -------------------------------- onSubscription -------------------------------- + +/** + * Returns a flow that invokes the given [action] **after** this shared flow starts to be collected + * (after the subscription is registered). + * + * The [action] is called before any value is emitted from the upstream + * flow to this subscription but after the subscription is established. It is guaranteed that all emissions to + * the upstream flow that happen inside or immediately after this `onSubscription` action will be + * collected by this subscription. + * + * The receiver of the [action] is [FlowCollector], so `onSubscription` can emit additional elements. + */ +public fun SharedFlow.onSubscription(action: suspend FlowCollector.() -> Unit): SharedFlow = + SubscribedSharedFlow(this, action) + +@OptIn(ExperimentalForInheritanceCoroutinesApi::class) +private class SubscribedSharedFlow( + private val sharedFlow: SharedFlow, + private val action: suspend FlowCollector.() -> Unit +) : SharedFlow by sharedFlow { + override suspend fun collect(collector: FlowCollector) = + sharedFlow.collect(SubscribedFlowCollector(collector, action)) +} + +internal class SubscribedFlowCollector( + private val collector: FlowCollector, + private val action: suspend FlowCollector.() -> Unit +) : FlowCollector by collector { + suspend fun onSubscription() { + val safeCollector = SafeCollector(collector, currentCoroutineContext()) + try { + safeCollector.action() + } finally { + safeCollector.releaseIntercepted() + } + if (collector is SubscribedFlowCollector) collector.onSubscription() + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Transform.kt b/kotlinx-coroutines-core/common/src/flow/operators/Transform.kt new file mode 100644 index 0000000000..f3c9be1c7e --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Transform.kt @@ -0,0 +1,166 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") +@file:Suppress("UNCHECKED_CAST") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.internal.* +import kotlin.jvm.* +import kotlin.reflect.* +import kotlinx.coroutines.flow.internal.unsafeFlow as flow +import kotlinx.coroutines.flow.unsafeTransform as transform + +/** + * Returns a flow containing only values of the original flow that match the given [predicate]. + */ +public inline fun Flow.filter(crossinline predicate: suspend (T) -> Boolean): Flow = transform { value -> + if (predicate(value)) return@transform emit(value) +} + +/** + * Returns a flow containing only values of the original flow that do not match the given [predicate]. + */ +public inline fun Flow.filterNot(crossinline predicate: suspend (T) -> Boolean): Flow = transform { value -> + if (!predicate(value)) return@transform emit(value) +} + +/** + * Returns a flow containing only values that are instances of specified type [R]. + */ +@Suppress("UNCHECKED_CAST") +public inline fun Flow<*>.filterIsInstance(): Flow = filter { it is R } as Flow + +/** + * Returns a flow containing only values that are instances of the given [klass]. + */ +public fun Flow<*>.filterIsInstance(klass: KClass): Flow = filter { klass.isInstance(it) } as Flow + +/** + * Returns a flow containing only values of the original flow that are not null. + */ +public fun Flow.filterNotNull(): Flow = transform { value -> + if (value != null) return@transform emit(value) +} + +/** + * Returns a flow containing the results of applying the given [transform] function to each value of the original flow. + */ +public inline fun Flow.map(crossinline transform: suspend (value: T) -> R): Flow = transform { value -> + return@transform emit(transform(value)) +} + +/** + * Returns a flow that contains only non-null results of applying the given [transform] function to each value of the original flow. + */ +public inline fun Flow.mapNotNull(crossinline transform: suspend (value: T) -> R?): Flow = transform { value -> + val transformed = transform(value) ?: return@transform + return@transform emit(transformed) +} + +/** + * Returns a flow that wraps each element into [IndexedValue], containing value and its index (starting from zero). + */ +public fun Flow.withIndex(): Flow> = flow { + var index = 0 + collect { value -> + emit(IndexedValue(checkIndexOverflow(index++), value)) + } +} + +/** + * Returns a flow that invokes the given [action] **before** each value of the upstream flow is emitted downstream. + */ +public fun Flow.onEach(action: suspend (T) -> Unit): Flow = transform { value -> + action(value) + return@transform emit(value) +} + +/** + * Folds the given flow with [operation], emitting every intermediate result, including [initial] value. + * Note that initial value should be immutable (or should not be mutated) as it is shared between different collectors. + * For example: + * ``` + * flowOf(1, 2, 3).scan(emptyList()) { acc, value -> acc + value }.toList() + * ``` + * will produce `[[], [1], [1, 2], [1, 2, 3]]`. + * + * This function is an alias to [runningFold] operator. + */ +public fun Flow.scan(initial: R, @BuilderInference operation: suspend (accumulator: R, value: T) -> R): Flow = runningFold(initial, operation) + +/** + * Folds the given flow with [operation], emitting every intermediate result, including [initial] value. + * Note that initial value should be immutable (or should not be mutated) as it is shared between different collectors. + * For example: + * ``` + * flowOf(1, 2, 3).runningFold(emptyList()) { acc, value -> acc + value }.toList() + * ``` + * will produce `[[], [1], [1, 2], [1, 2, 3]]`. + */ +public fun Flow.runningFold(initial: R, @BuilderInference operation: suspend (accumulator: R, value: T) -> R): Flow = flow { + var accumulator: R = initial + emit(accumulator) + collect { value -> + accumulator = operation(accumulator, value) + emit(accumulator) + } +} + +/** + * Reduces the given flow with [operation], emitting every intermediate result, including initial value. + * The first element is taken as initial value for operation accumulator. + * This operator has a sibling with initial value -- [scan]. + * + * For example: + * ``` + * flowOf(1, 2, 3, 4).runningReduce { acc, value -> acc + value }.toList() + * ``` + * will produce `[1, 3, 6, 10]` + */ +public fun Flow.runningReduce(operation: suspend (accumulator: T, value: T) -> T): Flow = flow { + var accumulator: Any? = NULL + collect { value -> + accumulator = if (accumulator === NULL) { + value + } else { + operation(accumulator as T, value) + } + emit(accumulator as T) + } +} + +/** + * Splits the given flow into a flow of non-overlapping lists each not exceeding the given [size] but never empty. + * The last emitted list may have fewer elements than the given size. + * + * Example of usage: + * ``` + * flowOf("a", "b", "c", "d", "e") + * .chunked(2) // ["a", "b"], ["c", "d"], ["e"] + * .map { it.joinToString(separator = "") } + * .collect { + * println(it) // Prints "ab", "cd", e" + * } + * ``` + * + * @throws IllegalArgumentException if [size] is not positive. + */ +@ExperimentalCoroutinesApi +public fun Flow.chunked(size: Int): Flow> { + require(size >= 1) { "Expected positive chunk size, but got $size" } + return flow { + var result: ArrayList? = null // Do not preallocate anything + collect { value -> + // Allocate if needed + val acc = result ?: ArrayList(size).also { result = it } + acc.add(value) + if (acc.size == size) { + emit(acc) + // Cleanup, but don't allocate -- it might've been the case this is the last element + result = null + } + } + result?.let { emit(it) } + } +} diff --git a/kotlinx-coroutines-core/common/src/flow/operators/Zip.kt b/kotlinx-coroutines-core/common/src/flow/operators/Zip.kt new file mode 100644 index 0000000000..3b9237eda8 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/operators/Zip.kt @@ -0,0 +1,327 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") +@file:Suppress("UNCHECKED_CAST") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.flow.internal.* +import kotlin.jvm.* +import kotlinx.coroutines.flow.flow as safeFlow +import kotlinx.coroutines.flow.internal.unsafeFlow as flow + +/** + * Returns a [Flow] whose values are generated with [transform] function by combining + * the most recently emitted values by each flow. + * + * It can be demonstrated with the following example: + * ``` + * val flow = flowOf(1, 2).onEach { delay(10) } + * val flow2 = flowOf("a", "b", "c").onEach { delay(15) } + * flow.combine(flow2) { i, s -> i.toString() + s }.collect { + * println(it) // Will print "1a 2a 2b 2c" + * } + * ``` + * + * This function is a shorthand for `flow.combineTransform(flow2) { a, b -> emit(transform(a, b)) } + */ +@JvmName("flowCombine") +public fun Flow.combine(flow: Flow, transform: suspend (a: T1, b: T2) -> R): Flow = flow { + combineInternal(arrayOf(this@combine, flow), nullArrayFactory(), { emit(transform(it[0] as T1, it[1] as T2)) }) +} + +/** + * Returns a [Flow] whose values are generated with [transform] function by combining + * the most recently emitted values by each flow. + * + * It can be demonstrated with the following example: + * ``` + * val flow = flowOf(1, 2).onEach { delay(10) } + * val flow2 = flowOf("a", "b", "c").onEach { delay(15) } + * combine(flow, flow2) { i, s -> i.toString() + s }.collect { + * println(it) // Will print "1a 2a 2b 2c" + * } + * ``` + * + * This function is a shorthand for `combineTransform(flow, flow2) { a, b -> emit(transform(a, b)) } + */ +public fun combine(flow: Flow, flow2: Flow, transform: suspend (a: T1, b: T2) -> R): Flow = + flow.combine(flow2, transform) + +/** + * Returns a [Flow] whose values are generated by [transform] function that process the most recently emitted values by each flow. + * + * The receiver of the [transform] is [FlowCollector] and thus `transform` is a + * generic function that may transform emitted element, skip it or emit it multiple times. + * + * Its usage can be demonstrated with the following example: + * ``` + * val flow = requestFlow() + * val flow2 = searchEngineFlow() + * flow.combineTransform(flow2) { request, searchEngine -> + * emit("Downloading in progress") + * val result = download(request, searchEngine) + * emit(result) + * } + * ``` + */ +@JvmName("flowCombineTransform") +public fun Flow.combineTransform( + flow: Flow, + @BuilderInference transform: suspend FlowCollector.(a: T1, b: T2) -> Unit +): Flow = combineTransformUnsafe(this, flow) { args: Array<*> -> + transform( + args[0] as T1, + args[1] as T2 + ) +} + +/** + * Returns a [Flow] whose values are generated by [transform] function that process the most recently emitted values by each flow. + * + * The receiver of the [transform] is [FlowCollector] and thus `transform` is a + * generic function that may transform emitted element, skip it or emit it multiple times. + * + * Its usage can be demonstrated with the following example: + * ``` + * val flow = requestFlow() + * val flow2 = searchEngineFlow() + * combineTransform(flow, flow2) { request, searchEngine -> + * emit("Downloading in progress") + * val result = download(request, searchEngine) + * emit(result) + * } + * ``` + */ +public fun combineTransform( + flow: Flow, + flow2: Flow, + @BuilderInference transform: suspend FlowCollector.(a: T1, b: T2) -> Unit +): Flow = combineTransformUnsafe(flow, flow2) { args: Array<*> -> + transform( + args[0] as T1, + args[1] as T2 + ) +} + +/** + * Returns a [Flow] whose values are generated with [transform] function by combining + * the most recently emitted values by each flow. + */ +public fun combine( + flow: Flow, + flow2: Flow, + flow3: Flow, + @BuilderInference transform: suspend (T1, T2, T3) -> R +): Flow = combineUnsafe(flow, flow2, flow3) { args: Array<*> -> + transform( + args[0] as T1, + args[1] as T2, + args[2] as T3 + ) +} + +/** + * Returns a [Flow] whose values are generated by [transform] function that process the most recently emitted values by each flow. + * + * The receiver of the [transform] is [FlowCollector] and thus `transform` is a + * generic function that may transform emitted element, skip it or emit it multiple times. + */ +public fun combineTransform( + flow: Flow, + flow2: Flow, + flow3: Flow, + @BuilderInference transform: suspend FlowCollector.(T1, T2, T3) -> Unit +): Flow = combineTransformUnsafe(flow, flow2, flow3) { args: Array<*> -> + transform( + args[0] as T1, + args[1] as T2, + args[2] as T3 + ) +} + +/** + * Returns a [Flow] whose values are generated with [transform] function by combining + * the most recently emitted values by each flow. + */ +public fun combine( + flow: Flow, + flow2: Flow, + flow3: Flow, + flow4: Flow, + transform: suspend (T1, T2, T3, T4) -> R +): Flow = combineUnsafe(flow, flow2, flow3, flow4) { args: Array<*> -> + transform( + args[0] as T1, + args[1] as T2, + args[2] as T3, + args[3] as T4 + ) +} + +/** + * Returns a [Flow] whose values are generated by [transform] function that process the most recently emitted values by each flow. + * + * The receiver of the [transform] is [FlowCollector] and thus `transform` is a + * generic function that may transform emitted element, skip it or emit it multiple times. + */ +public fun combineTransform( + flow: Flow, + flow2: Flow, + flow3: Flow, + flow4: Flow, + @BuilderInference transform: suspend FlowCollector.(T1, T2, T3, T4) -> Unit +): Flow = combineTransformUnsafe(flow, flow2, flow3, flow4) { args: Array<*> -> + transform( + args[0] as T1, + args[1] as T2, + args[2] as T3, + args[3] as T4 + ) +} + +/** + * Returns a [Flow] whose values are generated with [transform] function by combining + * the most recently emitted values by each flow. + */ +public fun combine( + flow: Flow, + flow2: Flow, + flow3: Flow, + flow4: Flow, + flow5: Flow, + transform: suspend (T1, T2, T3, T4, T5) -> R +): Flow = combineUnsafe(flow, flow2, flow3, flow4, flow5) { args: Array<*> -> + transform( + args[0] as T1, + args[1] as T2, + args[2] as T3, + args[3] as T4, + args[4] as T5 + ) +} + +/** + * Returns a [Flow] whose values are generated by [transform] function that process the most recently emitted values by each flow. + * + * The receiver of the [transform] is [FlowCollector] and thus `transform` is a + * generic function that may transform emitted element, skip it or emit it multiple times. + */ +public fun combineTransform( + flow: Flow, + flow2: Flow, + flow3: Flow, + flow4: Flow, + flow5: Flow, + @BuilderInference transform: suspend FlowCollector.(T1, T2, T3, T4, T5) -> Unit +): Flow = combineTransformUnsafe(flow, flow2, flow3, flow4, flow5) { args: Array<*> -> + transform( + args[0] as T1, + args[1] as T2, + args[2] as T3, + args[3] as T4, + args[4] as T5 + ) +} + +/** + * Returns a [Flow] whose values are generated with [transform] function by combining + * the most recently emitted values by each flow. + */ +public inline fun combine( + vararg flows: Flow, + crossinline transform: suspend (Array) -> R +): Flow = flow { + combineInternal(flows, { arrayOfNulls(flows.size) }, { emit(transform(it)) }) +} + +/** + * Returns a [Flow] whose values are generated by [transform] function that process the most recently emitted values by each flow. + * + * The receiver of the [transform] is [FlowCollector] and thus `transform` is a + * generic function that may transform emitted element, skip it or emit it multiple times. + */ +public inline fun combineTransform( + vararg flows: Flow, + @BuilderInference crossinline transform: suspend FlowCollector.(Array) -> Unit +): Flow = safeFlow { + combineInternal(flows, { arrayOfNulls(flows.size) }, { transform(it) }) +} + +/* + * Same as combine, but does not copy array each time, deconstructing existing + * array each time. Used in overloads that accept FunctionN instead of Function> + */ +private inline fun combineUnsafe( + vararg flows: Flow, + crossinline transform: suspend (Array) -> R +): Flow = flow { + combineInternal(flows, nullArrayFactory(), { emit(transform(it)) }) +} + +/* + * Same as combineTransform, but does not copy array each time, deconstructing existing + * array each time. Used in overloads that accept FunctionN instead of Function> + */ +private inline fun combineTransformUnsafe( + vararg flows: Flow, + @BuilderInference crossinline transform: suspend FlowCollector.(Array) -> Unit +): Flow = safeFlow { + combineInternal(flows, nullArrayFactory(), { transform(it) }) +} + +// Saves bunch of anonymous classes +private fun nullArrayFactory(): () -> Array? = { null } + +/** + * Returns a [Flow] whose values are generated with [transform] function by combining + * the most recently emitted values by each flow. + */ +public inline fun combine( + flows: Iterable>, + crossinline transform: suspend (Array) -> R +): Flow { + val flowArray = flows.toList().toTypedArray() + return flow { + combineInternal( + flowArray, + arrayFactory = { arrayOfNulls(flowArray.size) }, + transform = { emit(transform(it)) }) + } +} + +/** + * Returns a [Flow] whose values are generated by [transform] function that process the most recently emitted values by each flow. + * + * The receiver of the [transform] is [FlowCollector] and thus `transform` is a + * generic function that may transform emitted element, skip it or emit it multiple times. + */ +public inline fun combineTransform( + flows: Iterable>, + @BuilderInference crossinline transform: suspend FlowCollector.(Array) -> Unit +): Flow { + val flowArray = flows.toList().toTypedArray() + return safeFlow { + combineInternal(flowArray, { arrayOfNulls(flowArray.size) }, { transform(it) }) + } +} + +/** + * Zips values from the current flow (`this`) with [other] flow using provided [transform] function applied to each pair of values. + * The resulting flow completes as soon as one of the flows completes and cancel is called on the remaining flow. + * + * It can be demonstrated with the following example: + * ``` + * val flow = flowOf(1, 2, 3).onEach { delay(10) } + * val flow2 = flowOf("a", "b", "c", "d").onEach { delay(15) } + * flow.zip(flow2) { i, s -> i.toString() + s }.collect { + * println(it) // Will print "1a 2b 3c" + * } + * ``` + * + * ### Buffering + * + * The upstream flow is collected sequentially in the same coroutine without any buffering, while the + * [other] flow is collected concurrently as if `buffer(0)` is used. See documentation in the [buffer] operator + * for explanation. You can use additional calls to the [buffer] operator as needed for more concurrency. + */ +public fun Flow.zip(other: Flow, transform: suspend (T1, T2) -> R): Flow = zipImpl(this, other, transform) diff --git a/kotlinx-coroutines-core/common/src/flow/terminal/Collect.kt b/kotlinx-coroutines-core/common/src/flow/terminal/Collect.kt new file mode 100644 index 0000000000..f55970e3fb --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/terminal/Collect.kt @@ -0,0 +1,113 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.internal.* +import kotlin.jvm.* + +/** + * Terminal flow operator that collects the given flow but ignores all emitted values. + * If any exception occurs during collect or in the provided flow, this exception is rethrown from this method. + * + * It is a shorthand for `collect {}`. + * + * This operator is usually used with [onEach], [onCompletion] and [catch] operators to process all emitted values and + * handle an exception that might occur in the upstream flow or during processing, for example: + * + * ``` + * flow + * .onEach { value -> process(value) } + * .catch { e -> handleException(e) } + * .collect() // trigger collection of the flow + * ``` + */ +public suspend fun Flow<*>.collect(): Unit = collect(NopCollector) + +/** + * Terminal flow operator that [launches][launch] the [collection][collect] of the given flow in the [scope]. + * It is a shorthand for `scope.launch { flow.collect() }`. + * + * This operator is usually used with [onEach], [onCompletion] and [catch] operators to process all emitted values + * handle an exception that might occur in the upstream flow or during processing, for example: + * + * ``` + * flow + * .onEach { value -> updateUi(value) } + * .onCompletion { cause -> updateUi(if (cause == null) "Done" else "Failed") } + * .catch { cause -> LOG.error("Exception: $cause") } + * .launchIn(uiScope) + * ``` + * + * In this example, note that the `job` returned by [launchIn] is not used, and the provided scope takes care of cancellation. + */ +public fun Flow.launchIn(scope: CoroutineScope): Job = scope.launch { + collect() // tail-call +} + +/** + * Terminal flow operator that collects the given flow with a provided [action] that takes the index of an element (zero-based) and the element. + * If any exception occurs during collect or in the provided flow, this exception is rethrown from this method. + * + * See also [collect] and [withIndex]. + */ +public suspend inline fun Flow.collectIndexed(crossinline action: suspend (index: Int, value: T) -> Unit): Unit = + collect(object : FlowCollector { + private var index = 0 + override suspend fun emit(value: T) = action(checkIndexOverflow(index++), value) + }) + +/** + * Terminal flow operator that collects the given flow with a provided [action]. + * The crucial difference from [collect] is that when the original flow emits a new value + * then the [action] block for the previous value is cancelled. + * + * It can be demonstrated by the following example: + * + * ``` + * flow { + * emit(1) + * delay(50) + * emit(2) + * }.collectLatest { value -> + * println("Collecting $value") + * delay(100) // Emulate work + * println("$value collected") + * } + * ``` + * + * prints "Collecting 1, Collecting 2, 2 collected" + */ +public suspend fun Flow.collectLatest(action: suspend (value: T) -> Unit) { + /* + * Implementation note: + * buffer(0) is inserted here to fulfil user's expectations in sequential usages, e.g.: + * ``` + * flowOf(1, 2, 3).collectLatest { + * delay(1) + * println(it) // Expect only 3 to be printed + * } + * ``` + * + * It's not the case for intermediate operators which users mostly use for interactive UI, + * where performance of dispatch is more important. + */ + mapLatest(action).buffer(0).collect() +} + +/** + * Collects all the values from the given [flow] and emits them to the collector. + * It is a shorthand for `flow.collect { value -> emit(value) }`. + */ +public suspend fun FlowCollector.emitAll(flow: Flow) { + ensureActive() + flow.collect(this) +} + +/** @suppress */ +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Backwards compatibility with JS and K/N") +public suspend inline fun Flow.collect(crossinline action: suspend (value: T) -> Unit): Unit = + collect(object : FlowCollector { + override suspend fun emit(value: T) = action(value) + }) diff --git a/kotlinx-coroutines-core/common/src/flow/terminal/Collection.kt b/kotlinx-coroutines-core/common/src/flow/terminal/Collection.kt new file mode 100644 index 0000000000..33057124e9 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/terminal/Collection.kt @@ -0,0 +1,26 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlin.jvm.* + +/** + * Collects given flow into a [destination] + */ +public suspend fun Flow.toList(destination: MutableList = ArrayList()): List = toCollection(destination) + +/** + * Collects given flow into a [destination] + */ +public suspend fun Flow.toSet(destination: MutableSet = LinkedHashSet()): Set = toCollection(destination) + +/** + * Collects given flow into a [destination] + */ +public suspend fun > Flow.toCollection(destination: C): C { + collect { value -> + destination.add(value) + } + return destination +} diff --git a/kotlinx-coroutines-core/common/src/flow/terminal/Count.kt b/kotlinx-coroutines-core/common/src/flow/terminal/Count.kt new file mode 100644 index 0000000000..cfa11bce1c --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/terminal/Count.kt @@ -0,0 +1,32 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlin.jvm.* + +/** + * Returns the number of elements in this flow. + */ +public suspend fun Flow.count(): Int { + var i = 0 + collect { + ++i + } + + return i +} + +/** + * Returns the number of elements matching the given predicate. + */ +public suspend fun Flow.count(predicate: suspend (T) -> Boolean): Int { + var i = 0 + collect { value -> + if (predicate(value)) { + ++i + } + } + + return i +} diff --git a/kotlinx-coroutines-core/common/src/flow/terminal/Logic.kt b/kotlinx-coroutines-core/common/src/flow/terminal/Logic.kt new file mode 100644 index 0000000000..6d1cd6fee9 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/terminal/Logic.kt @@ -0,0 +1,107 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlin.jvm.* + + +/** + * A terminal operator that returns `true` and immediately cancels the flow + * if at least one element matches the given [predicate]. + * + * If the flow does not emit any elements or no element matches the predicate, the function returns `false`. + * + * Equivalent to `!all { !predicate(it) }` (see [Flow.all]) and `!none { predicate(it) }` (see [Flow.none]). + * + * Example: + * + * ``` + * val myFlow = flow { + * repeat(10) { + * emit(it) + * } + * throw RuntimeException("You still didn't find the required number? I gave you ten!") + * } + * println(myFlow.any { it > 5 }) // true + * println(flowOf(1, 2, 3).any { it > 5 }) // false + * ``` + * + * @see Iterable.any + * @see Sequence.any + */ +public suspend fun Flow.any(predicate: suspend (T) -> Boolean): Boolean { + var found = false + collectWhile { + val satisfies = predicate(it) + if (satisfies) found = true + !satisfies + } + return found +} + +/** + * A terminal operator that returns `true` if all elements match the given [predicate], + * or returns `false` and cancels the flow as soon as the first element not matching the predicate is encountered. + * + * If the flow terminates without emitting any elements, the function returns `true` because there + * are no elements in it that *do not* match the predicate. + * See a more detailed explanation of this logic concept in the + * ["Vacuous truth"](https://en.wikipedia.org/wiki/Vacuous_truth) article. + * + * Equivalent to `!any { !predicate(it) }` (see [Flow.any]) and `none { !predicate(it) }` (see [Flow.none]). + * + * Example: + * + * ``` + * val myFlow = flow { + * repeat(10) { + * emit(it) + * } + * throw RuntimeException("You still didn't find the required number? I gave you ten!") + * } + * println(myFlow.all { it <= 5 }) // false + * println(flowOf(1, 2, 3).all { it <= 5 }) // true + * ``` + * + * @see Iterable.all + * @see Sequence.all + */ +public suspend fun Flow.all(predicate: suspend (T) -> Boolean): Boolean { + var foundCounterExample = false + collectWhile { + val satisfies = predicate(it) + if (!satisfies) foundCounterExample = true + satisfies + } + return !foundCounterExample +} + +/** + * A terminal operator that returns `true` if no elements match the given [predicate], + * or returns `false` and cancels the flow as soon as the first element matching the predicate is encountered. + * + * If the flow terminates without emitting any elements, the function returns `true` because there + * are no elements in it that match the predicate. + * See a more detailed explanation of this logic concept in the + * ["Vacuous truth"](https://en.wikipedia.org/wiki/Vacuous_truth) article. + * + * Equivalent to `!any(predicate)` (see [Flow.any]) and `all { !predicate(it) }` (see [Flow.all]). + * + * Example: + * ``` + * val myFlow = flow { + * repeat(10) { + * emit(it) + * } + * throw RuntimeException("You still didn't find the required number? I gave you ten!") + * } + * println(myFlow.none { it > 5 }) // false + * println(flowOf(1, 2, 3).none { it > 5 }) // true + * ``` + * + * @see Iterable.none + * @see Sequence.none + */ +public suspend fun Flow.none(predicate: suspend (T) -> Boolean): Boolean = !any(predicate) diff --git a/kotlinx-coroutines-core/common/src/flow/terminal/Reduce.kt b/kotlinx-coroutines-core/common/src/flow/terminal/Reduce.kt new file mode 100644 index 0000000000..fae4525c64 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/flow/terminal/Reduce.kt @@ -0,0 +1,167 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") +@file:Suppress("UNCHECKED_CAST") + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.flow.internal.* +import kotlinx.coroutines.internal.Symbol +import kotlin.jvm.* + +/** + * Accumulates value starting with the first element and applying [operation] to current accumulator value and each element. + * Throws [NoSuchElementException] if flow was empty. + */ +public suspend fun Flow.reduce(operation: suspend (accumulator: S, value: T) -> S): S { + var accumulator: Any? = NULL + + collect { value -> + accumulator = if (accumulator !== NULL) { + @Suppress("UNCHECKED_CAST") + operation(accumulator as S, value) + } else { + value + } + } + + if (accumulator === NULL) throw NoSuchElementException("Empty flow can't be reduced") + @Suppress("UNCHECKED_CAST") + return accumulator as S +} + +/** + * Accumulates value starting with [initial] value and applying [operation] current accumulator value and each element + */ +public suspend inline fun Flow.fold( + initial: R, + crossinline operation: suspend (acc: R, value: T) -> R +): R { + var accumulator = initial + collect { value -> + accumulator = operation(accumulator, value) + } + return accumulator +} + +/** + * The terminal operator that awaits for one and only one value to be emitted. + * Throws [NoSuchElementException] for empty flow and [IllegalArgumentException] for flow + * that contains more than one element. + */ +public suspend fun Flow.single(): T { + var result: Any? = NULL + collect { value -> + require(result === NULL) { "Flow has more than one element" } + result = value + } + + if (result === NULL) throw NoSuchElementException("Flow is empty") + return result as T +} + +/** + * The terminal operator that awaits for one and only one value to be emitted. + * Returns the single value or `null`, if the flow was empty or emitted more than one value. + */ +public suspend fun Flow.singleOrNull(): T? { + var result: Any? = NULL + collectWhile { + // No values yet, update result + if (result === NULL) { + result = it + true + } else { + // Second value, reset result and bail out + result = NULL + false + } + } + return if (result === NULL) null else result as T +} + +/** + * The terminal operator that returns the first element emitted by the flow and then cancels flow's collection. + * Throws [NoSuchElementException] if the flow was empty. + */ +public suspend fun Flow.first(): T { + var result: Any? = NULL + collectWhile { + result = it + false + } + if (result === NULL) throw NoSuchElementException("Expected at least one element") + return result as T +} + +/** + * The terminal operator that returns the first element emitted by the flow matching the given [predicate] and then cancels flow's collection. + * Throws [NoSuchElementException] if the flow has not contained elements matching the [predicate]. + */ +public suspend fun Flow.first(predicate: suspend (T) -> Boolean): T { + var result: Any? = NULL + collectWhile { + if (predicate(it)) { + result = it + false + } else { + true + } + } + if (result === NULL) throw NoSuchElementException("Expected at least one element matching the predicate") + return result as T +} + +/** + * The terminal operator that returns the first element emitted by the flow and then cancels flow's collection. + * Returns `null` if the flow was empty. + */ +public suspend fun Flow.firstOrNull(): T? { + var result: T? = null + collectWhile { + result = it + false + } + return result +} + +/** + * The terminal operator that returns the first element emitted by the flow matching the given [predicate] and then cancels flow's collection. + * Returns `null` if the flow did not contain an element matching the [predicate]. + */ +public suspend fun Flow.firstOrNull(predicate: suspend (T) -> Boolean): T? { + var result: T? = null + collectWhile { + if (predicate(it)) { + result = it + false + } else { + true + } + } + return result +} + +/** + * The terminal operator that returns the last element emitted by the flow. + * + * Throws [NoSuchElementException] if the flow was empty. + */ +public suspend fun Flow.last(): T { + var result: Any? = NULL + collect { + result = it + } + if (result === NULL) throw NoSuchElementException("Expected at least one element") + return result as T +} + +/** + * The terminal operator that returns the last element emitted by the flow or `null` if the flow was empty. + */ +public suspend fun Flow.lastOrNull(): T? { + var result: T? = null + collect { + result = it + } + return result +} diff --git a/kotlinx-coroutines-core/common/src/internal/Concurrent.common.kt b/kotlinx-coroutines-core/common/src/internal/Concurrent.common.kt new file mode 100644 index 0000000000..0be8a104db --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/Concurrent.common.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.internal + +internal expect class ReentrantLock() { + fun tryLock(): Boolean + fun unlock() +} + +internal expect inline fun ReentrantLock.withLock(action: () -> T): T + +internal expect fun identitySet(expectedSize: Int): MutableSet + +/** + * Annotation indicating that the marked property is the subject of benign data race. + * LLVM does not support this notion, so on K/N platforms we alias it into `@Volatile` to prevent potential OoTA. + * + * The purpose of this annotation is not to save an extra-volatile on JVM platform, but rather to explicitly emphasize + * that data-race is benign. + */ +@OptionalExpectation +@Target(AnnotationTarget.FIELD) +internal expect annotation class BenignDataRace() + +// Used **only** as a workaround for #3820 in StateFlow. Do not use anywhere else +internal expect class WorkaroundAtomicReference(value: V) { + public fun get(): V + public fun set(value: V) + public fun getAndSet(value: V): V + public fun compareAndSet(expected: V, value: V): Boolean +} + +@Suppress("UNUSED_PARAMETER", "EXTENSION_SHADOWED_BY_MEMBER") +internal var WorkaroundAtomicReference.value: T + get() = this.get() + set(value) = this.set(value) + +internal inline fun WorkaroundAtomicReference.loop(action: WorkaroundAtomicReference.(value: T) -> Unit) { + while (true) { + action(value) + } +} diff --git a/kotlinx-coroutines-core/common/src/internal/ConcurrentLinkedList.kt b/kotlinx-coroutines-core/common/src/internal/ConcurrentLinkedList.kt new file mode 100644 index 0000000000..8f9b13f4ab --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/ConcurrentLinkedList.kt @@ -0,0 +1,264 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.jvm.* + +/** + * Returns the first segment `s` with `s.id >= id` or `CLOSED` + * if all the segments in this linked list have lower `id`, and the list is closed for further segment additions. + */ +internal fun > S.findSegmentInternal( + id: Long, + createNewSegment: (id: Long, prev: S) -> S +): SegmentOrClosed { + /* + Go through `next` references and add new segments if needed, similarly to the `push` in the Michael-Scott + queue algorithm. The only difference is that "CAS failure" means that the required segment has already been + added, so the algorithm just uses it. This way, only one segment with each id can be added. + */ + var cur: S = this + while (cur.id < id || cur.isRemoved) { + val next = cur.nextOrIfClosed { return SegmentOrClosed(CLOSED) } + if (next != null) { // there is a next node -- move there + cur = next + continue + } + val newTail = createNewSegment(cur.id + 1, cur) + if (cur.trySetNext(newTail)) { // successfully added new node -- move there + if (cur.isRemoved) cur.remove() + cur = newTail + } + } + return SegmentOrClosed(cur) +} + +/** + * Returns `false` if the segment `to` is logically removed, `true` on a successful update. + */ +@Suppress("NOTHING_TO_INLINE", "RedundantNullableReturnType") // Must be inline because it is an AtomicRef extension +internal inline fun > AtomicRef.moveForward(to: S): Boolean = loop { cur -> + if (cur.id >= to.id) return true + if (!to.tryIncPointers()) return false + if (compareAndSet(cur, to)) { // the segment is moved + if (cur.decPointers()) cur.remove() + return true + } + if (to.decPointers()) to.remove() // undo tryIncPointers +} + +/** + * Tries to find a segment with the specified [id] following by next references from the + * [startFrom] segment and creating new ones if needed. The typical use-case is reading this `AtomicRef` values, + * doing some synchronization, and invoking this function to find the required segment and update the pointer. + * At the same time, [Segment.cleanPrev] should also be invoked if the previous segments are no longer needed + * (e.g., queues should use it in dequeue operations). + * + * Since segments can be removed from the list, or it can be closed for further segment additions. + * Returns the segment `s` with `s.id >= id` or `CLOSED` if all the segments in this linked list have lower `id`, + * and the list is closed. + */ +@Suppress("NOTHING_TO_INLINE") +internal inline fun > AtomicRef.findSegmentAndMoveForward( + id: Long, + startFrom: S, + noinline createNewSegment: (id: Long, prev: S) -> S +): SegmentOrClosed { + while (true) { + val s = startFrom.findSegmentInternal(id, createNewSegment) + if (s.isClosed || moveForward(s.segment)) return s + } +} + +/** + * Closes this linked list of nodes by forbidding adding new ones, + * returns the last node in the list. + */ +internal fun > N.close(): N { + var cur: N = this + while (true) { + val next = cur.nextOrIfClosed { return cur } + if (next === null) { + if (cur.markAsClosed()) return cur + } else { + cur = next + } + } +} + +internal abstract class ConcurrentLinkedListNode>(prev: N?) { + // Pointer to the next node, updates similarly to the Michael-Scott queue algorithm. + private val _next = atomic(null) + // Pointer to the previous node, updates in [remove] function. + private val _prev = atomic(prev) + + private val nextOrClosed get() = _next.value + + /** + * Returns the next segment or `null` of the one does not exist, + * and invokes [onClosedAction] if this segment is marked as closed. + */ + @Suppress("UNCHECKED_CAST") + inline fun nextOrIfClosed(onClosedAction: () -> Nothing): N? = nextOrClosed.let { + if (it === CLOSED) { + onClosedAction() + } else { + it as N? + } + } + + val next: N? get() = nextOrIfClosed { return null } + + /** + * Tries to set the next segment if it is not specified and this segment is not marked as closed. + */ + fun trySetNext(value: N): Boolean = _next.compareAndSet(null, value) + + /** + * Checks whether this node is the physical tail of the current linked list. + */ + val isTail: Boolean get() = next == null + + val prev: N? get() = _prev.value + + /** + * Cleans the pointer to the previous node. + */ + fun cleanPrev() { _prev.lazySet(null) } + + /** + * Tries to mark the linked list as closed by forbidding adding new nodes after this one. + */ + fun markAsClosed() = _next.compareAndSet(null, CLOSED) + + /** + * This property indicates whether the current node is logically removed. + * The expected use-case is removing the node logically (so that [isRemoved] becomes true), + * and invoking [remove] after that. Note that this implementation relies on the contract + * that the physical tail cannot be logically removed. Please, do not break this contract; + * otherwise, memory leaks and unexpected behavior can occur. + */ + abstract val isRemoved: Boolean + + /** + * Removes this node physically from this linked list. The node should be + * logically removed (so [isRemoved] returns `true`) at the point of invocation. + */ + fun remove() { + assert { isRemoved || isTail } // The node should be logically removed at first. + // The physical tail cannot be removed. Instead, we remove it when + // a new segment is added and this segment is not the tail one anymore. + if (isTail) return + while (true) { + // Read `next` and `prev` pointers ignoring logically removed nodes. + val prev = aliveSegmentLeft + val next = aliveSegmentRight + // Link `next` and `prev`. + next._prev.update { if (it === null) null else prev } + if (prev !== null) prev._next.value = next + // Checks that prev and next are still alive. + if (next.isRemoved && !next.isTail) continue + if (prev !== null && prev.isRemoved) continue + // This node is removed. + return + } + } + + private val aliveSegmentLeft: N? get() { + var cur = prev + while (cur !== null && cur.isRemoved) + cur = cur._prev.value + return cur + } + + private val aliveSegmentRight: N get() { + assert { !isTail } // Should not be invoked on the tail node + var cur = next!! + while (cur.isRemoved) + cur = cur.next ?: return cur + return cur + } +} + +/** + * Each segment in the list has a unique id and is created by the provided to [findSegmentAndMoveForward] method. + * Essentially, this is a node in the Michael-Scott queue algorithm, + * but with maintaining [prev] pointer for efficient [remove] implementation. + * + * NB: this class cannot be public or leak into user's code as public type as [CancellableContinuationImpl] + * instance-check it and uses a separate code-path for that. + */ +internal abstract class Segment>( + @JvmField val id: Long, prev: S?, pointers: Int +) : ConcurrentLinkedListNode(prev), + // Segments typically store waiting continuations. Thus, on cancellation, the corresponding + // slot should be cleaned and the segment should be removed if it becomes full of cancelled cells. + // To install such a handler efficiently, without creating an extra object, we allow storing + // segments as cancellation handlers in [CancellableContinuationImpl] state, putting the slot + // index in another field. The details are here: https://github.com/Kotlin/kotlinx.coroutines/pull/3084. + // For that, we need segments to implement this internal marker interface. + NotCompleted +{ + /** + * This property should return the number of slots in this segment, + * it is used to define whether the segment is logically removed. + */ + abstract val numberOfSlots: Int + + /** + * Numbers of cleaned slots (the lowest bits) and AtomicRef pointers to this segment (the highest bits) + */ + private val cleanedAndPointers = atomic(pointers shl POINTERS_SHIFT) + + /** + * The segment is considered as removed if all the slots are cleaned + * and there are no pointers to this segment from outside. + */ + override val isRemoved get() = cleanedAndPointers.value == numberOfSlots && !isTail + + // increments the number of pointers if this segment is not logically removed. + internal fun tryIncPointers() = cleanedAndPointers.addConditionally(1 shl POINTERS_SHIFT) { it != numberOfSlots || isTail } + + // returns `true` if this segment is logically removed after the decrement. + internal fun decPointers() = cleanedAndPointers.addAndGet(-(1 shl POINTERS_SHIFT)) == numberOfSlots && !isTail + + /** + * This function is invoked on continuation cancellation when this segment + * with the specified [index] are installed as cancellation handler via + * `SegmentDisposable.disposeOnCancellation(Segment, Int)`. + * + * @param index the index under which the sement registered itself in the continuation. + * Indicies are opaque and arithmetics or numeric intepretation is not allowed on them, + * as they may encode additional metadata. + * @param cause the cause of the cancellation, with the same semantics as [CancellableContinuation.invokeOnCancellation] + * @param context the context of the cancellable continuation the segment was registered in + */ + abstract fun onCancellation(index: Int, cause: Throwable?, context: CoroutineContext) + + /** + * Invoked on each slot clean-up; should not be invoked twice for the same slot. + */ + fun onSlotCleaned() { + if (cleanedAndPointers.incrementAndGet() == numberOfSlots) remove() + } +} + +private inline fun AtomicInt.addConditionally(delta: Int, condition: (cur: Int) -> Boolean): Boolean { + while (true) { + val cur = this.value + if (!condition(cur)) return false + if (this.compareAndSet(cur, cur + delta)) return true + } +} + +@JvmInline +internal value class SegmentOrClosed>(private val value: Any?) { + val isClosed: Boolean get() = value === CLOSED + @Suppress("UNCHECKED_CAST") + val segment: S get() = if (value === CLOSED) error("Does not contain segment") else value as S +} + +private const val POINTERS_SHIFT = 16 + +private val CLOSED = Symbol("CLOSED") diff --git a/kotlinx-coroutines-core/common/src/internal/CoroutineExceptionHandlerImpl.common.kt b/kotlinx-coroutines-core/common/src/internal/CoroutineExceptionHandlerImpl.common.kt new file mode 100644 index 0000000000..25a3a2684d --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/CoroutineExceptionHandlerImpl.common.kt @@ -0,0 +1,68 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * The list of globally installed [CoroutineExceptionHandler] instances that will be notified of any exceptions that + * were not processed in any other manner. + */ +internal expect val platformExceptionHandlers: Collection + +/** + * Ensures that the given [callback] is present in the [platformExceptionHandlers] list. + */ +internal expect fun ensurePlatformExceptionHandlerLoaded(callback: CoroutineExceptionHandler) + +/** + * The platform-dependent global exception handler, used so that the exception is logged at least *somewhere*. + */ +internal expect fun propagateExceptionFinalResort(exception: Throwable) + +/** + * Deal with exceptions that happened in coroutines and weren't programmatically dealt with. + * + * First, it notifies every [CoroutineExceptionHandler] in the [platformExceptionHandlers] list. + * If one of them throws [ExceptionSuccessfullyProcessed], it means that that handler believes that the exception was + * dealt with sufficiently well and doesn't need any further processing. + * Otherwise, the platform-dependent global exception handler is also invoked. + */ +internal fun handleUncaughtCoroutineException(context: CoroutineContext, exception: Throwable) { + // use additional extension handlers + for (handler in platformExceptionHandlers) { + try { + handler.handleException(context, exception) + } catch (_: ExceptionSuccessfullyProcessed) { + return + } catch (t: Throwable) { + propagateExceptionFinalResort(handlerException(exception, t)) + } + } + + try { + exception.addSuppressed(DiagnosticCoroutineContextException(context)) + } catch (e: Throwable) { + // addSuppressed is never user-defined and cannot normally throw with the only exception being OOM + // we do ignore that just in case to definitely deliver the exception + } + propagateExceptionFinalResort(exception) +} + +/** + * Private exception that is added to suppressed exceptions of the original exception + * when it is reported to the last-ditch current thread 'uncaughtExceptionHandler'. + * + * The purpose of this exception is to add an otherwise inaccessible diagnostic information and to + * be able to poke the context of the failing coroutine in the debugger. + */ +internal expect class DiagnosticCoroutineContextException(context: CoroutineContext) : RuntimeException + +/** + * A dummy exception that signifies that the exception was successfully processed by the handler and no further + * action is required. + * + * Would be nicer if [CoroutineExceptionHandler] could return a boolean, but that would be a breaking change. + * For now, we will take solace in knowledge that such exceptions are exceedingly rare, even rarer than globally + * uncaught exceptions in general. + */ +internal object ExceptionSuccessfullyProcessed : Exception() diff --git a/kotlinx-coroutines-core/common/src/internal/DispatchedContinuation.kt b/kotlinx-coroutines-core/common/src/internal/DispatchedContinuation.kt new file mode 100644 index 0000000000..4c8f54e877 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/DispatchedContinuation.kt @@ -0,0 +1,312 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.jvm.* + +private val UNDEFINED = Symbol("UNDEFINED") +@JvmField +internal val REUSABLE_CLAIMED = Symbol("REUSABLE_CLAIMED") + +internal class DispatchedContinuation( + @JvmField internal val dispatcher: CoroutineDispatcher, + @JvmField val continuation: Continuation +) : DispatchedTask(MODE_UNINITIALIZED), CoroutineStackFrame, Continuation by continuation { + @JvmField + @Suppress("PropertyName") + internal var _state: Any? = UNDEFINED + override val callerFrame: CoroutineStackFrame? get() = continuation as? CoroutineStackFrame + override fun getStackTraceElement(): StackTraceElement? = null + @JvmField // pre-cached value to avoid ctx.fold on every resumption + internal val countOrElement = threadContextElements(context) + + /** + * Possible states of reusability: + * + * 1) `null`. Cancellable continuation wasn't yet attempted to be reused or + * was used and then invalidated (e.g. because of the cancellation). + * 2) [CancellableContinuation]. Continuation to be/that is being reused. + * 3) [REUSABLE_CLAIMED]. CC is currently being reused and its owner executes `suspend` block: + * ``` + * // state == null | CC + * suspendCancellableCoroutineReusable { cont -> + * // state == REUSABLE_CLAIMED + * block(cont) + * } + * // state == CC + * ``` + * 4) [Throwable] continuation was cancelled with this cause while being in [suspendCancellableCoroutineReusable], + * [CancellableContinuationImpl.getResult] will check for cancellation later. + * + * [REUSABLE_CLAIMED] state is required to prevent double-use of the reused continuation. + * In the `getResult`, we have the following code: + * ``` + * if (trySuspend()) { + * // <- at this moment current continuation can be redispatched and claimed again. + * attachChildToParent() + * releaseClaimedContinuation() + * } + * ``` + */ + private val _reusableCancellableContinuation = atomic(null) + + private val reusableCancellableContinuation: CancellableContinuationImpl<*>? + get() = _reusableCancellableContinuation.value as? CancellableContinuationImpl<*> + + internal fun isReusable(): Boolean { + /* + Invariant: caller.resumeMode.isReusableMode + * Reusability control: + * `null` -> no reusability at all, `false` + * anything else -> reusable. + */ + return _reusableCancellableContinuation.value != null + } + + /** + * Awaits until previous call to `suspendCancellableCoroutineReusable` will + * stop mutating cached instance + */ + internal fun awaitReusability() { + _reusableCancellableContinuation.loop { + if (it !== REUSABLE_CLAIMED) return + } + } + + internal fun release() { + /* + * Called from `releaseInterceptedContinuation`, can be concurrent with + * the code in `getResult` right after `trySuspend` returned `true`, so we have + * to wait for a release here. + */ + awaitReusability() + reusableCancellableContinuation?.detachChild() + } + + /** + * Claims the continuation for [suspendCancellableCoroutineReusable] block, + * so all cancellations will be postponed. + */ + @Suppress("UNCHECKED_CAST") + internal fun claimReusableCancellableContinuation(): CancellableContinuationImpl? { + /* + * Transitions: + * 1) `null` -> claimed, caller will instantiate CC instance + * 2) `CC` -> claimed, caller will reuse CC instance + */ + _reusableCancellableContinuation.loop { state -> + when { + state === null -> { + /* + * null -> CC was not yet published -> we do not compete with cancel + * -> can use plain store instead of CAS + */ + _reusableCancellableContinuation.value = REUSABLE_CLAIMED + return null + } + // potentially competing with cancel + state is CancellableContinuationImpl<*> -> { + if (_reusableCancellableContinuation.compareAndSet(state, REUSABLE_CLAIMED)) { + return state as CancellableContinuationImpl + } + } + state === REUSABLE_CLAIMED -> { + // Do nothing, wait until reusable instance will be returned from + // getResult() of a previous `suspendCancellableCoroutineReusable` + } + state is Throwable -> { + // Also do nothing, Throwable can only indicate that the CC + // is in REUSABLE_CLAIMED state, but with postponed cancellation + } + else -> error("Inconsistent state $state") + } + } + } + + /** + * Checks whether there were any attempts to cancel reusable CC while it was in [REUSABLE_CLAIMED] state + * and returns cancellation cause if so, `null` otherwise. + * If continuation was cancelled, it becomes non-reusable. + * + * ``` + * suspendCancellableCoroutineReusable { // <- claimed + * // Any asynchronous cancellation is "postponed" while this block + * // is being executed + * } // postponed cancellation is checked here in `getResult` + * ``` + * + * See [CancellableContinuationImpl.getResult]. + */ + internal fun tryReleaseClaimedContinuation(continuation: CancellableContinuation<*>): Throwable? { + _reusableCancellableContinuation.loop { state -> + // not when(state) to avoid Intrinsics.equals call + when { + state === REUSABLE_CLAIMED -> { + if (_reusableCancellableContinuation.compareAndSet(REUSABLE_CLAIMED, continuation)) return null + } + state is Throwable -> { + require(_reusableCancellableContinuation.compareAndSet(state, null)) + return state + } + else -> error("Inconsistent state $state") + } + } + } + + /** + * Tries to postpone cancellation if reusable CC is currently in [REUSABLE_CLAIMED] state. + * Returns `true` if cancellation is (or previously was) postponed, `false` otherwise. + */ + internal fun postponeCancellation(cause: Throwable): Boolean { + _reusableCancellableContinuation.loop { state -> + when (state) { + REUSABLE_CLAIMED -> { + if (_reusableCancellableContinuation.compareAndSet(REUSABLE_CLAIMED, cause)) + return true + } + is Throwable -> return true + else -> { + // Invalidate + if (_reusableCancellableContinuation.compareAndSet(state, null)) + return false + } + } + } + } + + override fun takeState(): Any? { + val state = _state + assert { state !== UNDEFINED } // fail-fast if repeatedly invoked + _state = UNDEFINED + return state + } + + override val delegate: Continuation + get() = this + + override fun resumeWith(result: Result) { + val state = result.toState() + if (dispatcher.safeIsDispatchNeeded(context)) { + _state = state + resumeMode = MODE_ATOMIC + dispatcher.safeDispatch(context, this) + } else { + executeUnconfined(state, MODE_ATOMIC) { + withCoroutineContext(context, countOrElement) { + continuation.resumeWith(result) + } + } + } + } + + // We inline it to save an entry on the stack in cases where it shows (unconfined dispatcher) + // It is used only in Continuation.resumeCancellableWith + @Suppress("NOTHING_TO_INLINE") + internal inline fun resumeCancellableWith(result: Result) { + val state = result.toState() + if (dispatcher.safeIsDispatchNeeded(context)) { + _state = state + resumeMode = MODE_CANCELLABLE + dispatcher.safeDispatch(context, this) + } else { + executeUnconfined(state, MODE_CANCELLABLE) { + if (!resumeCancelled(state)) { + resumeUndispatchedWith(result) + } + } + } + } + + // inline here is to save us an entry on the stack for the sake of better stacktraces + @Suppress("NOTHING_TO_INLINE") + internal inline fun resumeCancelled(state: Any?): Boolean { + val job = context[Job] + if (job != null && !job.isActive) { + val cause = job.getCancellationException() + cancelCompletedResult(state, cause) + resumeWithException(cause) + return true + } + return false + } + + @Suppress("NOTHING_TO_INLINE") + internal inline fun resumeUndispatchedWith(result: Result) { + withContinuationContext(continuation, countOrElement) { + continuation.resumeWith(result) + } + } + + // used by "yield" implementation + internal fun dispatchYield(context: CoroutineContext, value: T) { + _state = value + resumeMode = MODE_CANCELLABLE + dispatcher.dispatchYield(context, this) + } + + override fun toString(): String = + "DispatchedContinuation[$dispatcher, ${continuation.toDebugString()}]" +} + +internal fun CoroutineDispatcher.safeDispatch(context: CoroutineContext, runnable: Runnable) { + try { + dispatch(context, runnable) + } catch (e: Throwable) { + throw DispatchException(e, this, context) + } +} + +internal fun CoroutineDispatcher.safeIsDispatchNeeded(context: CoroutineContext): Boolean { + try { + return isDispatchNeeded(context) + } catch (e: Throwable) { + throw DispatchException(e, this, context) + } +} + +/** + * It is not inline to save bytecode (it is pretty big and used in many places) + * and we leave it public so that its name is not mangled in use stack traces if it shows there. + * It may appear in stack traces when coroutines are started/resumed with unconfined dispatcher. + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public fun Continuation.resumeCancellableWith( + result: Result, +): Unit = when (this) { + is DispatchedContinuation -> resumeCancellableWith(result) + else -> resumeWith(result) +} + +internal fun DispatchedContinuation.yieldUndispatched(): Boolean = + executeUnconfined(Unit, MODE_CANCELLABLE, doYield = true) { + run() + } + +/** + * Executes given [block] as part of current event loop, updating current continuation + * mode and state if continuation is not resumed immediately. + * [doYield] indicates whether current continuation is yielding (to provide fast-path if event-loop is empty). + * Returns `true` if execution of continuation was queued (trampolined) or `false` otherwise. + */ +private inline fun DispatchedContinuation<*>.executeUnconfined( + contState: Any?, mode: Int, doYield: Boolean = false, + block: () -> Unit +): Boolean { + assert { mode != MODE_UNINITIALIZED } // invalid execution mode + val eventLoop = ThreadLocalEventLoop.eventLoop + // If we are yielding and unconfined queue is empty, we can bail out as part of fast path + if (doYield && eventLoop.isUnconfinedQueueEmpty) return false + return if (eventLoop.isUnconfinedLoopActive) { + // When unconfined loop is active -- dispatch continuation for execution to avoid stack overflow + _state = contState + resumeMode = mode + eventLoop.dispatchUnconfined(this) + true // queued into the active loop + } else { + // Was not active -- run event loop until all unconfined tasks are executed + runUnconfinedEventLoop(eventLoop, block = block) + false + } +} diff --git a/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt b/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt new file mode 100644 index 0000000000..ad5fed1205 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt @@ -0,0 +1,219 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.jvm.* + +/** + * Non-cancellable dispatch mode. + * + * **DO NOT CHANGE THE CONSTANT VALUE**. It might be inlined into legacy user code that was calling + * inline `suspendAtomicCancellableCoroutine` function and did not support reuse. + */ +internal const val MODE_ATOMIC = 0 + +/** + * Cancellable dispatch mode. It is used by user-facing [suspendCancellableCoroutine]. + * Note, that implementation of cancellability checks mode via [Int.isCancellableMode] extension. + * + * **DO NOT CHANGE THE CONSTANT VALUE**. It is being into the user code from [suspendCancellableCoroutine]. + */ +@PublishedApi +internal const val MODE_CANCELLABLE: Int = 1 + +/** + * Cancellable dispatch mode for [suspendCancellableCoroutineReusable]. + * Note, that implementation of cancellability checks mode via [Int.isCancellableMode] extension; + * implementation of reuse checks mode via [Int.isReusableMode] extension. + */ +internal const val MODE_CANCELLABLE_REUSABLE = 2 + +/** + * Undispatched mode for [CancellableContinuation.resumeUndispatched]. + * It is used when the thread is right, but it needs to be marked with the current coroutine. + */ +internal const val MODE_UNDISPATCHED = 4 + +/** + * Initial mode for [DispatchedContinuation] implementation, should never be used for dispatch, because it is always + * overwritten when continuation is resumed with the actual resume mode. + */ +internal const val MODE_UNINITIALIZED = -1 + +internal val Int.isCancellableMode get() = this == MODE_CANCELLABLE || this == MODE_CANCELLABLE_REUSABLE +internal val Int.isReusableMode get() = this == MODE_CANCELLABLE_REUSABLE + +internal abstract class DispatchedTask internal constructor( + @JvmField var resumeMode: Int +) : SchedulerTask() { + internal abstract val delegate: Continuation + + internal abstract fun takeState(): Any? + + /** + * Called when this task was cancelled while it was being dispatched. + */ + internal open fun cancelCompletedResult(takenState: Any?, cause: Throwable) {} + + /** + * There are two implementations of `DispatchedTask`: + * - [DispatchedContinuation] keeps only simple values as successfully results. + * - [CancellableContinuationImpl] keeps additional data with values and overrides this method to unwrap it. + */ + @Suppress("UNCHECKED_CAST") + internal open fun getSuccessfulResult(state: Any?): T = + state as T + + /** + * There are two implementations of `DispatchedTask`: + * - [DispatchedContinuation] is just an intermediate storage that stores the exception that has its stack-trace + * properly recovered and is ready to pass to the [delegate] continuation directly. + * - [CancellableContinuationImpl] stores raw cause of the failure in its state; when it needs to be dispatched + * its stack-trace has to be recovered, so it overrides this method for that purpose. + */ + internal open fun getExceptionalResult(state: Any?): Throwable? = + (state as? CompletedExceptionally)?.cause + + final override fun run() { + assert { resumeMode != MODE_UNINITIALIZED } // should have been set before dispatching + try { + val delegate = delegate as DispatchedContinuation + val continuation = delegate.continuation + withContinuationContext(continuation, delegate.countOrElement) { + val context = continuation.context + val state = takeState() // NOTE: Must take state in any case, even if cancelled + val exception = getExceptionalResult(state) + /* + * Check whether continuation was originally resumed with an exception. + * If so, it dominates cancellation, otherwise the original exception + * will be silently lost. + */ + val job = if (exception == null && resumeMode.isCancellableMode) context[Job] else null + if (job != null && !job.isActive) { + val cause = job.getCancellationException() + cancelCompletedResult(state, cause) + continuation.resumeWithStackTrace(cause) + } else { + if (exception != null) { + continuation.resumeWithException(exception) + } else { + continuation.resume(getSuccessfulResult(state)) + } + } + } + } catch (e: DispatchException) { + handleCoroutineException(delegate.context, e.cause) + } catch (e: Throwable) { + handleFatalException(e) + } + } + + /** + * Machinery that handles fatal exceptions in kotlinx.coroutines. + * There are two kinds of fatal exceptions: + * + * 1) Exceptions from kotlinx.coroutines code. Such exceptions indicate that either + * the library or the compiler has a bug that breaks internal invariants. + * They usually have specific workarounds, but require careful study of the cause and should + * be reported to the maintainers and fixed on the library's side anyway. + * + * 2) Exceptions from [ThreadContextElement.updateThreadContext] and [ThreadContextElement.restoreThreadContext]. + * While a user code can trigger such exception by providing an improper implementation of [ThreadContextElement], + * we can't ignore it because it may leave coroutine in the inconsistent state. + * If you encounter such exception, you can either disable this context element or wrap it into + * another context element that catches all exceptions and handles it in the application specific manner. + * + * Fatal exception handling can be intercepted with [CoroutineExceptionHandler] element in the context of + * a failed coroutine, but such exceptions should be reported anyway. + */ + internal fun handleFatalException(exception: Throwable) { + val reason = CoroutinesInternalError("Fatal exception in coroutines machinery for $this. " + + "Please read KDoc to 'handleFatalException' method and report this incident to maintainers", exception) + handleCoroutineException(this.delegate.context, reason) + } +} + +internal fun DispatchedTask.dispatch(mode: Int) { + assert { mode != MODE_UNINITIALIZED } // invalid mode value for this method + val delegate = this.delegate + val undispatched = mode == MODE_UNDISPATCHED + if (!undispatched && delegate is DispatchedContinuation<*> && mode.isCancellableMode == resumeMode.isCancellableMode) { + // dispatch directly using this instance's Runnable implementation + val dispatcher = delegate.dispatcher + val context = delegate.context + if (dispatcher.safeIsDispatchNeeded(context)) { + dispatcher.safeDispatch(context, this) + } else { + resumeUnconfined() + } + } else { + // delegate is coming from 3rd-party interceptor implementation (and does not support cancellation) + // or undispatched mode was requested + resume(delegate, undispatched) + } +} + +internal fun DispatchedTask.resume(delegate: Continuation, undispatched: Boolean) { + // This resume is never cancellable. The result is always delivered to delegate continuation. + val state = takeState() + val exception = getExceptionalResult(state) + val result = if (exception != null) Result.failure(exception) else Result.success(getSuccessfulResult(state)) + when { + undispatched -> (delegate as DispatchedContinuation).resumeUndispatchedWith(result) + else -> delegate.resumeWith(result) + } +} + +private fun DispatchedTask<*>.resumeUnconfined() { + val eventLoop = ThreadLocalEventLoop.eventLoop + if (eventLoop.isUnconfinedLoopActive) { + // When unconfined loop is active -- dispatch continuation for execution to avoid stack overflow + eventLoop.dispatchUnconfined(this) + } else { + // Was not active -- run event loop until all unconfined tasks are executed + runUnconfinedEventLoop(eventLoop) { + resume(delegate, undispatched = true) + } + } +} + +internal inline fun DispatchedTask<*>.runUnconfinedEventLoop( + eventLoop: EventLoop, + block: () -> Unit +) { + eventLoop.incrementUseCount(unconfined = true) + try { + block() + while (true) { + // break when all unconfined continuations where executed + if (!eventLoop.processUnconfinedEvent()) break + } + } catch (e: Throwable) { + /* + * This exception doesn't happen normally, only if we have a bug in implementation. + * Report it as a fatal exception. + */ + handleFatalException(e) + } finally { + eventLoop.decrementUseCount(unconfined = true) + } +} + +@Suppress("NOTHING_TO_INLINE") +internal inline fun Continuation<*>.resumeWithStackTrace(exception: Throwable) { + resumeWith(Result.failure(recoverStackTrace(exception, this))) +} + +/** + * This exception holds an exception raised in [CoroutineDispatcher.dispatch] method. + * When dispatcher methods fail unexpectedly, it is likely a user-induced programmatic bug, + * such as calling `executor.close()` prematurely. To avoid reporting such exceptions as fatal errors, + * we handle them with a separate code path. See also #4091. + * + * @see safeDispatch + */ +internal class DispatchException( + override val cause: Throwable, + dispatcher: CoroutineDispatcher, + context: CoroutineContext, +) : Exception("Coroutine dispatcher $dispatcher threw an exception, context = $context", cause) diff --git a/kotlinx-coroutines-core/common/src/internal/InlineList.kt b/kotlinx-coroutines-core/common/src/internal/InlineList.kt new file mode 100644 index 0000000000..e2cae68d55 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/InlineList.kt @@ -0,0 +1,44 @@ +@file:Suppress("UNCHECKED_CAST") + +package kotlinx.coroutines.internal + +import kotlinx.coroutines.assert +import kotlin.jvm.* + +/* + * Inline class that represents a mutable list, but does not allocate an underlying storage + * for zero and one elements. + * Cannot be parametrized with `List<*>`. + */ +@JvmInline +internal value class InlineList(private val holder: Any? = null) { + operator fun plus(element: E): InlineList { + assert { element !is List<*> } // Lists are prohibited + return when (holder) { + null -> InlineList(element) + is ArrayList<*> -> { + (holder as ArrayList).add(element) + InlineList(holder) + } + else -> { + val list = ArrayList(4) + list.add(holder as E) + list.add(element) + InlineList(list) + } + } + } + + inline fun forEachReversed(action: (E) -> Unit) { + when (holder) { + null -> return + !is ArrayList<*> -> action(holder as E) + else -> { + val list = holder as ArrayList + for (i in (list.size - 1) downTo 0) { + action(list[i]) + } + } + } + } +} diff --git a/kotlinx-coroutines-core/common/src/internal/InternalAnnotations.common.kt b/kotlinx-coroutines-core/common/src/internal/InternalAnnotations.common.kt new file mode 100644 index 0000000000..21eb1ec0bf --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/InternalAnnotations.common.kt @@ -0,0 +1,13 @@ +package kotlinx.coroutines.internal + +// Ignore JRE requirements for animal-sniffer, compileOnly dependency +@Target( + AnnotationTarget.FUNCTION, + AnnotationTarget.PROPERTY_GETTER, + AnnotationTarget.PROPERTY_SETTER, + AnnotationTarget.CONSTRUCTOR, + AnnotationTarget.CLASS, + AnnotationTarget.FILE +) +@OptionalExpectation +internal expect annotation class IgnoreJreRequirement() diff --git a/kotlinx-coroutines-core/common/src/internal/LimitedDispatcher.kt b/kotlinx-coroutines-core/common/src/internal/LimitedDispatcher.kt new file mode 100644 index 0000000000..1361327b6f --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/LimitedDispatcher.kt @@ -0,0 +1,153 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * The result of .limitedParallelism(x) call, a dispatcher + * that wraps the given dispatcher, but limits the parallelism level, while + * trying to emulate fairness. + * + * ### Implementation details + * + * By design, 'LimitedDispatcher' never [dispatches][CoroutineDispatcher.dispatch] originally sent tasks + * to the underlying dispatcher. Instead, it maintains its own queue of tasks sent to this dispatcher and + * dispatches at most [parallelism] "worker-loop" tasks that poll the underlying queue and cooperatively preempt + * in order to avoid starvation of the underlying dispatcher. + * + * Such behavior is crucial to be compatible with any underlying dispatcher implementation without + * direct cooperation. + */ +internal class LimitedDispatcher( + private val dispatcher: CoroutineDispatcher, + private val parallelism: Int, + private val name: String? +) : CoroutineDispatcher(), Delay by (dispatcher as? Delay ?: DefaultDelay) { + + // Atomic is necessary here for the sake of K/N memory ordering, + // there is no need in atomic operations for this property + private val runningWorkers = atomic(0) + + private val queue = LockFreeTaskQueue(singleConsumer = false) + + // A separate object that we can synchronize on for K/N + private val workerAllocationLock = SynchronizedObject() + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + parallelism.checkParallelism() + if (parallelism >= this.parallelism) return namedOrThis(name) + return super.limitedParallelism(parallelism, name) + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + dispatchInternal(block) { worker -> + dispatcher.safeDispatch(this, worker) + } + } + + @InternalCoroutinesApi + override fun dispatchYield(context: CoroutineContext, block: Runnable) { + dispatchInternal(block) { worker -> + dispatcher.dispatchYield(this, worker) + } + } + + /** + * Tries to dispatch the given [block]. + * If there are not enough workers, it starts a new one via [startWorker]. + */ + private inline fun dispatchInternal(block: Runnable, startWorker: (Worker) -> Unit) { + // Add task to queue so running workers will be able to see that + queue.addLast(block) + if (runningWorkers.value >= parallelism) return + // allocation may fail if some workers were launched in parallel or a worker temporarily decreased + // `runningWorkers` when they observed an empty queue. + if (!tryAllocateWorker()) return + val task = obtainTaskOrDeallocateWorker() ?: return + try { + startWorker(Worker(task)) + } catch (e: Throwable) { + /* If we failed to start a worker, we should decrement the counter. + The queue is in an inconsistent state--it's non-empty despite the target parallelism not having been + reached--but at least a properly functioning worker will have a chance to correct this if some future + dispatch does succeed. + If we don't decrement the counter, it will be impossible to ever reach the target parallelism again. */ + runningWorkers.decrementAndGet() + throw e + } + } + + /** + * Tries to obtain the permit to start a new worker. + */ + private fun tryAllocateWorker(): Boolean { + synchronized(workerAllocationLock) { + if (runningWorkers.value >= parallelism) return false + runningWorkers.incrementAndGet() + return true + } + } + + /** + * Obtains the next task from the queue, or logically deallocates the worker if the queue is empty. + */ + private fun obtainTaskOrDeallocateWorker(): Runnable? { + while (true) { + when (val nextTask = queue.removeFirstOrNull()) { + null -> synchronized(workerAllocationLock) { + runningWorkers.decrementAndGet() + if (queue.size == 0) return null + runningWorkers.incrementAndGet() + } + else -> return nextTask + } + } + } + + override fun toString() = name ?: "$dispatcher.limitedParallelism($parallelism)" + + /** + * A worker that polls the queue and runs tasks until there are no more of them. + * + * It always stores the next task to run. This is done in order to prevent the possibility of the fairness + * re-dispatch happening when there are no more tasks in the queue. This is important because, after all the + * actual tasks are done, nothing prevents the user from closing the dispatcher and making it incorrect to + * perform any more dispatches. + */ + private inner class Worker(private var currentTask: Runnable) : Runnable { + override fun run() { + try { + var fairnessCounter = 0 + while (true) { + try { + currentTask.run() + } catch (e: Throwable) { + handleCoroutineException(EmptyCoroutineContext, e) + } + currentTask = obtainTaskOrDeallocateWorker() ?: return + // 16 is our out-of-thin-air constant to emulate fairness. Used in JS dispatchers as well + if (++fairnessCounter >= 16 && dispatcher.safeIsDispatchNeeded(this@LimitedDispatcher)) { + // Do "yield" to let other views execute their runnable as well + // Note that we do not decrement 'runningWorkers' as we are still committed to our part of work + dispatcher.safeDispatch(this@LimitedDispatcher, this) + return + } + } + } catch (e: Throwable) { + // If the worker failed, we should deallocate its slot + synchronized(workerAllocationLock) { + runningWorkers.decrementAndGet() + } + throw e + } + } + } +} + +internal fun Int.checkParallelism() = require(this >= 1) { "Expected positive parallelism level, but got $this" } + +internal fun CoroutineDispatcher.namedOrThis(name: String?): CoroutineDispatcher { + if (name != null) return NamedDispatcher(this, name) + return this +} diff --git a/kotlinx-coroutines-core/common/src/internal/LocalAtomics.common.kt b/kotlinx-coroutines-core/common/src/internal/LocalAtomics.common.kt new file mode 100644 index 0000000000..aea07bed0b --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/LocalAtomics.common.kt @@ -0,0 +1,15 @@ +package kotlinx.coroutines.internal + +/* + * These are atomics that are used as local variables + * where atomicfu doesn't support its tranformations. + * + * Have `Local` prefix to avoid AFU clashes during star-imports + * + * TODO: remove after https://youtrack.jetbrains.com/issue/KT-62423/ + */ +internal expect class LocalAtomicInt(value: Int) { + fun get(): Int + fun set(value: Int) + fun decrementAndGet(): Int +} diff --git a/kotlinx-coroutines-core/common/src/internal/LockFreeLinkedList.common.kt b/kotlinx-coroutines-core/common/src/internal/LockFreeLinkedList.common.kt new file mode 100644 index 0000000000..b925089b0c --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/LockFreeLinkedList.common.kt @@ -0,0 +1,23 @@ +package kotlinx.coroutines.internal + +/** @suppress **This is unstable API and it is subject to change.** */ +public expect open class LockFreeLinkedListNode() { + public val isRemoved: Boolean + public val nextNode: LockFreeLinkedListNode + public val prevNode: LockFreeLinkedListNode + public fun addLast(node: LockFreeLinkedListNode, permissionsBitmask: Int): Boolean + public fun addOneIfEmpty(node: LockFreeLinkedListNode): Boolean + public open fun remove(): Boolean + + /** + * Closes the list for anything that requests the permission [forbiddenElementsBit]. + * Only a single permission can be forbidden at a time, but this isn't checked. + */ + public fun close(forbiddenElementsBit: Int) +} + +/** @suppress **This is unstable API and it is subject to change.** */ +public expect open class LockFreeLinkedListHead() : LockFreeLinkedListNode { + public inline fun forEach(block: (LockFreeLinkedListNode) -> Unit) + public final override fun remove(): Nothing +} diff --git a/kotlinx-coroutines-core/common/src/internal/LockFreeTaskQueue.kt b/kotlinx-coroutines-core/common/src/internal/LockFreeTaskQueue.kt new file mode 100644 index 0000000000..b056bfd33a --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/LockFreeTaskQueue.kt @@ -0,0 +1,303 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlin.jvm.* + +private typealias Core = LockFreeTaskQueueCore + +/** + * Lock-free Multiply-Producer xxx-Consumer Queue for task scheduling purposes. + * + * **Note 1: This queue is NOT linearizable. It provides only quiescent consistency for its operations.** + * However, this guarantee is strong enough for task-scheduling purposes. + * In particular, the following execution is permitted for this queue, but is not permitted for a linearizable queue: + * + * ``` + * Thread 1: addLast(1) = true, removeFirstOrNull() = null + * Thread 2: addLast(2) = 2 // this operation is concurrent with both operations in the first thread + * ``` + * + * **Note 2: When this queue is used with multiple consumers (`singleConsumer == false`) this it is NOT lock-free.** + * In particular, consumer spins until producer finishes its operation in the case of near-empty queue. + * It is a very short window that could manifest itself rarely and only under specific load conditions, + * but it still deprives this algorithm of its lock-freedom. + */ +internal open class LockFreeTaskQueue( + singleConsumer: Boolean // true when there is only a single consumer (slightly faster & lock-free) +) { + private val _cur = atomic(Core(Core.INITIAL_CAPACITY, singleConsumer)) + + // Note: it is not atomic w.r.t. remove operation (remove can transiently fail when isEmpty is false) + val isEmpty: Boolean get() = _cur.value.isEmpty + val size: Int get() = _cur.value.size + + fun close() { + _cur.loop { cur -> + if (cur.close()) return // closed this copy + _cur.compareAndSet(cur, cur.next()) // move to next + } + } + + fun addLast(element: E): Boolean { + _cur.loop { cur -> + when (cur.addLast(element)) { + Core.ADD_SUCCESS -> return true + Core.ADD_CLOSED -> return false + Core.ADD_FROZEN -> _cur.compareAndSet(cur, cur.next()) // move to next + } + } + } + + @Suppress("UNCHECKED_CAST") + fun removeFirstOrNull(): E? { + _cur.loop { cur -> + val result = cur.removeFirstOrNull() + if (result !== Core.REMOVE_FROZEN) return result as E? + _cur.compareAndSet(cur, cur.next()) + } + } + + // Used for validation in tests only + fun map(transform: (E) -> R): List = _cur.value.map(transform) + + // Used for validation in tests only + fun isClosed(): Boolean = _cur.value.isClosed() +} + +/** + * Lock-free Multiply-Producer xxx-Consumer Queue core. + * @see LockFreeTaskQueue + */ +internal class LockFreeTaskQueueCore( + private val capacity: Int, + private val singleConsumer: Boolean // true when there is only a single consumer (slightly faster) +) { + private val mask = capacity - 1 + private val _next = atomic?>(null) + private val _state = atomic(0L) + private val array = atomicArrayOfNulls(capacity) + + init { + check(mask <= MAX_CAPACITY_MASK) + check(capacity and mask == 0) + } + + // Note: it is not atomic w.r.t. remove operation (remove can transiently fail when isEmpty is false) + val isEmpty: Boolean get() = _state.value.withState { head, tail -> head == tail } + val size: Int get() = _state.value.withState { head, tail -> (tail - head) and MAX_CAPACITY_MASK } + + fun close(): Boolean { + _state.update { state -> + if (state and CLOSED_MASK != 0L) return true // ok - already closed + if (state and FROZEN_MASK != 0L) return false // frozen -- try next + state or CLOSED_MASK // try set closed bit + } + return true + } + + // ADD_CLOSED | ADD_FROZEN | ADD_SUCCESS + fun addLast(element: E): Int { + _state.loop { state -> + if (state and (FROZEN_MASK or CLOSED_MASK) != 0L) return state.addFailReason() // cannot add + state.withState { head, tail -> + val mask = this.mask // manually move instance field to local for performance + // If queue is Single-Consumer then there could be one element beyond head that we cannot overwrite, + // so we check for full queue with an extra margin of one element + if ((tail + 2) and mask == head and mask) return ADD_FROZEN // overfull, so do freeze & copy + // If queue is Multi-Consumer then the consumer could still have not cleared element + // despite the above check for one free slot. + if (!singleConsumer && array[tail and mask].value != null) { + // There are two options in this situation + // 1. Spin-wait until consumer clears the slot + // 2. Freeze & resize to avoid spinning + // We use heuristic here to avoid memory-overallocation + // Freeze & reallocate when queue is small or more than half of the queue is used + if (capacity < MIN_ADD_SPIN_CAPACITY || (tail - head) and MAX_CAPACITY_MASK > capacity shr 1) { + return ADD_FROZEN + } + // otherwise spin + return@loop + } + val newTail = (tail + 1) and MAX_CAPACITY_MASK + if (_state.compareAndSet(state, state.updateTail(newTail))) { + // successfully added + array[tail and mask].value = element + // could have been frozen & copied before this item was set -- correct it by filling placeholder + var cur = this + while(true) { + if (cur._state.value and FROZEN_MASK == 0L) break // all fine -- not frozen yet + cur = cur.next().fillPlaceholder(tail, element) ?: break + } + return ADD_SUCCESS // added successfully + } + } + } + } + + private fun fillPlaceholder(index: Int, element: E): Core? { + val old = array[index and mask].value + /* + * addLast actions: + * 1) Commit tail slot + * 2) Write element to array slot + * 3) Check for array copy + * + * If copy happened between 2 and 3 then the consumer might have consumed our element, + * then another producer might have written its placeholder in our slot, so we should + * perform *unique* check that current placeholder is our to avoid overwriting another producer placeholder + */ + if (old is Placeholder && old.index == index) { + array[index and mask].value = element + // we've corrected missing element, should check if that propagated to further copies, just in case + return this + } + // it is Ok, no need for further action + return null + } + + // REMOVE_FROZEN | null (EMPTY) | E (SUCCESS) + fun removeFirstOrNull(): Any? { + _state.loop { state -> + if (state and FROZEN_MASK != 0L) return REMOVE_FROZEN // frozen -- cannot modify + state.withState { head, tail -> + if ((tail and mask) == (head and mask)) return null // empty + val element = array[head and mask].value + if (element == null) { + // If queue is Single-Consumer, then element == null only when add has not finished yet + if (singleConsumer) return null // consider it not added yet + // retry (spin) until consumer adds it + return@loop + } + // element == Placeholder can only be when add has not finished yet + if (element is Placeholder) return null // consider it not added yet + // we cannot put null into array here, because copying thread could replace it with Placeholder and that is a disaster + val newHead = (head + 1) and MAX_CAPACITY_MASK + if (_state.compareAndSet(state, state.updateHead(newHead))) { + // Array could have been copied by another thread and it is perfectly fine, since only elements + // between head and tail were copied and there are no extra steps we should take here + array[head and mask].value = null // now can safely put null (state was updated) + return element // successfully removed in fast-path + } + // Multi-Consumer queue must retry this loop on CAS failure (another consumer might have removed element) + if (!singleConsumer) return@loop + // Single-consumer queue goes to slow-path for remove in case of interference + var cur = this + while (true) { + @Suppress("UNUSED_VALUE") + cur = cur.removeSlowPath(head, newHead) ?: return element + } + } + } + } + + private fun removeSlowPath(oldHead: Int, newHead: Int): Core? { + _state.loop { state -> + state.withState { head, _ -> + assert { head == oldHead } // "This queue can have only one consumer" + if (state and FROZEN_MASK != 0L) { + // state was already frozen, so removed element was copied to next + return next() // continue to correct head in next + } + if (_state.compareAndSet(state, state.updateHead(newHead))) { + array[head and mask].value = null // now can safely put null (state was updated) + return null + } + } + } + } + + fun next(): LockFreeTaskQueueCore = allocateOrGetNextCopy(markFrozen()) + + private fun markFrozen(): Long = + _state.updateAndGet { state -> + if (state and FROZEN_MASK != 0L) return state // already marked + state or FROZEN_MASK + } + + private fun allocateOrGetNextCopy(state: Long): Core { + _next.loop { next -> + if (next != null) return next // already allocated & copied + _next.compareAndSet(null, allocateNextCopy(state)) + } + } + + private fun allocateNextCopy(state: Long): Core { + val next = LockFreeTaskQueueCore(capacity * 2, singleConsumer) + state.withState { head, tail -> + var index = head + while (index and mask != tail and mask) { + // replace nulls with placeholders on copy + val value = array[index and mask].value ?: Placeholder(index) + next.array[index and next.mask].value = value + index++ + } + next._state.value = state wo FROZEN_MASK + } + return next + } + + // Used for validation in tests only + fun map(transform: (E) -> R): List { + val res = ArrayList(capacity) + _state.value.withState { head, tail -> + var index = head + while (index and mask != tail and mask) { + // replace nulls with placeholders on copy + val element = array[index and mask].value + @Suppress("UNCHECKED_CAST") + if (element != null && element !is Placeholder) res.add(transform(element as E)) + index++ + } + } + return res + } + + // Used for validation in tests only + fun isClosed(): Boolean = _state.value and CLOSED_MASK != 0L + + + // Instance of this class is placed into array when we have to copy array, but addLast is in progress -- + // it had already reserved a slot in the array (with null) and have not yet put its value there. + // Placeholder keeps the actual index (not masked) to distinguish placeholders on different wraparounds of array + // Internal because of inlining + internal class Placeholder(@JvmField val index: Int) + + @Suppress("PrivatePropertyName", "MemberVisibilityCanBePrivate") + internal companion object { + const val INITIAL_CAPACITY = 8 + + const val CAPACITY_BITS = 30 + const val MAX_CAPACITY_MASK = (1 shl CAPACITY_BITS) - 1 + const val HEAD_SHIFT = 0 + const val HEAD_MASK = MAX_CAPACITY_MASK.toLong() shl HEAD_SHIFT + const val TAIL_SHIFT = HEAD_SHIFT + CAPACITY_BITS + const val TAIL_MASK = MAX_CAPACITY_MASK.toLong() shl TAIL_SHIFT + + const val FROZEN_SHIFT = TAIL_SHIFT + CAPACITY_BITS + const val FROZEN_MASK = 1L shl FROZEN_SHIFT + const val CLOSED_SHIFT = FROZEN_SHIFT + 1 + const val CLOSED_MASK = 1L shl CLOSED_SHIFT + + const val MIN_ADD_SPIN_CAPACITY = 1024 + + @JvmField val REMOVE_FROZEN = Symbol("REMOVE_FROZEN") + + const val ADD_SUCCESS = 0 + const val ADD_FROZEN = 1 + const val ADD_CLOSED = 2 + + infix fun Long.wo(other: Long) = this and other.inv() + fun Long.updateHead(newHead: Int) = (this wo HEAD_MASK) or (newHead.toLong() shl HEAD_SHIFT) + fun Long.updateTail(newTail: Int) = (this wo TAIL_MASK) or (newTail.toLong() shl TAIL_SHIFT) + + inline fun Long.withState(block: (head: Int, tail: Int) -> T): T { + val head = ((this and HEAD_MASK) shr HEAD_SHIFT).toInt() + val tail = ((this and TAIL_MASK) shr TAIL_SHIFT).toInt() + return block(head, tail) + } + + // FROZEN | CLOSED + fun Long.addFailReason(): Int = if (this and CLOSED_MASK != 0L) ADD_CLOSED else ADD_FROZEN + } +} diff --git a/kotlinx-coroutines-core/common/src/internal/MainDispatcherFactory.kt b/kotlinx-coroutines-core/common/src/internal/MainDispatcherFactory.kt new file mode 100644 index 0000000000..c05b183ed3 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/MainDispatcherFactory.kt @@ -0,0 +1,25 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* + +/** @suppress */ +@InternalCoroutinesApi // Emulating DI for Kotlin object's +public interface MainDispatcherFactory { + public val loadPriority: Int // higher priority wins + + /** + * Creates the main dispatcher. [allFactories] parameter contains all factories found by service loader. + * This method is not guaranteed to be idempotent. + * + * It is required that this method fails with an exception instead of returning an instance that doesn't work + * correctly as a [Delay]. + * The reason for this is that, on the JVM, [DefaultDelay] will use [Dispatchers.Main] for most delays by default + * if this method returns an instance without throwing. + */ + public fun createDispatcher(allFactories: List): MainCoroutineDispatcher + + /** + * Hint used along with error message when the factory failed to create a dispatcher. + */ + public fun hintOnError(): String? = null +} diff --git a/kotlinx-coroutines-core/common/src/internal/NamedDispatcher.kt b/kotlinx-coroutines-core/common/src/internal/NamedDispatcher.kt new file mode 100644 index 0000000000..72dbd65380 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/NamedDispatcher.kt @@ -0,0 +1,25 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.DefaultDelay +import kotlin.coroutines.* + +/** + * Wrapping dispatcher that has a nice user-supplied `toString()` representation + */ +internal class NamedDispatcher( + private val dispatcher: CoroutineDispatcher, + private val name: String +) : CoroutineDispatcher(), Delay by (dispatcher as? Delay ?: DefaultDelay) { + + override fun isDispatchNeeded(context: CoroutineContext): Boolean = dispatcher.isDispatchNeeded(context) + + override fun dispatch(context: CoroutineContext, block: Runnable) = dispatcher.dispatch(context, block) + + @InternalCoroutinesApi + override fun dispatchYield(context: CoroutineContext, block: Runnable) = dispatcher.dispatchYield(context, block) + + override fun toString(): String { + return name + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/src/internal/OnUndeliveredElement.kt b/kotlinx-coroutines-core/common/src/internal/OnUndeliveredElement.kt new file mode 100644 index 0000000000..5ed99d3d4d --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/OnUndeliveredElement.kt @@ -0,0 +1,36 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +internal typealias OnUndeliveredElement = (E) -> Unit + +internal fun OnUndeliveredElement.callUndeliveredElementCatchingException( + element: E, + undeliveredElementException: UndeliveredElementException? = null +): UndeliveredElementException? { + try { + invoke(element) + } catch (ex: Throwable) { + // undeliveredElementException.cause !== ex is an optimization in case the same exception is thrown + // over and over again by on OnUndeliveredElement + if (undeliveredElementException != null && undeliveredElementException.cause !== ex) { + undeliveredElementException.addSuppressed(ex) + } else { + return UndeliveredElementException("Exception in undelivered element handler for $element", ex) + } + } + return undeliveredElementException +} + +internal fun OnUndeliveredElement.callUndeliveredElement(element: E, context: CoroutineContext) { + callUndeliveredElementCatchingException(element, null)?.let { ex -> + handleCoroutineException(context, ex) + } +} + +/** + * Internal exception that is thrown when [OnUndeliveredElement] handler in + * a [kotlinx.coroutines.channels.Channel] throws an exception. + */ +internal class UndeliveredElementException(message: String, cause: Throwable) : RuntimeException(message, cause) diff --git a/kotlinx-coroutines-core/common/src/internal/ProbesSupport.common.kt b/kotlinx-coroutines-core/common/src/internal/ProbesSupport.common.kt new file mode 100644 index 0000000000..a76364d549 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/ProbesSupport.common.kt @@ -0,0 +1,7 @@ +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +internal expect inline fun probeCoroutineCreated(completion: Continuation): Continuation + +internal expect inline fun probeCoroutineResumed(completion: Continuation): Unit diff --git a/kotlinx-coroutines-core/common/src/internal/Scopes.kt b/kotlinx-coroutines-core/common/src/internal/Scopes.kt new file mode 100644 index 0000000000..9b830bd5c9 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/Scopes.kt @@ -0,0 +1,43 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlin.jvm.* + +/** + * This is a coroutine instance that is created by [coroutineScope] builder. + */ +internal open class ScopeCoroutine( + context: CoroutineContext, + @JvmField val uCont: Continuation // unintercepted continuation +) : AbstractCoroutine(context, true, true), CoroutineStackFrame { + + final override val callerFrame: CoroutineStackFrame? get() = uCont as? CoroutineStackFrame + final override fun getStackTraceElement(): StackTraceElement? = null + + final override val isScopedCoroutine: Boolean get() = true + + override fun afterCompletion(state: Any?) { + // Resume in a cancellable way by default when resuming from another context + uCont.intercepted().resumeCancellableWith(recoverResult(state, uCont)) + } + + /** + * Invoked when a scoped coorutine was completed in an undispatched manner directly + * at the place of its start because it never suspended. + */ + open fun afterCompletionUndispatched() { + } + + override fun afterResume(state: Any?) { + // Resume direct because scope is already in the correct context + uCont.resumeWith(recoverResult(state, uCont)) + } +} + +internal class ContextScope(context: CoroutineContext) : CoroutineScope { + override val coroutineContext: CoroutineContext = context + // CoroutineScope is used intentionally for user-friendly representation + override fun toString(): String = "CoroutineScope(coroutineContext=$coroutineContext)" +} diff --git a/kotlinx-coroutines-core/common/src/internal/StackTraceRecovery.common.kt b/kotlinx-coroutines-core/common/src/internal/StackTraceRecovery.common.kt new file mode 100644 index 0000000000..f5e0ab8f1d --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/StackTraceRecovery.common.kt @@ -0,0 +1,47 @@ +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +/** + * Tries to recover stacktrace for given [exception] and [continuation]. + * Stacktrace recovery tries to restore [continuation] stack frames using its debug metadata with [CoroutineStackFrame] API + * and then reflectively instantiate exception of given type with original exception as a cause and + * sets new stacktrace for wrapping exception. + * Some frames may be missing due to tail-call elimination. + * + * Works only on JVM with enabled debug-mode. + */ +internal expect fun recoverStackTrace(exception: E, continuation: Continuation<*>): E + +/** + * initCause on JVM, nop on other platforms + */ +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +internal expect fun Throwable.initCause(cause: Throwable) + +/** + * Tries to recover stacktrace for given [exception]. Used in non-suspendable points of awaiting. + * Stacktrace recovery tries to instantiate exception of given type with original exception as a cause. + * Wrapping exception will have proper stacktrace as it's instantiated in the right context. + * + * Works only on JVM with enabled debug-mode. + */ +internal expect fun recoverStackTrace(exception: E): E + +// Name conflict with recoverStackTrace +@Suppress("NOTHING_TO_INLINE") +internal expect suspend inline fun recoverAndThrow(exception: Throwable): Nothing + +/** + * The opposite of [recoverStackTrace]. + * It is guaranteed that `unwrap(recoverStackTrace(e)) === e` + */ +@PublishedApi // Used from kotlinx-coroutines-test and reactor modules via suppress, not part of ABI +internal expect fun unwrap(exception: E): E + +internal expect class StackTraceElement + +internal expect interface CoroutineStackFrame { + public val callerFrame: CoroutineStackFrame? + public fun getStackTraceElement(): StackTraceElement? +} diff --git a/kotlinx-coroutines-core/common/src/internal/Symbol.kt b/kotlinx-coroutines-core/common/src/internal/Symbol.kt new file mode 100644 index 0000000000..96e3480c1c --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/Symbol.kt @@ -0,0 +1,15 @@ +package kotlinx.coroutines.internal + +import kotlin.jvm.* + +/** + * A symbol class that is used to define unique constants that are self-explanatory in debugger. + * + * @suppress **This is unstable API and it is subject to change.** + */ +internal class Symbol(@JvmField val symbol: String) { + override fun toString(): String = "<$symbol>" + + @Suppress("UNCHECKED_CAST", "NOTHING_TO_INLINE") + inline fun unbox(value: Any?): T = if (value === this) null as T else value as T +} diff --git a/kotlinx-coroutines-core/common/src/internal/Synchronized.common.kt b/kotlinx-coroutines-core/common/src/internal/Synchronized.common.kt new file mode 100644 index 0000000000..43777f20db --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/Synchronized.common.kt @@ -0,0 +1,30 @@ +@file:Suppress("LEAKED_IN_PLACE_LAMBDA", "WRONG_INVOCATION_KIND") + +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.contracts.* + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public expect open class SynchronizedObject() // marker abstract class + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public expect inline fun synchronizedImpl(lock: SynchronizedObject, block: () -> T): T + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@OptIn(ExperimentalContracts::class) +@InternalCoroutinesApi +public inline fun synchronized(lock: SynchronizedObject, block: () -> T): T { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + return synchronizedImpl(lock, block) +} diff --git a/kotlinx-coroutines-core/common/src/internal/SystemProps.common.kt b/kotlinx-coroutines-core/common/src/internal/SystemProps.common.kt new file mode 100644 index 0000000000..61b7dc9bfa --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/SystemProps.common.kt @@ -0,0 +1,72 @@ +@file:JvmName("SystemPropsKt") +@file:JvmMultifileClass + +package kotlinx.coroutines.internal + +import kotlin.jvm.* + +/** + * Gets the system property indicated by the specified [property name][propertyName], + * or returns [defaultValue] if there is no property with that key. + * + * **Note: this function should be used in JVM tests only, other platforms use the default value.** + */ +internal fun systemProp( + propertyName: String, + defaultValue: Boolean +): Boolean = systemProp(propertyName)?.toBoolean() ?: defaultValue + +/** + * Gets the system property indicated by the specified [property name][propertyName], + * or returns [defaultValue] if there is no property with that key. It also checks that the result + * is between [minValue] and [maxValue] (inclusively), throws [IllegalStateException] if it is not. + * + * **Note: this function should be used in JVM tests only, other platforms use the default value.** + */ +internal fun systemProp( + propertyName: String, + defaultValue: Int, + minValue: Int = 1, + maxValue: Int = Int.MAX_VALUE +): Int = systemProp(propertyName, defaultValue.toLong(), minValue.toLong(), maxValue.toLong()).toInt() + +/** + * Gets the system property indicated by the specified [property name][propertyName], + * or returns [defaultValue] if there is no property with that key. It also checks that the result + * is between [minValue] and [maxValue] (inclusively), throws [IllegalStateException] if it is not. + * + * **Note: this function should be used in JVM tests only, other platforms use the default value.** + */ +internal fun systemProp( + propertyName: String, + defaultValue: Long, + minValue: Long = 1, + maxValue: Long = Long.MAX_VALUE +): Long { + val value = systemProp(propertyName) ?: return defaultValue + val parsed = value.toLongOrNull() + ?: error("System property '$propertyName' has unrecognized value '$value'") + if (parsed !in minValue..maxValue) { + error("System property '$propertyName' should be in range $minValue..$maxValue, but is '$parsed'") + } + return parsed +} + +/** + * Gets the system property indicated by the specified [property name][propertyName], + * or returns [defaultValue] if there is no property with that key. + * + * **Note: this function should be used in JVM tests only, other platforms use the default value.** + */ +internal fun systemProp( + propertyName: String, + defaultValue: String +): String = systemProp(propertyName) ?: defaultValue + +/** + * Gets the system property indicated by the specified [property name][propertyName], + * or returns `null` if there is no property with that key. + * + * **Note: this function should be used in JVM tests only, other platforms use the default value.** + */ +internal expect fun systemProp(propertyName: String): String? diff --git a/kotlinx-coroutines-core/common/src/internal/ThreadContext.common.kt b/kotlinx-coroutines-core/common/src/internal/ThreadContext.common.kt new file mode 100644 index 0000000000..c52d35c128 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/ThreadContext.common.kt @@ -0,0 +1,5 @@ +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +internal expect fun threadContextElements(context: CoroutineContext): Any diff --git a/kotlinx-coroutines-core/common/src/internal/ThreadLocal.common.kt b/kotlinx-coroutines-core/common/src/internal/ThreadLocal.common.kt new file mode 100644 index 0000000000..15622597d5 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/ThreadLocal.common.kt @@ -0,0 +1,15 @@ +package kotlinx.coroutines.internal + +internal expect class CommonThreadLocal { + fun get(): T + fun set(value: T) +} + +/** + * Create a thread-local storage for an object of type [T]. + * + * If two different thread-local objects share the same [name], they will not necessarily share the same value, + * but they may. + * Therefore, use a unique [name] for each thread-local object. + */ +internal expect fun commonThreadLocal(name: Symbol): CommonThreadLocal diff --git a/kotlinx-coroutines-core/common/src/internal/ThreadSafeHeap.kt b/kotlinx-coroutines-core/common/src/internal/ThreadSafeHeap.kt new file mode 100644 index 0000000000..c8d4cfe1af --- /dev/null +++ b/kotlinx-coroutines-core/common/src/internal/ThreadSafeHeap.kt @@ -0,0 +1,158 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public interface ThreadSafeHeapNode { + public var heap: ThreadSafeHeap<*>? + public var index: Int +} + +/** + * Synchronized binary heap. + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public open class ThreadSafeHeap : SynchronizedObject() where T: ThreadSafeHeapNode, T: Comparable { + private var a: Array? = null + + private val _size = atomic(0) + + public var size: Int + get() = _size.value + private set(value) { _size.value = value } + + public val isEmpty: Boolean get() = size == 0 + + public fun find( + predicate: (value: T) -> Boolean + ): T? = synchronized(this) block@{ + for (i in 0 until size) { + val value = a?.get(i)!! + if (predicate(value)) return@block value + } + null + } + + public fun peek(): T? = synchronized(this) { firstImpl() } + + public fun removeFirstOrNull(): T? = synchronized(this) { + if (size > 0) { + removeAtImpl(0) + } else { + null + } + } + + public inline fun removeFirstIf(predicate: (T) -> Boolean): T? = synchronized(this) { + val first = firstImpl() ?: return null + if (predicate(first)) { + removeAtImpl(0) + } else { + null + } + } + + public fun addLast(node: T): Unit = synchronized(this) { addImpl(node) } + + // Condition also receives current first node in the heap + public inline fun addLastIf(node: T, cond: (T?) -> Boolean): Boolean = synchronized(this) { + if (cond(firstImpl())) { + addImpl(node) + true + } else { + false + } + } + + public fun remove(node: T): Boolean = synchronized(this) { + return if (node.heap == null) { + false + } else { + val index = node.index + assert { index >= 0 } + removeAtImpl(index) + true + } + } + + @PublishedApi + internal fun firstImpl(): T? = a?.get(0) + + @PublishedApi + internal fun removeAtImpl(index: Int): T { + assert { size > 0 } + val a = this.a!! + size-- + if (index < size) { + swap(index, size) + val j = (index - 1) / 2 + if (index > 0 && a[index]!! < a[j]!!) { + swap(index, j) + siftUpFrom(j) + } else { + siftDownFrom(index) + } + } + val result = a[size]!! + assert { result.heap === this } + result.heap = null + result.index = -1 + a[size] = null + return result + } + + @PublishedApi + internal fun addImpl(node: T) { + assert { node.heap == null } + node.heap = this + val a = realloc() + val i = size++ + a[i] = node + node.index = i + siftUpFrom(i) + } + + private tailrec fun siftUpFrom(i: Int) { + if (i <= 0) return + val a = a!! + val j = (i - 1) / 2 + if (a[j]!! <= a[i]!!) return + swap(i, j) + siftUpFrom(j) + } + + private tailrec fun siftDownFrom(i: Int) { + var j = 2 * i + 1 + if (j >= size) return + val a = a!! + if (j + 1 < size && a[j + 1]!! < a[j]!!) j++ + if (a[i]!! <= a[j]!!) return + swap(i, j) + siftDownFrom(j) + } + + @Suppress("UNCHECKED_CAST") + private fun realloc(): Array { + val a = this.a + return when { + a == null -> (arrayOfNulls(4) as Array).also { this.a = it } + size >= a.size -> a.copyOf(size * 2).also { this.a = it } + else -> a + } + } + + private fun swap(i: Int, j: Int) { + val a = a!! + val ni = a[j]!! + val nj = a[i]!! + a[i] = ni + a[j] = nj + ni.index = i + nj.index = j + } +} diff --git a/kotlinx-coroutines-core/common/src/intrinsics/Cancellable.kt b/kotlinx-coroutines-core/common/src/intrinsics/Cancellable.kt new file mode 100644 index 0000000000..1e87d767af --- /dev/null +++ b/kotlinx-coroutines-core/common/src/intrinsics/Cancellable.kt @@ -0,0 +1,64 @@ +package kotlinx.coroutines.intrinsics + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +/** + * Use this function to start coroutine in a cancellable way, so that it can be cancelled + * while waiting to be dispatched. + * + * @suppress **This is internal API and it is subject to change.** + */ +@InternalCoroutinesApi +public fun (suspend () -> T).startCoroutineCancellable(completion: Continuation): Unit = runSafely(completion) { + createCoroutineUnintercepted(completion).intercepted().resumeCancellableWith(Result.success(Unit)) +} + +/** + * Use this function to start coroutine in a cancellable way, so that it can be cancelled + * while waiting to be dispatched. + */ +internal fun (suspend (R) -> T).startCoroutineCancellable( + receiver: R, completion: Continuation, +) = runSafely(completion) { + createCoroutineUnintercepted(receiver, completion).intercepted().resumeCancellableWith(Result.success(Unit)) +} + +/** + * Similar to [startCoroutineCancellable], but for already created coroutine. + * [fatalCompletion] is used only when interception machinery throws an exception + */ +internal fun Continuation.startCoroutineCancellable(fatalCompletion: Continuation<*>) = + runSafely(fatalCompletion) { + intercepted().resumeCancellableWith(Result.success(Unit)) + } + +/** + * Runs given block and completes completion with its exception if it occurs. + * Rationale: [startCoroutineCancellable] is invoked when we are about to run coroutine asynchronously in its own dispatcher. + * Thus if dispatcher throws an exception during coroutine start, coroutine never completes, so we should treat dispatcher exception + * as its cause and resume completion. + */ +private inline fun runSafely(completion: Continuation<*>, block: () -> Unit) { + try { + block() + } catch (e: Throwable) { + dispatcherFailure(completion, e) + } +} + +private fun dispatcherFailure(completion: Continuation<*>, e: Throwable) { + /* + * This method is invoked when we failed to start a coroutine due to the throwing + * dispatcher implementation or missing Dispatchers.Main. + * This situation is not recoverable, so we are trying to deliver the exception by all means: + * 1) Resume the coroutine with an exception, so it won't prevent its parent from completion + * 2) Rethrow the exception immediately, so it will crash the caller (e.g. when the coroutine had + * no parent or it was async/produce over MainScope). + */ + val reportException = if (e is DispatchException) e.cause else e + completion.resumeWith(Result.failure(reportException)) + throw reportException +} diff --git a/kotlinx-coroutines-core/common/src/intrinsics/Undispatched.kt b/kotlinx-coroutines-core/common/src/intrinsics/Undispatched.kt new file mode 100644 index 0000000000..bb81d442d4 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/intrinsics/Undispatched.kt @@ -0,0 +1,104 @@ +package kotlinx.coroutines.intrinsics + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +/** + * Use this function to start a new coroutine in [CoroutineStart.UNDISPATCHED] mode — + * immediately execute the coroutine in the current thread until the next suspension. + * It does not use [ContinuationInterceptor], but updates the context of the current thread for the new coroutine. + */ +internal fun (suspend (R) -> T).startCoroutineUndispatched(receiver: R, completion: Continuation) { + val actualCompletion = probeCoroutineCreated(completion) + val value = try { + /* The code below is started immediately in the current stack-frame + * and runs until the first suspension point. */ + withCoroutineContext(actualCompletion.context, null) { + probeCoroutineResumed(actualCompletion) + startCoroutineUninterceptedOrReturn(receiver, actualCompletion) + } + } catch (e: Throwable) { + val reportException = if (e is DispatchException) e.cause else e + actualCompletion.resumeWithException(reportException) + return + } + if (value !== COROUTINE_SUSPENDED) { + @Suppress("UNCHECKED_CAST") + actualCompletion.resume(value as T) + } +} + +/** + * Starts this coroutine with the given code [block] in the same context and returns the coroutine result when it + * completes without suspension. + * This function shall be invoked at most once on this coroutine. + * This function checks cancellation of the outer [Job] on fast-path. + * + * It starts the coroutine using [startCoroutineUninterceptedOrReturn]. + */ +internal fun ScopeCoroutine.startUndispatchedOrReturn( + receiver: R, block: suspend R.() -> T +): Any? = startUndspatched(alwaysRethrow = true, receiver, block) + +/** + * Same as [startUndispatchedOrReturn], but ignores [TimeoutCancellationException] on fast-path. + */ +internal fun ScopeCoroutine.startUndispatchedOrReturnIgnoreTimeout( + receiver: R, block: suspend R.() -> T +): Any? = startUndspatched(alwaysRethrow = false, receiver, block) + +/** + * Starts and handles the result of an undispatched coroutine, potentially with children. + * For example, it handles `coroutineScope { ...suspend of throw, maybe start children... }` + * and `launch(start = UNDISPATCHED) { ... }` + * + * @param alwaysRethrow specifies whether an exception should be unconditioanlly rethrown. + * It is a tweak for 'withTimeout' in order to successfully return values when the block was cancelled: + * i.e. `withTimeout(1ms) { Thread.sleep(1000); 42 }` should not fail. + */ +private fun ScopeCoroutine.startUndspatched( + alwaysRethrow: Boolean, + receiver: R, block: suspend R.() -> T +): Any? { + val result = try { + block.startCoroutineUninterceptedOrReturn(receiver, this) + } catch (e: DispatchException) { + // Special codepath for failing CoroutineDispatcher: rethrow an exception + // immediately without waiting for children to indicate something is wrong + dispatchExceptionAndMakeCompleting(e) + } catch (e: Throwable) { + CompletedExceptionally(e) + } + + /* + * We are trying to complete our undispatched block with the following possible codepaths: + * 1) The coroutine just suspended. I.e. `coroutineScope { .. suspend here }`. + * Then just suspend + * 2) The coroutine completed with something, but has active children. Wait for them, also suspend + * 3) The coroutine succesfully completed. Return or rethrow its result. + */ + if (result === COROUTINE_SUSPENDED) return COROUTINE_SUSPENDED // (1) + val state = makeCompletingOnce(result) + if (state === COMPLETING_WAITING_CHILDREN) return COROUTINE_SUSPENDED // (2) + afterCompletionUndispatched() + return if (state is CompletedExceptionally) { // (3) + when { + alwaysRethrow || notOwnTimeout(state.cause) -> throw recoverStackTrace(state.cause, uCont) + result is CompletedExceptionally -> throw recoverStackTrace(result.cause, uCont) + else -> result + } + } else { + state.unboxState() + } +} + +private fun ScopeCoroutine<*>.notOwnTimeout(cause: Throwable): Boolean { + return cause !is TimeoutCancellationException || cause.coroutine !== this +} + +private fun ScopeCoroutine<*>.dispatchExceptionAndMakeCompleting(e: DispatchException): Nothing { + makeCompleting(CompletedExceptionally(e.cause)) + throw recoverStackTrace(e.cause, uCont) +} diff --git a/kotlinx-coroutines-core/common/src/selects/OnTimeout.kt b/kotlinx-coroutines-core/common/src/selects/OnTimeout.kt new file mode 100644 index 0000000000..449972648d --- /dev/null +++ b/kotlinx-coroutines-core/common/src/selects/OnTimeout.kt @@ -0,0 +1,61 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.* +import kotlin.time.* + +/** + * Clause that selects the given [block] after a specified timeout passes. + * If timeout is negative or zero, [block] is selected immediately. + * + * **Note: This is an experimental api.** It may be replaced with light-weight timer/timeout channels in the future. + * + * @param timeMillis timeout time in milliseconds. + */ +@ExperimentalCoroutinesApi +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +public fun SelectBuilder.onTimeout(timeMillis: Long, block: suspend () -> R): Unit = + OnTimeout(timeMillis).selectClause.invoke(block) + +/** + * Clause that selects the given [block] after the specified [timeout] passes. + * If timeout is negative or zero, [block] is selected immediately. + * + * **Note: This is an experimental api.** It may be replaced with light-weight timer/timeout channels in the future. + */ +@ExperimentalCoroutinesApi +public fun SelectBuilder.onTimeout(timeout: Duration, block: suspend () -> R): Unit = + onTimeout(timeout.toDelayMillis(), block) + +/** + * We implement [SelectBuilder.onTimeout] as a clause, so each invocation creates + * an instance of [OnTimeout] that specifies the registration part according to + * the [timeout][timeMillis] parameter. + */ +private class OnTimeout( + private val timeMillis: Long +) { + @Suppress("UNCHECKED_CAST") + val selectClause: SelectClause0 + get() = SelectClause0Impl( + clauseObject = this@OnTimeout, + regFunc = OnTimeout::register as RegistrationFunction + ) + + @Suppress("UNUSED_PARAMETER") + private fun register(select: SelectInstance<*>, ignoredParam: Any?) { + // Should this clause complete immediately? + if (timeMillis <= 0) { + select.selectInRegistrationPhase(Unit) + return + } + // Invoke `trySelect` after the timeout is reached. + val action = Runnable { + select.trySelect(this@OnTimeout, Unit) + } + select as SelectImplementation<*> + val context = select.context + val disposableHandle = context.delay.invokeOnTimeout(timeMillis, action, context) + // Do not forget to clean-up when this `select` is completed or cancelled. + select.disposeOnCompletion(disposableHandle) + } +} diff --git a/kotlinx-coroutines-core/common/src/selects/Select.kt b/kotlinx-coroutines-core/common/src/selects/Select.kt new file mode 100644 index 0000000000..a13338c32b --- /dev/null +++ b/kotlinx-coroutines-core/common/src/selects/Select.kt @@ -0,0 +1,903 @@ +package kotlinx.coroutines.selects + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.TrySelectDetailedResult.* +import kotlin.contracts.* +import kotlin.coroutines.* +import kotlin.internal.* +import kotlin.jvm.* + +/** + * Waits for the result of multiple suspending functions simultaneously, which are specified using _clauses_ + * in the [builder] scope of this select invocation. The caller is suspended until one of the clauses + * is either _selected_ or _fails_. + * + * At most one clause is *atomically* selected and its block is executed. The result of the selected clause + * becomes the result of the select. If any clause _fails_, then the select invocation produces the + * corresponding exception. No clause is selected in this case. + * + * This select function is _biased_ to the first clause. When multiple clauses can be selected at the same time, + * the first one of them gets priority. Use [selectUnbiased] for an unbiased (randomized) selection among + * the clauses. + + * There is no `default` clause for select expression. Instead, each selectable suspending function has the + * corresponding non-suspending version that can be used with a regular `when` expression to select one + * of the alternatives or to perform the default (`else`) action if none of them can be immediately selected. + * + * ### List of supported select methods + * + * | **Receiver** | **Suspending function** | **Select clause** + * | ---------------- | --------------------------------------------- | ----------------------------------------------------- + * | [Job] | [join][Job.join] | [onJoin][Job.onJoin] + * | [Deferred] | [await][Deferred.await] | [onAwait][Deferred.onAwait] + * | [SendChannel] | [send][SendChannel.send] | [onSend][SendChannel.onSend] + * | [ReceiveChannel] | [receive][ReceiveChannel.receive] | [onReceive][ReceiveChannel.onReceive] + * | [ReceiveChannel] | [receiveCatching][ReceiveChannel.receiveCatching] | [onReceiveCatching][ReceiveChannel.onReceiveCatching] + * | none | [delay] | [onTimeout][SelectBuilder.onTimeout] + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + * + * Note that this function does not check for cancellation when it is not suspended. + * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. + */ +@OptIn(ExperimentalContracts::class) +public suspend inline fun select(crossinline builder: SelectBuilder.() -> Unit): R { + contract { + callsInPlace(builder, InvocationKind.EXACTLY_ONCE) + } + return SelectImplementation(coroutineContext).run { + builder(this) + // TAIL-CALL OPTIMIZATION: the only + // suspend call is at the last position. + doSelect() + } +} + +/** + * Scope for [select] invocation. + * + * An instance of [SelectBuilder] can only be retrieved as a receiver of a [select] block call, + * and it is only valid during the registration phase of the select builder. + * Any uses outside it lead to unspecified behaviour and are prohibited. + * + * The general rule of thumb is that instances of this type should always be used + * implicitly and there shouldn't be any signatures mentioning this type, + * whether explicitly (e.g. function signature) or implicitly (e.g. inferred `val` type). + */ +public sealed interface SelectBuilder { + /** + * Registers a clause in this [select] expression without additional parameters that does not select any value. + */ + public operator fun SelectClause0.invoke(block: suspend () -> R) + + /** + * Registers clause in this [select] expression without additional parameters that selects value of type [Q]. + */ + public operator fun SelectClause1.invoke(block: suspend (Q) -> R) + + /** + * Registers clause in this [select] expression with additional parameter of type [P] that selects value of type [Q]. + */ + public operator fun SelectClause2.invoke(param: P, block: suspend (Q) -> R) + + /** + * Registers clause in this [select] expression with additional nullable parameter of type [P] + * with the `null` value for this parameter that selects value of type [Q]. + */ + public operator fun SelectClause2.invoke(block: suspend (Q) -> R): Unit = invoke(null, block) + + /** + * Clause that selects the given [block] after a specified timeout passes. + * If timeout is negative or zero, [block] is selected immediately. + * + * **Note: This is an experimental api.** It may be replaced with light-weight timer/timeout channels in the future. + * + * @param timeMillis timeout time in milliseconds. + */ + @ExperimentalCoroutinesApi + @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + @LowPriorityInOverloadResolution + @Deprecated( + message = "Replaced with the same extension function", + level = DeprecationLevel.ERROR, + replaceWith = ReplaceWith(expression = "onTimeout", imports = ["kotlinx.coroutines.selects.onTimeout"]) + ) // Since 1.7.0, was experimental + public fun onTimeout(timeMillis: Long, block: suspend () -> R): Unit = onTimeout(timeMillis, block) +} + +/** + * Each [select] clause is specified with: + * 1) the [object of this clause][clauseObject], + * such as the channel instance for [SendChannel.onSend]; + * 2) the function that specifies how this clause + * should be registered in the object above; + * 3) the function that modifies the internal result + * (passed via [SelectInstance.trySelect] or + * [SelectInstance.selectInRegistrationPhase]) + * to the argument of the user-specified block. + * 4) the function that specifies how the internal result provided via + * [SelectInstance.trySelect] or [SelectInstance.selectInRegistrationPhase] + * should be processed in case of this `select` cancellation while dispatching. + * + * @suppress **This is unstable API, and it is subject to change.** + */ +@InternalCoroutinesApi +public sealed interface SelectClause { + public val clauseObject: Any + public val regFunc: RegistrationFunction + public val processResFunc: ProcessResultFunction + public val onCancellationConstructor: OnCancellationConstructor? +} + +/** + * The registration function specifies how the `select` instance should be registered into + * the specified clause object. In case of channels, the registration logic + * coincides with the plain `send/receive` operation with the only difference that + * the `select` instance is stored as a waiter instead of continuation. + * + * @suppress **This is unstable API, and it is subject to change.** + */ +@InternalCoroutinesApi +public typealias RegistrationFunction = (clauseObject: Any, select: SelectInstance<*>, param: Any?) -> Unit + +/** + * This function specifies how the _internal_ result, provided via [SelectInstance.selectInRegistrationPhase] + * or [SelectInstance.trySelect] should be processed. For example, both [ReceiveChannel.onReceive] and + * [ReceiveChannel.onReceiveCatching] clauses perform exactly the same synchronization logic, + * but differ when the channel has been discovered in the closed or cancelled state. + * + * @suppress **This is unstable API, and it is subject to change.** + */ +@InternalCoroutinesApi +public typealias ProcessResultFunction = (clauseObject: Any, param: Any?, clauseResult: Any?) -> Any? + +/** + * This function specifies how the internal result, provided via [SelectInstance.trySelect] + * or [SelectInstance.selectInRegistrationPhase], should be processed in case of this `select` + * cancellation while dispatching. Unfortunately, we cannot pass this function only in [SelectInstance.trySelect], + * as [SelectInstance.selectInRegistrationPhase] can be called when the coroutine is already cancelled. + * + * @suppress **This is unstable API, and it is subject to change.** + */ +@InternalCoroutinesApi +public typealias OnCancellationConstructor = (select: SelectInstance<*>, param: Any?, internalResult: Any?) -> + (Throwable, Any?, CoroutineContext) -> Unit + +/** + * Clause for [select] expression without additional parameters that does not select any value. + */ +public sealed interface SelectClause0 : SelectClause + +internal class SelectClause0Impl( + override val clauseObject: Any, + override val regFunc: RegistrationFunction, + override val onCancellationConstructor: OnCancellationConstructor? = null +) : SelectClause0 { + override val processResFunc: ProcessResultFunction = DUMMY_PROCESS_RESULT_FUNCTION +} + +private val DUMMY_PROCESS_RESULT_FUNCTION: ProcessResultFunction = { _, _, _ -> null } + +/** + * Clause for [select] expression without additional parameters that selects value of type [Q]. + */ +public sealed interface SelectClause1 : SelectClause + +internal class SelectClause1Impl( + override val clauseObject: Any, + override val regFunc: RegistrationFunction, + override val processResFunc: ProcessResultFunction, + override val onCancellationConstructor: OnCancellationConstructor? = null +) : SelectClause1 + +/** + * Clause for [select] expression with additional parameter of type [P] that selects value of type [Q]. + */ +public sealed interface SelectClause2 : SelectClause + +internal class SelectClause2Impl( + override val clauseObject: Any, + override val regFunc: RegistrationFunction, + override val processResFunc: ProcessResultFunction, + override val onCancellationConstructor: OnCancellationConstructor? = null +) : SelectClause2 + +/** + * Internal representation of `select` instance. + * + * @suppress **This is unstable API, and it is subject to change.** + */ +@InternalCoroutinesApi +public sealed interface SelectInstance { + /** + * The context of the coroutine that is performing this `select` operation. + */ + public val context: CoroutineContext + + /** + * This function should be called by other operations, + * which are trying to perform a rendezvous with this `select`. + * Returns `true` if the rendezvous succeeds, `false` otherwise. + * + * Note that according to the current implementation, a rendezvous attempt can fail + * when either another clause is already selected or this `select` is still in + * REGISTRATION phase. To distinguish the reasons, [SelectImplementation.trySelectDetailed] + * function can be used instead. + */ + public fun trySelect(clauseObject: Any, result: Any?): Boolean + + /** + * When this `select` instance is stored as a waiter, the specified [handle][disposableHandle] + * defines how the stored `select` should be removed in case of cancellation or another clause selection. + */ + public fun disposeOnCompletion(disposableHandle: DisposableHandle) + + /** + * When a clause becomes selected during registration, the corresponding internal result + * (which is further passed to the clause's [ProcessResultFunction]) should be provided + * via this function. After that, other clause registrations are ignored and [trySelect] fails. + */ + public fun selectInRegistrationPhase(internalResult: Any?) +} + +internal interface SelectInstanceInternal : SelectInstance, Waiter + +@PublishedApi +internal open class SelectImplementation( + override val context: CoroutineContext +) : CancelHandler, SelectBuilder, SelectInstanceInternal { + + /** + * Essentially, the `select` operation is split into three phases: REGISTRATION, WAITING, and COMPLETION. + * + * == Phase 1: REGISTRATION == + * In the first REGISTRATION phase, the user-specified [SelectBuilder] is applied, and all the listed clauses + * are registered via the provided [registration functions][SelectClause.regFunc]. Intuitively, `select` clause + * registration is similar to the plain blocking operation, with the only difference that this [SelectInstance] + * is stored as a waiter instead of continuation, and [SelectInstance.trySelect] is used to make a rendezvous. + * Also, when registering, it is possible for the operation to complete immediately, without waiting. In this case, + * [SelectInstance.selectInRegistrationPhase] should be used. Otherwise, when no rendezvous happens and this `select` + * instance is stored as a waiter, a completion handler for the registering clause should be specified via + * [SelectInstance.disposeOnCompletion]; this handler specifies how to remove this `select` instance from the + * clause object when another clause becomes selected or the operation cancels. + * + * After a clause registration is completed, another coroutine can attempt to make a rendezvous with this `select`. + * However, to resolve a race between clauses registration and [SelectInstance.trySelect], the latter fails when + * this `select` is still in REGISTRATION phase. Thus, the corresponding clause has to be registered again. + * + * In this phase, the `state` field stores either a special [STATE_REG] marker or + * a list of clauses to be re-registered due to failed rendezvous attempts. + * + * == Phase 2: WAITING == + * If no rendezvous happens in REGISTRATION phase, the `select` operation moves to WAITING one and suspends until + * [SelectInstance.trySelect] is called. Also, when waiting, this `select` can be cancelled. In the latter case, + * further [SelectInstance.trySelect] attempts fail, and all the completion handlers, specified via + * [SelectInstance.disposeOnCompletion], are invoked to remove this `select` instance from the corresponding + * clause objects. + * + * In this phase, the `state` field stores either the continuation to be later resumed or a special `Cancelled` + * object (with the cancellation cause inside) when this `select` becomes cancelled. + * + * == Phase 3: COMPLETION == + * Once a rendezvous happens either in REGISTRATION phase (via [SelectInstance.selectInRegistrationPhase]) or + * in WAITING phase (via [SelectInstance.trySelect]), this `select` moves to the final `COMPLETION` phase. + * First, the provided internal result is processed via the [ProcessResultFunction] of the selected clause; + * it returns the argument for the user-specified block or throws an exception (see [SendChannel.onSend] as + * an example). After that, this `select` should be removed from all other clause objects by calling the + * corresponding [DisposableHandle]-s, provided via [SelectInstance.disposeOnCompletion] during registration. + * At the end, the user-specified block is called and this `select` finishes. + * + * In this phase, once a rendezvous is happened, the `state` field stores the corresponding clause. + * After that, it moves to [STATE_COMPLETED] to avoid memory leaks. + * + * + * + * The state machine is listed below: + * + * REGISTRATION PHASE WAITING PHASE COMPLETION PHASE + * ⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢ ⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢ ⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢⌢ + * + * +-----------+ +-----------+ + * | CANCELLED | | COMPLETED | + * +-----------+ +-----------+ + * ^ ^ + * INITIAL STATE | | this `select` + * ------------+ | cancelled | is completed + * \ | | + * +=============+ move to +------+ successful +------------+ + * +--| STATE_REG |---------------> | cont |-----------------| ClauseData | + * | +=============+ WAITING phase +------+ trySelect(..) +------------+ + * | ^ | ^ + * | | | some clause has been selected during registration | + * add a | | +-------------------------------------------------------+ + * clause to be | | | + * re-registered | | re-register some clause has been selected | + * | | clauses during registration while there | + * v | are clauses to be re-registered; | + * +------------------+ ignore the latter | + * +--| List |-----------------------------------------------------+ + * | +------------------+ + * | ^ + * | | add one more clause + * | | for re-registration + * +------------+ + * + * One of the most valuable benefits of this `select` design is that it allows processing clauses + * in a way similar to plain operations, such as `send` or `receive` on channels. The only difference + * is that instead of continuation, the operation should store the provided `select` instance object. + * Thus, this design makes it possible to support the `select` expression for any blocking data structure + * in Kotlin Coroutines. + * + * It is worth mentioning that the algorithm above provides "obstruction-freedom" non-blocking guarantee + * instead of the standard "lock-freedom" to avoid using heavy descriptors. In practice, this relaxation + * does not make significant difference. However, it is vital for Kotlin Coroutines to provide some + * non-blocking guarantee, as users may add blocking code in [SelectBuilder], and this blocking code + * should not cause blocking behaviour in other places, such as an attempt to make a rendezvous with + * the `select` that is hang in REGISTRATION phase. + * + * Also, this implementation is NOT linearizable under some circumstances. The reason is that a rendezvous + * attempt with `select` (via [SelectInstance.trySelect]) may fail when this `select` operation is still + * in REGISTRATION phase. Consider the following situation on two empty rendezvous channels `c1` and `c2` + * and the `select` operation that tries to send an element to one of these channels. First, this `select` + * instance is registered as a waiter in `c1`. After that, another thread can observe that `c1` is no longer + * empty and try to receive an element from `c1` -- this receive attempt fails due to the `select` operation + * being in REGISTRATION phase. + * It is also possible to observe that this `select` operation registered in `c2` first, and only after that in + * `c1` (it has to re-register in `c1` after the unsuccessful rendezvous attempt), which is also non-linearizable. + * We, however, find such a non-linearizable behaviour not so important in practice and leverage the correctness + * relaxation for the algorithm simplicity and the non-blocking progress guarantee. + */ + + /** + * The state of this `select` operation. See the description above for details. + */ + private val state = atomic(STATE_REG) + + /** + * Returns `true` if this `select` instance is in the REGISTRATION phase; + * otherwise, returns `false`. + */ + private val inRegistrationPhase + get() = state.value.let { + it === STATE_REG || it is List<*> + } + + /** + * Returns `true` if this `select` is already selected; + * thus, other parties are bound to fail when making a rendezvous with it. + */ + private val isSelected + get() = state.value is SelectImplementation<*>.ClauseData + + /** + * Returns `true` if this `select` is cancelled. + */ + private val isCancelled + get() = state.value === STATE_CANCELLED + + /** + * List of clauses waiting on this `select` instance. + * + * This property is the subject to bening data race: concurrent cancellation might null-out this property + * while [trySelect] operation reads it and iterates over its content. + * A logical race is resolved by the consensus on [state] property. + */ + @BenignDataRace + private var clauses: MutableList? = ArrayList(2) + + /** + * Stores the completion action provided through [disposeOnCompletion] or [invokeOnCancellation] + * during clause registration. After that, if the clause is successfully registered + * (so, it has not completed immediately), this handler is stored into + * the corresponding [ClauseData] instance. + * + * Note that either [DisposableHandle] is provided, or a [Segment] instance with + * the index in it, which specify the location of storing this `select`. + * In the latter case, [Segment.onCancellation] should be called on completion/cancellation. + */ + private var disposableHandleOrSegment: Any? = null + + /** + * In case the disposable handle is specified via [Segment] + * and index in it, implying calling [Segment.onCancellation], + * the corresponding index is stored in this field. + * The segment is stored in [disposableHandleOrSegment]. + */ + private var indexInSegment: Int = -1 + + /** + * Stores the result passed via [selectInRegistrationPhase] during clause registration + * or [trySelect], which is called by another coroutine trying to make a rendezvous + * with this `select` instance. Further, this result is processed via the + * [ProcessResultFunction] of the selected clause. + * + * Unfortunately, we cannot store the result in the [state] field, as the latter stores + * the clause object upon selection (see [ClauseData.clauseObject] and [SelectClause.clauseObject]). + * Instead, it is possible to merge the [internalResult] and [disposableHandle] fields into + * one that stores either result when the clause is successfully registered ([inRegistrationPhase] is `true`), + * or [DisposableHandle] instance when the clause is completed during registration ([inRegistrationPhase] is `false`). + * Yet, this optimization is omitted for code simplicity. + * + * This property is the subject to benign data race: + * [Cleanup][cleanup] procedure can be invoked both as part of the completion sequence + * and as a cancellation handler triggered by an external cancellation. + * In both scenarios, [NO_RESULT] is written to this property via race. + */ + @BenignDataRace + private var internalResult: Any? = NO_RESULT + + /** + * This function is called after the [SelectBuilder] is applied. In case one of the clauses is already selected, + * the algorithm applies the corresponding [ProcessResultFunction] and invokes the user-specified [block][ClauseData.block]. + * Otherwise, it moves this `select` to WAITING phase (re-registering clauses if needed), suspends until a rendezvous + * is happened, and then completes the operation by applying the corresponding [ProcessResultFunction] and + * invoking the user-specified [block][ClauseData.block]. + */ + @PublishedApi + internal open suspend fun doSelect(): R = + if (isSelected) complete() // Fast path + else doSelectSuspend() // Slow path + + // We separate the following logic as it has two suspension points + // and, therefore, breaks the tail-call optimization if it were + // inlined in [doSelect] + private suspend fun doSelectSuspend(): R { + // In case no clause has been selected during registration, + // the `select` operation suspends and waits for a rendezvous. + waitUntilSelected() // <-- suspend call => no tail-call optimization here + // There is a selected clause! Apply the corresponding + // [ProcessResultFunction] and invoke the user-specified block. + return complete() // <-- one more suspend call + } + + // ======================== + // = CLAUSES REGISTRATION = + // ======================== + + override fun SelectClause0.invoke(block: suspend () -> R) = + ClauseData(clauseObject, regFunc, processResFunc, PARAM_CLAUSE_0, block, onCancellationConstructor).register() + + override fun SelectClause1.invoke(block: suspend (Q) -> R) = + ClauseData(clauseObject, regFunc, processResFunc, null, block, onCancellationConstructor).register() + + override fun SelectClause2.invoke(param: P, block: suspend (Q) -> R) = + ClauseData(clauseObject, regFunc, processResFunc, param, block, onCancellationConstructor).register() + + /** + * Attempts to register this `select` clause. If another clause is already selected, + * this function does nothing and completes immediately. + * Otherwise, it registers this `select` instance in + * the [clause object][ClauseData.clauseObject] + * according to the provided [registration function][ClauseData.regFunc]. + * On success, this `select` instance is stored as a waiter + * in the clause object -- the algorithm also stores + * the provided via [disposeOnCompletion] completion action + * and adds the clause to the list of registered one. + * In case of registration failure, the internal result + * (not processed by [ProcessResultFunction] yet) must be + * provided via [selectInRegistrationPhase] -- the algorithm + * updates the state to this clause reference. + */ + @JvmName("register") + internal fun ClauseData.register(reregister: Boolean = false) { + assert { state.value !== STATE_CANCELLED } + // Is there already selected clause? + if (state.value.let { it is SelectImplementation<*>.ClauseData }) return + // For new clauses, check that there does not exist + // another clause with the same object. + if (!reregister) checkClauseObject(clauseObject) + // Try to register in the corresponding object. + if (tryRegisterAsWaiter(this@SelectImplementation)) { + // Successfully registered, and this `select` instance + // is stored as a waiter. Add this clause to the list + // of registered clauses and store the provided via + // [invokeOnCompletion] completion action into the clause. + // + // Importantly, the [waitUntilSelected] function is implemented + // carefully to ensure that the cancellation handler has not been + // installed when clauses re-register, so the logic below cannot + // be invoked concurrently with the clean-up procedure. + // This also guarantees that the list of clauses cannot be cleared + // in the registration phase, so it is safe to read it with "!!". + if (!reregister) clauses!! += this + disposableHandleOrSegment = this@SelectImplementation.disposableHandleOrSegment + indexInSegment = this@SelectImplementation.indexInSegment + this@SelectImplementation.disposableHandleOrSegment = null + this@SelectImplementation.indexInSegment = -1 + } else { + // This clause has been selected! + // Update the state correspondingly. + state.value = this + } + } + + /** + * Checks that there does not exist another clause with the same object. + */ + private fun checkClauseObject(clauseObject: Any) { + // Read the list of clauses, it is guaranteed that it is non-null. + // In fact, it can become `null` only in the clean-up phase, while + // this check can be called only in the registration one. + val clauses = clauses!! + // Check that there does not exist another clause with the same object. + check(clauses.none { it.clauseObject === clauseObject }) { + "Cannot use select clauses on the same object: $clauseObject" + } + } + + override fun disposeOnCompletion(disposableHandle: DisposableHandle) { + this.disposableHandleOrSegment = disposableHandle + } + + /** + * An optimized version for the code below that does not allocate + * a cancellation handler object and efficiently stores the specified + * [segment] and [index]. + * + * ``` + * disposeOnCompletion { + * segment.onCancellation(index, null) + * } + * ``` + */ + override fun invokeOnCancellation(segment: Segment<*>, index: Int) { + this.disposableHandleOrSegment = segment + this.indexInSegment = index + } + + override fun selectInRegistrationPhase(internalResult: Any?) { + this.internalResult = internalResult + } + + // ========================= + // = WAITING FOR SELECTION = + // ========================= + + /** + * Suspends and waits until some clause is selected. However, it is possible for a concurrent + * coroutine to invoke [trySelect] while this `select` is still in REGISTRATION phase. + * In this case, [trySelect] marks the corresponding select clause to be re-registered, and + * this function performs registration of such clauses. After that, it atomically stores + * the continuation into the [state] field if there is no more clause to be re-registered. + */ + private suspend fun waitUntilSelected() = suspendCancellableCoroutine sc@{ cont -> + // Update the state. + state.loop { curState -> + when { + // This `select` is in REGISTRATION phase, and there is no clause to be re-registered. + // Perform a transition to WAITING phase by storing the current continuation. + curState === STATE_REG -> if (state.compareAndSet(curState, cont)) { + // Perform a clean-up in case of cancellation. + // + // Importantly, we MUST install the cancellation handler + // only when the algorithm is bound to suspend. Otherwise, + // a race with [tryRegister] is possible, and the provided + // via [disposeOnCompletion] cancellation action can be ignored. + // Also, we MUST guarantee that this dispose handle is _visible_ + // according to the memory model, and we CAN guarantee this when + // the state is updated. + cont.invokeOnCancellation(this) + return@sc + } + // This `select` is in REGISTRATION phase, but there are clauses that has to be registered again. + // Perform the required registrations and try again. + curState is List<*> -> if (state.compareAndSet(curState, STATE_REG)) { + @Suppress("UNCHECKED_CAST") + curState as List + curState.forEach { reregisterClause(it) } + } + // This `select` operation became completed during clauses re-registration. + curState is SelectImplementation<*>.ClauseData -> { + cont.resume(Unit, curState.createOnCancellationAction(this, internalResult)) + return@sc + } + // This `select` cannot be in any other state. + else -> error("unexpected state: $curState") + } + } + } + + /** + * Re-registers the clause with the specified + * [clause object][clauseObject] after unsuccessful + * [trySelect] of this clause while the `select` + * was still in REGISTRATION phase. + */ + private fun reregisterClause(clauseObject: Any) { + val clause = findClause(clauseObject)!! // it is guaranteed that the corresponding clause is presented + clause.disposableHandleOrSegment = null + clause.indexInSegment = -1 + clause.register(reregister = true) + } + + // ============== + // = RENDEZVOUS = + // ============== + + override fun trySelect(clauseObject: Any, result: Any?): Boolean = + trySelectInternal(clauseObject, result) == TRY_SELECT_SUCCESSFUL + + /** + * Similar to [trySelect] but provides a failure reason + * if this rendezvous is unsuccessful. We need this function + * in the channel implementation. + */ + fun trySelectDetailed(clauseObject: Any, result: Any?) = + TrySelectDetailedResult(trySelectInternal(clauseObject, result)) + + private fun trySelectInternal(clauseObject: Any, internalResult: Any?): Int { + while (true) { + when (val curState = state.value) { + // Perform a rendezvous with this select if it is in WAITING state. + is CancellableContinuation<*> -> { + val clause = findClause(clauseObject) ?: continue // retry if `clauses` is already `null` + val onCancellation = clause.createOnCancellationAction(this@SelectImplementation, internalResult) + if (state.compareAndSet(curState, clause)) { + @Suppress("UNCHECKED_CAST") + val cont = curState as CancellableContinuation + // Success! Store the resumption value and + // try to resume the continuation. + this.internalResult = internalResult + if (cont.tryResume(onCancellation)) return TRY_SELECT_SUCCESSFUL + // If the resumption failed, we need to clean the [result] field to avoid memory leaks. + this.internalResult = NO_RESULT + return TRY_SELECT_CANCELLED + } + } + // Already selected. + STATE_COMPLETED, is SelectImplementation<*>.ClauseData -> return TRY_SELECT_ALREADY_SELECTED + // Already cancelled. + STATE_CANCELLED -> return TRY_SELECT_CANCELLED + // This select is still in REGISTRATION phase, re-register the clause + // in order not to wait until this select moves to WAITING phase. + // This is a rare race, so we do not need to worry about performance here. + STATE_REG -> if (state.compareAndSet(curState, listOf(clauseObject))) return TRY_SELECT_REREGISTER + // This select is still in REGISTRATION phase, and the state stores a list of clauses + // for re-registration, add the selecting clause to this list. + // This is a rare race, so we do not need to worry about performance here. + is List<*> -> if (state.compareAndSet(curState, curState + clauseObject)) return TRY_SELECT_REREGISTER + // Another state? Something went really wrong. + else -> error("Unexpected state: $curState") + } + } + } + + /** + * Finds the clause with the corresponding [clause object][SelectClause.clauseObject]. + * If the reference to the list of clauses is already cleared due to completion/cancellation, + * this function returns `null` + */ + private fun findClause(clauseObject: Any): ClauseData? { + // Read the list of clauses. If the `clauses` field is already `null`, + // the clean-up phase has already completed, and this function returns `null`. + val clauses = this.clauses ?: return null + // Find the clause with the specified clause object. + return clauses.find { it.clauseObject === clauseObject } + ?: error("Clause with object $clauseObject is not found") + } + + // ============== + // = COMPLETION = + // ============== + + /** + * Completes this `select` operation after the internal result is provided + * via [SelectInstance.trySelect] or [SelectInstance.selectInRegistrationPhase]. + * (1) First, this function applies the [ProcessResultFunction] of the selected clause + * to the internal result. + * (2) After that, the [clean-up procedure][cleanup] + * is called to remove this `select` instance from other clause objects, and + * make it possible to collect it by GC after this `select` finishes. + * (3) Finally, the user-specified block is invoked + * with the processed result as an argument. + */ + private suspend fun complete(): R { + assert { isSelected } + // Get the selected clause. + @Suppress("UNCHECKED_CAST") + val selectedClause = state.value as SelectImplementation.ClauseData + // Perform the clean-up before the internal result processing and + // the user-specified block invocation to guarantee the absence + // of memory leaks. Collect the internal result before that. + val internalResult = this.internalResult + cleanup(selectedClause) + // Process the internal result and invoke the user's block. + return if (!RECOVER_STACK_TRACES) { + // TAIL-CALL OPTIMIZATION: the `suspend` block + // is invoked at the very end. + val blockArgument = selectedClause.processResult(internalResult) + selectedClause.invokeBlock(blockArgument) + } else { + // TAIL-CALL OPTIMIZATION: the `suspend` + // function is invoked at the very end. + // However, internally this `suspend` function + // constructs a state machine to recover a + // possible stack-trace. + processResultAndInvokeBlockRecoveringException(selectedClause, internalResult) + } + } + + private suspend fun processResultAndInvokeBlockRecoveringException(clause: ClauseData, internalResult: Any?): R = + try { + val blockArgument = clause.processResult(internalResult) + clause.invokeBlock(blockArgument) + } catch (e: Throwable) { + // In the debug mode, we need to properly recover + // the stack-trace of the exception; the tail-call + // optimization cannot be applied here. + recoverAndThrow(e) + } + + /** + * Invokes all [DisposableHandle]-s provided via + * [SelectInstance.disposeOnCompletion] during + * clause registrations. + */ + private fun cleanup(selectedClause: ClauseData) { + assert { state.value == selectedClause } + // Read the list of clauses. If the `clauses` field is already `null`, + // a concurrent clean-up procedure has already completed, and it is safe to finish. + val clauses = this.clauses ?: return + // Invoke all cancellation handlers except for the + // one related to the selected clause, if specified. + clauses.forEach { clause -> + if (clause !== selectedClause) clause.dispose() + } + // We do need to clean all the data to avoid memory leaks. + this.state.value = STATE_COMPLETED + this.internalResult = NO_RESULT + this.clauses = null + } + + // [CompletionHandler] implementation, must be invoked on cancellation. + override fun invoke(cause: Throwable?) { + // Update the state. + state.update { cur -> + // Finish immediately when this `select` is already completed. + // Notably, this select might be logically completed + // (the `state` field stores the selected `ClauseData`), + // while the continuation is already cancelled. + // We need to invoke the cancellation handler in this case. + if (cur === STATE_COMPLETED) return + STATE_CANCELLED + } + // Read the list of clauses. If the `clauses` field is already `null`, + // a concurrent clean-up procedure has already completed, and it is safe to finish. + val clauses = this.clauses ?: return + // Remove this `select` instance from all the clause object (channels, mutexes, etc.). + clauses.forEach { it.dispose() } + // We do need to clean all the data to avoid memory leaks. + this.internalResult = NO_RESULT + this.clauses = null + } + + /** + * Each `select` clause is internally represented with a [ClauseData] instance. + */ + internal inner class ClauseData( + @JvmField val clauseObject: Any, // the object of this `select` clause: Channel, Mutex, Job, ... + private val regFunc: RegistrationFunction, + private val processResFunc: ProcessResultFunction, + private val param: Any?, // the user-specified param + private val block: Any, // the user-specified block, which should be called if this clause becomes selected + @JvmField val onCancellationConstructor: OnCancellationConstructor? + ) { + @JvmField + var disposableHandleOrSegment: Any? = null + @JvmField + var indexInSegment: Int = -1 + + /** + * Tries to register the specified [select] instance in [clauseObject] and check + * whether the registration succeeded or a rendezvous has happened during the registration. + * This function returns `true` if this [select] is successfully registered and + * is _waiting_ for a rendezvous, or `false` when this clause becomes + * selected during registration. + * + * For example, the [Channel.onReceive] clause registration + * on a non-empty channel retrieves the first element and completes + * the corresponding [select] via [SelectInstance.selectInRegistrationPhase]. + */ + fun tryRegisterAsWaiter(select: SelectImplementation): Boolean { + assert { select.inRegistrationPhase || select.isCancelled } + assert { select.internalResult === NO_RESULT } + regFunc(clauseObject, select, param) + return select.internalResult === NO_RESULT + } + + /** + * Processes the internal result provided via either + * [SelectInstance.selectInRegistrationPhase] or + * [SelectInstance.trySelect] and returns an argument + * for the user-specified [block]. + * + * Importantly, this function may throw an exception + * (e.g., when the channel is closed in [Channel.onSend], the + * corresponding [ProcessResultFunction] is bound to fail). + */ + fun processResult(result: Any?) = processResFunc(clauseObject, param, result) + + /** + * Invokes the user-specified block and returns + * the final result of this `select` clause. + */ + @Suppress("UNCHECKED_CAST") + suspend fun invokeBlock(argument: Any?): R { + val block = block + // We distinguish no-argument and 1-argument + // lambdas via special markers for the clause + // parameters. Specifically, PARAM_CLAUSE_0 + // is always used with [SelectClause0], which + // takes a no-argument lambda. + // + // TAIL-CALL OPTIMIZATION: we invoke + // the `suspend` block at the very end. + return if (this.param === PARAM_CLAUSE_0) { + block as suspend () -> R + block() + } else { + block as suspend (Any?) -> R + block(argument) + } + } + + fun dispose() { + with(disposableHandleOrSegment) { + if (this is Segment<*>) { + this.onCancellation(indexInSegment, null, context) + } else { + (this as? DisposableHandle)?.dispose() + } + } + } + + fun createOnCancellationAction(select: SelectInstance<*>, internalResult: Any?) = + onCancellationConstructor?.invoke(select, param, internalResult) + } +} + +private fun CancellableContinuation.tryResume( + onCancellation: ((cause: Throwable, value: Any?, context: CoroutineContext) -> Unit)? +): Boolean { + val token = + tryResume(Unit, null, onCancellation) ?: return false + completeResume(token) + return true +} + +// trySelectInternal(..) results. +private const val TRY_SELECT_SUCCESSFUL = 0 +private const val TRY_SELECT_REREGISTER = 1 +private const val TRY_SELECT_CANCELLED = 2 +private const val TRY_SELECT_ALREADY_SELECTED = 3 + +// trySelectDetailed(..) results. +internal enum class TrySelectDetailedResult { + SUCCESSFUL, REREGISTER, CANCELLED, ALREADY_SELECTED +} +private fun TrySelectDetailedResult(trySelectInternalResult: Int): TrySelectDetailedResult = when(trySelectInternalResult) { + TRY_SELECT_SUCCESSFUL -> SUCCESSFUL + TRY_SELECT_REREGISTER -> REREGISTER + TRY_SELECT_CANCELLED -> CANCELLED + TRY_SELECT_ALREADY_SELECTED -> ALREADY_SELECTED + else -> error("Unexpected internal result: $trySelectInternalResult") +} + +// Markers for REGISTRATION, COMPLETED, and CANCELLED states. +private val STATE_REG = Symbol("STATE_REG") +private val STATE_COMPLETED = Symbol("STATE_COMPLETED") +private val STATE_CANCELLED = Symbol("STATE_CANCELLED") + +// As the selection result is nullable, we use this special +// marker for the absence of result. +private val NO_RESULT = Symbol("NO_RESULT") + +// We use this marker parameter objects to distinguish +// SelectClause[0,1,2] and invoke the user-specified block correctly. +internal val PARAM_CLAUSE_0 = Symbol("PARAM_CLAUSE_0") diff --git a/kotlinx-coroutines-core/common/src/selects/SelectOld.kt b/kotlinx-coroutines-core/common/src/selects/SelectOld.kt new file mode 100644 index 0000000000..e636a63233 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/selects/SelectOld.kt @@ -0,0 +1,143 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +/* + * For binary compatibility, we need to maintain the previous `select` implementations. + * Thus, we keep [SelectBuilderImpl] and [UnbiasedSelectBuilderImpl] and implement the + * functions marked with `@PublishedApi`. + * + * We keep the old `select` functions as [selectOld] and [selectUnbiasedOld] for test purpose. + */ + +@PublishedApi +internal class SelectBuilderImpl( + uCont: Continuation // unintercepted delegate continuation +) : SelectImplementation(uCont.context) { + private val cont = CancellableContinuationImpl(uCont.intercepted(), MODE_CANCELLABLE) + + @PublishedApi + internal fun getResult(): Any? { + // In the current `select` design, the [select] and [selectUnbiased] functions + // do not wrap the operation in `suspendCoroutineUninterceptedOrReturn` and + // suspend explicitly via [doSelect] call, which returns the final result. + // However, [doSelect] is a suspend function, so it cannot be invoked directly. + // In addition, the `select` builder is eligible to throw an exception, which + // should be handled properly. + // + // As a solution, we: + // 1) check whether the `select` building is already completed with exception, finishing immediately in this case; + // 2) create a CancellableContinuationImpl with the provided unintercepted continuation as a delegate; + // 3) wrap the [doSelect] call in an additional coroutine, which we launch in UNDISPATCHED mode; + // 4) resume the created CancellableContinuationImpl after the [doSelect] invocation completes; + // 5) use CancellableContinuationImpl.getResult() as a result of this function. + if (cont.isCompleted) return cont.getResult() + CoroutineScope(context).launch(start = CoroutineStart.UNDISPATCHED) { + val result = try { + doSelect() + } catch (e: Throwable) { + cont.resumeUndispatchedWithException(e) + return@launch + } + cont.resumeUndispatched(result) + } + return cont.getResult() + } + + @PublishedApi + internal fun handleBuilderException(e: Throwable) { + cont.resumeWithException(e) // will be thrown later via `cont.getResult()` + } +} + +@PublishedApi +internal class UnbiasedSelectBuilderImpl( + uCont: Continuation // unintercepted delegate continuation +) : UnbiasedSelectImplementation(uCont.context) { + private val cont = CancellableContinuationImpl(uCont.intercepted(), MODE_CANCELLABLE) + + @PublishedApi + internal fun initSelectResult(): Any? { + // Here, we do the same trick as in [SelectBuilderImpl]. + if (cont.isCompleted) return cont.getResult() + CoroutineScope(context).launch(start = CoroutineStart.UNDISPATCHED) { + val result = try { + doSelect() + } catch (e: Throwable) { + cont.resumeUndispatchedWithException(e) + return@launch + } + cont.resumeUndispatched(result) + } + return cont.getResult() + } + + @PublishedApi + internal fun handleBuilderException(e: Throwable) { + cont.resumeWithException(e) + } +} + +/* + * This is the old version of `select`. It should work to guarantee binary compatibility. + * + * Internal note: + * We do test it manually by changing the implementation of **new** select with the following: + * ``` + * public suspend inline fun select(crossinline builder: SelectBuilder.() -> Unit): R { + * contract { + * callsInPlace(builder, InvocationKind.EXACTLY_ONCE) + * } + * return selectOld(builder) + * } + * ``` + * + * These signatures are not used by the already compiled code, but their body is. + */ +@PublishedApi +internal suspend inline fun selectOld(crossinline builder: SelectBuilder.() -> Unit): R { + return suspendCoroutineUninterceptedOrReturn { uCont -> + val scope = SelectBuilderImpl(uCont) + try { + builder(scope) + } catch (e: Throwable) { + scope.handleBuilderException(e) + } + scope.getResult() + } +} + +// This is the old version of `selectUnbiased`. It should work to guarantee binary compatibility. +@PublishedApi +internal suspend inline fun selectUnbiasedOld(crossinline builder: SelectBuilder.() -> Unit): R = + suspendCoroutineUninterceptedOrReturn { uCont -> + val scope = UnbiasedSelectBuilderImpl(uCont) + try { + builder(scope) + } catch (e: Throwable) { + scope.handleBuilderException(e) + } + scope.initSelectResult() + } + +@OptIn(ExperimentalStdlibApi::class) +private fun CancellableContinuation.resumeUndispatched(result: T) { + val dispatcher = context[CoroutineDispatcher] + if (dispatcher != null) { + dispatcher.resumeUndispatched(result) + } else { + resume(result) + } +} + +@OptIn(ExperimentalStdlibApi::class) +private fun CancellableContinuation<*>.resumeUndispatchedWithException(exception: Throwable) { + val dispatcher = context[CoroutineDispatcher] + if (dispatcher != null) { + dispatcher.resumeUndispatchedWithException(exception) + } else { + resumeWithException(exception) + } +} diff --git a/kotlinx-coroutines-core/common/src/selects/SelectUnbiased.kt b/kotlinx-coroutines-core/common/src/selects/SelectUnbiased.kt new file mode 100644 index 0000000000..e72c6ed6eb --- /dev/null +++ b/kotlinx-coroutines-core/common/src/selects/SelectUnbiased.kt @@ -0,0 +1,64 @@ +@file:OptIn(ExperimentalContracts::class) + +package kotlinx.coroutines.selects + +import kotlin.contracts.* +import kotlin.coroutines.* + +/** + * Waits for the result of multiple suspending functions simultaneously like [select], but in an _unbiased_ + * way when multiple clauses are selectable at the same time. + * + * This unbiased implementation of `select` expression randomly shuffles the clauses before checking + * if they are selectable, thus ensuring that there is no statistical bias to the selection of the first + * clauses. + * + * See [select] function description for all the other details. + */ +@OptIn(ExperimentalContracts::class) +public suspend inline fun selectUnbiased(crossinline builder: SelectBuilder.() -> Unit): R { + contract { + callsInPlace(builder, InvocationKind.EXACTLY_ONCE) + } + return UnbiasedSelectImplementation(coroutineContext).run { + builder(this) + doSelect() + } +} + +/** + * The unbiased `select` inherits the [standard one][SelectImplementation], + * but does not register clauses immediately. Instead, it stores all of them + * in [clausesToRegister] lists, shuffles and registers them in the beginning of [doSelect] + * (see [shuffleAndRegisterClauses]), and then delegates the rest + * to the parent's [doSelect] implementation. + */ +@PublishedApi +internal open class UnbiasedSelectImplementation(context: CoroutineContext) : SelectImplementation(context) { + private val clausesToRegister: MutableList = arrayListOf() + + override fun SelectClause0.invoke(block: suspend () -> R) { + clausesToRegister += ClauseData(clauseObject, regFunc, processResFunc, PARAM_CLAUSE_0, block, onCancellationConstructor) + } + + override fun SelectClause1.invoke(block: suspend (Q) -> R) { + clausesToRegister += ClauseData(clauseObject, regFunc, processResFunc, null, block, onCancellationConstructor) + } + + override fun SelectClause2.invoke(param: P, block: suspend (Q) -> R) { + clausesToRegister += ClauseData(clauseObject, regFunc, processResFunc, param, block, onCancellationConstructor) + } + + @PublishedApi + override suspend fun doSelect(): R { + shuffleAndRegisterClauses() + return super.doSelect() + } + + private fun shuffleAndRegisterClauses() = try { + clausesToRegister.shuffle() + clausesToRegister.forEach { it.register() } + } finally { + clausesToRegister.clear() + } +} diff --git a/kotlinx-coroutines-core/common/src/selects/WhileSelect.kt b/kotlinx-coroutines-core/common/src/selects/WhileSelect.kt new file mode 100644 index 0000000000..43bb187d7a --- /dev/null +++ b/kotlinx-coroutines-core/common/src/selects/WhileSelect.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.* + +/** + * Loops while [select] expression returns `true`. + * + * The statement of the form: + * + * ``` + * whileSelect { + * /*body*/ + * } + * ``` + * + * is a shortcut for: + * + * ``` + * while(select { + * /*body*/ + * }) {} + * + * **Note: This is an experimental api.** It may be replaced with a higher-performance DSL for selection from loops. + */ +@ExperimentalCoroutinesApi +public suspend inline fun whileSelect(crossinline builder: SelectBuilder.() -> Unit) { + while(select(builder)) { /* do nothing */ } +} diff --git a/kotlinx-coroutines-core/common/src/sync/Mutex.kt b/kotlinx-coroutines-core/common/src/sync/Mutex.kt new file mode 100644 index 0000000000..093c367970 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/sync/Mutex.kt @@ -0,0 +1,313 @@ +package kotlinx.coroutines.sync + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlin.contracts.* +import kotlin.coroutines.CoroutineContext +import kotlin.jvm.* + +/** + * Mutual exclusion for coroutines. + * + * Mutex has two states: _locked_ and _unlocked_. + * It is **non-reentrant**, that is invoking [lock] even from the same thread/coroutine that currently holds + * the lock still suspends the invoker. + * + * JVM API note: + * Memory semantic of the [Mutex] is similar to `synchronized` block on JVM: + * An unlock operation on a [Mutex] happens-before every subsequent successful lock on that [Mutex]. + * Unsuccessful call to [tryLock] do not have any memory effects. + */ +public interface Mutex { + /** + * Returns `true` if this mutex is locked. + */ + public val isLocked: Boolean + + /** + * Tries to lock this mutex, returning `false` if this mutex is already locked. + * + * It is recommended to use [withLock] for safety reasons, so that the acquired lock is always + * released at the end of your critical section, and [unlock] is never invoked before a successful + * lock acquisition. + * + * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex + * is already locked with the same token (same identity), this function throws [IllegalStateException]. + */ + public fun tryLock(owner: Any? = null): Boolean + + /** + * Locks this mutex, suspending caller until the lock is acquired (in other words, while the lock is held elsewhere). + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + * This function releases the lock if it was already acquired by this function before the [CancellationException] + * was thrown. + * + * Note that this function does not check for cancellation when it is not suspended. + * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. + * + * Use [tryLock] to try acquiring the lock without waiting. + * + * This function is fair; suspended callers are resumed in first-in-first-out order. + * + * It is recommended to use [withLock] for safety reasons, so that the acquired lock is always + * released at the end of the critical section, and [unlock] is never invoked before a successful + * lock acquisition. + * + * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex + * is already locked with the same token (same identity), this function throws [IllegalStateException]. + */ + public suspend fun lock(owner: Any? = null) + + /** + * Clause for [select] expression of [lock] suspending function that selects when the mutex is locked. + * Additional parameter for the clause in the `owner` (see [lock]) and when the clause is selected + * the reference to this mutex is passed into the corresponding block. + */ + @Deprecated(level = DeprecationLevel.WARNING, message = "Mutex.onLock deprecated without replacement. " + + "For additional details please refer to #2794") // WARNING since 1.6.0 + public val onLock: SelectClause2 + + /** + * Checks whether this mutex is locked by the specified owner. + * + * @return `true` when this mutex is locked by the specified owner; + * `false` if the mutex is not locked or locked by another owner. + */ + public fun holdsLock(owner: Any): Boolean + + /** + * Unlocks this mutex. Throws [IllegalStateException] if invoked on a mutex that is not locked or + * was locked with a different owner token (by identity). + * + * It is recommended to use [withLock] for safety reasons, so that the acquired lock is always + * released at the end of the critical section, and [unlock] is never invoked before a successful + * lock acquisition. + * + * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex + * was locked with the different token (by identity), this function throws [IllegalStateException]. + */ + public fun unlock(owner: Any? = null) +} + +/** + * Creates a [Mutex] instance. + * The mutex created is fair: lock is granted in first come, first served order. + * + * @param locked initial state of the mutex. + */ +@Suppress("FunctionName") +public fun Mutex(locked: Boolean = false): Mutex = + MutexImpl(locked) + +/** + * Executes the given [action] under this mutex's lock. + * + * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex + * is already locked with the same token (same identity), this function throws [IllegalStateException]. + * + * @return the return value of the action. + */ +@OptIn(ExperimentalContracts::class) +public suspend inline fun Mutex.withLock(owner: Any? = null, action: () -> T): T { + contract { + callsInPlace(action, InvocationKind.EXACTLY_ONCE) + } + lock(owner) + return try { + action() + } finally { + unlock(owner) + } +} + + +internal open class MutexImpl(locked: Boolean) : SemaphoreAndMutexImpl(1, if (locked) 1 else 0), Mutex { + /** + * After the lock is acquired, the corresponding owner is stored in this field. + * The [unlock] operation checks the owner and either re-sets it to [NO_OWNER], + * if there is no waiting request, or to the owner of the suspended [lock] operation + * to be resumed, otherwise. + */ + private val owner = atomic(if (locked) null else NO_OWNER) + + private val onSelectCancellationUnlockConstructor: OnCancellationConstructor = + { _: SelectInstance<*>, owner: Any?, _: Any? -> + { _, _, _ -> unlock(owner) } + } + + override val isLocked: Boolean get() = + availablePermits == 0 + + override fun holdsLock(owner: Any): Boolean = holdsLockImpl(owner) == HOLDS_LOCK_YES + + /** + * [HOLDS_LOCK_UNLOCKED] if the mutex is unlocked + * [HOLDS_LOCK_YES] if the mutex is held with the specified [owner] + * [HOLDS_LOCK_ANOTHER_OWNER] if the mutex is held with a different owner + */ + private fun holdsLockImpl(owner: Any?): Int { + while (true) { + // Is this mutex locked? + if (!isLocked) return HOLDS_LOCK_UNLOCKED + val curOwner = this.owner.value + // Wait in a spin-loop until the owner is set + if (curOwner === NO_OWNER) continue // <-- ATTENTION, BLOCKING PART HERE + // Check the owner + return if (curOwner === owner) HOLDS_LOCK_YES else HOLDS_LOCK_ANOTHER_OWNER + } + } + + override suspend fun lock(owner: Any?) { + if (tryLock(owner)) return + lockSuspend(owner) + } + + private suspend fun lockSuspend(owner: Any?) = suspendCancellableCoroutineReusable { cont -> + val contWithOwner = CancellableContinuationWithOwner(cont, owner) + acquire(contWithOwner) + } + + override fun tryLock(owner: Any?): Boolean = when (tryLockImpl(owner)) { + TRY_LOCK_SUCCESS -> true + TRY_LOCK_FAILED -> false + TRY_LOCK_ALREADY_LOCKED_BY_OWNER -> error("This mutex is already locked by the specified owner: $owner") + else -> error("unexpected") + } + + private fun tryLockImpl(owner: Any?): Int { + while (true) { + if (tryAcquire()) { + assert { this.owner.value === NO_OWNER } + this.owner.value = owner + return TRY_LOCK_SUCCESS + } else { + // The semaphore permit acquisition has failed. + // However, we need to check that this mutex is not + // locked by our owner. + if (owner == null) return TRY_LOCK_FAILED + when (holdsLockImpl(owner)) { + // This mutex is already locked by our owner. + HOLDS_LOCK_YES -> return TRY_LOCK_ALREADY_LOCKED_BY_OWNER + // This mutex is locked by another owner, `trylock(..)` must return `false`. + HOLDS_LOCK_ANOTHER_OWNER -> return TRY_LOCK_FAILED + // This mutex is no longer locked, restart the operation. + HOLDS_LOCK_UNLOCKED -> continue + } + } + } + } + + override fun unlock(owner: Any?) { + while (true) { + // Is this mutex locked? + check(isLocked) { "This mutex is not locked" } + // Read the owner, waiting until it is set in a spin-loop if required. + val curOwner = this.owner.value + if (curOwner === NO_OWNER) continue // <-- ATTENTION, BLOCKING PART HERE + // Check the owner. + check(curOwner === owner || owner == null) { "This mutex is locked by $curOwner, but $owner is expected" } + // Try to clean the owner first. We need to use CAS here to synchronize with concurrent `unlock(..)`-s. + if (!this.owner.compareAndSet(curOwner, NO_OWNER)) continue + // Release the semaphore permit at the end. + release() + return + } + } + + @Suppress("UNCHECKED_CAST", "OverridingDeprecatedMember", "OVERRIDE_DEPRECATION") + override val onLock: SelectClause2 get() = SelectClause2Impl( + clauseObject = this, + regFunc = MutexImpl::onLockRegFunction as RegistrationFunction, + processResFunc = MutexImpl::onLockProcessResult as ProcessResultFunction, + onCancellationConstructor = onSelectCancellationUnlockConstructor + ) + + protected open fun onLockRegFunction(select: SelectInstance<*>, owner: Any?) { + if (owner != null && holdsLock(owner)) { + select.selectInRegistrationPhase(ON_LOCK_ALREADY_LOCKED_BY_OWNER) + } else { + onAcquireRegFunction(SelectInstanceWithOwner(select as SelectInstanceInternal<*>, owner), owner) + } + } + + protected open fun onLockProcessResult(owner: Any?, result: Any?): Any? { + if (result == ON_LOCK_ALREADY_LOCKED_BY_OWNER) { + error("This mutex is already locked by the specified owner: $owner") + } + return this + } + + @OptIn(InternalForInheritanceCoroutinesApi::class) + private inner class CancellableContinuationWithOwner( + @JvmField + val cont: CancellableContinuationImpl, + @JvmField + val owner: Any? + ) : CancellableContinuation by cont, Waiter by cont { + override fun tryResume( + value: R, + idempotent: Any?, + onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)? + ): Any? { + assert { this@MutexImpl.owner.value === NO_OWNER } + val token = cont.tryResume(value, idempotent) { _, _, _ -> + assert { this@MutexImpl.owner.value.let { it === NO_OWNER || it === owner } } + this@MutexImpl.owner.value = owner + unlock(owner) + } + if (token != null) { + assert { this@MutexImpl.owner.value === NO_OWNER } + this@MutexImpl.owner.value = owner + } + return token + } + + override fun resume( + value: R, + onCancellation: ((cause: Throwable, value: R, context: CoroutineContext) -> Unit)? + ) { + assert { this@MutexImpl.owner.value === NO_OWNER } + this@MutexImpl.owner.value = owner + cont.resume(value) { unlock(owner) } + } + } + + private inner class SelectInstanceWithOwner( + @JvmField + val select: SelectInstanceInternal, + @JvmField + val owner: Any? + ) : SelectInstanceInternal by select { + override fun trySelect(clauseObject: Any, result: Any?): Boolean { + assert { this@MutexImpl.owner.value === NO_OWNER } + return select.trySelect(clauseObject, result).also { success -> + if (success) this@MutexImpl.owner.value = owner + } + } + + override fun selectInRegistrationPhase(internalResult: Any?) { + assert { this@MutexImpl.owner.value === NO_OWNER } + this@MutexImpl.owner.value = owner + select.selectInRegistrationPhase(internalResult) + } + } + + override fun toString() = "Mutex@${hexAddress}[isLocked=$isLocked,owner=${owner.value}]" +} + +private val NO_OWNER = Symbol("NO_OWNER") +private val ON_LOCK_ALREADY_LOCKED_BY_OWNER = Symbol("ALREADY_LOCKED_BY_OWNER") + +private const val TRY_LOCK_SUCCESS = 0 +private const val TRY_LOCK_FAILED = 1 +private const val TRY_LOCK_ALREADY_LOCKED_BY_OWNER = 2 + +private const val HOLDS_LOCK_UNLOCKED = 0 +private const val HOLDS_LOCK_YES = 1 +private const val HOLDS_LOCK_ANOTHER_OWNER = 2 diff --git a/kotlinx-coroutines-core/common/src/sync/Semaphore.kt b/kotlinx-coroutines-core/common/src/sync/Semaphore.kt new file mode 100644 index 0000000000..7cc13f81f8 --- /dev/null +++ b/kotlinx-coroutines-core/common/src/sync/Semaphore.kt @@ -0,0 +1,395 @@ +package kotlinx.coroutines.sync + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlin.contracts.* +import kotlin.coroutines.* +import kotlin.js.* +import kotlin.math.* + +/** + * A counting semaphore for coroutines that logically maintains a number of available permits. + * Each [acquire] takes a single permit or suspends until it is available. + * Each [release] adds a permit, potentially releasing a suspended acquirer. + * Semaphore is fair and maintains a FIFO order of acquirers. + * + * Semaphores are mostly used to limit the number of coroutines that have access to particular resource. + * Semaphore with `permits = 1` is essentially a [Mutex]. + **/ +public interface Semaphore { + /** + * Returns the current number of permits available in this semaphore. + */ + public val availablePermits: Int + + /** + * Acquires a permit from this semaphore, suspending until one is available. + * All suspending acquirers are processed in first-in-first-out (FIFO) order. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + * This function releases the semaphore if it was already acquired by this function before the [CancellationException] + * was thrown. + * + * Note that this function does not check for cancellation when it does not suspend. + * Use [CoroutineScope.isActive] or [CoroutineScope.ensureActive] to periodically + * check for cancellation in tight loops if needed. + * + * Use [tryAcquire] to try to acquire a permit of this semaphore without suspension. + */ + public suspend fun acquire() + + /** + * Tries to acquire a permit from this semaphore without suspension. + * + * @return `true` if a permit was acquired, `false` otherwise. + */ + public fun tryAcquire(): Boolean + + /** + * Releases a permit, returning it into this semaphore. Resumes the first + * suspending acquirer if there is one at the point of invocation. + * Throws [IllegalStateException] if the number of [release] invocations is greater than the number of preceding [acquire]. + */ + public fun release() +} + +/** + * Creates new [Semaphore] instance. + * @param permits the number of permits available in this semaphore. + * @param acquiredPermits the number of already acquired permits, + * should be between `0` and `permits` (inclusively). + */ +@Suppress("FunctionName") +public fun Semaphore(permits: Int, acquiredPermits: Int = 0): Semaphore = SemaphoreImpl(permits, acquiredPermits) + +/** + * Executes the given [action], acquiring a permit from this semaphore at the beginning + * and releasing it after the [action] is completed. + * + * @return the return value of the [action]. + */ +@OptIn(ExperimentalContracts::class) +public suspend inline fun Semaphore.withPermit(action: () -> T): T { + contract { + callsInPlace(action, InvocationKind.EXACTLY_ONCE) + } + acquire() + return try { + action() + } finally { + release() + } +} + +@Suppress("UNCHECKED_CAST") +internal open class SemaphoreAndMutexImpl(private val permits: Int, acquiredPermits: Int) { + /* + The queue of waiting acquirers is essentially an infinite array based on the list of segments + (see `SemaphoreSegment`); each segment contains a fixed number of slots. To determine a slot for each enqueue + and dequeue operation, we increment the corresponding counter at the beginning of the operation + and use the value before the increment as a slot number. This way, each enqueue-dequeue pair + works with an individual cell. We use the corresponding segment pointers to find the required ones. + + Here is a state machine for cells. Note that only one `acquire` and at most one `release` operation + can deal with each cell, and that `release` uses `getAndSet(PERMIT)` to perform transitions for performance reasons + so that the state `PERMIT` represents different logical states. + + +------+ `acquire` suspends +------+ `release` tries +--------+ // if `cont.tryResume(..)` succeeds, then + | NULL | -------------------> | cont | -------------------> | PERMIT | (cont RETRIEVED) // the corresponding `acquire` operation gets + +------+ +------+ to resume `cont` +--------+ // a permit and the `release` one completes. + | | + | | `acquire` request is cancelled and the continuation is + | `release` comes | replaced with a special `CANCEL` token to avoid memory leaks + | to the slot before V + | `acquire` and puts +-----------+ `release` has +--------+ + | a permit into the | CANCELLED | -----------------> | PERMIT | (RElEASE FAILED) + | slot, waiting for +-----------+ failed +--------+ + | `acquire` after + | that. + | + | `acquire` gets +-------+ + | +-----------------> | TAKEN | (ELIMINATION HAPPENED) + V | the permit +-------+ + +--------+ | + | PERMIT | -< + +--------+ | + | `release` has waited a bounded time, +--------+ + +---------------------------------------> | BROKEN | (BOTH RELEASE AND ACQUIRE FAILED) + but `acquire` has not come +--------+ + */ + + private val head: AtomicRef + private val deqIdx = atomic(0L) + private val tail: AtomicRef + private val enqIdx = atomic(0L) + + init { + require(permits > 0) { "Semaphore should have at least 1 permit, but had $permits" } + require(acquiredPermits in 0..permits) { "The number of acquired permits should be in 0..$permits" } + val s = SemaphoreSegment(0, null, 2) + head = atomic(s) + tail = atomic(s) + } + + /** + * This counter indicates the number of available permits if it is positive, + * or the negated number of waiters on this semaphore otherwise. + * Note, that 32-bit counter is enough here since the maximal number of available + * permits is [permits] which is [Int], and the maximum number of waiting acquirers + * cannot be greater than 2^31 in any real application. + */ + private val _availablePermits = atomic(permits - acquiredPermits) + val availablePermits: Int get() = max(_availablePermits.value, 0) + + private val onCancellationRelease = { _: Throwable, _: Unit, _: CoroutineContext -> release() } + + fun tryAcquire(): Boolean { + while (true) { + // Get the current number of available permits. + val p = _availablePermits.value + // Is the number of available permits greater + // than the maximal one because of an incorrect + // `release()` call without a preceding `acquire()`? + // Change it to `permits` and start from the beginning. + if (p > permits) { + coerceAvailablePermitsAtMaximum() + continue + } + // Try to decrement the number of available + // permits if it is greater than zero. + if (p <= 0) return false + if (_availablePermits.compareAndSet(p, p - 1)) return true + } + } + + suspend fun acquire() { + // Decrement the number of available permits. + val p = decPermits() + // Is the permit acquired? + if (p > 0) return // permit acquired + // Try to suspend otherwise. + // While it looks better when the following function is inlined, + // it is important to make `suspend` function invocations in a way + // so that the tail-call optimization can be applied here. + acquireSlowPath() + } + + private suspend fun acquireSlowPath() = suspendCancellableCoroutineReusable sc@ { cont -> + // Try to suspend. + if (addAcquireToQueue(cont)) return@sc + // The suspension has been failed + // due to the synchronous resumption mode. + // Restart the whole `acquire`. + acquire(cont) + } + + @JsName("acquireCont") + protected fun acquire(waiter: CancellableContinuation) = acquire( + waiter = waiter, + suspend = { cont -> addAcquireToQueue(cont as Waiter) }, + onAcquired = { cont -> cont.resume(Unit, onCancellationRelease) } + ) + + @JsName("acquireInternal") + private inline fun acquire(waiter: W, suspend: (waiter: W) -> Boolean, onAcquired: (waiter: W) -> Unit) { + while (true) { + // Decrement the number of available permits at first. + val p = decPermits() + // Is the permit acquired? + if (p > 0) { + onAcquired(waiter) + return + } + // Permit has not been acquired, try to suspend. + if (suspend(waiter)) return + } + } + + // We do not fully support `onAcquire` as it is needed only for `Mutex.onLock`. + @Suppress("UNUSED_PARAMETER") + protected fun onAcquireRegFunction(select: SelectInstance<*>, ignoredParam: Any?) = + acquire( + waiter = select, + suspend = { s -> addAcquireToQueue(s as Waiter) }, + onAcquired = { s -> s.selectInRegistrationPhase(Unit) } + ) + + /** + * Decrements the number of available permits + * and ensures that it is not greater than [permits] + * at the point of decrement. The last may happen + * due to an incorrect `release()` call without + * a preceding `acquire()`. + */ + private fun decPermits(): Int { + while (true) { + // Decrement the number of available permits. + val p = _availablePermits.getAndDecrement() + // Is the number of available permits greater + // than the maximal one due to an incorrect + // `release()` call without a preceding `acquire()`? + if (p > permits) continue + // The number of permits is correct, return it. + return p + } + } + + fun release() { + while (true) { + // Increment the number of available permits. + val p = _availablePermits.getAndIncrement() + // Is this `release` call correct and does not + // exceed the maximal number of permits? + if (p >= permits) { + // Revert the number of available permits + // back to the correct one and fail with error. + coerceAvailablePermitsAtMaximum() + error("The number of released permits cannot be greater than $permits") + } + // Is there a waiter that should be resumed? + if (p >= 0) return + // Try to resume the first waiter, and + // restart the operation if either this + // first waiter is cancelled or + // due to `SYNC` resumption mode. + if (tryResumeNextFromQueue()) return + } + } + + /** + * Changes the number of available permits to + * [permits] if it became greater due to an + * incorrect [release] call. + */ + private fun coerceAvailablePermitsAtMaximum() { + while (true) { + val cur = _availablePermits.value + if (cur <= permits) break + if (_availablePermits.compareAndSet(cur, permits)) break + } + } + + /** + * Returns `false` if the received permit cannot be used and the calling operation should restart. + */ + private fun addAcquireToQueue(waiter: Waiter): Boolean { + val curTail = this.tail.value + val enqIdx = enqIdx.getAndIncrement() + val createNewSegment = ::createSegment + val segment = this.tail.findSegmentAndMoveForward(id = enqIdx / SEGMENT_SIZE, startFrom = curTail, + createNewSegment = createNewSegment).segment // cannot be closed + val i = (enqIdx % SEGMENT_SIZE).toInt() + // the regular (fast) path -- if the cell is empty, try to install continuation + if (segment.cas(i, null, waiter)) { // installed continuation successfully + waiter.invokeOnCancellation(segment, i) + return true + } + // On CAS failure -- the cell must be either PERMIT or BROKEN + // If the cell already has PERMIT from tryResumeNextFromQueue, try to grab it + if (segment.cas(i, PERMIT, TAKEN)) { // took permit thus eliminating acquire/release pair + /// This continuation is not yet published, but still can be cancelled via outer job + when (waiter) { + is CancellableContinuation<*> -> { + waiter as CancellableContinuation + waiter.resume(Unit, onCancellationRelease) + } + is SelectInstance<*> -> { + waiter.selectInRegistrationPhase(Unit) + } + else -> error("unexpected: $waiter") + } + return true + } + assert { segment.get(i) === BROKEN } // it must be broken in this case, no other way around it + return false // broken cell, need to retry on a different cell + } + + @Suppress("UNCHECKED_CAST") + private fun tryResumeNextFromQueue(): Boolean { + val curHead = this.head.value + val deqIdx = deqIdx.getAndIncrement() + val id = deqIdx / SEGMENT_SIZE + val createNewSegment = ::createSegment + val segment = this.head.findSegmentAndMoveForward(id, startFrom = curHead, + createNewSegment = createNewSegment).segment // cannot be closed + segment.cleanPrev() + if (segment.id > id) return false + val i = (deqIdx % SEGMENT_SIZE).toInt() + val cellState = segment.getAndSet(i, PERMIT) // set PERMIT and retrieve the prev cell state + when { + cellState === null -> { + // Acquire has not touched this cell yet, wait until it comes for a bounded time + // The cell state can only transition from PERMIT to TAKEN by addAcquireToQueue + repeat(MAX_SPIN_CYCLES) { + if (segment.get(i) === TAKEN) return true + } + // Try to break the slot in order not to wait + return !segment.cas(i, PERMIT, BROKEN) + } + cellState === CANCELLED -> return false // the acquirer has already been cancelled + else -> return cellState.tryResumeAcquire() + } + } + + private fun Any.tryResumeAcquire(): Boolean = when(this) { + is CancellableContinuation<*> -> { + this as CancellableContinuation + val token = tryResume(Unit, null, onCancellationRelease) + if (token != null) { + completeResume(token) + true + } else false + } + is SelectInstance<*> -> { + trySelect(this@SemaphoreAndMutexImpl, Unit) + } + else -> error("unexpected: $this") + } +} + +private class SemaphoreImpl( + permits: Int, acquiredPermits: Int +): SemaphoreAndMutexImpl(permits, acquiredPermits), Semaphore + +private fun createSegment(id: Long, prev: SemaphoreSegment?) = SemaphoreSegment(id, prev, 0) + +private class SemaphoreSegment(id: Long, prev: SemaphoreSegment?, pointers: Int) : Segment(id, prev, pointers) { + val acquirers = atomicArrayOfNulls(SEGMENT_SIZE) + override val numberOfSlots: Int get() = SEGMENT_SIZE + + @Suppress("NOTHING_TO_INLINE") + inline fun get(index: Int): Any? = acquirers[index].value + + @Suppress("NOTHING_TO_INLINE") + inline fun set(index: Int, value: Any?) { + acquirers[index].value = value + } + + @Suppress("NOTHING_TO_INLINE") + inline fun cas(index: Int, expected: Any?, value: Any?): Boolean = acquirers[index].compareAndSet(expected, value) + + @Suppress("NOTHING_TO_INLINE") + inline fun getAndSet(index: Int, value: Any?) = acquirers[index].getAndSet(value) + + // Cleans the acquirer slot located by the specified index + // and removes this segment physically if all slots are cleaned. + override fun onCancellation(index: Int, cause: Throwable?, context: CoroutineContext) { + // Clean the slot + set(index, CANCELLED) + // Remove this segment if needed + onSlotCleaned() + } + + override fun toString() = "SemaphoreSegment[id=$id, hashCode=${hashCode()}]" +} +private val MAX_SPIN_CYCLES = systemProp("kotlinx.coroutines.semaphore.maxSpinCycles", 100) +private val PERMIT = Symbol("PERMIT") +private val TAKEN = Symbol("TAKEN") +private val BROKEN = Symbol("BROKEN") +private val CANCELLED = Symbol("CANCELLED") +private val SEGMENT_SIZE = systemProp("kotlinx.coroutines.semaphore.segmentSize", 16) diff --git a/kotlinx-coroutines-core/common/test/AbstractCoroutineTest.kt b/kotlinx-coroutines-core/common/test/AbstractCoroutineTest.kt new file mode 100644 index 0000000000..ed33729cd3 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/AbstractCoroutineTest.kt @@ -0,0 +1,91 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +@Suppress("DEPRECATION") // cancel(cause) +class AbstractCoroutineTest : TestBase() { + @Test + fun testNotifications() = runTest { + expect(1) + val coroutineContext = coroutineContext // workaround for KT-22984 + val coroutine = object : AbstractCoroutine(coroutineContext, true, false) { + override fun onStart() { + expect(3) + } + + override fun onCancelling(cause: Throwable?) { + assertNull(cause) + expect(5) + } + + override fun onCompleted(value: String) { + assertEquals("OK", value) + expect(6) + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + expectUnreached() + } + } + + coroutine.invokeOnCompletion(onCancelling = true) { + assertNull(it) + expect(7) + } + + coroutine.invokeOnCompletion { + assertNull(it) + expect(8) + } + expect(2) + coroutine.start() + expect(4) + coroutine.resume("OK") + finish(9) + } + + @Test + fun testNotificationsWithException() = runTest { + expect(1) + val coroutineContext = coroutineContext // workaround for KT-22984 + val coroutine = object : AbstractCoroutine(coroutineContext + NonCancellable, true, false) { + override fun onStart() { + expect(3) + } + + override fun onCancelling(cause: Throwable?) { + assertIs(cause) + expect(5) + } + + override fun onCompleted(value: String) { + expectUnreached() + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + assertIs(cause) + expect(8) + } + } + + coroutine.invokeOnCompletion(onCancelling = true) { + assertIs(it) + expect(6) + } + + coroutine.invokeOnCompletion { + assertIs(it) + expect(9) + } + + expect(2) + coroutine.start() + expect(4) + coroutine.cancelCoroutine(TestException1()) + expect(7) + coroutine.resumeWithException(TestException2()) + finish(10) + } +} diff --git a/kotlinx-coroutines-core/common/test/AsyncLazyTest.kt b/kotlinx-coroutines-core/common/test/AsyncLazyTest.kt new file mode 100644 index 0000000000..e87e46158a --- /dev/null +++ b/kotlinx-coroutines-core/common/test/AsyncLazyTest.kt @@ -0,0 +1,184 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class AsyncLazyTest : TestBase() { + + @Test + fun testSimple() = runTest { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + expect(3) + 42 + } + expect(2) + assertTrue(!d.isActive && !d.isCompleted) + assertEquals(d.await(), 42) + assertTrue(!d.isActive && d.isCompleted && !d.isCancelled) + expect(4) + assertEquals(d.await(), 42) // second await -- same result + finish(5) + } + + @Test + fun testLazyDeferAndYield() = runTest { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + expect(3) + yield() // this has not effect, because parent coroutine is waiting + expect(4) + 42 + } + expect(2) + assertTrue(!d.isActive && !d.isCompleted) + assertEquals(d.await(), 42) + assertTrue(!d.isActive && d.isCompleted && !d.isCancelled) + expect(5) + assertEquals(d.await(), 42) // second await -- same result + finish(6) + } + + @Test + fun testLazyDeferAndYield2() = runTest { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + expect(7) + 42 + } + expect(2) + assertTrue(!d.isActive && !d.isCompleted) + launch { // see how it looks from another coroutine + expect(4) + assertTrue(!d.isActive && !d.isCompleted) + yield() // yield back to main + expect(6) + assertTrue(d.isActive && !d.isCompleted) // implicitly started by main's await + yield() // yield to d + } + expect(3) + assertTrue(!d.isActive && !d.isCompleted) + yield() // yield to second child (lazy async is not computing yet) + expect(5) + assertTrue(!d.isActive && !d.isCompleted) + assertEquals(d.await(), 42) // starts computing + assertTrue(!d.isActive && d.isCompleted && !d.isCancelled) + finish(8) + } + + @Test + fun testSimpleException() = runTest( + expected = { it is TestException } + ) { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + finish(3) + throw TestException() + } + expect(2) + assertTrue(!d.isActive && !d.isCompleted) + d.await() // will throw IOException + } + + @Test + fun testLazyDeferAndYieldException() = runTest( + expected = { it is TestException } + ) { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + expect(3) + yield() // this has not effect, because parent coroutine is waiting + finish(4) + throw TestException() + } + expect(2) + assertTrue(!d.isActive && !d.isCompleted) + d.await() // will throw IOException + } + + @Test + fun testCatchException() = runTest { + expect(1) + val d = async(NonCancellable, start = CoroutineStart.LAZY) { + expect(3) + throw TestException() + } + expect(2) + assertTrue(!d.isActive && !d.isCompleted) + try { + d.await() // will throw IOException + } catch (e: TestException) { + assertTrue(!d.isActive && d.isCompleted && d.isCancelled) + expect(4) + } + finish(5) + } + + @Test + fun testStart() = runTest { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + expect(4) + 42 + } + expect(2) + assertTrue(!d.isActive && !d.isCompleted) + assertTrue(d.start()) + assertTrue(d.isActive && !d.isCompleted) + expect(3) + assertTrue(!d.start()) + yield() // yield to started coroutine + assertTrue(!d.isActive && d.isCompleted && !d.isCancelled) // and it finishes + expect(5) + assertEquals(d.await(), 42) // await sees result + finish(6) + } + + @Test + fun testCancelBeforeStart() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + expectUnreached() + 42 + } + expect(2) + assertTrue(!d.isActive && !d.isCompleted) + d.cancel() + assertTrue(!d.isActive && d.isCompleted && d.isCancelled) + assertTrue(!d.start()) + finish(3) + assertEquals(d.await(), 42) // await shall throw CancellationException + expectUnreached() + } + + @Test + fun testCancelWhileComputing() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + expect(4) + yield() // yield to main, that is going to cancel us + expectUnreached() + 42 + } + expect(2) + assertTrue(!d.isActive && !d.isCompleted && !d.isCancelled) + assertTrue(d.start()) + assertTrue(d.isActive && !d.isCompleted && !d.isCancelled) + expect(3) + yield() // yield to d + expect(5) + assertTrue(d.isActive && !d.isCompleted && !d.isCancelled) + d.cancel() + assertTrue(!d.isActive && d.isCancelled) // cancelling ! + assertTrue(!d.isActive && d.isCancelled) // still cancelling + finish(6) + assertEquals(d.await(), 42) // await shall throw CancellationException + expectUnreached() + } +} diff --git a/kotlinx-coroutines-core/common/test/AsyncTest.kt b/kotlinx-coroutines-core/common/test/AsyncTest.kt new file mode 100644 index 0000000000..b9c4797508 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/AsyncTest.kt @@ -0,0 +1,299 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED", "UNREACHABLE_CODE", "USELESS_IS_CHECK") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class AsyncTest : TestBase() { + + @Test + fun testSimple() = runTest { + expect(1) + val d = async { + expect(3) + 42 + } + expect(2) + assertTrue(d.isActive) + assertEquals(d.await(), 42) + assertTrue(!d.isActive) + expect(4) + assertEquals(d.await(), 42) // second await -- same result + finish(5) + } + + @Test + fun testUndispatched() = runTest { + expect(1) + val d = async(start = CoroutineStart.UNDISPATCHED) { + expect(2) + 42 + } + expect(3) + assertTrue(!d.isActive) + assertEquals(d.await(), 42) + finish(4) + } + + @Test + fun testSimpleException() = runTest(expected = { it is TestException }) { + expect(1) + val d = async { + finish(3) + throw TestException() + } + expect(2) + d.await() // will throw TestException + } + + @Test + fun testCancellationWithCause() = runTest { + expect(1) + val d = async(NonCancellable, start = CoroutineStart.ATOMIC) { + expect(3) + yield() + } + expect(2) + d.cancel(TestCancellationException("TEST")) + try { + d.await() + } catch (e: TestCancellationException) { + finish(4) + assertEquals("TEST", e.message) + } + } + + @Test + fun testLostException() = runTest { + expect(1) + val deferred = async(Job()) { + expect(2) + throw Exception() + } + + // Exception is not consumed -> nothing is reported + deferred.join() + finish(3) + } + + @Test + fun testParallelDecompositionCaughtException() = runTest { + val deferred = async(NonCancellable) { + val decomposed = async(NonCancellable) { + throw TestException() + 1 + } + try { + decomposed.await() + } catch (e: TestException) { + 42 + } + } + assertEquals(42, deferred.await()) + } + + @Test + fun testParallelDecompositionCaughtExceptionWithInheritedParent() = runTest { + expect(1) + val deferred = async(NonCancellable) { + expect(2) + val decomposed = async { // inherits parent job! + expect(3) + throw TestException() + 1 + } + try { + decomposed.await() + } catch (e: TestException) { + expect(4) // Should catch this exception, but parent is already cancelled + 42 + } + } + try { + // This will fail + assertEquals(42, deferred.await()) + } catch (e: TestException) { + finish(5) + } + } + + @Test + fun testParallelDecompositionUncaughtExceptionWithInheritedParent() = runTest(expected = { it is TestException }) { + val deferred = async(NonCancellable) { + val decomposed = async { + throw TestException() + 1 + } + + decomposed.await() + } + + deferred.await() + expectUnreached() + } + + @Test + fun testParallelDecompositionUncaughtException() = runTest(expected = { it is TestException }) { + val deferred = async(NonCancellable) { + val decomposed = async { + throw TestException() + 1 + } + + decomposed.await() + } + + deferred.await() + expectUnreached() + } + + @Test + fun testCancellationTransparency() = runTest { + val deferred = async(NonCancellable, start = CoroutineStart.ATOMIC) { + expect(2) + throw TestException() + } + expect(1) + deferred.cancel() + try { + deferred.await() + } catch (e: TestException) { + finish(3) + } + } + + @Test + fun testDeferAndYieldException() = runTest(expected = { it is TestException }) { + expect(1) + val d = async { + expect(3) + yield() // no effect, parent waiting + finish(4) + throw TestException() + } + expect(2) + d.await() // will throw IOException + } + + @Test + fun testDeferWithTwoWaiters() = runTest { + expect(1) + val d = async { + expect(5) + yield() + expect(9) + 42 + } + expect(2) + launch { + expect(6) + assertEquals(d.await(), 42) + expect(11) + } + expect(3) + launch { + expect(7) + assertEquals(d.await(), 42) + expect(12) + } + expect(4) + yield() // this actually yields control to async, which produces results and resumes both waiters (in order) + expect(8) + yield() // yield again to "d", which completes + expect(10) + yield() // yield to both waiters + finish(13) + } + + @Test + fun testDeferBadClass() = runTest { + val bad = BadClass() + val d = async { + expect(1) + bad + } + assertSame(d.await(), bad) + finish(2) + } + + @Test + fun testOverriddenParent() = runTest { + val parent = Job() + val deferred = async(parent, CoroutineStart.ATOMIC) { + expect(2) + delay(Long.MAX_VALUE) + } + + parent.cancel() + try { + expect(1) + deferred.await() + } catch (e: CancellationException) { + finish(3) + } + } + + @Test + fun testIncompleteAsyncState() = runTest { + val deferred = async { + coroutineContext[Job]!!.invokeOnCompletion { } + } + + deferred.await().dispose() + assertIs(deferred.getCompleted()) + assertNull(deferred.getCompletionExceptionOrNull()) + assertTrue(deferred.isCompleted) + assertFalse(deferred.isActive) + assertFalse(deferred.isCancelled) + } + + @Test + fun testIncompleteAsyncFastPath() = runTest { + val deferred = async(Dispatchers.Unconfined) { + coroutineContext[Job]!!.invokeOnCompletion { } + } + + deferred.await().dispose() + assertIs(deferred.getCompleted()) + assertNull(deferred.getCompletionExceptionOrNull()) + assertTrue(deferred.isCompleted) + assertFalse(deferred.isActive) + assertFalse(deferred.isCancelled) + } + + @Test + fun testAsyncWithFinally() = runTest { + expect(1) + + @Suppress("UNREACHABLE_CODE") + val d = async { + expect(3) + try { + yield() // to main, will cancel + } finally { + expect(6) // will go there on await + return@async "Fail" // result will not override cancellation + } + expectUnreached() + "Fail2" + } + expect(2) + yield() // to async + expect(4) + check(d.isActive && !d.isCompleted && !d.isCancelled) + d.cancel() + check(!d.isActive && !d.isCompleted && d.isCancelled) + check(!d.isActive && !d.isCompleted && d.isCancelled) + expect(5) + try { + d.await() // awaits + expectUnreached() // does not complete normally + } catch (e: Throwable) { + expect(7) + check(e is CancellationException) + } + check(!d.isActive && d.isCompleted && d.isCancelled) + finish(8) + } +} diff --git a/kotlinx-coroutines-core/common/test/AtomicCancellationCommonTest.kt b/kotlinx-coroutines-core/common/test/AtomicCancellationCommonTest.kt new file mode 100644 index 0000000000..52d3b8e3a9 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/AtomicCancellationCommonTest.kt @@ -0,0 +1,154 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.selects.* +import kotlinx.coroutines.sync.* +import kotlin.test.* + +class AtomicCancellationCommonTest : TestBase() { + @Test + fun testCancellableLaunch() = runTest { + expect(1) + val job = launch { + expectUnreached() // will get cancelled before start + } + expect(2) + job.cancel() + finish(3) + } + + @Test + fun testAtomicLaunch() = runTest { + expect(1) + val job = launch(start = CoroutineStart.ATOMIC) { + finish(4) // will execute even after it was cancelled + } + expect(2) + job.cancel() + expect(3) + } + + @Test + fun testUndispatchedLaunch() = runTest { + expect(1) + assertFailsWith { + withContext(Job()) { + cancel() + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + yield() + expectUnreached() + } + } + } + finish(3) + } + + @Test + fun testUndispatchedLaunchWithUnconfinedContext() = runTest { + expect(1) + assertFailsWith { + withContext(Dispatchers.Unconfined + Job()) { + cancel() + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + yield() + expectUnreached() + } + } + } + finish(3) + } + + @Test + fun testDeferredAwaitCancellable() = runTest { + expect(1) + val deferred = async { // deferred, not yet complete + expect(4) + "OK" + } + assertEquals(false, deferred.isCompleted) + var job: Job? = null + launch { // will cancel job as soon as deferred completes + expect(5) + assertEquals(true, deferred.isCompleted) + job!!.cancel() + } + job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + deferred.await() // suspends + expectUnreached() // will not execute -- cancelled while dispatched + } finally { + finish(7) // but will execute finally blocks + } + } + expect(3) // continues to execute when the job suspends + yield() // to deferred & canceller + expect(6) + } + + @Test + fun testJobJoinCancellable() = runTest { + expect(1) + val jobToJoin = launch { // not yet complete + expect(4) + } + assertEquals(false, jobToJoin.isCompleted) + var job: Job? = null + launch { // will cancel job as soon as jobToJoin completes + expect(5) + assertEquals(true, jobToJoin.isCompleted) + job!!.cancel() + } + job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + jobToJoin.join() // suspends + expectUnreached() // will not execute -- cancelled while dispatched + } finally { + finish(7) // but will execute finally blocks + } + } + expect(3) // continues to execute when the job suspends + yield() // to jobToJoin & canceller + expect(6) + } + + @Test + fun testLockCancellable() = runTest { + expect(1) + val mutex = Mutex(true) // locked mutex + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + mutex.lock() // suspends + expectUnreached() // should NOT execute because of cancellation + } + expect(3) + mutex.unlock() // unlock mutex first + job.cancel() // cancel the job next + yield() // now yield + finish(4) + } + + @Test + fun testSelectLockCancellable() = runTest { + expect(1) + val mutex = Mutex(true) // locked mutex + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + select { // suspends + mutex.onLock { + expect(4) + "OK" + } + } + expectUnreached() // should NOT execute because of cancellation + } + expect(3) + mutex.unlock() // unlock mutex first + job.cancel() // cancel the job next + yield() // now yield + finish(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/AwaitCancellationTest.kt b/kotlinx-coroutines-core/common/test/AwaitCancellationTest.kt new file mode 100644 index 0000000000..51376d50a3 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/AwaitCancellationTest.kt @@ -0,0 +1,24 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class AwaitCancellationTest : TestBase() { + + @Test + fun testCancellation() = runTest(expected = { it is CancellationException }) { + expect(1) + coroutineScope { + val deferred: Deferred = async { + expect(2) + awaitCancellation() + } + yield() + expect(3) + require(deferred.isActive) + deferred.cancel() + finish(4) + deferred.await() + } + } +} diff --git a/kotlinx-coroutines-core/common/test/AwaitTest.kt b/kotlinx-coroutines-core/common/test/AwaitTest.kt new file mode 100644 index 0000000000..2004285da2 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/AwaitTest.kt @@ -0,0 +1,381 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class AwaitTest : TestBase() { + + @Test + fun testAwaitAll() = runTest { + expect(1) + val d = async { + expect(3) + "OK" + } + + val d2 = async { + yield() + expect(4) + 1L + } + + expect(2) + require(d2.isActive && !d2.isCompleted) + + assertEquals(listOf("OK", 1L), awaitAll(d, d2)) + expect(5) + + require(d.isCompleted && d2.isCompleted) + require(!d.isCancelled && !d2.isCancelled) + finish(6) + } + + @Test + fun testAwaitAllLazy() = runTest { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + expect(2) + 1 + } + val d2 = async(start = CoroutineStart.LAZY) { + expect(3) + 2 + } + assertEquals(listOf(1, 2), awaitAll(d, d2)) + finish(4) + } + + @Test + fun testAwaitAllTyped() = runTest { + val d1 = async { 1L } + val d2 = async { "" } + val d3 = async { } + + assertEquals(listOf(1L, ""), listOf(d1, d2).awaitAll()) + assertEquals(listOf(1L, Unit), listOf(d1, d3).awaitAll()) + assertEquals(listOf("", Unit), listOf(d2, d3).awaitAll()) + } + + @Test + fun testAwaitAllExceptionally() = runTest { + expect(1) + val d = async { + expect(3) + "OK" + } + + val d2 = async(NonCancellable) { + yield() + throw TestException() + } + + val d3 = async { + expect(4) + delay(Long.MAX_VALUE) + 1 + } + + expect(2) + try { + awaitAll(d, d2, d3) + } catch (e: TestException) { + expect(5) + } + + yield() + require(d.isCompleted && d2.isCancelled && d3.isActive) + d3.cancel() + finish(6) + } + + @Test + fun testAwaitAllMultipleExceptions() = runTest { + val d = async(NonCancellable) { + expect(2) + throw TestException() + } + + val d2 = async(NonCancellable) { + yield() + throw TestException() + } + + val d3 = async { + yield() + } + + expect(1) + try { + awaitAll(d, d2, d3) + } catch (e: TestException) { + expect(3) + } + + finish(4) + } + + @Test + fun testAwaitAllCancellation() = runTest { + val outer = async { + + expect(1) + val inner = async { + expect(4) + delay(Long.MAX_VALUE) + } + + expect(2) + awaitAll(inner) + expectUnreached() + } + + yield() + expect(3) + yield() + require(outer.isActive) + outer.cancel() + require(outer.isCancelled) + finish(5) + } + + @Test + fun testAwaitAllPartiallyCompleted() = runTest { + val d1 = async { expect(1); 1 } + d1.await() + val d2 = async { expect(3); 2 } + expect(2) + assertEquals(listOf(1, 2), awaitAll(d1, d2)) + require(d1.isCompleted && d2.isCompleted) + finish(4) + } + + @Test + fun testAwaitAllPartiallyCompletedExceptionally() = runTest { + val d1 = async(NonCancellable) { + expect(1) + throw TestException() + } + + yield() + + // This job is called after exception propagation + val d2 = async { expect(4) } + + expect(2) + try { + awaitAll(d1, d2) + expectUnreached() + } catch (e: TestException) { + expect(3) + } + + require(d2.isActive) + d2.await() + require(d1.isCompleted && d2.isCompleted) + finish(5) + } + + @Test + fun testAwaitAllFullyCompleted() = runTest { + val d1 = CompletableDeferred(Unit) + val d2 = CompletableDeferred(Unit) + val job = async { expect(3) } + expect(1) + awaitAll(d1, d2) + expect(2) + job.await() + finish(4) + } + + @Test + fun testAwaitOnSet() = runTest { + val d1 = CompletableDeferred(Unit) + val d2 = CompletableDeferred(Unit) + val job = async { expect(2) } + expect(1) + listOf(d1, d2, job).awaitAll() + finish(3) + } + + @Test + fun testAwaitAllFullyCompletedExceptionally() = runTest { + val d1 = CompletableDeferred(parent = null) + .apply { completeExceptionally(TestException()) } + val d2 = CompletableDeferred(parent = null) + .apply { completeExceptionally(TestException()) } + val job = async { expect(3) } + expect(1) + try { + awaitAll(d1, d2) + } catch (e: TestException) { + expect(2) + } + + job.await() + finish(4) + } + + @Test + fun testAwaitAllSameJobMultipleTimes() = runTest { + val d = async { "OK" } + // Duplicates are allowed though kdoc doesn't guarantee that + assertEquals(listOf("OK", "OK", "OK"), awaitAll(d, d, d)) + } + + @Test + fun testAwaitAllSameThrowingJobMultipleTimes() = runTest { + val d1 = + async(NonCancellable) { throw TestException() } + val d2 = async { } // do nothing + + try { + expect(1) + // Duplicates are allowed though kdoc doesn't guarantee that + awaitAll(d1, d2, d1, d2) + expectUnreached() + } catch (e: TestException) { + finish(2) + } + } + + @Test + fun testAwaitAllEmpty() = runTest { + expect(1) + assertEquals(emptyList(), awaitAll()) + assertEquals(emptyList(), emptyList>().awaitAll()) + finish(2) + } + + // joinAll + + @Test + fun testJoinAll() = runTest { + val d1 = launch { expect(2) } + val d2 = async { + expect(3) + "OK" + } + val d3 = launch { expect(4) } + + expect(1) + joinAll(d1, d2, d3) + finish(5) + } + + @Test + fun testJoinAllLazy() = runTest { + expect(1) + val d = async(start = CoroutineStart.LAZY) { + expect(2) + } + val d2 = launch(start = CoroutineStart.LAZY) { + expect(3) + } + joinAll(d, d2) + finish(4) + } + + @Test + fun testJoinAllExceptionally() = runTest { + val d1 = launch { + expect(2) + } + val d2 = async(NonCancellable) { + expect(3) + throw TestException() + } + val d3 = async { + expect(4) + } + + expect(1) + joinAll(d1, d2, d3) + finish(5) + } + + @Test + fun testJoinAllCancellation() = runTest { + val outer = launch { + expect(2) + val inner = launch { + expect(3) + delay(Long.MAX_VALUE) + } + + joinAll(inner) + expectUnreached() + } + + expect(1) + yield() + require(outer.isActive) + yield() + outer.cancel() + outer.join() + finish(4) + } + + @Test + fun testJoinAllAlreadyCompleted() = runTest { + val job = launch { + expect(1) + } + + job.join() + expect(2) + + joinAll(job) + finish(3) + } + + @Test + fun testJoinAllEmpty() = runTest { + expect(1) + joinAll() + listOf().joinAll() + finish(2) + } + + @Test + fun testJoinAllSameJob() = runTest { + val job = launch { } + joinAll(job, job, job) + } + + @Test + fun testJoinAllSameJobExceptionally() = runTest { + val job = + async(NonCancellable) { throw TestException() } + joinAll(job, job, job) + } + + @Test + fun testAwaitAllDelegates() = runTest { + expect(1) + val deferred = CompletableDeferred() + @OptIn(InternalForInheritanceCoroutinesApi::class) + val delegate = object : Deferred by deferred {} + launch { + expect(3) + deferred.complete("OK") + } + expect(2) + awaitAll(delegate) + finish(4) + } + + @Test + fun testCancelAwaitAllDelegate() = runTest { + expect(1) + val deferred = CompletableDeferred() + @OptIn(InternalForInheritanceCoroutinesApi::class) + val delegate = object : Deferred by deferred {} + launch { + expect(3) + deferred.cancel() + } + expect(2) + assertFailsWith { awaitAll(delegate) } + finish(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/BuilderContractsTest.kt b/kotlinx-coroutines-core/common/test/BuilderContractsTest.kt new file mode 100644 index 0000000000..b732819ca2 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/BuilderContractsTest.kt @@ -0,0 +1,63 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import kotlin.test.* + +class BuilderContractsTest : TestBase() { + + @Test + fun testContracts() = runTest { + // Coroutine scope + val cs: Int + coroutineScope { + cs = 42 + } + consume(cs) + + // Supervisor scope + val svs: Int + supervisorScope { + svs = 21 + } + consume(svs) + + // with context scope + val wctx: Int + withContext(Dispatchers.Unconfined) { + wctx = 239 + } + consume(wctx) + + val wt: Int + withTimeout(Long.MAX_VALUE) { + wt = 123 + } + consume(wt) + + val s: Int + select { + s = 42 + Job().apply { complete() }.onJoin {} + } + consume(s) + + + val ch: Int + val i = Channel() + i.consume { + ch = 321 + } + consume(ch) + } + + private fun consume(a: Int) { + /* + * Verify the value is actually set correctly + * (non-zero, VerificationError is not triggered, can be read) + */ + assertNotEquals(0, a) + assertEquals(a.hashCode(), a) + } +} diff --git a/kotlinx-coroutines-core/common/test/CancellableContinuationHandlersTest.kt b/kotlinx-coroutines-core/common/test/CancellableContinuationHandlersTest.kt new file mode 100644 index 0000000000..f005c93e5e --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CancellableContinuationHandlersTest.kt @@ -0,0 +1,187 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.test.* + +class CancellableContinuationHandlersTest : TestBase() { + + @Test + fun testDoubleSubscription() = runTest({ it is IllegalStateException }) { + suspendCancellableCoroutine { c -> + c.invokeOnCancellation { finish(1) } + c.invokeOnCancellation { expectUnreached() } + } + } + + @Test + fun testDoubleSubscriptionAfterCompletion() = runTest { + suspendCancellableCoroutine { c -> + c.resume(Unit) + // First invokeOnCancellation is Ok + c.invokeOnCancellation { expectUnreached() } + // Second invokeOnCancellation is not allowed + assertFailsWith { c.invokeOnCancellation { expectUnreached() } } + } + } + + @Test + fun testDoubleSubscriptionAfterCompletionWithException() = runTest { + assertFailsWith { + suspendCancellableCoroutine { c -> + c.resumeWithException(TestException()) + // First invokeOnCancellation is Ok + c.invokeOnCancellation { expectUnreached() } + // Second invokeOnCancellation is not allowed + assertFailsWith { c.invokeOnCancellation { expectUnreached() } } + } + } + } + + @Test + fun testDoubleSubscriptionAfterCancellation() = runTest { + try { + suspendCancellableCoroutine { c -> + c.cancel() + c.invokeOnCancellation { + assertIs(it) + expect(1) + } + assertFailsWith { c.invokeOnCancellation { expectUnreached() } } + } + } catch (e: CancellationException) { + finish(2) + } + } + + @Test + fun testSecondSubscriptionAfterCancellation() = runTest { + try { + suspendCancellableCoroutine { c -> + // Set IOC first + c.invokeOnCancellation { + assertNull(it) + expect(2) + } + expect(1) + // then cancel (it gets called) + c.cancel() + // then try to install another one + assertFailsWith { c.invokeOnCancellation { expectUnreached() } } + } + } catch (e: CancellationException) { + finish(3) + } + } + + @Test + fun testSecondSubscriptionAfterResumeCancelAndDispatch() = runTest { + var cont: CancellableContinuation? = null + val job = launch(start = CoroutineStart.UNDISPATCHED) { + // will be cancelled during dispatch + assertFailsWith { + suspendCancellableCoroutine { c -> + cont = c + // Set IOC first -- not called (completed) + c.invokeOnCancellation { + assertIs(it) + expect(4) + } + expect(1) + } + } + expect(5) + } + expect(2) + // then resume it + cont!!.resume(Unit) // schedule cancelled continuation for dispatch + // then cancel the job during dispatch + job.cancel() + expect(3) + yield() // finish dispatching (will call IOC handler here!) + expect(6) + // then try to install another one after we've done dispatching it + assertFailsWith { + cont!!.invokeOnCancellation { expectUnreached() } + } + finish(7) + } + + @Test + fun testDoubleSubscriptionAfterCancellationWithCause() = runTest { + try { + suspendCancellableCoroutine { c -> + c.cancel(AssertionError()) + c.invokeOnCancellation { + require(it is AssertionError) + expect(1) + } + assertFailsWith { c.invokeOnCancellation { expectUnreached() } } + } + } catch (e: AssertionError) { + finish(2) + } + } + + @Test + fun testDoubleSubscriptionMixed() = runTest { + try { + suspendCancellableCoroutine { c -> + c.invokeOnCancellation { + require(it is IndexOutOfBoundsException) + expect(1) + } + c.cancel(IndexOutOfBoundsException()) + assertFailsWith { c.invokeOnCancellation { expectUnreached() } } + } + } catch (e: IndexOutOfBoundsException) { + finish(2) + } + } + + @Test + fun testExceptionInHandler() = runTest( + unhandled = listOf({ it -> it is CompletionHandlerException }) + ) { + expect(1) + try { + suspendCancellableCoroutine { c -> + c.invokeOnCancellation { throw AssertionError() } + c.cancel() + } + } catch (e: CancellationException) { + expect(2) + } + finish(3) + } + + @Test + fun testSegmentAsHandler() = runTest { + class MySegment : Segment(0, null, 0) { + override val numberOfSlots: Int get() = 0 + + var invokeOnCancellationCalled = false + override fun onCancellation(index: Int, cause: Throwable?, context: CoroutineContext) { + invokeOnCancellationCalled = true + } + } + val s = MySegment() + expect(1) + try { + suspendCancellableCoroutine { c -> + expect(2) + c as CancellableContinuationImpl<*> + c.invokeOnCancellation(s, 0) + c.cancel() + } + } catch (e: CancellationException) { + expect(3) + } + expect(4) + check(s.invokeOnCancellationCalled) + finish(5) + } +} diff --git a/kotlinx-coroutines-core/common/test/CancellableContinuationTest.kt b/kotlinx-coroutines-core/common/test/CancellableContinuationTest.kt new file mode 100644 index 0000000000..43ad996b70 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CancellableContinuationTest.kt @@ -0,0 +1,139 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +class CancellableContinuationTest : TestBase() { + @Test + fun testResumeWithExceptionAndResumeWithException() = runTest { + var continuation: Continuation? = null + val job = launch { + try { + expect(2) + suspendCancellableCoroutine { c -> + continuation = c + } + } catch (e: TestException) { + expect(3) + } + } + expect(1) + yield() + continuation!!.resumeWithException(TestException()) + yield() + assertFailsWith { continuation!!.resumeWithException(TestException()) } + job.join() + finish(4) + } + + @Test + fun testResumeAndResumeWithException() = runTest { + var continuation: Continuation? = null + val job = launch { + expect(2) + suspendCancellableCoroutine { c -> + continuation = c + } + expect(3) + } + expect(1) + yield() + continuation!!.resume(Unit) + job.join() + assertFailsWith { continuation!!.resumeWithException(TestException()) } + finish(4) + } + + @Test + fun testResumeAndResume() = runTest { + var continuation: Continuation? = null + val job = launch { + expect(2) + suspendCancellableCoroutine { c -> + continuation = c + } + expect(3) + } + expect(1) + yield() + continuation!!.resume(Unit) + job.join() + assertFailsWith { continuation!!.resume(Unit) } + finish(4) + } + + /** + * Cancelling outer job may, in practise, race with attempt to resume continuation and resumes + * should be ignored. Here suspended coroutine is cancelled but then resumed with exception. + */ + @Test + fun testCancelAndResumeWithException() = runTest { + var continuation: Continuation? = null + val job = launch { + try { + expect(2) + suspendCancellableCoroutine { c -> + continuation = c + } + } catch (e: CancellationException) { + expect(3) + } + } + expect(1) + yield() + job.cancel() // Cancel job + yield() + continuation!!.resumeWithException(TestException()) // Should not fail + finish(4) + } + + /** + * Cancelling outer job may, in practise, race with attempt to resume continuation and resumes + * should be ignored. Here suspended coroutine is cancelled but then resumed with exception. + */ + @Test + fun testCancelAndResume() = runTest { + var continuation: Continuation? = null + val job = launch { + try { + expect(2) + suspendCancellableCoroutine { c -> + continuation = c + } + } catch (e: CancellationException) { + expect(3) + } + } + expect(1) + yield() + job.cancel() // Cancel job + yield() + continuation!!.resume(Unit) // Should not fail + finish(4) + } + + @Test + fun testCompleteJobWhileSuspended() = runTest { + expect(1) + val completableJob = Job() + val coroutineBlock = suspend { + assertFailsWith { + suspendCancellableCoroutine { cont -> + expect(2) + assertSame(completableJob, cont.context[Job]) + completableJob.complete() + } + expectUnreached() + } + expect(3) + } + coroutineBlock.startCoroutine(Continuation(completableJob) { + assertEquals(Unit, it.getOrNull()) + expect(4) + }) + finish(5) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/CancellableResumeOldTest.kt b/kotlinx-coroutines-core/common/test/CancellableResumeOldTest.kt new file mode 100644 index 0000000000..501d033111 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CancellableResumeOldTest.kt @@ -0,0 +1,291 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +/** + * Test for [CancellableContinuation.resume] with `onCancellation` parameter. + */ +@Suppress("DEPRECATION") +class CancellableResumeOldTest : TestBase() { + @Test + fun testResumeImmediateNormally() = runTest { + expect(1) + val ok = suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { expectUnreached() } + cont.resume("OK") { expectUnreached() } + expect(3) + } + assertEquals("OK", ok) + finish(4) + } + + @Test + fun testResumeImmediateAfterCancel() = runTest( + expected = { it is TestException } + ) { + expect(1) + suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { expect(3) } + cont.cancel(TestException("FAIL")) + expect(4) + cont.resume("OK") { cause -> + expect(5) + assertIs(cause) + } + finish(6) + } + expectUnreached() + } + + @Test + fun testResumeImmediateAfterCancelWithHandlerFailure() = runTest( + expected = { it is TestException }, + unhandled = listOf( + { it is CompletionHandlerException && it.cause is TestException2 }, + { it is CompletionHandlerException && it.cause is TestException3 } + ) + ) { + expect(1) + suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { + expect(3) + throw TestException2("FAIL") // invokeOnCancellation handler fails with exception + } + cont.cancel(TestException("FAIL")) + expect(4) + cont.resume("OK") { cause -> + expect(5) + assertIs(cause) + throw TestException3("FAIL") // onCancellation block fails with exception + } + finish(6) + } + expectUnreached() + } + + @Test + fun testResumeImmediateAfterIndirectCancel() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + val ctx = coroutineContext + suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { expect(3) } + ctx.cancel() + expect(4) + cont.resume("OK") { + expect(5) + } + finish(6) + } + expectUnreached() + } + + @Test + fun testResumeImmediateAfterIndirectCancelWithHandlerFailure() = runTest( + expected = { it is CancellationException }, + unhandled = listOf( + { it is CompletionHandlerException && it.cause is TestException2 }, + { it is CompletionHandlerException && it.cause is TestException3 } + ) + ) { + expect(1) + val ctx = coroutineContext + suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { + expect(3) + throw TestException2("FAIL") // invokeOnCancellation handler fails with exception + } + ctx.cancel() + expect(4) + cont.resume("OK") { + expect(5) + throw TestException3("FAIL") // onCancellation block fails with exception + } + finish(6) + } + expectUnreached() + } + + @Test + fun testResumeLaterNormally() = runTest { + expect(1) + lateinit var cc: CancellableContinuation + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + val ok = suspendCancellableCoroutine { cont -> + expect(3) + cont.invokeOnCancellation { expectUnreached() } + cc = cont + } + assertEquals("OK", ok) + finish(6) + } + expect(4) + cc.resume("OK") { expectUnreached() } + expect(5) + } + + @Test + fun testResumeLaterAfterCancel() = runTest { + expect(1) + lateinit var cc: CancellableContinuation + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + suspendCancellableCoroutine { cont -> + expect(3) + cont.invokeOnCancellation { expect(5) } + cc = cont + } + expectUnreached() + } catch (e: CancellationException) { + finish(9) + } + } + expect(4) + job.cancel(TestCancellationException()) + expect(6) + cc.resume("OK") { cause -> + expect(7) + assertIs(cause) + } + expect(8) + } + + @Test + fun testResumeLaterAfterCancelWithHandlerFailure() = runTest( + unhandled = listOf( + { it is CompletionHandlerException && it.cause is TestException2 }, + { it is CompletionHandlerException && it.cause is TestException3 } + ) + ) { + expect(1) + lateinit var cc: CancellableContinuation + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + suspendCancellableCoroutine { cont -> + expect(3) + cont.invokeOnCancellation { + expect(5) + throw TestException2("FAIL") // invokeOnCancellation handler fails with exception + } + cc = cont + } + expectUnreached() + } catch (e: CancellationException) { + finish(9) + } + } + expect(4) + job.cancel(TestCancellationException()) + expect(6) + cc.resume("OK") { cause -> + expect(7) + assertIs(cause) + throw TestException3("FAIL") // onCancellation block fails with exception + } + expect(8) + } + + @Test + fun testResumeCancelWhileDispatched() = runTest { + expect(1) + lateinit var cc: CancellableContinuation + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + suspendCancellableCoroutine { cont -> + expect(3) + // resumed first, dispatched, then cancelled, but still got invokeOnCancellation call + cont.invokeOnCancellation { cause -> + // Note: invokeOnCancellation is called before cc.resume(value) { ... } handler + expect(7) + assertIs(cause) + } + cc = cont + } + expectUnreached() + } catch (e: CancellationException) { + expect(9) + } + } + expect(4) + cc.resume("OK") { cause -> + // Note: this handler is called after invokeOnCancellation handler + expect(8) + assertIs(cause) + } + expect(5) + job.cancel(TestCancellationException()) // cancel while execution is dispatched + expect(6) + yield() // to coroutine -- throws cancellation exception + finish(10) + } + + @Test + fun testResumeCancelWhileDispatchedWithHandlerFailure() = runTest( + unhandled = listOf( + { it is CompletionHandlerException && it.cause is TestException2 }, + { it is CompletionHandlerException && it.cause is TestException3 } + ) + ) { + expect(1) + lateinit var cc: CancellableContinuation + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + suspendCancellableCoroutine { cont -> + expect(3) + // resumed first, dispatched, then cancelled, but still got invokeOnCancellation call + cont.invokeOnCancellation { cause -> + // Note: invokeOnCancellation is called before cc.resume(value) { ... } handler + expect(7) + assertIs(cause) + throw TestException2("FAIL") // invokeOnCancellation handler fails with exception + } + cc = cont + } + expectUnreached() + } catch (e: CancellationException) { + expect(9) + } + } + expect(4) + cc.resume("OK") { cause -> + // Note: this handler is called after invokeOnCancellation handler + expect(8) + assertIs(cause) + throw TestException3("FAIL") // onCancellation block fails with exception + } + expect(5) + job.cancel(TestCancellationException()) // cancel while execution is dispatched + expect(6) + yield() // to coroutine -- throws cancellation exception + finish(10) + } + + @Test + fun testResumeUnconfined() = runTest { + val outerScope = this + withContext(Dispatchers.Unconfined) { + val result = suspendCancellableCoroutine { + outerScope.launch { + it.resume("OK") { + expectUnreached() + } + } + } + assertEquals("OK", result) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/CancellableResumeTest.kt b/kotlinx-coroutines-core/common/test/CancellableResumeTest.kt new file mode 100644 index 0000000000..c048a8dddf --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CancellableResumeTest.kt @@ -0,0 +1,316 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +/** + * Test for [CancellableContinuation.resume] with `onCancellation` parameter. + */ +class CancellableResumeTest : TestBase() { + @Test + fun testResumeImmediateNormally() = runTest { + expect(1) + val ok = suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { expectUnreached() } + cont.resume("OK") { _, _, _ -> expectUnreached() } + expect(3) + } + assertEquals("OK", ok) + finish(4) + } + + @Test + fun testResumeImmediateAfterCancel() = runTest( + expected = { it is TestException } + ) { + expect(1) + suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { expect(3) } + cont.cancel(TestException("FAIL")) + expect(4) + val value = "OK" + cont.resume(value) { cause, valueToClose, context -> + expect(5) + assertSame(value, valueToClose) + assertSame(context, cont.context) + assertIs(cause) + } + finish(6) + } + expectUnreached() + } + + @Test + fun testResumeImmediateAfterCancelWithHandlerFailure() = runTest( + expected = { it is TestException }, + unhandled = listOf( + { it is CompletionHandlerException && it.cause is TestException2 }, + { it is CompletionHandlerException && it.cause is TestException3 } + ) + ) { + expect(1) + suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { + expect(3) + throw TestException2("FAIL") // invokeOnCancellation handler fails with exception + } + cont.cancel(TestException("FAIL")) + expect(4) + val value = "OK" + cont.resume(value) { cause, valueToClose, context -> + expect(5) + assertSame(value, valueToClose) + assertSame(context, cont.context) + assertIs(cause) + throw TestException3("FAIL") // onCancellation block fails with exception + } + finish(6) + } + expectUnreached() + } + + @Test + fun testResumeImmediateAfterIndirectCancel() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + val ctx = coroutineContext + suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { expect(3) } + ctx.cancel() + expect(4) + val value = "OK" + cont.resume(value) { cause, valueToClose, context -> + expect(5) + assertSame(value, valueToClose) + assertSame(context, cont.context) + assertIs(cause) + } + finish(6) + } + expectUnreached() + } + + @Test + fun testResumeImmediateAfterIndirectCancelWithHandlerFailure() = runTest( + expected = { it is CancellationException }, + unhandled = listOf( + { it is CompletionHandlerException && it.cause is TestException2 }, + { it is CompletionHandlerException && it.cause is TestException3 } + ) + ) { + expect(1) + val ctx = coroutineContext + suspendCancellableCoroutine { cont -> + expect(2) + cont.invokeOnCancellation { + expect(3) + throw TestException2("FAIL") // invokeOnCancellation handler fails with exception + } + ctx.cancel() + expect(4) + val value = "OK" + cont.resume(value) { cause, valueToClose, context -> + expect(5) + assertSame(value, valueToClose) + assertSame(context, cont.context) + assertIs(cause) + throw TestException3("FAIL") // onCancellation block fails with exception + } + finish(6) + } + expectUnreached() + } + + @Test + fun testResumeLaterNormally() = runTest { + expect(1) + lateinit var cc: CancellableContinuation + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + val ok = suspendCancellableCoroutine { cont -> + expect(3) + cont.invokeOnCancellation { expectUnreached() } + cc = cont + } + assertEquals("OK", ok) + finish(6) + } + expect(4) + cc.resume("OK") { _, _, _ -> expectUnreached() } + expect(5) + } + + @Test + fun testResumeLaterAfterCancel() = runTest { + expect(1) + lateinit var cc: CancellableContinuation + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + suspendCancellableCoroutine { cont -> + expect(3) + cont.invokeOnCancellation { expect(5) } + cc = cont + } + expectUnreached() + } catch (_: CancellationException) { + finish(9) + } + } + expect(4) + job.cancel(TestCancellationException()) + expect(6) + val value = "OK" + cc.resume(value) { cause, valueToClose, context -> + expect(7) + assertSame(value, valueToClose) + assertSame(context, cc.context) + assertIs(cause) + } + expect(8) + } + + @Test + fun testResumeLaterAfterCancelWithHandlerFailure() = runTest( + unhandled = listOf( + { it is CompletionHandlerException && it.cause is TestException2 }, + { it is CompletionHandlerException && it.cause is TestException3 } + ) + ) { + expect(1) + lateinit var cc: CancellableContinuation + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + suspendCancellableCoroutine { cont -> + expect(3) + cont.invokeOnCancellation { + expect(5) + throw TestException2("FAIL") // invokeOnCancellation handler fails with exception + } + cc = cont + } + expectUnreached() + } catch (_: CancellationException) { + finish(9) + } + } + expect(4) + job.cancel(TestCancellationException()) + expect(6) + val value = "OK" + cc.resume(value) { cause, valueToClose, context -> + expect(7) + assertSame(value, valueToClose) + assertSame(context, cc.context) + assertIs(cause) + throw TestException3("FAIL") // onCancellation block fails with exception + } + expect(8) + } + + @Test + fun testResumeCancelWhileDispatched() = runTest { + expect(1) + lateinit var cc: CancellableContinuation + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + suspendCancellableCoroutine { cont -> + expect(3) + // resumed first, dispatched, then cancelled, but still got invokeOnCancellation call + cont.invokeOnCancellation { cause -> + // Note: invokeOnCancellation is called before cc.resume(value) { ... } handler + expect(7) + assertIs(cause) + } + cc = cont + } + expectUnreached() + } catch (_: CancellationException) { + expect(9) + } + } + expect(4) + val value = "OK" + cc.resume("OK") { cause, valueToClose, context -> + // Note: this handler is called after invokeOnCancellation handler + expect(8) + assertSame(value, valueToClose) + assertSame(context, cc.context) + assertIs(cause) + } + expect(5) + job.cancel(TestCancellationException()) // cancel while execution is dispatched + expect(6) + yield() // to coroutine -- throws cancellation exception + finish(10) + } + + @Test + fun testResumeCancelWhileDispatchedWithHandlerFailure() = runTest( + unhandled = listOf( + { it is CompletionHandlerException && it.cause is TestException2 }, + { it is CompletionHandlerException && it.cause is TestException3 } + ) + ) { + expect(1) + lateinit var cc: CancellableContinuation + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + suspendCancellableCoroutine { cont -> + expect(3) + // resumed first, dispatched, then cancelled, but still got invokeOnCancellation call + cont.invokeOnCancellation { cause -> + // Note: invokeOnCancellation is called before cc.resume(value) { ... } handler + expect(7) + assertIs(cause) + throw TestException2("FAIL") // invokeOnCancellation handler fails with exception + } + cc = cont + } + expectUnreached() + } catch (_: CancellationException) { + expect(9) + } + } + expect(4) + val value = "OK" + cc.resume(value) { cause, valueToClose, context -> + // Note: this handler is called after invokeOnCancellation handler + expect(8) + assertSame(value, valueToClose) + assertSame(context, cc.context) + assertIs(cause) + throw TestException3("FAIL") // onCancellation block fails with exception + } + expect(5) + job.cancel(TestCancellationException()) // cancel while execution is dispatched + expect(6) + yield() // to coroutine -- throws cancellation exception + finish(10) + } + + @Test + fun testResumeUnconfined() = runTest { + val outerScope = this + withContext(Dispatchers.Unconfined) { + val result = suspendCancellableCoroutine { + outerScope.launch { + it.resume("OK") { _, _, _ -> + expectUnreached() + } + } + } + assertEquals("OK", result) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/CancelledParentAttachTest.kt b/kotlinx-coroutines-core/common/test/CancelledParentAttachTest.kt new file mode 100644 index 0000000000..e34cba4e14 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CancelledParentAttachTest.kt @@ -0,0 +1,113 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.internal.* +import kotlin.test.* + +class CancelledParentAttachTest : TestBase() { + + @Test + fun testAsync() = runTest { + CoroutineStart.entries.forEach { testAsyncCancelledParent(it) } + } + + private suspend fun testAsyncCancelledParent(start: CoroutineStart) { + try { + withContext(Job()) { + cancel() + expect(1) + val d = async(start = start) { 42 } + expect(2) + d.invokeOnCompletion { + finish(3) + reset() + } + } + expectUnreached() + } catch (_: CancellationException) { + // Expected + } + } + + @Test + fun testLaunch() = runTest { + CoroutineStart.entries.forEach { testLaunchCancelledParent(it) } + } + + private suspend fun testLaunchCancelledParent(start: CoroutineStart) { + try { + withContext(Job()) { + cancel() + expect(1) + val d = launch(start = start) { } + expect(2) + d.invokeOnCompletion { + finish(3) + reset() + } + } + expectUnreached() + } catch (_: CancellationException) { + // Expected + } + } + + @Test + fun testProduce() = runTest({ it is CancellationException }) { + cancel() + expect(1) + val d = produce { } + expect(2) + (d as Job).invokeOnCompletion { + finish(3) + reset() + } + } + + @Test + fun testBroadcast() = runTest { + CoroutineStart.entries.forEach { testBroadcastCancelledParent(it) } + } + + @Suppress("DEPRECATION_ERROR") + private suspend fun testBroadcastCancelledParent(start: CoroutineStart) { + try { + withContext(Job()) { + cancel() + expect(1) + val bc = broadcast(start = start) {} + expect(2) + (bc as Job).invokeOnCompletion { + finish(3) + reset() + } + } + expectUnreached() + } catch (_: CancellationException) { + // Expected + } + } + + @Test + fun testScopes() = runTest { + testScope { coroutineScope { } } + testScope { supervisorScope { } } + testScope { flowScope { } } + testScope { withTimeout(Long.MAX_VALUE) { } } + testScope { withContext(Job()) { } } + testScope { withContext(CoroutineName("")) { } } + } + + private suspend inline fun testScope(crossinline block: suspend () -> Unit) { + try { + withContext(Job()) { + cancel() + block() + } + expectUnreached() + } catch (_: CancellationException) { + // Expected + } + } +} diff --git a/kotlinx-coroutines-core/common/test/CompletableDeferredTest.kt b/kotlinx-coroutines-core/common/test/CompletableDeferredTest.kt new file mode 100644 index 0000000000..b2a39218eb --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CompletableDeferredTest.kt @@ -0,0 +1,215 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED", "DEPRECATION") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class CompletableDeferredTest : TestBase() { + @Test + fun testFresh() { + val c = CompletableDeferred() + checkFresh(c) + } + + @Test + fun testComplete() { + val c = CompletableDeferred() + assertEquals(true, c.complete("OK")) + checkCompleteOk(c) + assertEquals("OK", c.getCompleted()) + assertEquals(false, c.complete("OK")) + checkCompleteOk(c) + assertEquals("OK", c.getCompleted()) + } + + @Test + fun testCompleteWithIncompleteResult() { + val c = CompletableDeferred() + assertEquals(true, c.complete(c.invokeOnCompletion { })) + checkCompleteOk(c) + assertEquals(false, c.complete(c.invokeOnCompletion { })) + checkCompleteOk(c) + assertIs(c.getCompleted()) + } + + private fun checkFresh(c: CompletableDeferred<*>) { + assertEquals(true, c.isActive) + assertEquals(false, c.isCancelled) + assertEquals(false, c.isCompleted) + assertThrows { c.getCancellationException() } + assertThrows { c.getCompleted() } + assertThrows { c.getCompletionExceptionOrNull() } + } + + private fun checkCompleteOk(c: CompletableDeferred<*>) { + assertEquals(false, c.isActive) + assertEquals(false, c.isCancelled) + assertEquals(true, c.isCompleted) + assertIs(c.getCancellationException()) + assertNull(c.getCompletionExceptionOrNull()) + } + + private fun checkCancel(c: CompletableDeferred) { + assertEquals(false, c.isActive) + assertEquals(true, c.isCancelled) + assertEquals(true, c.isCompleted) + assertThrows { c.getCompleted() } + assertIs(c.getCompletionExceptionOrNull()) + } + + @Test + fun testCancelWithException() { + val c = CompletableDeferred() + assertEquals(true, c.completeExceptionally(TestException())) + checkCancelWithException(c) + assertEquals(false, c.completeExceptionally(TestException())) + checkCancelWithException(c) + } + + private fun checkCancelWithException(c: CompletableDeferred) { + assertEquals(false, c.isActive) + assertEquals(true, c.isCancelled) + assertEquals(true, c.isCompleted) + assertIs(c.getCancellationException()) + assertThrows { c.getCompleted() } + assertIs(c.getCompletionExceptionOrNull()) + } + + @Test + fun testCompleteWithResultOK() { + val c = CompletableDeferred() + assertEquals(true, c.completeWith(Result.success("OK"))) + checkCompleteOk(c) + assertEquals("OK", c.getCompleted()) + assertEquals(false, c.completeWith(Result.success("OK"))) + checkCompleteOk(c) + assertEquals("OK", c.getCompleted()) + } + + @Test + fun testCompleteWithResultException() { + val c = CompletableDeferred() + assertEquals(true, c.completeWith(Result.failure(TestException()))) + checkCancelWithException(c) + assertEquals(false, c.completeWith(Result.failure(TestException()))) + checkCancelWithException(c) + } + + @Test + fun testParentCancelsChild() { + val parent = Job() + val c = CompletableDeferred(parent) + checkFresh(c) + parent.cancel() + assertEquals(false, parent.isActive) + assertEquals(true, parent.isCancelled) + assertEquals(false, c.isActive) + assertEquals(true, c.isCancelled) + assertEquals(true, c.isCompleted) + assertThrows { c.getCompleted() } + assertIs(c.getCompletionExceptionOrNull()) + } + + @Test + fun testParentActiveOnChildCompletion() { + val parent = Job() + val c = CompletableDeferred(parent) + checkFresh(c) + assertEquals(true, parent.isActive) + assertEquals(true, c.complete("OK")) + checkCompleteOk(c) + assertEquals(true, parent.isActive) + } + + @Test + fun testParentCancelledOnChildException() { + val parent = Job() + val c = CompletableDeferred(parent) + checkFresh(c) + assertEquals(true, parent.isActive) + assertEquals(true, c.completeExceptionally(TestException())) + checkCancelWithException(c) + assertEquals(false, parent.isActive) + assertEquals(true, parent.isCancelled) + } + + @Test + fun testParentActiveOnChildCancellation() { + val parent = Job() + val c = CompletableDeferred(parent) + checkFresh(c) + assertEquals(true, parent.isActive) + c.cancel() + checkCancel(c) + assertEquals(true, parent.isActive) + } + + @Test + fun testAwait() = runTest { + expect(1) + val c = CompletableDeferred() + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + assertEquals("OK", c.await()) // suspends + expect(5) + assertEquals("OK", c.await()) // does not suspend + expect(6) + } + expect(3) + c.complete("OK") + expect(4) + yield() // to launch + finish(7) + } + + @Test + fun testCancelAndAwaitParentWaitChildren() = runTest { + expect(1) + val parent = CompletableDeferred() + launch(parent, start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + yield() // will get cancelled + } finally { + expect(5) + } + } + expect(3) + parent.cancel() + expect(4) + try { + parent.await() + } catch (e: CancellationException) { + finish(6) + } + } + + @Test + fun testCompleteAndAwaitParentWaitChildren() = runTest { + expect(1) + val parent = CompletableDeferred() + launch(parent, start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + yield() // will get cancelled + } finally { + expect(5) + } + } + expect(3) + parent.complete("OK") + expect(4) + assertEquals("OK", parent.await()) + finish(6) + } + + private inline fun assertThrows(block: () -> Unit) { + try { + block() + fail("Should not complete normally") + } catch (e: Throwable) { + assertIs(e) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/CompletableJobTest.kt b/kotlinx-coroutines-core/common/test/CompletableJobTest.kt new file mode 100644 index 0000000000..697581aeb2 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CompletableJobTest.kt @@ -0,0 +1,99 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class CompletableJobTest : TestBase() { + @Test + fun testComplete() { + val job = Job() + assertTrue(job.isActive) + assertFalse(job.isCompleted) + assertTrue(job.complete()) + assertTrue(job.isCompleted) + assertFalse(job.isActive) + assertFalse(job.isCancelled) + assertFalse(job.complete()) + } + + @Test + fun testCompleteWithException() { + val job = Job() + assertTrue(job.isActive) + assertFalse(job.isCompleted) + assertTrue(job.completeExceptionally(TestException())) + assertTrue(job.isCompleted) + assertFalse(job.isActive) + assertTrue(job.isCancelled) + assertFalse(job.completeExceptionally(TestException())) + assertFalse(job.complete()) + } + + @Test + fun testCompleteWithChildren() { + val parent = Job() + val child = Job(parent) + assertTrue(parent.complete()) + assertFalse(parent.complete()) + assertTrue(parent.isActive) + assertFalse(parent.isCompleted) + assertTrue(child.complete()) + assertTrue(child.isCompleted) + assertTrue(parent.isCompleted) + assertFalse(child.isActive) + assertFalse(parent.isActive) + } + + @Test + fun testExceptionIsNotReportedToChildren() = parametrized { job -> + expect(1) + val child = launch(job) { + expect(2) + try { + // KT-33840 + hang {} + } catch (e: Throwable) { + assertIs(e) + assertIs(if (RECOVER_STACK_TRACES) e.cause?.cause else e.cause) + expect(4) + throw e + } + } + yield() + expect(3) + job.completeExceptionally(TestException()) + child.join() + finish(5) + } + + @Test + fun testCompleteExceptionallyDoesntAffectDeferred() = parametrized { job -> + expect(1) + val child = async(job) { + expect(2) + try { + // KT-33840 + hang {} + } catch (e: Throwable) { + assertIs(e) + assertIs(if (RECOVER_STACK_TRACES) e.cause?.cause else e.cause) + expect(4) + throw e + } + } + yield() + expect(3) + job.completeExceptionally(TestException()) + child.join() + assertTrue { child.getCompletionExceptionOrNull() is CancellationException } + finish(5) + } + + private fun parametrized(block: suspend CoroutineScope.(CompletableJob) -> Unit) { + runTest { + block(Job()) + reset() + block(SupervisorJob()) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/CoroutineDispatcherOperatorFunInvokeTest.kt b/kotlinx-coroutines-core/common/test/CoroutineDispatcherOperatorFunInvokeTest.kt new file mode 100644 index 0000000000..0a643cb788 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CoroutineDispatcherOperatorFunInvokeTest.kt @@ -0,0 +1,74 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.ContinuationInterceptor +import kotlin.coroutines.CoroutineContext +import kotlin.test.* + +class CoroutineDispatcherOperatorFunInvokeTest : TestBase() { + + /** + * Copy pasted from [WithContextTest.testThrowException], + * then edited to use operator. + */ + @Test + fun testThrowException() = runTest { + expect(1) + try { + (wrappedCurrentDispatcher()) { + expect(2) + throw AssertionError() + } + } catch (e: AssertionError) { + expect(3) + } + + yield() + finish(4) + } + + /** + * Copy pasted from [WithContextTest.testWithContextChildWaitSameContext], + * then edited to use operator fun invoke for [CoroutineDispatcher]. + */ + @Test + fun testWithContextChildWaitSameContext() = runTest { + expect(1) + (wrappedCurrentDispatcher()) { + expect(2) + launch { + // ^^^ schedules to main thread + expect(4) // waits before return + } + expect(3) + "OK".wrap() + }.unwrap() + finish(5) + } + + private class Wrapper(val value: String) : Incomplete { + override val isActive: Boolean + get() = error("") + override val list: NodeList? + get() = error("") + } + + private fun String.wrap() = Wrapper(this) + private fun Wrapper.unwrap() = value + + private fun CoroutineScope.wrappedCurrentDispatcher() = object : CoroutineDispatcher() { + val dispatcher = coroutineContext[ContinuationInterceptor] as CoroutineDispatcher + override fun dispatch(context: CoroutineContext, block: Runnable) { + dispatcher.dispatch(context, block) + } + + override fun isDispatchNeeded(context: CoroutineContext): Boolean { + return dispatcher.isDispatchNeeded(context) + } + + @InternalCoroutinesApi + override fun dispatchYield(context: CoroutineContext, block: Runnable) { + dispatcher.dispatchYield(context, block) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/CoroutineExceptionHandlerTest.kt b/kotlinx-coroutines-core/common/test/CoroutineExceptionHandlerTest.kt new file mode 100644 index 0000000000..5549b08836 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CoroutineExceptionHandlerTest.kt @@ -0,0 +1,44 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class CoroutineExceptionHandlerTest : TestBase() { + // Parent Job() does not handle exception --> handler is invoked on child crash + @Test + fun testJob() = runTest { + expect(1) + var coroutineException: Throwable? = null + val handler = CoroutineExceptionHandler { _, ex -> + coroutineException = ex + expect(3) + } + val parent = Job() + val job = launch(handler + parent) { + throw TestException() + } + expect(2) + job.join() + finish(4) + assertIs(coroutineException) + assertTrue(parent.isCancelled) + } + + // Parent CompletableDeferred() "handles" exception --> handler is NOT invoked on child crash + @Test + fun testCompletableDeferred() = runTest { + expect(1) + val handler = CoroutineExceptionHandler { _, _ -> + expectUnreached() + } + val parent = CompletableDeferred() + val job = launch(handler + parent) { + throw TestException() + } + expect(2) + job.join() + finish(3) + assertTrue(parent.isCancelled) + assertIs(parent.getCompletionExceptionOrNull()) + } +} diff --git a/kotlinx-coroutines-core/common/test/CoroutineScopeTest.kt b/kotlinx-coroutines-core/common/test/CoroutineScopeTest.kt new file mode 100644 index 0000000000..94a9d0cedd --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CoroutineScopeTest.kt @@ -0,0 +1,313 @@ +@file:Suppress("UNREACHABLE_CODE") + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.test.* + +class CoroutineScopeTest : TestBase() { + @Test + fun testScope() = runTest { + suspend fun callJobScoped() = coroutineScope { + expect(2) + launch { + expect(4) + } + launch { + expect(5) + + launch { + expect(7) + } + + expect(6) + + } + expect(3) + 42 + } + expect(1) + val result = callJobScoped() + assertEquals(42, result) + yield() // Check we're not cancelled + finish(8) + } + + @Test + fun testScopeCancelledFromWithin() = runTest { + expect(1) + suspend fun callJobScoped() = coroutineScope { + launch { + expect(2) + delay(Long.MAX_VALUE) + } + launch { + expect(3) + throw TestException2() + } + } + + try { + callJobScoped() + expectUnreached() + } catch (e: TestException2) { + expect(4) + } + yield() // Check we're not cancelled + finish(5) + } + + @Test + fun testExceptionFromWithin() = runTest { + expect(1) + try { + expect(2) + coroutineScope { + expect(3) + throw TestException1() + } + expectUnreached() + } catch (e: TestException1) { + finish(4) + } + } + + @Test + fun testScopeBlockThrows() = runTest { + expect(1) + suspend fun callJobScoped(): Unit = coroutineScope { + launch { + expect(2) + delay(Long.MAX_VALUE) + } + yield() // let launch sleep + throw TestException1() + } + try { + callJobScoped() + expectUnreached() + } catch (e: TestException1) { + expect(3) + } + yield() // Check we're not cancelled + finish(4) + } + + @Test + fun testOuterJobIsCancelled() = runTest { + suspend fun callJobScoped() = coroutineScope { + launch { + expect(3) + try { + delay(Long.MAX_VALUE) + } finally { + expect(4) + } + } + + expect(2) + delay(Long.MAX_VALUE) + 42 + } + + val outerJob = launch(NonCancellable) { + expect(1) + try { + callJobScoped() + expectUnreached() + } catch (e: JobCancellationException) { + expect(5) + if (RECOVER_STACK_TRACES) { + val cause = e.cause as JobCancellationException // shall be recovered JCE + assertNull(cause.cause) + } else { + assertNull(e.cause) + } + } + } + repeat(3) { yield() } // let everything to start properly + outerJob.cancel() + outerJob.join() + finish(6) + } + + @Test + fun testAsyncCancellationFirst() = runTest { + try { + expect(1) + failedConcurrentSumFirst() + expectUnreached() + } catch (e: TestException1) { + finish(6) + } + } + + // First async child fails -> second is cancelled + private suspend fun failedConcurrentSumFirst(): Int = coroutineScope { + val one = async { + expect(3) + throw TestException1() + } + val two = async(start = CoroutineStart.ATOMIC) { + try { + expect(4) + delay(Long.MAX_VALUE) // Emulates very long computation + 42 + } finally { + expect(5) + } + } + expect(2) + one.await() + two.await() + } + + @Test + fun testAsyncCancellationSecond() = runTest { + try { + expect(1) + failedConcurrentSumSecond() + expectUnreached() + } catch (e: TestException1) { + finish(6) + } + } + + // Second async child fails -> fist is cancelled + private suspend fun failedConcurrentSumSecond(): Int = coroutineScope { + val one = async { + try { + expect(3) + delay(Long.MAX_VALUE) // Emulates very long computation + 42 + } finally { + expect(5) + } + } + val two = async(start = CoroutineStart.ATOMIC) { + expect(4) + throw TestException1() + } + expect(2) + one.await() + two.await() + } + + @Test + @Suppress("UNREACHABLE_CODE") + fun testDocumentationExample() = runTest { + suspend fun loadData() = coroutineScope { + expect(1) + val data = async { + try { + delay(Long.MAX_VALUE) + } finally { + expect(3) + } + } + yield() + // UI updater + withContext(coroutineContext) { + expect(2) + throw TestException1() + data.await() // Actually unreached + expectUnreached() + } + } + + try { + loadData() + expectUnreached() + } catch (e: TestException1) { + finish(4) + } + } + + @Test + fun testCoroutineScopeCancellationVsException() = runTest { + expect(1) + var job: Job? = null + job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + coroutineScope { + expect(3) + yield() // must suspend + expect(5) + job!!.cancel() // cancel this job _before_ it throws + throw TestException1() + } + } catch (e: TestException1) { + // must have caught TextException + expect(6) + } + } + expect(4) + yield() // to coroutineScope + finish(7) + } + + @Test + fun testLaunchContainsDefaultDispatcher() = runTest { + val scopeWithoutDispatcher = CoroutineScope(coroutineContext.minusKey(ContinuationInterceptor)) + scopeWithoutDispatcher.launch(Dispatchers.Default) { + assertSame(Dispatchers.Default, coroutineContext[ContinuationInterceptor]) + }.join() + scopeWithoutDispatcher.launch { + assertSame(Dispatchers.Default, coroutineContext[ContinuationInterceptor]) + }.join() + } + + @Test + fun testNewCoroutineContextDispatcher() { + fun newContextDispatcher(c1: CoroutineContext, c2: CoroutineContext) = + ContextScope(c1).newCoroutineContext(c2)[ContinuationInterceptor] + + assertSame(Dispatchers.Default, newContextDispatcher(EmptyCoroutineContext, EmptyCoroutineContext)) + assertSame(Dispatchers.Default, newContextDispatcher(EmptyCoroutineContext, Dispatchers.Default)) + assertSame(Dispatchers.Default, newContextDispatcher(Dispatchers.Default, EmptyCoroutineContext)) + assertSame(Dispatchers.Default, newContextDispatcher(Dispatchers.Default, Dispatchers.Default)) + assertSame(Dispatchers.Default, newContextDispatcher(Dispatchers.Unconfined, Dispatchers.Default)) + assertSame(Dispatchers.Unconfined, newContextDispatcher(Dispatchers.Default, Dispatchers.Unconfined)) + assertSame(Dispatchers.Unconfined, newContextDispatcher(Dispatchers.Unconfined, Dispatchers.Unconfined)) + } + + @Test + fun testScopePlusContext() { + assertSame(EmptyCoroutineContext, scopePlusContext(EmptyCoroutineContext, EmptyCoroutineContext)) + assertSame(Dispatchers.Default, scopePlusContext(EmptyCoroutineContext, Dispatchers.Default)) + assertSame(Dispatchers.Default, scopePlusContext(Dispatchers.Default, EmptyCoroutineContext)) + assertSame(Dispatchers.Default, scopePlusContext(Dispatchers.Default, Dispatchers.Default)) + assertSame(Dispatchers.Default, scopePlusContext(Dispatchers.Unconfined, Dispatchers.Default)) + assertSame(Dispatchers.Unconfined, scopePlusContext(Dispatchers.Default, Dispatchers.Unconfined)) + assertSame(Dispatchers.Unconfined, scopePlusContext(Dispatchers.Unconfined, Dispatchers.Unconfined)) + } + + @Test + fun testIncompleteScopeState() = runTest { + lateinit var scopeJob: Job + coroutineScope { + scopeJob = coroutineContext[Job]!! + scopeJob.invokeOnCompletion { } + } + + scopeJob.join() + assertTrue(scopeJob.isCompleted) + assertFalse(scopeJob.isActive) + assertFalse(scopeJob.isCancelled) + } + + private fun scopePlusContext(c1: CoroutineContext, c2: CoroutineContext) = + (ContextScope(c1) + c2).coroutineContext + + @Test + fun testIsActiveWithoutJob() { + var invoked = false + suspend fun testIsActive() { + assertTrue(coroutineContext.isActive) + invoked = true + } + ::testIsActive.startCoroutine(Continuation(EmptyCoroutineContext){}) + assertTrue(invoked) + } +} diff --git a/kotlinx-coroutines-core/common/test/CoroutinesTest.kt b/kotlinx-coroutines-core/common/test/CoroutinesTest.kt new file mode 100644 index 0000000000..8cd149ef23 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/CoroutinesTest.kt @@ -0,0 +1,332 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class CoroutinesTest : TestBase() { + + @Test + fun testSimple() = runTest { + expect(1) + finish(2) + } + + @Test + fun testYield() = runTest { + expect(1) + yield() // effectively does nothing, as we don't have other coroutines + finish(2) + } + + @Test + fun testLaunchAndYieldJoin() = runTest { + expect(1) + val job = launch { + expect(3) + yield() + expect(4) + } + expect(2) + assertTrue(job.isActive && !job.isCompleted) + job.join() + assertTrue(!job.isActive && job.isCompleted) + finish(5) + } + + @Test + fun testLaunchUndispatched() = runTest { + expect(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + yield() + expect(4) + } + expect(3) + assertTrue(job.isActive && !job.isCompleted) + job.join() + assertTrue(!job.isActive && job.isCompleted) + finish(5) + } + + @Test + fun testNested() = runTest { + expect(1) + val j1 = launch { + expect(3) + val j2 = launch { + expect(5) + } + expect(4) + j2.join() + expect(6) + } + expect(2) + j1.join() + finish(7) + } + + @Test + fun testWaitChild() = runTest { + expect(1) + launch { + expect(3) + yield() // to parent + finish(5) + } + expect(2) + yield() + expect(4) + // parent waits for child's completion + } + + @Test + fun testCancelChildExplicit() = runTest { + expect(1) + val job = launch { + expect(3) + yield() + expectUnreached() + } + expect(2) + yield() + expect(4) + job.cancel() + finish(5) + } + + @Test + fun testCancelChildWithFinally() = runTest { + expect(1) + val job = launch { + expect(3) + try { + yield() + } finally { + finish(6) // cancelled child will still execute finally + } + expectUnreached() + } + expect(2) + yield() + expect(4) + job.cancel() + expect(5) + } + + @Test + fun testWaitNestedChild() = runTest { + expect(1) + launch { + expect(3) + launch { + expect(6) + yield() // to parent + expect(9) + } + expect(4) + yield() + expect(7) + yield() // to parent + finish(10) // the last one to complete + } + expect(2) + yield() + expect(5) + yield() + expect(8) + // parent waits for child + } + + @Test + fun testExceptionPropagation() = runTest( + expected = { it is TestException } + ) { + finish(1) + throw TestException() + } + + @Test + fun testCancelParentOnChildException() = runTest(expected = { it is TestException }) { + expect(1) + launch { + finish(3) + throwTestException() // does not propagate exception to launch, but cancels parent (!) + expectUnreached() + } + expect(2) + yield() + expectUnreached() // because of exception in child + } + + @Test + fun testCancelParentOnNestedException() = runTest(expected = { it is TestException }) { + expect(1) + launch { + expect(3) + launch { + finish(6) + throwTestException() // unhandled exception kills all parents + expectUnreached() + } + expect(4) + yield() + expectUnreached() // because of exception in child + } + expect(2) + yield() + expect(5) + yield() + expectUnreached() // because of exception in child + } + + @Test + fun testJoinWithFinally() = runTest { + expect(1) + val job = launch { + expect(3) + try { + yield() // to main, will cancel us + } finally { + expect(7) // join is waiting + } + } + expect(2) + yield() // to job + expect(4) + assertTrue(job.isActive && !job.isCompleted) + job.cancel() // cancels job + expect(5) // still here + assertTrue(!job.isActive && !job.isCompleted) + expect(6) // we're still here + job.join() // join the job, let job complete its "finally" section + expect(8) + assertTrue(!job.isActive && job.isCompleted) + finish(9) + } + + @Test + fun testCancelAndJoin() = runTest { + expect(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + yield() + expectUnreached() // will get cancelled + } finally { + expect(4) + } + } + expect(3) + job.cancelAndJoin() + finish(5) + } + + @Test + fun testCancelAndJoinChildCrash() = runTest(expected = { it is TestException }) { + expect(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + throwTestException() + expectUnreached() + } + // now we have a failed job with TestException + finish(3) + try { + job.cancelAndJoin() // join should crash on child's exception but it will be wrapped into CancellationException + } catch (e: Throwable) { + e as CancellationException // type assertion + assertIs(e.cause) + throw e + } + expectUnreached() + } + + @Test + fun testYieldInFinally() = runTest( + expected = { it is TestException } + ) { + expect(1) + try { + expect(2) + throwTestException() + } finally { + expect(3) + yield() + finish(4) + } + expectUnreached() + } + + @Test + fun testCancelAndJoinChildren() = runTest { + expect(1) + val parent = Job() + launch(parent, CoroutineStart.UNDISPATCHED) { + expect(2) + try { + yield() // to be cancelled + } finally { + expect(5) + } + expectUnreached() + } + expect(3) + parent.cancelChildren() + expect(4) + parent.children.forEach { it.join() } // will yield to child + assertTrue(parent.isActive) // make sure it did not cancel parent + finish(6) + } + + @Test + fun testParentCrashCancelsChildren() = runTest( + unhandled = listOf({ it -> it is TestException }) + ) { + expect(1) + val parent = launch(Job()) { + expect(4) + throw TestException("Crashed") + } + val child = launch(parent, CoroutineStart.UNDISPATCHED) { + expect(2) + try { + yield() // to test + } finally { + expect(5) + withContext(NonCancellable) { yield() } // to test + expect(7) + } + expectUnreached() // will get cancelled, because parent crashes + } + expect(3) + yield() // to parent + expect(6) + parent.join() // make sure crashed parent still waits for its child + finish(8) + // make sure is cancelled + assertTrue(child.isCancelled) + } + + @Test + fun testNotCancellableChildWithExceptionCancelled() = runTest( + expected = { it is TestException } + ) { + expect(1) + // CoroutineStart.ATOMIC makes sure it will not get cancelled for it starts executing + val d = async(NonCancellable, start = CoroutineStart.ATOMIC) { + finish(4) + throwTestException() // will throw + expectUnreached() + } + expect(2) + // now cancel with some other exception + d.cancel(TestCancellationException()) + // now await to see how it got crashed -- TestCancellationException should have been suppressed by TestException + expect(3) + d.await() + } + + private fun throwTestException() { throw TestException() } +} diff --git a/kotlinx-coroutines-core/common/test/DelayDurationTest.kt b/kotlinx-coroutines-core/common/test/DelayDurationTest.kt new file mode 100644 index 0000000000..dad01eb285 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/DelayDurationTest.kt @@ -0,0 +1,70 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED", "DEPRECATION") + +// KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.seconds +import kotlin.time.Duration.Companion.nanoseconds + +class DelayDurationTest : TestBase() { + + @Test + fun testCancellation() = runTest(expected = { it is CancellationException }) { + runAndCancel(1.seconds) + } + + @Test + fun testInfinite() = runTest(expected = { it is CancellationException }) { + runAndCancel(Duration.INFINITE) + } + + @Test + fun testRegularDelay() = runTest { + val deferred = async { + expect(2) + delay(1.seconds) + expect(4) + } + + expect(1) + yield() + expect(3) + deferred.await() + finish(5) + } + + @Test + fun testNanoDelay() = runTest { + val deferred = async { + expect(2) + delay(1.nanoseconds) + expect(4) + } + + expect(1) + yield() + expect(3) + deferred.await() + finish(5) + } + + private suspend fun runAndCancel(time: Duration) = coroutineScope { + expect(1) + val deferred = async { + expect(2) + delay(time) + expectUnreached() + } + + yield() + expect(3) + require(deferred.isActive) + deferred.cancel() + finish(4) + deferred.await() + } +} diff --git a/kotlinx-coroutines-core/common/test/DelayTest.kt b/kotlinx-coroutines-core/common/test/DelayTest.kt new file mode 100644 index 0000000000..007a272f4f --- /dev/null +++ b/kotlinx-coroutines-core/common/test/DelayTest.kt @@ -0,0 +1,55 @@ + +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED", "DEPRECATION") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class DelayTest : TestBase() { + + @Test + fun testCancellation() = runTest(expected = {it is CancellationException }) { + runAndCancel(1000) + } + + @Test + fun testMaxLongValue()= runTest(expected = {it is CancellationException }) { + runAndCancel(Long.MAX_VALUE) + } + + @Test + fun testMaxIntValue()= runTest(expected = {it is CancellationException }) { + runAndCancel(Int.MAX_VALUE.toLong()) + } + + @Test + fun testRegularDelay() = runTest { + val deferred = async { + expect(2) + delay(1) + expect(3) + } + + expect(1) + yield() + deferred.await() + finish(4) + } + + private suspend fun runAndCancel(time: Long) = coroutineScope { + expect(1) + val deferred = async { + expect(2) + delay(time) + expectUnreached() + } + + yield() + expect(3) + require(deferred.isActive) + deferred.cancel() + finish(4) + deferred.await() + } +} diff --git a/kotlinx-coroutines-core/common/test/DispatchedContinuationTest.kt b/kotlinx-coroutines-core/common/test/DispatchedContinuationTest.kt new file mode 100644 index 0000000000..2035ef5bab --- /dev/null +++ b/kotlinx-coroutines-core/common/test/DispatchedContinuationTest.kt @@ -0,0 +1,75 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +/** + * When using [suspendCoroutine] from the standard library the continuation must be dispatched atomically, + * without checking for cancellation at any point in time. + */ +class DispatchedContinuationTest : TestBase() { + private lateinit var cont: Continuation + + @Test + fun testCancelThenResume() = runTest { + expect(1) + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + coroutineContext[Job]!!.cancel() + // a regular suspendCoroutine will still suspend despite the fact that coroutine was cancelled + val value = suspendCoroutine { + expect(3) + cont = it + } + expect(6) + assertEquals("OK", value) + } + expect(4) + cont.resume("OK") + expect(5) + yield() // to the launched job + finish(7) + } + + @Test + fun testCancelThenResumeUnconfined() = runTest { + expect(1) + launch(Dispatchers.Unconfined) { + expect(2) + coroutineContext[Job]!!.cancel() + // a regular suspendCoroutine will still suspend despite the fact that coroutine was cancelled + val value = suspendCoroutine { + expect(3) + cont = it + } + expect(5) + assertEquals("OK", value) + } + expect(4) + cont.resume("OK") // immediately resumes -- because unconfined + finish(6) + } + + @Test + fun testResumeThenCancel() = runTest { + expect(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + val value = suspendCoroutine { + expect(3) + cont = it + } + expect(7) + assertEquals("OK", value) + } + expect(4) + cont.resume("OK") + expect(5) + // now cancel the job, which the coroutine is waiting to be dispatched + job.cancel() + expect(6) + yield() // to the launched job + finish(8) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/DurationToMillisTest.kt b/kotlinx-coroutines-core/common/test/DurationToMillisTest.kt new file mode 100644 index 0000000000..4815419f9b --- /dev/null +++ b/kotlinx-coroutines-core/common/test/DurationToMillisTest.kt @@ -0,0 +1,66 @@ +package kotlinx.coroutines + +import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.nanoseconds +import kotlin.time.Duration.Companion.seconds + +class DurationToMillisTest { + + @Test + fun testNegativeDurationCoercedToZeroMillis() { + assertEquals(0L, (-1).seconds.toDelayMillis()) + } + + @Test + fun testZeroDurationCoercedToZeroMillis() { + assertEquals(0L, 0.seconds.toDelayMillis()) + } + + @Test + fun testOneNanosecondCoercedToOneMillisecond() { + assertEquals(1L, 1.nanoseconds.toDelayMillis()) + } + + @Test + fun testOneSecondCoercedTo1000Milliseconds() { + assertEquals(1_000L, 1.seconds.toDelayMillis()) + } + + @Test + fun testMixedComponentDurationRoundedUpToNextMillisecond() { + assertEquals(999L, (998.milliseconds + 75909.nanoseconds).toDelayMillis()) + } + + @Test + fun testOneExtraNanosecondRoundedUpToNextMillisecond() { + assertEquals(999L, (998.milliseconds + 1.nanoseconds).toDelayMillis()) + } + + @Test + fun testInfiniteDurationCoercedToLongMaxValue() { + assertEquals(Long.MAX_VALUE, Duration.INFINITE.toDelayMillis()) + } + + @Test + fun testNegativeInfiniteDurationCoercedToZero() { + assertEquals(0L, (-Duration.INFINITE).toDelayMillis()) + } + + @Test + fun testNanosecondOffByOneInfinityDoesNotOverflow() { + assertEquals(Long.MAX_VALUE / 1_000_000, (Long.MAX_VALUE - 1L).nanoseconds.toDelayMillis()) + } + + @Test + fun testMillisecondOffByOneInfinityDoesNotIncrement() { + assertEquals((Long.MAX_VALUE / 2) - 1, ((Long.MAX_VALUE / 2) - 1).milliseconds.toDelayMillis()) + } + + @Test + fun testOutOfBoundsNanosecondsButFiniteDoesNotIncrement() { + val milliseconds = Long.MAX_VALUE / 10 + assertEquals(milliseconds, milliseconds.milliseconds.toDelayMillis()) + } +} diff --git a/kotlinx-coroutines-core/common/test/EmptyContext.kt b/kotlinx-coroutines-core/common/test/EmptyContext.kt new file mode 100644 index 0000000000..abec3d3de7 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/EmptyContext.kt @@ -0,0 +1,30 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.probeCoroutineCreated +import kotlinx.coroutines.internal.probeCoroutineResumed +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +suspend fun withEmptyContext(block: suspend () -> T): T = suspendCoroutine { cont -> + block.startCoroutineUnintercepted(Continuation(EmptyCoroutineContext) { cont.resumeWith(it) }) +} + +/** + * Use this function to restart a coroutine directly from inside of [suspendCoroutine], + * when the code is already in the context of this coroutine. + * It does not use [ContinuationInterceptor] and does not update the context of the current thread. + */ +fun (suspend () -> T).startCoroutineUnintercepted(completion: Continuation) { + val actualCompletion = probeCoroutineCreated(completion) + val value = try { + probeCoroutineResumed(actualCompletion) + startCoroutineUninterceptedOrReturn(actualCompletion) + } catch (e: Throwable) { + actualCompletion.resumeWithException(e) + return + } + if (value !== COROUTINE_SUSPENDED) { + @Suppress("UNCHECKED_CAST") + actualCompletion.resume(value as T) + } +} diff --git a/kotlinx-coroutines-core/common/test/FailedJobTest.kt b/kotlinx-coroutines-core/common/test/FailedJobTest.kt new file mode 100644 index 0000000000..77656ee6ae --- /dev/null +++ b/kotlinx-coroutines-core/common/test/FailedJobTest.kt @@ -0,0 +1,59 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +// see https://github.com/Kotlin/kotlinx.coroutines/issues/585 +class FailedJobTest : TestBase() { + @Test + fun testCancelledJob() = runTest { + expect(1) + val job = launch { + expectUnreached() + } + expect(2) + job.cancelAndJoin() + finish(3) + assertTrue(job.isCompleted) + assertTrue(!job.isActive) + assertTrue(job.isCancelled) + } + + @Test + fun testFailedJob() = runTest( + unhandled = listOf({it -> it is TestException }) + ) { + expect(1) + val job = launch(NonCancellable) { + expect(3) + throw TestException() + } + expect(2) + job.join() + finish(4) + assertTrue(job.isCompleted) + assertTrue(!job.isActive) + assertTrue(job.isCancelled) + } + + @Test + fun testFailedChildJob() = runTest( + unhandled = listOf({it -> it is TestException }) + ) { + expect(1) + val job = launch(NonCancellable) { + expect(3) + launch { + throw TestException() + } + } + expect(2) + job.join() + finish(4) + assertTrue(job.isCompleted) + assertTrue(!job.isActive) + assertTrue(job.isCancelled) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/ImmediateYieldTest.kt b/kotlinx-coroutines-core/common/test/ImmediateYieldTest.kt new file mode 100644 index 0000000000..c2c21489cf --- /dev/null +++ b/kotlinx-coroutines-core/common/test/ImmediateYieldTest.kt @@ -0,0 +1,54 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +class ImmediateYieldTest : TestBase() { + + // See https://github.com/Kotlin/kotlinx.coroutines/issues/1474 + @Test + fun testImmediateYield() = runTest { + expect(1) + launch(ImmediateDispatcher(coroutineContext[ContinuationInterceptor])) { + expect(2) + yield() + expect(4) + } + expect(3) // after yield + yield() // yield back + finish(5) + } + + // imitate immediate dispatcher + private class ImmediateDispatcher(job: ContinuationInterceptor?) : CoroutineDispatcher() { + val delegate: CoroutineDispatcher = job as CoroutineDispatcher + + override fun isDispatchNeeded(context: CoroutineContext): Boolean = false + + override fun dispatch(context: CoroutineContext, block: Runnable) = + delegate.dispatch(context, block) + } + + @Test + fun testWrappedUnconfinedDispatcherYield() = runTest { + expect(1) + launch(wrapperDispatcher(Dispatchers.Unconfined)) { + expect(2) + yield() // Would not work with wrapped unconfined dispatcher + expect(3) + } + finish(4) // after launch + } + + @Test + fun testWrappedUnconfinedDispatcherYieldStackOverflow() = runTest { + expect(1) + withContext(wrapperDispatcher(Dispatchers.Unconfined)) { + repeat(100_000) { + yield() + } + } + finish(2) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/JobExtensionsTest.kt b/kotlinx-coroutines-core/common/test/JobExtensionsTest.kt new file mode 100644 index 0000000000..aacb022ac4 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/JobExtensionsTest.kt @@ -0,0 +1,92 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +class JobExtensionsTest : TestBase() { + + private val job = Job() + private val scope = CoroutineScope(job + CoroutineExceptionHandler { _, _ -> }) + + @Test + fun testIsActive() = runTest { + expect(1) + scope.launch(Dispatchers.Unconfined) { + ensureActive() + coroutineContext.ensureActive() + coroutineContext[Job]!!.ensureActive() + expect(2) + delay(Long.MAX_VALUE) + } + + expect(3) + job.ensureActive() + scope.ensureActive() + scope.coroutineContext.ensureActive() + job.cancelAndJoin() + finish(4) + } + + @Test + fun testIsCompleted() = runTest { + expect(1) + scope.launch(Dispatchers.Unconfined) { + ensureActive() + coroutineContext.ensureActive() + coroutineContext[Job]!!.ensureActive() + expect(2) + } + + expect(3) + job.complete() + job.join() + assertFailsWith { job.ensureActive() } + assertFailsWith { scope.ensureActive() } + assertFailsWith { scope.coroutineContext.ensureActive() } + finish(4) + } + + + @Test + fun testIsCancelled() = runTest { + expect(1) + scope.launch(Dispatchers.Unconfined) { + ensureActive() + coroutineContext.ensureActive() + coroutineContext[Job]!!.ensureActive() + expect(2) + throw TestException() + } + + expect(3) + checkException { job.ensureActive() } + checkException { scope.ensureActive() } + checkException { scope.coroutineContext.ensureActive() } + finish(4) + } + + @Test + fun testEnsureActiveWithEmptyContext() = runTest { + withEmptyContext { + ensureActive() // should not do anything + } + } + + private inline fun checkException(block: () -> Unit) { + val result = runCatching(block) + val exception = result.exceptionOrNull() ?: fail() + assertIs(exception) + assertIs(exception.cause) + } + + @Test + fun testJobExtension() = runTest { + assertSame(coroutineContext[Job]!!, coroutineContext.job) + assertSame(NonCancellable, NonCancellable.job) + assertSame(job, job.job) + assertFailsWith { EmptyCoroutineContext.job } + assertFailsWith { Dispatchers.Default.job } + assertFailsWith { (Dispatchers.Default + CoroutineName("")).job } + } +} diff --git a/kotlinx-coroutines-core/common/test/JobStatesTest.kt b/kotlinx-coroutines-core/common/test/JobStatesTest.kt new file mode 100644 index 0000000000..6f18afd326 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/JobStatesTest.kt @@ -0,0 +1,164 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +/** + * Tests that the transitions to the state of the [Job] correspond to documentation in the + * table that is presented in the [Job] documentation. + */ +class JobStatesTest : TestBase() { + @Test + public fun testNormalCompletion() = runTest { + expect(1) + val parent = coroutineContext.job + val job = launch(start = CoroutineStart.LAZY) { + expect(2) + // launches child + launch { + expect(4) + } + // completes normally + } + // New job + assertFalse(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + assertSame(parent, job.parent) + // New -> Active + job.start() + assertTrue(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + assertSame(parent, job.parent) + // Active -> Completing + yield() // scheduled & starts child + expect(3) + assertTrue(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + assertSame(parent, job.parent) + // Completing -> Completed + yield() + finish(5) + assertFalse(job.isActive) + assertTrue(job.isCompleted) + assertFalse(job.isCancelled) + assertNull(job.parent) + } + + @Test + public fun testCompletingFailed() = runTest( + unhandled = listOf({ it -> it is TestException }) + ) { + expect(1) + val job = launch(NonCancellable, start = CoroutineStart.LAZY) { + expect(2) + // launches child + launch { + expect(4) + throw TestException() + } + // completes normally + } + // New job + assertFalse(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + // New -> Active + job.start() + assertTrue(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + // Active -> Completing + yield() // scheduled & starts child + expect(3) + assertTrue(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + // Completing -> Cancelled + yield() + finish(5) + assertFalse(job.isActive) + assertTrue(job.isCompleted) + assertTrue(job.isCancelled) + } + + @Test + public fun testFailed() = runTest( + unhandled = listOf({ it -> it is TestException }) + ) { + expect(1) + val job = launch(NonCancellable, start = CoroutineStart.LAZY) { + expect(2) + // launches child + launch(start = CoroutineStart.ATOMIC) { + expect(4) + } + // failing + throw TestException() + } + // New job + assertFalse(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + // New -> Active + job.start() + assertTrue(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + // Active -> Cancelling + yield() // scheduled & starts child + expect(3) + assertFalse(job.isActive) + assertFalse(job.isCompleted) + assertTrue(job.isCancelled) + // Cancelling -> Cancelled + yield() + finish(5) + assertFalse(job.isActive) + assertTrue(job.isCompleted) + assertTrue(job.isCancelled) + } + + @Test + public fun testCancelling() = runTest { + expect(1) + val job = launch(NonCancellable, start = CoroutineStart.LAZY) { + expect(2) + // launches child + launch(start = CoroutineStart.ATOMIC) { + expect(4) + } + // completes normally + } + // New job + assertFalse(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + // New -> Active + job.start() + assertTrue(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + // Active -> Completing + yield() // scheduled & starts child + expect(3) + assertTrue(job.isActive) + assertFalse(job.isCompleted) + assertFalse(job.isCancelled) + // Completing -> Cancelling + job.cancel() + assertFalse(job.isActive) + assertFalse(job.isCompleted) + assertTrue(job.isCancelled) + // Cancelling -> Cancelled + yield() + finish(5) + assertFalse(job.isActive) + assertTrue(job.isCompleted) + assertTrue(job.isCancelled) + } +} diff --git a/kotlinx-coroutines-core/common/test/JobTest.kt b/kotlinx-coroutines-core/common/test/JobTest.kt new file mode 100644 index 0000000000..55119ab65c --- /dev/null +++ b/kotlinx-coroutines-core/common/test/JobTest.kt @@ -0,0 +1,250 @@ +@file:Suppress("DEPRECATION") + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class JobTest : TestBase() { + @Test + fun testState() { + val job = Job() + assertNull(job.parent) + assertTrue(job.isActive) + job.cancel() + assertTrue(!job.isActive) + } + + @Test + fun testHandler() { + val job = Job() + var fireCount = 0 + job.invokeOnCompletion { fireCount++ } + assertTrue(job.isActive) + assertEquals(0, fireCount) + // cancel once + job.cancel() + assertTrue(!job.isActive) + assertEquals(1, fireCount) + // cancel again + job.cancel() + assertTrue(!job.isActive) + assertEquals(1, fireCount) + } + + @Test + fun testManyHandlers() { + val job = Job() + val n = 100 * stressTestMultiplier + val fireCount = IntArray(n) + for (i in 0 until n) job.invokeOnCompletion { fireCount[i]++ } + assertTrue(job.isActive) + for (i in 0 until n) assertEquals(0, fireCount[i]) + // cancel once + job.cancel() + assertTrue(!job.isActive) + for (i in 0 until n) assertEquals(1, fireCount[i]) + // cancel again + job.cancel() + assertTrue(!job.isActive) + for (i in 0 until n) assertEquals(1, fireCount[i]) + } + + @Test + fun testUnregisterInHandler() { + val job = Job() + val n = 100 * stressTestMultiplier + val fireCount = IntArray(n) + for (i in 0 until n) { + var registration: DisposableHandle? = null + registration = job.invokeOnCompletion { + fireCount[i]++ + registration!!.dispose() + } + } + assertTrue(job.isActive) + for (i in 0 until n) assertEquals(0, fireCount[i]) + // cancel once + job.cancel() + assertTrue(!job.isActive) + for (i in 0 until n) assertEquals(1, fireCount[i]) + // cancel again + job.cancel() + assertTrue(!job.isActive) + for (i in 0 until n) assertEquals(1, fireCount[i]) + } + + @Test + fun testManyHandlersWithUnregister() { + val job = Job() + val n = 100 * stressTestMultiplier + val fireCount = IntArray(n) + val registrations = Array(n) { i -> job.invokeOnCompletion { fireCount[i]++ } } + assertTrue(job.isActive) + fun unreg(i: Int) = i % 4 <= 1 + for (i in 0 until n) if (unreg(i)) registrations[i].dispose() + for (i in 0 until n) assertEquals(0, fireCount[i]) + job.cancel() + assertTrue(!job.isActive) + for (i in 0 until n) assertEquals(if (unreg(i)) 0 else 1, fireCount[i]) + } + + @Test + fun testExceptionsInHandler() { + val job = Job() + val n = 100 * stressTestMultiplier + val fireCount = IntArray(n) + for (i in 0 until n) job.invokeOnCompletion { + fireCount[i]++ + throw TestException() + } + assertTrue(job.isActive) + for (i in 0 until n) assertEquals(0, fireCount[i]) + val cancelResult = runCatching { job.cancel() } + assertTrue(!job.isActive) + for (i in 0 until n) assertEquals(1, fireCount[i]) + assertIs(cancelResult.exceptionOrNull()) + assertIs(cancelResult.exceptionOrNull()!!.cause) + } + + @Test + fun testCancelledParent() { + val parent = Job() + parent.cancel() + assertTrue(!parent.isActive) + val child = Job(parent) + assertTrue(!child.isActive) + } + + @Test + fun testDisposeSingleHandler() { + val job = Job() + var fireCount = 0 + val handler = job.invokeOnCompletion { fireCount++ } + handler.dispose() + job.cancel() + assertEquals(0, fireCount) + } + + @Test + fun testDisposeMultipleHandler() { + val job = Job() + val handlerCount = 10 + var fireCount = 0 + val handlers = Array(handlerCount) { job.invokeOnCompletion { fireCount++ } } + handlers.forEach { it.dispose() } + job.cancel() + assertEquals(0, fireCount) + } + + @Test + fun testCancelAndJoinParentWaitChildren() = runTest { + expect(1) + val parent = Job() + launch(parent, start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + yield() // will get cancelled + } finally { + expect(5) + } + } + expect(3) + parent.cancel() + expect(4) + parent.join() + finish(6) + } + + @Test + fun testOnCancellingHandler() = runTest { + val job = launch { + expect(2) + delay(Long.MAX_VALUE) + } + + job.invokeOnCompletion(onCancelling = true) { + assertNotNull(it) + expect(3) + } + + expect(1) + yield() + job.cancelAndJoin() + finish(4) + } + + @Test + fun testInvokeOnCancellingFiringOnNormalExit() = runTest { + val job = launch { + expect(2) + } + job.invokeOnCompletion(onCancelling = true) { + assertNull(it) + expect(3) + } + expect(1) + job.join() + finish(4) + } + + @Test + fun testOverriddenParent() = runTest { + val parent = Job() + val deferred = launch(parent, CoroutineStart.ATOMIC) { + expect(2) + delay(Long.MAX_VALUE) + } + + parent.cancel() + expect(1) + deferred.join() + finish(3) + } + + @Test + fun testJobWithParentCancelNormally() { + val parent = Job() + val job = Job(parent) + job.cancel() + assertTrue(job.isCancelled) + assertFalse(parent.isCancelled) + } + + @Test + fun testJobWithParentCancelException() { + val parent = Job() + val job = Job(parent) + job.completeExceptionally(TestException()) + assertTrue(job.isCancelled) + assertTrue(parent.isCancelled) + } + + @Test + fun testIncompleteJobState() = runTest { + val parent = coroutineContext.job + val job = launch { + coroutineContext[Job]!!.invokeOnCompletion { } + } + assertSame(parent, job.parent) + job.join() + assertNull(job.parent) + assertTrue(job.isCompleted) + assertFalse(job.isActive) + assertFalse(job.isCancelled) + } + + @Test + fun testChildrenWithIncompleteState() = runTest { + val job = async { Wrapper() } + job.join() + assertTrue(job.children.toList().isEmpty()) + } + + private class Wrapper : Incomplete { + override val isActive: Boolean + get() = error("") + override val list: NodeList? + get() = error("") + } +} diff --git a/kotlinx-coroutines-core/common/test/LaunchLazyTest.kt b/kotlinx-coroutines-core/common/test/LaunchLazyTest.kt new file mode 100644 index 0000000000..8f200faa35 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/LaunchLazyTest.kt @@ -0,0 +1,68 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class LaunchLazyTest : TestBase() { + @Test + fun testLaunchAndYieldJoin() = runTest { + expect(1) + val job = launch(start = CoroutineStart.LAZY) { + expect(4) + yield() // does nothing -- main waits + expect(5) + } + expect(2) + yield() // does nothing, was not started yet + expect(3) + assertTrue(!job.isActive && !job.isCompleted) + job.join() + assertTrue(!job.isActive && job.isCompleted) + finish(6) + } + + @Test + fun testStart() = runTest { + expect(1) + val job = launch(start = CoroutineStart.LAZY) { + expect(5) + yield() // yields back to main + expect(7) + } + expect(2) + yield() // does nothing, was not started yet + expect(3) + assertTrue(!job.isActive && !job.isCompleted) + assertTrue(job.start()) + assertTrue(job.isActive && !job.isCompleted) + assertTrue(!job.start()) // start again -- does nothing + assertTrue(job.isActive && !job.isCompleted) + expect(4) + yield() // now yield to started coroutine + expect(6) + assertTrue(job.isActive && !job.isCompleted) + yield() // yield again + assertTrue(!job.isActive && job.isCompleted) // it completes this time + expect(8) + job.join() // immediately returns + finish(9) + } + + @Test + fun testInvokeOnCompletionAndStart() = runTest { + expect(1) + val job = launch(start = CoroutineStart.LAZY) { + expect(5) + } + yield() // no started yet! + expect(2) + job.invokeOnCompletion { + expect(6) + } + expect(3) + job.start() + expect(4) + yield() + finish(7) + } +} diff --git a/kotlinx-coroutines-core/common/test/LimitedParallelismSharedTest.kt b/kotlinx-coroutines-core/common/test/LimitedParallelismSharedTest.kt new file mode 100644 index 0000000000..2d3dea634d --- /dev/null +++ b/kotlinx-coroutines-core/common/test/LimitedParallelismSharedTest.kt @@ -0,0 +1,59 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.CoroutineContext +import kotlin.coroutines.EmptyCoroutineContext +import kotlin.test.* + +class LimitedParallelismSharedTest : TestBase() { + + @Test + fun testLimitedDefault() = runTest { + // Test that evaluates the very basic completion of tasks in limited dispatcher + // for all supported platforms. + // For more specific and concurrent tests, see 'concurrent' package. + val view = Dispatchers.Default.limitedParallelism(1) + val view2 = Dispatchers.Default.limitedParallelism(1) + val j1 = launch(view) { + while (true) { + yield() + } + } + val j2 = launch(view2) { j1.cancel() } + joinAll(j1, j2) + } + + @Test + fun testParallelismSpec() { + assertFailsWith { Dispatchers.Default.limitedParallelism(0) } + assertFailsWith { Dispatchers.Default.limitedParallelism(-1) } + assertFailsWith { Dispatchers.Default.limitedParallelism(Int.MIN_VALUE) } + Dispatchers.Default.limitedParallelism(Int.MAX_VALUE) + } + + /** + * Checks that even if the dispatcher sporadically fails, the limited dispatcher will still allow reaching the + * target parallelism level. + */ + @Test + fun testLimitedParallelismOfOccasionallyFailingDispatcher() { + val limit = 5 + var doFail = false + val workerQueue = mutableListOf() + val limited = object: CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + if (doFail) throw TestException() + workerQueue.add(block) + } + }.limitedParallelism(limit) + repeat(6 * limit) { + try { + limited.dispatch(EmptyCoroutineContext, Runnable { /* do nothing */ }) + } catch (_: DispatchException) { + // ignore + } + doFail = !doFail + } + assertEquals(limit, workerQueue.size) + } +} diff --git a/kotlinx-coroutines-core/common/test/NonCancellableTest.kt b/kotlinx-coroutines-core/common/test/NonCancellableTest.kt new file mode 100644 index 0000000000..bede537379 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/NonCancellableTest.kt @@ -0,0 +1,134 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class NonCancellableTest : TestBase() { + @Test + fun testNonCancellable() = runTest { + expect(1) + val job = async { + withContext(NonCancellable) { + expect(2) + yield() + expect(4) + } + + expect(5) + yield() + expectUnreached() + } + + yield() + job.cancel() + expect(3) + assertTrue(job.isCancelled) + try { + job.await() + expectUnreached() + } catch (e: JobCancellationException) { + if (RECOVER_STACK_TRACES) { + val cause = e.cause as JobCancellationException // shall be recovered JCE + assertNull(cause.cause) + } else { + assertNull(e.cause) + } + finish(6) + } + } + + @Test + fun testNonCancellableWithException() = runTest { + expect(1) + val deferred = async(NonCancellable) { + withContext(NonCancellable) { + expect(2) + yield() + expect(4) + } + + expect(5) + yield() + expectUnreached() + } + + yield() + deferred.cancel(TestCancellationException("TEST")) + expect(3) + assertTrue(deferred.isCancelled) + try { + deferred.await() + expectUnreached() + } catch (e: TestCancellationException) { + assertEquals("TEST", e.message) + finish(6) + } + } + + @Test + fun testNonCancellableFinally() = runTest { + expect(1) + val job = async { + try { + expect(2) + yield() + expectUnreached() + } finally { + withContext(NonCancellable) { + expect(4) + yield() + expect(5) + } + } + + expectUnreached() + } + + yield() + job.cancel() + expect(3) + assertTrue(job.isCancelled) + + try { + job.await() + expectUnreached() + } catch (e: CancellationException) { + finish(6) + } + } + + @Test + fun testNonCancellableTwice() = runTest { + expect(1) + val job = async { + withContext(NonCancellable) { + expect(2) + yield() + expect(4) + } + + withContext(NonCancellable) { + expect(5) + yield() + expect(6) + } + } + + yield() + job.cancel() + expect(3) + assertTrue(job.isCancelled) + try { + job.await() + expectUnreached() + } catch (e: JobCancellationException) { + if (RECOVER_STACK_TRACES) { + val cause = e.cause as JobCancellationException // shall be recovered JCE + assertNull(cause.cause) + } else { + assertNull(e.cause) + } + finish(7) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/ParentCancellationTest.kt b/kotlinx-coroutines-core/common/test/ParentCancellationTest.kt new file mode 100644 index 0000000000..30a53209d7 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/ParentCancellationTest.kt @@ -0,0 +1,169 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +/** + * Systematically tests that various builders cancel parent on failure. + */ +class ParentCancellationTest : TestBase() { + @Test + fun testJobChild() = runTest { + testParentCancellation(expectUnhandled = false) { fail -> + val child = Job(coroutineContext[Job]) + CoroutineScope(coroutineContext + child).fail() + } + } + + @Test + fun testSupervisorJobChild() = runTest { + testParentCancellation(expectParentActive = true, expectUnhandled = true) { fail -> + val child = SupervisorJob(coroutineContext[Job]) + CoroutineScope(coroutineContext + child).fail() + } + } + + @Test + fun testCompletableDeferredChild() = runTest { + testParentCancellation { fail -> + val child = CompletableDeferred(coroutineContext[Job]) + CoroutineScope(coroutineContext + child).fail() + } + } + + @Test + fun testLaunchChild() = runTest { + testParentCancellation(runsInScopeContext = true) { fail -> + launch { fail() } + } + } + + @Test + fun testAsyncChild() = runTest { + testParentCancellation(runsInScopeContext = true) { fail -> + async { fail() } + } + } + + @Test + fun testProduceChild() = runTest { + testParentCancellation(runsInScopeContext = true) { fail -> + produce { fail() } + } + } + + @Test + @Suppress("DEPRECATION_ERROR") + fun testBroadcastChild() = runTest { + testParentCancellation(runsInScopeContext = true) { fail -> + broadcast { fail() }.openSubscription() + } + } + + @Test + fun testSupervisorChild() = runTest { + testParentCancellation(expectParentActive = true, expectUnhandled = true, runsInScopeContext = true) { fail -> + supervisorScope { fail() } + } + } + + @Test + fun testCoroutineScopeChild() = runTest { + testParentCancellation(expectParentActive = true, expectRethrows = true, runsInScopeContext = true) { fail -> + coroutineScope { fail() } + } + } + + @Test + fun testWithContextChild() = runTest { + testParentCancellation(expectParentActive = true, expectRethrows = true, runsInScopeContext = true) { fail -> + withContext(CoroutineName("fail")) { fail() } + } + } + + @Test + fun testWithTimeoutChild() = runTest { + testParentCancellation(expectParentActive = true, expectRethrows = true, runsInScopeContext = true) { fail -> + withTimeout(1000) { fail() } + } + } + + private suspend fun CoroutineScope.testParentCancellation( + expectParentActive: Boolean = false, + expectRethrows: Boolean = false, + expectUnhandled: Boolean = false, + runsInScopeContext: Boolean = false, + child: suspend CoroutineScope.(block: suspend CoroutineScope.() -> Unit) -> Unit + ) { + testWithException( + expectParentActive, + expectRethrows, + expectUnhandled, + runsInScopeContext, + TestException(), + child + ) + testWithException( + true, + expectRethrows, + false, + runsInScopeContext, + CancellationException("Test"), + child + ) + } + + private suspend fun CoroutineScope.testWithException( + expectParentActive: Boolean, + expectRethrows: Boolean, + expectUnhandled: Boolean, + runsInScopeContext: Boolean, + throwException: Throwable, + child: suspend CoroutineScope.(block: suspend CoroutineScope.() -> Unit) -> Unit + ) { + reset() + expect(1) + val parent = CompletableDeferred() // parent that handles exception (!) + val scope = CoroutineScope(coroutineContext + parent) + try { + scope.child { + // launch failing grandchild + var unhandledException: Throwable? = null + val handler = CoroutineExceptionHandler { _, e -> unhandledException = e } + val grandchild = launch(handler) { + throw throwException + } + grandchild.join() + when { + !expectParentActive && runsInScopeContext -> expectUnreached() + expectUnhandled -> assertSame(throwException, unhandledException) + else -> assertNull(unhandledException) + } + } + if (expectRethrows && throwException !is CancellationException) { + expectUnreached() + } else { + expect(2) + } + } catch (e: Throwable) { + if (expectRethrows) { + expect(2) + assertSame(throwException, e) + } else { + expectUnreached() + } + } + if (expectParentActive) { + assertTrue(parent.isActive) + parent.cancelAndJoin() + } else { + parent.join() + assertFalse(parent.isActive) + assertTrue(parent.isCancelled) + } + finish(3) + } +} diff --git a/kotlinx-coroutines-core/common/test/SupervisorTest.kt b/kotlinx-coroutines-core/common/test/SupervisorTest.kt new file mode 100644 index 0000000000..f528059577 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/SupervisorTest.kt @@ -0,0 +1,266 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class SupervisorTest : TestBase() { + @Test + fun testSupervisorJob() = runTest( + unhandled = listOf( + { it -> it is TestException2 }, + { it -> it is TestException1 } + ) + ) { + expect(1) + val supervisor = SupervisorJob() + val job1 = launch(supervisor + CoroutineName("job1")) { + expect(2) + yield() // to second child + expect(4) + throw TestException1() + } + val job2 = launch(supervisor + CoroutineName("job2")) { + expect(3) + throw TestException2() + } + joinAll(job1, job2) + finish(5) + assertTrue(job1.isCancelled) + assertTrue(job2.isCancelled) + assertFalse(supervisor.isCancelled) + assertFalse(supervisor.isCompleted) + } + + @Test + fun testSupervisorScope() = runTest( + unhandled = listOf( + { it -> it is TestException1 }, + { it -> it is TestException2 } + ) + ) { + val result = supervisorScope { + launch { + throw TestException1() + } + launch { + throw TestException2() + } + "OK" + } + assertEquals("OK", result) + } + + @Test + fun testSupervisorScopeIsolation() = runTest( + unhandled = listOf( + { it -> it is TestException2 }) + ) { + val result = supervisorScope { + expect(1) + val job = launch { + expect(2) + delay(Long.MAX_VALUE) + } + + val failingJob = launch { + expect(3) + throw TestException2() + } + + failingJob.join() + yield() + expect(4) + assertTrue(job.isActive) + assertFalse(job.isCancelled) + job.cancel() + "OK" + } + assertEquals("OK", result) + finish(5) + } + + @Test + fun testThrowingSupervisorScope() = runTest { + var childJob: Job? = null + var supervisorJob: Job? = null + try { + expect(1) + supervisorScope { + childJob = async { + try { + delay(Long.MAX_VALUE) + } finally { + expect(3) + } + } + + expect(2) + yield() + supervisorJob = coroutineContext.job + throw TestException2() + } + } catch (e: Throwable) { + assertIs(e) + assertTrue(childJob!!.isCancelled) + assertTrue(supervisorJob!!.isCancelled) + finish(4) + } + } + + @Test + fun testSupervisorThrows() = runTest { + try { + supervisorScope { + expect(1) + launch { + expect(2) + delay(Long.MAX_VALUE) + } + + launch { + expect(3) + delay(Long.MAX_VALUE) + } + + yield() + expect(4) + throw TestException1() + } + } catch (e: TestException1) { + finish(5) + } + } + + @Test + fun testSupervisorThrowsWithFailingChild() = runTest(unhandled = listOf({e -> e is TestException2})) { + try { + supervisorScope { + expect(1) + launch { + expect(2) + delay(Long.MAX_VALUE) + } + + launch { + expect(3) + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() + } + } + + yield() + expect(4) + throw TestException1() + } + } catch (e: TestException1) { + finish(5) + } + } + + /** + * Tests that [supervisorScope] cancels all its children when the current coroutine is cancelled. + */ + @Test + fun testSupervisorScopeExternalCancellation() = runTest { + var childJob: Job? = null + val job = launch { + supervisorScope { + childJob = launch(start = CoroutineStart.UNDISPATCHED) { + try { + delay(Long.MAX_VALUE) + } finally { + expect(2) + } + } + } + } + while (childJob == null) yield() + expect(1) + job.cancel() + assertTrue(childJob!!.isCancelled) + job.join() + finish(3) + } + + @Test + fun testAsyncCancellation() = runTest { + val parent = SupervisorJob() + val deferred = async(parent) { + expect(2) + delay(Long.MAX_VALUE) + } + expect(1) + yield() + parent.completeExceptionally(TestException1()) + try { + deferred.await() + expectUnreached() + } catch (e: CancellationException) { + val cause = if (RECOVER_STACK_TRACES) e.cause?.cause!! else e.cause + assertIs(cause) + finish(3) + } + } + + @Test + fun testSupervisorWithParentCancelNormally() { + val parent = Job() + val supervisor = SupervisorJob(parent) + supervisor.cancel() + assertTrue(supervisor.isCancelled) + assertFalse(parent.isCancelled) + } + + @Test + fun testSupervisorWithParentCancelException() { + val parent = Job() + val supervisor = SupervisorJob(parent) + supervisor.completeExceptionally(TestException1()) + assertTrue(supervisor.isCancelled) + assertTrue(parent.isCancelled) + } + + @Test + fun testSupervisorScopeCancellationVsException() = runTest { + expect(1) + var job: Job? = null + job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + supervisorScope { + expect(3) + yield() // must suspend + expect(5) + job!!.cancel() // cancel this job _before_ it throws + throw TestException1() + } + } catch (e: TestException1) { + // must have caught TextException + expect(6) + } + } + expect(4) + yield() // to coroutineScope + finish(7) + } + + @Test + fun testSupervisorJobCancellationException() = runTest { + val job = SupervisorJob() + val child = launch(job + CoroutineExceptionHandler { _, _ -> expectUnreached() }) { + expect(1) + hang { + expect(3) + } + } + + yield() + expect(2) + child.cancelAndJoin() + job.complete() + job.join() + finish(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/UnconfinedCancellationTest.kt b/kotlinx-coroutines-core/common/test/UnconfinedCancellationTest.kt new file mode 100644 index 0000000000..59845d841b --- /dev/null +++ b/kotlinx-coroutines-core/common/test/UnconfinedCancellationTest.kt @@ -0,0 +1,99 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class UnconfinedCancellationTest : TestBase() { + @Test + fun testUnconfinedCancellation() = runTest { + val parent = Job() + launch(parent) { + expect(1) + parent.cancel() + launch(Dispatchers.Unconfined) { + expectUnreached() + } + + }.join() + finish(2) + } + + @Test + fun testUnconfinedCancellationState() = runTest { + val parent = Job() + launch(parent) { + expect(1) + parent.cancel() + val job = launch(Dispatchers.Unconfined) { + expectUnreached() + } + + assertTrue(job.isCancelled) + assertTrue(job.isCompleted) + assertFalse(job.isActive) + }.join() + finish(2) + } + + @Test + fun testUnconfinedCancellationLazy() = runTest { + val parent = Job() + launch(parent) { + expect(1) + val job = launch(Dispatchers.Unconfined, start = CoroutineStart.LAZY) { + expectUnreached() + } + job.invokeOnCompletion { expect(2) } + assertFalse(job.isCompleted) + + parent.cancel() + job.join() + }.join() + finish(3) + } + + @Test + fun testUndispatchedCancellation() = runTest { + val parent = Job() + launch(parent) { + expect(1) + parent.cancel() + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + yield() + expectUnreached() + } + + }.join() + finish(3) + } + + @Test + fun testCancelledAtomicUnconfined() = runTest { + val parent = Job() + launch(parent) { + expect(1) + parent.cancel() + launch(Dispatchers.Unconfined, start = CoroutineStart.ATOMIC) { + expect(2) + yield() + expectUnreached() + } + }.join() + finish(3) + } + + + @Test + fun testCancelledWithContextUnconfined() = runTest { + val parent = Job() + launch(parent) { + expect(1) + parent.cancel() + withContext(Dispatchers.Unconfined) { + expectUnreached() + } + }.join() + finish(2) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/UnconfinedTest.kt b/kotlinx-coroutines-core/common/test/UnconfinedTest.kt new file mode 100644 index 0000000000..b22e4bf4dc --- /dev/null +++ b/kotlinx-coroutines-core/common/test/UnconfinedTest.kt @@ -0,0 +1,110 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class UnconfinedTest : TestBase() { + + @Test + fun testOrder() = runTest { + expect(1) + launch(Dispatchers.Unconfined) { + expect(2) + launch { + expect(4) + launch { + expect(6) + } + + launch { + expect(7) + } + expect(5) + } + + expect(3) + } + + finish(8) + } + + @Test + fun testBlockThrows() = runTest { + expect(1) + try { + withContext(Dispatchers.Unconfined) { + expect(2) + withContext(Dispatchers.Unconfined + CoroutineName("a")) { + expect(3) + } + + expect(4) + launch(start = CoroutineStart.ATOMIC) { + expect(5) + } + + throw TestException() + } + } catch (e: TestException) { + finish(6) + } + } + + @Test + fun testEnterMultipleTimes() = runTest { + launch(Unconfined) { + expect(1) + } + + launch(Unconfined) { + expect(2) + } + + launch(Unconfined) { + expect(3) + } + + finish(4) + } + + @Test + fun testYield() = runTest { + expect(1) + launch(Dispatchers.Unconfined) { + expect(2) + yield() + launch { + expect(4) + } + expect(3) + yield() + expect(5) + }.join() + + finish(6) + } + + @Test + fun testCancellationWihYields() = runTest { + expect(1) + GlobalScope.launch(Dispatchers.Unconfined) { + val job = coroutineContext[Job]!! + expect(2) + yield() + GlobalScope.launch(Dispatchers.Unconfined) { + expect(4) + job.cancel() + expect(5) + } + expect(3) + + try { + yield() + } finally { + expect(6) + } + } + + finish(7) + } +} diff --git a/kotlinx-coroutines-core/common/test/UndispatchedResultTest.kt b/kotlinx-coroutines-core/common/test/UndispatchedResultTest.kt new file mode 100644 index 0000000000..cec00e7032 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/UndispatchedResultTest.kt @@ -0,0 +1,64 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +class UndispatchedResultTest : TestBase() { + + @Test + fun testWithContext() = runTest { + invokeTest { block -> withContext(wrapperDispatcher(coroutineContext), block) } + } + + @Test + fun testWithContextFastPath() = runTest { + invokeTest { block -> withContext(coroutineContext, block) } + } + + @Test + fun testWithTimeout() = runTest { + invokeTest { block -> withTimeout(Long.MAX_VALUE, block) } + } + + @Test + fun testAsync() = runTest { + invokeTest { block -> async(NonCancellable, block = block).await() } + } + + @Test + fun testCoroutineScope() = runTest { + invokeTest { block -> coroutineScope(block) } + } + + private suspend fun invokeTest(scopeProvider: suspend (suspend CoroutineScope.() -> Unit) -> Unit) { + invokeTest(EmptyCoroutineContext, scopeProvider) + invokeTest(Unconfined, scopeProvider) + } + + private suspend fun invokeTest( + context: CoroutineContext, + scopeProvider: suspend (suspend CoroutineScope.() -> Unit) -> Unit + ) { + try { + scopeProvider { block(context) } + } catch (e: TestException) { + finish(5) + reset() + } + } + + private suspend fun CoroutineScope.block(context: CoroutineContext) { + try { + expect(1) + // Will cancel its parent + async(context) { + expect(2) + throw TestException() + }.await() + } catch (e: TestException) { + expect(3) + } + expect(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/WithContextTest.kt b/kotlinx-coroutines-core/common/test/WithContextTest.kt new file mode 100644 index 0000000000..9e9fd35fd5 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/WithContextTest.kt @@ -0,0 +1,378 @@ + +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-22237 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class WithContextTest : TestBase() { + + @Test + fun testThrowException() = runTest { + expect(1) + try { + withContext(coroutineContext) { + expect(2) + throw AssertionError() + } + } catch (e: AssertionError) { + expect(3) + } + + yield() + finish(4) + } + + @Test + fun testThrowExceptionFromWrappedContext() = runTest { + expect(1) + try { + withContext(wrapperDispatcher(coroutineContext)) { + expect(2) + throw AssertionError() + } + } catch (e: AssertionError) { + expect(3) + } + + yield() + finish(4) + } + + @Test + fun testSameContextNoSuspend() = runTest { + expect(1) + launch(coroutineContext) { // make sure there is not early dispatch here + finish(5) // after main exits + } + expect(2) + val result = withContext(coroutineContext) { // same context! + expect(3) // still here + "OK".wrap() + }.unwrap() + assertEquals("OK", result) + expect(4) + // will wait for the first coroutine + } + + @Test + fun testSameContextWithSuspend() = runTest { + expect(1) + launch(coroutineContext) { // make sure there is not early dispatch here + expect(4) + } + expect(2) + val result = withContext(coroutineContext) { // same context! + expect(3) // still here + yield() // now yields to launch! + expect(5) + "OK".wrap() + }.unwrap() + assertEquals("OK", result) + finish(6) + } + + @Test + fun testCancelWithJobNoSuspend() = runTest { + expect(1) + launch(coroutineContext) { // make sure there is not early dispatch to here + finish(6) // after main exits + } + expect(2) + val job = Job() + try { + withContext(coroutineContext + job) { + // same context + new job + expect(3) // still here + job.cancel() // cancel out job! + try { + yield() // shall throw CancellationException + expectUnreached() + } catch (e: CancellationException) { + expect(4) + } + "OK".wrap() + } + + expectUnreached() + } catch (e: CancellationException) { + expect(5) + // will wait for the first coroutine + } + } + + @Test + fun testCancelWithJobWithSuspend() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + launch(coroutineContext) { // make sure there is not early dispatch to here + expect(4) + } + expect(2) + val job = Job() + withContext(coroutineContext + job) { // same context + new job + expect(3) // still here + yield() // now yields to launch! + expect(5) + job.cancel() // cancel out job! + try { + yield() // shall throw CancellationException + expectUnreached() + } catch (e: CancellationException) { + finish(6) + } + "OK".wrap() + } + // still fails, because parent job was cancelled + expectUnreached() + } + + @Test + fun testRunCancellableDefault() = runTest( + expected = { it is CancellationException } + ) { + val job = Job() + job.cancel() // cancel before it has a chance to run + withContext(job + wrapperDispatcher(coroutineContext)) { + expectUnreached() // will get cancelled + } + } + + @Test + fun testRunCancellationUndispatchedVsException() = runTest { + expect(1) + var job: Job? = null + job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + // Same dispatcher, different context + withContext(CoroutineName("testRunCancellationUndispatchedVsException")) { + expect(3) + yield() // must suspend + expect(5) + job!!.cancel() // cancel this job _before_ it throws + throw TestException() + } + } catch (e: TestException) { + // must have caught TextException + expect(6) + } + } + expect(4) + yield() // to coroutineScope + finish(7) + } + + @Test + fun testRunCancellationDispatchedVsException() = runTest { + expect(1) + var job: Job? = null + job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + // "Different" dispatcher (schedules execution back and forth) + withContext(wrapperDispatcher(coroutineContext)) { + expect(4) + yield() // must suspend + expect(6) + job!!.cancel() // cancel this job _before_ it throws + throw TestException() + } + } catch (e: TestException) { + // must have caught TextException + expect(8) + } + } + expect(3) + yield() // withContext is next + expect(5) + yield() // withContext again + expect(7) + yield() // to catch block + finish(9) + } + + @Test + fun testRunSelfCancellationWithException() = runTest { + expect(1) + var job: Job? = null + job = launch(Job()) { + try { + expect(3) + withContext(wrapperDispatcher(coroutineContext)) { + require(isActive) + expect(5) + job!!.cancel() + require(!isActive) + throw TestException() // but throw an exception + } + } catch (e: Throwable) { + expect(7) + // make sure TestException, not CancellationException is thrown + assertIs(e, "Caught $e") + } + } + expect(2) + yield() // to the launched job + expect(4) + yield() // again to the job + expect(6) + yield() // again to exception handler + finish(8) + } + + @Test + fun testRunSelfCancellation() = runTest { + expect(1) + var job: Job? = null + job = launch(Job()) { + try { + expect(3) + withContext(wrapperDispatcher(coroutineContext)) { + require(isActive) + expect(5) + job!!.cancel() // cancel itself + require(!isActive) + "OK".wrap() + } + expectUnreached() + } catch (e: Throwable) { + expect(7) + // make sure CancellationException is thrown + assertIs(e, "Caught $e") + } + } + + expect(2) + yield() // to the launched job + expect(4) + yield() // again to the job + expect(6) + yield() // again to exception handler + finish(8) + } + + @Test + fun testWithContextScopeFailure() = runTest { + expect(1) + try { + withContext(wrapperDispatcher(coroutineContext)) { + expect(2) + // launch a child that fails + launch { + expect(4) + throw TestException() + } + expect(3) + "OK".wrap() + } + expectUnreached() + } catch (e: TestException) { + // ensure that we can catch exception outside of the scope + expect(5) + } + finish(6) + } + + @Test + fun testWithContextChildWaitSameContext() = runTest { + expect(1) + withContext(coroutineContext) { + expect(2) + launch { + // ^^^ schedules to main thread + expect(4) // waits before return + } + expect(3) + "OK".wrap() + }.unwrap() + finish(5) + } + + @Test + fun testWithContextChildWaitWrappedContext() = runTest { + expect(1) + withContext(wrapperDispatcher(coroutineContext)) { + expect(2) + launch { + // ^^^ schedules to main thread + expect(4) // waits before return + } + expect(3) + "OK".wrap() + }.unwrap() + finish(5) + } + + @Test + fun testIncompleteWithContextState() = runTest { + lateinit var ctxJob: Job + withContext(wrapperDispatcher(coroutineContext)) { + ctxJob = coroutineContext[Job]!! + ctxJob.invokeOnCompletion { } + } + + ctxJob.join() + assertTrue(ctxJob.isCompleted) + assertFalse(ctxJob.isActive) + assertFalse(ctxJob.isCancelled) + } + + @Test + fun testWithContextCancelledJob() = runTest { + expect(1) + val job = Job() + job.cancel() + try { + withContext(job) { + expectUnreached() + } + } catch (e: CancellationException) { + expect(2) + } + finish(3) + } + + @Test + fun testWithContextCancelledThisJob() = runTest( + expected = { it is CancellationException } + ) { + coroutineContext.cancel() + withContext(wrapperDispatcher(coroutineContext)) { + expectUnreached() + } + expectUnreached() + } + + @Test + fun testSequentialCancellation() = runTest { + val job = launch { + expect(1) + withContext(wrapperDispatcher()) { + expect(2) + } + expectUnreached() + } + + yield() + val job2 = launch { + expect(3) + job.cancel() + } + + joinAll(job, job2) + finish(4) + } + + private class Wrapper(val value: String) : Incomplete { + override val isActive: Boolean + get() = error("") + override val list: NodeList? + get() = error("") + } + + private fun String.wrap() = Wrapper(this) + private fun Wrapper.unwrap() = value +} diff --git a/kotlinx-coroutines-core/common/test/WithTimeoutDurationTest.kt b/kotlinx-coroutines-core/common/test/WithTimeoutDurationTest.kt new file mode 100644 index 0000000000..855b00f2c7 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/WithTimeoutDurationTest.kt @@ -0,0 +1,212 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED", "UNREACHABLE_CODE") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.seconds + +class WithTimeoutDurationTest : TestBase() { + /** + * Tests a case of no timeout and no suspension inside. + */ + @Test + fun testBasicNoSuspend() = runTest { + expect(1) + val result = withTimeout(10.seconds) { + expect(2) + "OK" + } + assertEquals("OK", result) + finish(3) + } + + /** + * Tests a case of no timeout and one suspension inside. + */ + @Test + fun testBasicSuspend() = runTest { + expect(1) + val result = withTimeout(10.seconds) { + expect(2) + yield() + expect(3) + "OK" + } + assertEquals("OK", result) + finish(4) + } + + /** + * Tests proper dispatching of `withTimeout` blocks + */ + @Test + fun testDispatch() = runTest { + expect(1) + launch { + expect(4) + yield() // back to main + expect(7) + } + expect(2) + // test that it does not yield to the above job when started + val result = withTimeout(1.seconds) { + expect(3) + yield() // yield only now + expect(5) + "OK" + } + assertEquals("OK", result) + expect(6) + yield() // back to launch + finish(8) + } + + + /** + * Tests that a 100% CPU-consuming loop will react on timeout if it has yields. + */ + @Test + fun testYieldBlockingWithTimeout() = runTest( + expected = { it is CancellationException } + ) { + withTimeout(100.milliseconds) { + while (true) { + yield() + } + } + } + + /** + * Tests that [withTimeout] waits for children coroutines to complete. + */ + @Test + fun testWithTimeoutChildWait() = runTest { + expect(1) + withTimeout(100.milliseconds) { + expect(2) + // launch child with timeout + launch { + expect(4) + } + expect(3) + // now will wait for child before returning + } + finish(5) + } + + @Test + fun testBadClass() = runTest { + val bad = BadClass() + val result = withTimeout(100.milliseconds) { + bad + } + assertSame(bad, result) + } + + class BadClass { + override fun equals(other: Any?): Boolean = error("Should not be called") + override fun hashCode(): Int = error("Should not be called") + override fun toString(): String = error("Should not be called") + } + + @Test + fun testExceptionOnTimeout() = runTest { + expect(1) + try { + withTimeout(100.milliseconds) { + expect(2) + delay(1000.milliseconds) + expectUnreached() + "OK" + } + } catch (e: CancellationException) { + assertEquals("Timed out waiting for 100 ms", e.message) + finish(3) + } + } + + @Test + fun testSuppressExceptionWithResult() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + withTimeout(100.milliseconds) { + expect(2) + try { + delay(1000.milliseconds) + } catch (e: CancellationException) { + finish(3) + } + "OK" + } + expectUnreached() + } + + @Test + fun testSuppressExceptionWithAnotherException() = runTest { + expect(1) + try { + withTimeout(100.milliseconds) { + expect(2) + try { + delay(1000.milliseconds) + } catch (e: CancellationException) { + expect(3) + throw TestException() + } + expectUnreached() + "OK" + } + expectUnreached() + } catch (e: TestException) { + finish(4) + } + } + + @Test + fun testNegativeTimeout() = runTest { + expect(1) + try { + withTimeout(-1.milliseconds) { + expectUnreached() + "OK" + } + } catch (e: TimeoutCancellationException) { + assertEquals("Timed out immediately", e.message) + finish(2) + } + } + + @Test + fun testExceptionFromWithinTimeout() = runTest { + expect(1) + try { + expect(2) + withTimeout(1.seconds) { + expect(3) + throw TestException() + } + expectUnreached() + } catch (e: TestException) { + finish(4) + } + } + + @Test + fun testIncompleteWithTimeoutState() = runTest { + lateinit var timeoutJob: Job + val handle = withTimeout(Duration.INFINITE) { + timeoutJob = coroutineContext[Job]!! + timeoutJob.invokeOnCompletion { } + } + + handle.dispose() + timeoutJob.join() + assertTrue(timeoutJob.isCompleted) + assertFalse(timeoutJob.isActive) + assertFalse(timeoutJob.isCancelled) + } +} diff --git a/kotlinx-coroutines-core/common/test/WithTimeoutOrNullDurationTest.kt b/kotlinx-coroutines-core/common/test/WithTimeoutOrNullDurationTest.kt new file mode 100644 index 0000000000..f92a4c2654 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/WithTimeoutOrNullDurationTest.kt @@ -0,0 +1,243 @@ + +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.seconds + +class WithTimeoutOrNullDurationTest : TestBase() { + /** + * Tests a case of no timeout and no suspension inside. + */ + @Test + fun testBasicNoSuspend() = runTest { + expect(1) + val result = withTimeoutOrNull(10.seconds) { + expect(2) + "OK" + } + assertEquals("OK", result) + finish(3) + } + + /** + * Tests a case of no timeout and one suspension inside. + */ + @Test + fun testBasicSuspend() = runTest { + expect(1) + val result = withTimeoutOrNull(10.seconds) { + expect(2) + yield() + expect(3) + "OK" + } + assertEquals("OK", result) + finish(4) + } + + /** + * Tests property dispatching of `withTimeoutOrNull` blocks + */ + @Test + fun testDispatch() = runTest { + expect(1) + launch { + expect(4) + yield() // back to main + expect(7) + } + expect(2) + // test that it does not yield to the above job when started + val result = withTimeoutOrNull(1.seconds) { + expect(3) + yield() // yield only now + expect(5) + "OK" + } + assertEquals("OK", result) + expect(6) + yield() // back to launch + finish(8) + } + + /** + * Tests that a 100% CPU-consuming loop will react on timeout if it has yields. + */ + @Test + fun testYieldBlockingWithTimeout() = runTest { + expect(1) + val result = withTimeoutOrNull(100.milliseconds) { + while (true) { + yield() + } + } + assertNull(result) + finish(2) + } + + @Test + fun testSmallTimeout() = runTest { + val channel = Channel(1) + val value = withTimeoutOrNull(1.milliseconds) { + channel.receive() + } + assertNull(value) + } + + @Test + fun testThrowException() = runTest(expected = {it is AssertionError}) { + withTimeoutOrNull(Duration.INFINITE) { + throw AssertionError() + } + } + + @Test + fun testInnerTimeout() = runTest( + expected = { it is CancellationException } + ) { + withTimeoutOrNull(1000.milliseconds) { + withTimeout(10.milliseconds) { + while (true) { + yield() + } + } + @Suppress("UNREACHABLE_CODE") + expectUnreached() // will timeout + } + expectUnreached() // will timeout + } + + @Test + fun testNestedTimeout() = runTest(expected = { it is TimeoutCancellationException }) { + withTimeoutOrNull(Duration.INFINITE) { + // Exception from this withTimeout is not suppressed by withTimeoutOrNull + withTimeout(10.milliseconds) { + delay(Duration.INFINITE) + 1 + } + } + + expectUnreached() + } + + @Test + fun testOuterTimeout() = runTest { + if (isJavaAndWindows) return@runTest + var counter = 0 + val result = withTimeoutOrNull(320.milliseconds) { + while (true) { + val inner = withTimeoutOrNull(150.milliseconds) { + while (true) { + yield() + } + } + assertNull(inner) + counter++ + } + } + assertNull(result) + check(counter in 1..2) {"Executed: $counter times"} + } + + @Test + fun testBadClass() = runTest { + val bad = BadClass() + val result = withTimeoutOrNull(100.milliseconds) { + bad + } + assertSame(bad, result) + } + + class BadClass { + override fun equals(other: Any?): Boolean = error("Should not be called") + override fun hashCode(): Int = error("Should not be called") + override fun toString(): String = error("Should not be called") + } + + @Test + fun testNullOnTimeout() = runTest { + expect(1) + val result = withTimeoutOrNull(100.milliseconds) { + expect(2) + delay(1000.milliseconds) + expectUnreached() + "OK" + } + assertNull(result) + finish(3) + } + + @Test + fun testSuppressExceptionWithResult() = runTest { + expect(1) + val result = withTimeoutOrNull(100.milliseconds) { + expect(2) + try { + delay(1000.milliseconds) + } catch (e: CancellationException) { + expect(3) + } + "OK" + } + assertNull(result) + finish(4) + } + + @Test + fun testSuppressExceptionWithAnotherException() = runTest { + expect(1) + try { + withTimeoutOrNull(100.milliseconds) { + expect(2) + try { + delay(1000.milliseconds) + } catch (e: CancellationException) { + expect(3) + throw TestException() + } + expectUnreached() + "OK" + } + expectUnreached() + } catch (e: TestException) { + // catches TestException + finish(4) + + } + } + + @Test + fun testNegativeTimeout() = runTest { + expect(1) + var result = withTimeoutOrNull(-1.milliseconds) { + expectUnreached() + } + assertNull(result) + result = withTimeoutOrNull(0.milliseconds) { + expectUnreached() + } + assertNull(result) + finish(2) + } + + @Test + fun testExceptionFromWithinTimeout() = runTest { + expect(1) + try { + expect(2) + withTimeoutOrNull(1000.milliseconds) { + expect(3) + throw TestException() + } + expectUnreached() + } catch (e: TestException) { + finish(4) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/WithTimeoutOrNullTest.kt b/kotlinx-coroutines-core/common/test/WithTimeoutOrNullTest.kt new file mode 100644 index 0000000000..51a6a38daf --- /dev/null +++ b/kotlinx-coroutines-core/common/test/WithTimeoutOrNullTest.kt @@ -0,0 +1,234 @@ + +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class WithTimeoutOrNullTest : TestBase() { + /** + * Tests a case of no timeout and no suspension inside. + */ + @Test + fun testBasicNoSuspend() = runTest { + expect(1) + val result = withTimeoutOrNull(10_000) { + expect(2) + "OK" + } + assertEquals("OK", result) + finish(3) + } + + /** + * Tests a case of no timeout and one suspension inside. + */ + @Test + fun testBasicSuspend() = runTest { + expect(1) + val result = withTimeoutOrNull(10_000) { + expect(2) + yield() + expect(3) + "OK" + } + assertEquals("OK", result) + finish(4) + } + + /** + * Tests property dispatching of `withTimeoutOrNull` blocks + */ + @Test + fun testDispatch() = runTest { + expect(1) + launch { + expect(4) + yield() // back to main + expect(7) + } + expect(2) + // test that it does not yield to the above job when started + val result = withTimeoutOrNull(1000) { + expect(3) + yield() // yield only now + expect(5) + "OK" + } + assertEquals("OK", result) + expect(6) + yield() // back to launch + finish(8) + } + + /** + * Tests that a 100% CPU-consuming loop will react on timeout if it has yields. + */ + @Test + fun testYieldBlockingWithTimeout() = runTest { + expect(1) + val result = withTimeoutOrNull(100) { + while (true) { + yield() + } + } + assertNull(result) + finish(2) + } + + @Test + fun testSmallTimeout() = runTest { + val channel = Channel(1) + val value = withTimeoutOrNull(1) { + channel.receive() + } + assertNull(value) + } + + @Test + fun testThrowException() = runTest(expected = {it is AssertionError}) { + withTimeoutOrNull(Long.MAX_VALUE) { + throw AssertionError() + } + } + + @Test + fun testInnerTimeout() = runTest( + expected = { it is CancellationException } + ) { + withTimeoutOrNull(1000) { + withTimeout(10) { + while (true) { + yield() + } + } + @Suppress("UNREACHABLE_CODE") + expectUnreached() // will timeout + } + expectUnreached() // will timeout + } + + @Test + fun testNestedTimeout() = runTest(expected = { it is TimeoutCancellationException }) { + withTimeoutOrNull(Long.MAX_VALUE) { + // Exception from this withTimeout is not suppressed by withTimeoutOrNull + withTimeout(10) { + delay(Long.MAX_VALUE) + 1 + } + } + + expectUnreached() + } + + @Test + fun testOuterTimeout() = runTest { + if (isJavaAndWindows) return@runTest + var counter = 0 + val result = withTimeoutOrNull(320) { + while (true) { + val inner = withTimeoutOrNull(150) { + while (true) { + yield() + } + } + assertNull(inner) + counter++ + } + } + assertNull(result) + check(counter in 1..2) {"Executed: $counter times"} + } + + @Test + fun testBadClass() = runTest { + val bad = BadClass() + val result = withTimeoutOrNull(100) { + bad + } + assertSame(bad, result) + } + + @Test + fun testNullOnTimeout() = runTest { + expect(1) + val result = withTimeoutOrNull(100) { + expect(2) + delay(1000) + expectUnreached() + "OK" + } + assertNull(result) + finish(3) + } + + @Test + fun testSuppressExceptionWithResult() = runTest { + expect(1) + val result = withTimeoutOrNull(100) { + expect(2) + try { + delay(1000) + } catch (e: CancellationException) { + expect(3) + } + "OK" + } + assertNull(result) + finish(4) + } + + @Test + fun testSuppressExceptionWithAnotherException() = runTest { + expect(1) + try { + withTimeoutOrNull(100) { + expect(2) + try { + delay(1000) + } catch (e: CancellationException) { + expect(3) + throw TestException() + } + expectUnreached() + "OK" + } + expectUnreached() + } catch (e: TestException) { + // catches TestException + finish(4) + + } + } + + @Test + fun testNegativeTimeout() = runTest { + expect(1) + var result = withTimeoutOrNull(-1) { + expectUnreached() + } + assertNull(result) + result = withTimeoutOrNull(0) { + expectUnreached() + } + assertNull(result) + finish(2) + } + + @Test + fun testExceptionFromWithinTimeout() = runTest { + expect(1) + try { + expect(2) + withTimeoutOrNull(1000) { + expect(3) + throw TestException() + } + expectUnreached() + } catch (e: TestException) { + finish(4) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/WithTimeoutTest.kt b/kotlinx-coroutines-core/common/test/WithTimeoutTest.kt new file mode 100644 index 0000000000..5f2690c198 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/WithTimeoutTest.kt @@ -0,0 +1,204 @@ + +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED", "UNREACHABLE_CODE") // KT-21913 + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class WithTimeoutTest : TestBase() { + /** + * Tests a case of no timeout and no suspension inside. + */ + @Test + fun testBasicNoSuspend() = runTest { + expect(1) + val result = withTimeout(10_000) { + expect(2) + "OK" + } + assertEquals("OK", result) + finish(3) + } + + /** + * Tests a case of no timeout and one suspension inside. + */ + @Test + fun testBasicSuspend() = runTest { + expect(1) + val result = withTimeout(10_000) { + expect(2) + yield() + expect(3) + "OK" + } + assertEquals("OK", result) + finish(4) + } + + /** + * Tests proper dispatching of `withTimeout` blocks + */ + @Test + fun testDispatch() = runTest { + expect(1) + launch { + expect(4) + yield() // back to main + expect(7) + } + expect(2) + // test that it does not yield to the above job when started + val result = withTimeout(1000) { + expect(3) + yield() // yield only now + expect(5) + "OK" + } + assertEquals("OK", result) + expect(6) + yield() // back to launch + finish(8) + } + + + /** + * Tests that a 100% CPU-consuming loop will react on timeout if it has yields. + */ + @Test + fun testYieldBlockingWithTimeout() = runTest( + expected = { it is CancellationException } + ) { + withTimeout(100) { + while (true) { + yield() + } + } + } + + /** + * Tests that [withTimeout] waits for children coroutines to complete. + */ + @Test + fun testWithTimeoutChildWait() = runTest { + expect(1) + withTimeout(100) { + expect(2) + // launch child with timeout + launch { + expect(4) + } + expect(3) + // now will wait for child before returning + } + finish(5) + } + + @Test + fun testBadClass() = runTest { + val bad = BadClass() + val result = withTimeout(100) { + bad + } + assertSame(bad, result) + } + + @Test + fun testExceptionOnTimeout() = runTest { + expect(1) + try { + withTimeout(100) { + expect(2) + delay(1000) + expectUnreached() + "OK" + } + } catch (e: CancellationException) { + assertEquals("Timed out waiting for 100 ms", e.message) + finish(3) + } + } + + @Test + fun testSuppressExceptionWithResult() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + withTimeout(100) { + expect(2) + try { + delay(1000) + } catch (e: CancellationException) { + finish(3) + } + "OK" + } + expectUnreached() + } + + @Test + fun testSuppressExceptionWithAnotherException() = runTest{ + expect(1) + try { + withTimeout(100) { + expect(2) + try { + delay(1000) + } catch (e: CancellationException) { + expect(3) + throw TestException() + } + expectUnreached() + "OK" + } + expectUnreached() + } catch (e: TestException) { + finish(4) + } + } + + @Test + fun testNegativeTimeout() = runTest { + expect(1) + try { + withTimeout(-1) { + expectUnreached() + "OK" + } + } catch (e: TimeoutCancellationException) { + assertEquals("Timed out immediately", e.message) + finish(2) + } + } + + @Test + fun testExceptionFromWithinTimeout() = runTest { + expect(1) + try { + expect(2) + withTimeout(1000) { + expect(3) + throw TestException() + } + expectUnreached() + } catch (e: TestException) { + finish(4) + } + } + + @Test + fun testIncompleteWithTimeoutState() = runTest { + lateinit var timeoutJob: Job + val handle = withTimeout(Long.MAX_VALUE) { + timeoutJob = coroutineContext[Job]!! + timeoutJob.invokeOnCompletion { } + } + + handle.dispose() + timeoutJob.join() + assertTrue(timeoutJob.isCompleted) + assertFalse(timeoutJob.isActive) + assertFalse(timeoutJob.isCancelled) + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/BasicOperationsTest.kt b/kotlinx-coroutines-core/common/test/channels/BasicOperationsTest.kt new file mode 100644 index 0000000000..fb9e0d9cf1 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/BasicOperationsTest.kt @@ -0,0 +1,222 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class BasicOperationsTest : TestBase() { + @Test + fun testSimpleSendReceive() = runTest { + // Parametrized common test :( + TestChannelKind.values().forEach { kind -> testSendReceive(kind, 20) } + } + + @Test + fun testTrySendToFullChannel() = runTest { + TestChannelKind.values().forEach { kind -> testTrySendToFullChannel(kind) } + } + + @Test + fun testTrySendAfterClose() = runTest { + TestChannelKind.values().forEach { kind -> testTrySendAfterClose(kind) } + } + + @Test + fun testSendAfterClose() = runTest { + TestChannelKind.values().forEach { kind -> testSendAfterClose(kind) } + } + + @Test + fun testReceiveCatching() = runTest { + TestChannelKind.values().forEach { kind -> testReceiveCatching(kind) } + } + + @Test + fun testInvokeOnClose() = TestChannelKind.values().forEach { kind -> + reset() + val channel = kind.create() + channel.invokeOnClose { + if (it is AssertionError) { + expect(3) + } + } + expect(1) + channel.trySend(42) + expect(2) + channel.close(AssertionError()) + finish(4) + } + + @Test + fun testInvokeOnClosed() = TestChannelKind.values().forEach { kind -> + reset() + expect(1) + val channel = kind.create() + channel.close() + channel.invokeOnClose { expect(2) } + assertFailsWith { channel.invokeOnClose { expect(3) } } + finish(3) + } + + @Test + fun testMultipleInvokeOnClose() = TestChannelKind.values().forEach { kind -> + reset() + val channel = kind.create() + channel.invokeOnClose { expect(3) } + expect(1) + assertFailsWith { channel.invokeOnClose { expect(4) } } + expect(2) + channel.close() + finish(4) + } + + @Test + fun testIterator() = runTest { + TestChannelKind.values().forEach { kind -> + val channel = kind.create() + val iterator = channel.iterator() + assertFailsWith { iterator.next() } + channel.close() + assertFailsWith { iterator.next() } + assertFalse(iterator.hasNext()) + } + } + + @Test + fun testCancelledChannelInvokeOnClose() { + val ch = Channel() + ch.invokeOnClose { assertIs(it) } + ch.cancel() + } + + @Test + fun testCancelledChannelWithCauseInvokeOnClose() { + val ch = Channel() + ch.invokeOnClose { assertIs(it) } + ch.cancel(TimeoutCancellationException("")) + } + + @Test + fun testThrowingInvokeOnClose() = runTest { + val channel = Channel() + channel.invokeOnClose { + assertNull(it) + expect(3) + throw TestException() + } + + launch { + try { + expect(2) + channel.close() + } catch (e: TestException) { + expect(4) + } + } + expect(1) + yield() + assertTrue(channel.isClosedForReceive) + assertTrue(channel.isClosedForSend) + assertFalse(channel.close()) + finish(5) + } + + @Suppress("ReplaceAssertBooleanWithAssertEquality") + private suspend fun testReceiveCatching(kind: TestChannelKind) = coroutineScope { + reset() + val channel = kind.create() + launch { + expect(2) + channel.send(1) + } + + expect(1) + val result = channel.receiveCatching() + assertEquals(1, result.getOrThrow()) + assertEquals(1, result.getOrNull()) + assertTrue(ChannelResult.success(1) == result) + + expect(3) + launch { + expect(4) + channel.close() + } + val closed = channel.receiveCatching() + expect(5) + assertNull(closed.getOrNull()) + assertTrue(closed.isClosed) + assertNull(closed.exceptionOrNull()) + assertTrue(ChannelResult.closed(closed.exceptionOrNull()) == closed) + finish(6) + } + + private suspend fun testTrySendAfterClose(kind: TestChannelKind) = coroutineScope { + val channel = kind.create() + val d = async { channel.send(42) } + yield() + channel.close() + + assertTrue(channel.isClosedForSend) + channel.trySend(2) + .onSuccess { expectUnreached() } + .onClosed { + assertTrue { it is ClosedSendChannelException } + if (!kind.isConflated) { + assertEquals(42, channel.receive()) + } + } + d.await() + } + + private suspend fun testTrySendToFullChannel(kind: TestChannelKind) = coroutineScope { + if (kind.isConflated || kind.capacity == Int.MAX_VALUE) return@coroutineScope + val channel = kind.create() + // Make it full + repeat(11) { + channel.trySend(42) + } + channel.trySend(1) + .onSuccess { expectUnreached() } + .onFailure { assertNull(it) } + .onClosed { + expectUnreached() + } + } + + /** + * [ClosedSendChannelException] should not be eaten. + * See [https://github.com/Kotlin/kotlinx.coroutines/issues/957] + */ + private suspend fun testSendAfterClose(kind: TestChannelKind) { + assertFailsWith { + coroutineScope { + val channel = kind.create() + channel.close() + + launch { + channel.send(1) + } + } + } + } + + private suspend fun testSendReceive(kind: TestChannelKind, iterations: Int) = coroutineScope { + val channel = kind.create() + launch { + repeat(iterations) { channel.send(it) } + channel.close() + } + var expected = 0 + for (x in channel) { + if (!kind.isConflated) { + assertEquals(expected++, x) + } else { + assertTrue(x >= expected) + expected = x + 1 + } + } + if (!kind.isConflated) { + assertEquals(iterations, expected) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/BroadcastChannelFactoryTest.kt b/kotlinx-coroutines-core/common/test/channels/BroadcastChannelFactoryTest.kt new file mode 100644 index 0000000000..f13aae5ed8 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/BroadcastChannelFactoryTest.kt @@ -0,0 +1,34 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +@Suppress("DEPRECATION_ERROR") +class BroadcastChannelFactoryTest : TestBase() { + + @Test + fun testRendezvousChannelNotSupported() { + assertFailsWith { BroadcastChannel(0) } + } + + @Test + fun testUnlimitedChannelNotSupported() { + assertFailsWith { BroadcastChannel(Channel.UNLIMITED) } + } + + @Test + fun testConflatedBroadcastChannel() { + assertTrue { BroadcastChannel(Channel.CONFLATED) is ConflatedBroadcastChannel } + } + + @Test + fun testBufferedBroadcastChannel() { + assertTrue { BroadcastChannel(1) is BroadcastChannelImpl } + assertTrue { BroadcastChannel(10) is BroadcastChannelImpl } + } + + @Test + fun testInvalidCapacityNotSupported() { + assertFailsWith { BroadcastChannel(-3) } + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/BroadcastTest.kt b/kotlinx-coroutines-core/common/test/channels/BroadcastTest.kt new file mode 100644 index 0000000000..3e8514b0be --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/BroadcastTest.kt @@ -0,0 +1,139 @@ +@file:Suppress("DEPRECATION_ERROR") + +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import kotlin.test.* + +class BroadcastTest : TestBase() { + @Test + fun testBroadcastBasic() = runTest { + expect(1) + val b = broadcast { + expect(4) + send(1) // goes to receiver + expect(5) + select { onSend(2) {} } // goes to buffer + expect(6) + send(3) // suspends, will not be consumes, but will not be cancelled either + expect(10) + } + yield() // has no effect, because default is lazy + expect(2) + + val subscription = b.openSubscription() + expect(3) + assertEquals(1, subscription.receive()) // suspends + expect(7) + assertEquals(2, subscription.receive()) // suspends + expect(8) + subscription.cancel() + expect(9) + yield() // to broadcast + finish(11) + } + + /** + * See https://github.com/Kotlin/kotlinx.coroutines/issues/1713 + */ + @Test + fun testChannelBroadcastLazyCancel() = runTest { + expect(1) + val a = produce { + expect(3) + assertFailsWith { send("MSG") } + expect(5) + } + expect(2) + yield() // to produce + val b = a.broadcast() + b.cancel() + expect(4) + yield() // to abort produce + assertTrue(a.isClosedForReceive) // the source channel was consumed + finish(6) + } + + @Test + fun testChannelBroadcastLazyClose() = runTest { + expect(1) + val a = produce { + expect(3) + send("MSG") + expectUnreached() // is not executed, because send is cancelled + } + expect(2) + yield() // to produce + val b = a.broadcast() + b.close() + expect(4) + yield() // to abort produce + assertTrue(a.isClosedForReceive) // the source channel was consumed + finish(5) + } + + @Test + fun testChannelBroadcastEagerCancel() = runTest { + expect(1) + val a = produce { + expect(3) + yield() // back to main + expectUnreached() // will be cancelled + } + expect(2) + val b = a.broadcast(start = CoroutineStart.DEFAULT) + yield() // to produce + expect(4) + b.cancel() + yield() // to produce (cancelled) + assertTrue(a.isClosedForReceive) // the source channel was consumed + finish(5) + } + + @Test + fun testChannelBroadcastEagerClose() = runTest { + expect(1) + val a = produce { + expect(3) + yield() // back to main + // shall eventually get cancelled + assertFailsWith { + while (true) { send(Unit) } + } + } + expect(2) + val b = a.broadcast(start = CoroutineStart.DEFAULT) + yield() // to produce + expect(4) + b.close() + yield() // to produce (closed) + assertTrue(a.isClosedForReceive) // the source channel was consumed + finish(5) + } + + @Test + fun testBroadcastCloseWithException() = runTest { + expect(1) + val b = broadcast(NonCancellable, capacity = 1) { + expect(2) + send(1) + expect(3) + send(2) // suspends + expect(5) + // additional attempts to send fail + assertFailsWith { send(3) } + } + val sub = b.openSubscription() + yield() // into broadcast + expect(4) + b.close(TestException()) // close broadcast channel with exception + assertTrue(b.isClosedForSend) // sub was also closed + assertEquals(1, sub.receive()) // 1st element received + assertEquals(2, sub.receive()) // 2nd element received + assertFailsWith { sub.receive() } // then closed with exception + yield() // to cancel broadcast + finish(6) + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/BufferedBroadcastChannelTest.kt b/kotlinx-coroutines-core/common/test/channels/BufferedBroadcastChannelTest.kt new file mode 100644 index 0000000000..9680232d5d --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/BufferedBroadcastChannelTest.kt @@ -0,0 +1,211 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +@Suppress("DEPRECATION_ERROR") +class BufferedBroadcastChannelTest : TestBase() { + + @Test + fun testConcurrentModification() = runTest { + val channel = BroadcastChannel(1) + val s1 = channel.openSubscription() + val s2 = channel.openSubscription() + + val job1 = launch(Dispatchers.Unconfined, CoroutineStart.UNDISPATCHED) { + expect(1) + s1.receive() + s1.cancel() + } + + val job2 = launch(Dispatchers.Unconfined, CoroutineStart.UNDISPATCHED) { + expect(2) + s2.receive() + } + + expect(3) + channel.send(1) + joinAll(job1, job2) + finish(4) + } + + @Test + fun testBasic() = runTest { + expect(1) + val broadcast = BroadcastChannel(1) + assertFalse(broadcast.isClosedForSend) + val first = broadcast.openSubscription() + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + assertEquals(1, first.receive()) // suspends + assertFalse(first.isClosedForReceive) + expect(5) + assertEquals(2, first.receive()) // suspends + assertFalse(first.isClosedForReceive) + expect(10) + assertTrue(first.receiveCatching().isClosed) // suspends + assertTrue(first.isClosedForReceive) + expect(14) + } + expect(3) + broadcast.send(1) + expect(4) + yield() // to the first receiver + expect(6) + + val second = broadcast.openSubscription() + launch(start = CoroutineStart.UNDISPATCHED) { + expect(7) + assertEquals(2, second.receive()) // suspends + assertFalse(second.isClosedForReceive) + expect(11) + assertNull(second.receiveCatching().getOrNull()) // suspends + assertTrue(second.isClosedForReceive) + expect(15) + } + expect(8) + broadcast.send(2) + expect(9) + yield() // to first & second receivers + expect(12) + broadcast.close() + expect(13) + assertTrue(broadcast.isClosedForSend) + yield() // to first & second receivers + finish(16) + } + + @Test + fun testSendSuspend() = runTest { + expect(1) + val broadcast = BroadcastChannel(1) + val first = broadcast.openSubscription() + launch { + expect(4) + assertEquals(1, first.receive()) + expect(5) + assertEquals(2, first.receive()) + expect(6) + } + expect(2) + broadcast.send(1) // puts to buffer, receiver not running yet + expect(3) + broadcast.send(2) // suspends + finish(7) + } + + @Test + fun testConcurrentSendCompletion() = runTest { + expect(1) + val broadcast = BroadcastChannel(1) + val sub = broadcast.openSubscription() + // launch 3 concurrent senders (one goes buffer, two other suspend) + for (x in 1..3) { + launch(start = CoroutineStart.UNDISPATCHED) { + expect(x + 1) + broadcast.send(x) + } + } + // and close it for send + expect(5) + broadcast.close() + // now must receive all 3 items + expect(6) + assertFalse(sub.isClosedForReceive) + for (x in 1..3) + assertEquals(x, sub.receiveCatching().getOrNull()) + // and receive close signal + assertNull(sub.receiveCatching().getOrNull()) + assertTrue(sub.isClosedForReceive) + finish(7) + } + + @Test + fun testForgetUnsubscribed() = runTest { + expect(1) + val broadcast = BroadcastChannel(1) + broadcast.send(1) + broadcast.send(2) + broadcast.send(3) + expect(2) // should not suspend anywhere above + val sub = broadcast.openSubscription() + launch(start = CoroutineStart.UNDISPATCHED) { + expect(3) + assertEquals(4, sub.receive()) // suspends + expect(5) + } + expect(4) + broadcast.send(4) // sends + yield() + finish(6) + } + + @Test + fun testReceiveFullAfterClose() = runTest { + val channel = BroadcastChannel(10) + val sub = channel.openSubscription() + // generate into buffer & close + for (x in 1..5) channel.send(x) + channel.close() + // make sure all of them are consumed + check(!sub.isClosedForReceive) + for (x in 1..5) check(sub.receive() == x) + check(sub.receiveCatching().getOrNull() == null) + check(sub.isClosedForReceive) + } + + @Test + fun testCloseSubDuringIteration() = runTest { + val channel = BroadcastChannel(1) + // launch generator (for later) in this context + launch { + for (x in 1..5) { + channel.send(x) + } + channel.close() + } + // start consuming + val sub = channel.openSubscription() + var expected = 0 + assertFailsWith { + sub.consumeEach { + check(it == ++expected) + if (it == 2) { + sub.cancel() + } + } + } + check(expected == 2) + } + + @Test + fun testReceiveFromCancelledSub() = runTest { + val channel = BroadcastChannel(1) + val sub = channel.openSubscription() + assertFalse(sub.isClosedForReceive) + sub.cancel() + assertTrue(sub.isClosedForReceive) + assertFailsWith { sub.receive() } + } + + @Test + fun testCancelWithCause() = runTest({ it is TestCancellationException }) { + val channel = BroadcastChannel(1) + val subscription = channel.openSubscription() + subscription.cancel(TestCancellationException()) + subscription.receive() + } + + @Test + fun testReceiveNoneAfterCancel() = runTest { + val channel = BroadcastChannel(10) + val sub = channel.openSubscription() + // generate into buffer & cancel + for (x in 1..5) channel.send(x) + channel.cancel() + assertTrue(channel.isClosedForSend) + assertTrue(sub.isClosedForReceive) + check(sub.receiveCatching().getOrNull() == null) + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/BufferedChannelTest.kt b/kotlinx-coroutines-core/common/test/channels/BufferedChannelTest.kt new file mode 100644 index 0000000000..d314f8b1f4 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/BufferedChannelTest.kt @@ -0,0 +1,250 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class BufferedChannelTest : TestBase() { + + /** Tests that a buffered channel does not consume enough memory to fail with an OOM. */ + @Test + fun testMemoryConsumption() = runTest { + val largeChannel = Channel(Int.MAX_VALUE / 2) + repeat(10_000) { + largeChannel.send(it) + } + repeat(10_000) { + val element = largeChannel.receive() + assertEquals(it, element) + } + } + + @Test + fun testIteratorHasNextIsIdempotent() = runTest { + val q = Channel() + check(q.isEmpty) + val iter = q.iterator() + expect(1) + val sender = launch { + expect(4) + q.send(1) // sent + expect(10) + q.close() + expect(11) + } + expect(2) + val receiver = launch { + expect(5) + check(iter.hasNext()) + expect(6) + check(iter.hasNext()) + expect(7) + check(iter.hasNext()) + expect(8) + check(iter.next() == 1) + expect(9) + check(!iter.hasNext()) + expect(12) + } + expect(3) + sender.join() + receiver.join() + check(q.isClosedForReceive) + finish(13) + } + + @Test + fun testSimple() = runTest { + val q = Channel(1) + check(q.isEmpty) + expect(1) + val sender = launch { + expect(4) + q.send(1) // success -- buffered + check(!q.isEmpty) + expect(5) + q.send(2) // suspends (buffer full) + expect(9) + } + expect(2) + val receiver = launch { + expect(6) + check(q.receive() == 1) // does not suspend -- took from buffer + check(!q.isEmpty) // waiting sender's element moved to buffer + expect(7) + check(q.receive() == 2) // does not suspend (takes from sender) + expect(8) + } + expect(3) + sender.join() + receiver.join() + check(q.isEmpty) + (q as BufferedChannel<*>).checkSegmentStructureInvariants() + finish(10) + } + + @Test + fun testClosedBufferedReceiveCatching() = runTest { + val q = Channel(1) + check(q.isEmpty && !q.isClosedForSend && !q.isClosedForReceive) + expect(1) + launch { + expect(5) + check(!q.isEmpty && q.isClosedForSend && !q.isClosedForReceive) + assertEquals(42, q.receiveCatching().getOrNull()) + expect(6) + check(!q.isEmpty && q.isClosedForSend && q.isClosedForReceive) + assertNull(q.receiveCatching().getOrNull()) + expect(7) + } + expect(2) + q.send(42) // buffers + expect(3) + q.close() // goes on + expect(4) + check(!q.isEmpty && q.isClosedForSend && !q.isClosedForReceive) + yield() + check(!q.isEmpty && q.isClosedForSend && q.isClosedForReceive) + (q as BufferedChannel<*>).checkSegmentStructureInvariants() + finish(8) + } + + @Test + fun testClosedExceptions() = runTest { + val q = Channel(1) + expect(1) + launch { + expect(4) + try { q.receive() } + catch (e: ClosedReceiveChannelException) { + expect(5) + } + } + expect(2) + + require(q.close()) + expect(3) + yield() + expect(6) + try { q.send(42) } + catch (e: ClosedSendChannelException) { + (q as BufferedChannel<*>).checkSegmentStructureInvariants() + finish(7) + } + } + + @Test + fun testTryOp() = runTest { + val q = Channel(1) + assertTrue(q.trySend(1).isSuccess) + expect(1) + launch { + expect(3) + assertEquals(1, q.tryReceive().getOrNull()) + expect(4) + assertNull(q.tryReceive().getOrNull()) + expect(5) + assertEquals(2, q.receive()) // suspends + expect(9) + assertEquals(3, q.tryReceive().getOrNull()) + expect(10) + assertNull(q.tryReceive().getOrNull()) + expect(11) + } + expect(2) + yield() + expect(6) + assertTrue(q.trySend(2).isSuccess) + expect(7) + assertTrue(q.trySend(3).isSuccess) + expect(8) + assertFalse(q.trySend(4).isSuccess) + yield() + (q as BufferedChannel<*>).checkSegmentStructureInvariants() + finish(12) + } + + @Test + fun testConsumeAll() = runTest { + val q = Channel(5) + for (i in 1..10) { + if (i <= 5) { + expect(i) + q.send(i) // shall buffer + } else { + launch(start = CoroutineStart.UNDISPATCHED) { + expect(i) + q.send(i) // suspends + expectUnreached() // will get cancelled by cancel + } + } + } + expect(11) + q.cancel() + check(q.isClosedForSend) + check(q.isClosedForReceive) + assertFailsWith { q.receiveCatching().getOrThrow() } + (q as BufferedChannel<*>).checkSegmentStructureInvariants() + finish(12) + } + + @Test + fun testCancelWithCause() = runTest({ it is TestCancellationException }) { + val channel = Channel(5) + channel.cancel(TestCancellationException()) + channel.receive() + } + + @Test + fun testBufferSize() = runTest { + val capacity = 42 + val channel = Channel(capacity) + checkBufferChannel(channel, capacity) + } + + @Test + fun testBufferSizeFromTheMiddle() = runTest { + val capacity = 42 + val channel = Channel(capacity) + repeat(4) { + channel.trySend(-1) + } + repeat(4) { + channel.receiveCatching().getOrNull() + } + checkBufferChannel(channel, capacity) + } + + @Test + fun testBufferIsNotPreallocated() { + (0..100_000).map { Channel(Int.MAX_VALUE / 2) } + } + + private suspend fun CoroutineScope.checkBufferChannel( + channel: Channel, + capacity: Int + ) { + launch { + expect(2) + repeat(42) { + channel.send(it) + } + expect(3) + channel.send(42) + expect(5) + channel.close() + } + + expect(1) + yield() + + expect(4) + val result = ArrayList(42) + channel.consumeEach { + result.add(it) + } + assertEquals((0..capacity).toList(), result) + (channel as BufferedChannel<*>).checkSegmentStructureInvariants() + finish(6) + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelBufferOverflowTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelBufferOverflowTest.kt new file mode 100644 index 0000000000..a6ae885a66 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ChannelBufferOverflowTest.kt @@ -0,0 +1,37 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class ChannelBufferOverflowTest : TestBase() { + @Test + fun testDropLatest() = runTest { + val c = Channel(2, BufferOverflow.DROP_LATEST) + assertTrue(c.trySend(1).isSuccess) + assertTrue(c.trySend(2).isSuccess) + assertTrue(c.trySend(3).isSuccess) // overflows, dropped + c.send(4) // overflows dropped + assertEquals(1, c.receive()) + assertTrue(c.trySend(5).isSuccess) + assertTrue(c.trySend(6).isSuccess) // overflows, dropped + assertEquals(2, c.receive()) + assertEquals(5, c.receive()) + assertEquals(null, c.tryReceive().getOrNull()) + } + + @Test + fun testDropOldest() = runTest { + val c = Channel(2, BufferOverflow.DROP_OLDEST) + assertTrue(c.trySend(1).isSuccess) + assertTrue(c.trySend(2).isSuccess) + assertTrue(c.trySend(3).isSuccess) // overflows, keeps 2, 3 + c.send(4) // overflows, keeps 3, 4 + assertEquals(3, c.receive()) + assertTrue(c.trySend(5).isSuccess) + assertTrue(c.trySend(6).isSuccess) // overflows, keeps 5, 6 + assertEquals(5, c.receive()) + assertEquals(6, c.receive()) + assertEquals(null, c.tryReceive().getOrNull()) + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelFactoryTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelFactoryTest.kt new file mode 100644 index 0000000000..9260c2d48a --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ChannelFactoryTest.kt @@ -0,0 +1,45 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + + +class ChannelFactoryTest : TestBase() { + @Test + fun testRendezvousChannel() { + assertIs>(Channel()) + assertIs>(Channel(0)) + } + + @Test + fun testUnlimitedChannel() { + assertIs>(Channel(Channel.UNLIMITED)) + assertIs>(Channel(Channel.UNLIMITED, BufferOverflow.DROP_OLDEST)) + assertIs>(Channel(Channel.UNLIMITED, BufferOverflow.DROP_LATEST)) + } + + @Test + fun testConflatedChannel() { + assertIs>(Channel(Channel.CONFLATED)) + assertIs>(Channel(1, BufferOverflow.DROP_OLDEST)) + } + + @Test + fun testBufferedChannel() { + assertIs>(Channel(1)) + assertIs>(Channel(1, BufferOverflow.DROP_LATEST)) + assertIs>(Channel(10)) + } + + @Test + fun testInvalidCapacityNotSupported() { + assertFailsWith { Channel(-3) } + } + + @Test + fun testUnsupportedBufferOverflow() { + assertFailsWith { Channel(Channel.CONFLATED, BufferOverflow.DROP_OLDEST) } + assertFailsWith { Channel(Channel.CONFLATED, BufferOverflow.DROP_LATEST) } + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelReceiveCatchingTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelReceiveCatchingTest.kt new file mode 100644 index 0000000000..bb06a808ff --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ChannelReceiveCatchingTest.kt @@ -0,0 +1,143 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class ChannelReceiveCatchingTest : TestBase() { + @Test + fun testChannelOfThrowables() = runTest { + val channel = Channel() + launch { + channel.send(TestException1()) + channel.close(TestException2()) + } + + val element = channel.receiveCatching() + assertIs(element.getOrThrow()) + assertIs(element.getOrNull()) + + val closed = channel.receiveCatching() + assertTrue(closed.isClosed) + assertTrue(closed.isFailure) + assertIs(closed.exceptionOrNull()) + } + + @Test + @Suppress("ReplaceAssertBooleanWithAssertEquality") // inline classes test + fun testNullableIntChanel() = runTest { + val channel = Channel() + launch { + expect(2) + channel.send(1) + expect(3) + channel.send(null) + + expect(6) + channel.close() + } + + expect(1) + val element = channel.receiveCatching() + assertEquals(1, element.getOrThrow()) + assertEquals(1, element.getOrNull()) + assertEquals("Value(1)", element.toString()) + assertTrue(ChannelResult.success(1) == element) // Don't box + assertFalse(element.isFailure) + assertFalse(element.isClosed) + + expect(4) + val nullElement = channel.receiveCatching() + assertNull(nullElement.getOrThrow()) + assertNull(nullElement.getOrNull()) + assertEquals("Value(null)", nullElement.toString()) + assertTrue(ChannelResult.success(null) == nullElement) // Don't box + assertFalse(element.isFailure) + assertFalse(element.isClosed) + + expect(5) + val closed = channel.receiveCatching() + assertTrue(closed.isClosed) + assertTrue(closed.isFailure) + + val closed2 = channel.receiveCatching() + assertTrue(closed2.isClosed) + assertTrue(closed.isFailure) + assertNull(closed2.exceptionOrNull()) + finish(7) + } + + @Test + @ExperimentalUnsignedTypes + fun testUIntChannel() = runTest { + val channel = Channel() + launch { + expect(2) + channel.send(1u) + yield() + expect(4) + channel.send((Long.MAX_VALUE - 1).toUInt()) + expect(5) + } + + expect(1) + val element = channel.receiveCatching() + assertEquals(1u, element.getOrThrow()) + + expect(3) + val element2 = channel.receiveCatching() + assertEquals((Long.MAX_VALUE - 1).toUInt(), element2.getOrThrow()) + finish(6) + } + + @Test + fun testCancelChannel() = runTest { + val channel = Channel() + launch { + expect(2) + channel.cancel() + } + + expect(1) + val closed = channel.receiveCatching() + assertTrue(closed.isClosed) + assertTrue(closed.isFailure) + finish(3) + } + + @Test + @ExperimentalUnsignedTypes + fun testReceiveResultChannel() = runTest { + val channel = Channel>() + launch { + channel.send(ChannelResult.success(1u)) + channel.send(ChannelResult.closed(TestException1())) + channel.close(TestException2()) + } + + val intResult = channel.receiveCatching() + assertEquals(1u, intResult.getOrThrow().getOrThrow()) + assertFalse(intResult.isFailure) + assertFalse(intResult.isClosed) + + val closeCauseResult = channel.receiveCatching() + assertIs(closeCauseResult.getOrThrow().exceptionOrNull()) + + val closeCause = channel.receiveCatching() + assertTrue(closeCause.isClosed) + assertTrue(closeCause.isFailure) + assertIs(closeCause.exceptionOrNull()) + } + + @Test + fun testToString() = runTest { + val channel = Channel(1) + channel.send("message") + channel.close(TestException1("OK")) + assertEquals("Value(message)", channel.receiveCatching().toString()) + // toString implementation for exception differs on every platform + val str = channel.receiveCatching().toString() + if (!str.matches("Closed\\(.*TestException1: OK\\)".toRegex())) + error("Unexpected string: '$str'") + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementFailureTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementFailureTest.kt new file mode 100644 index 0000000000..c45ff29613 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementFailureTest.kt @@ -0,0 +1,173 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlin.test.* + +/** + * Tests for failures inside `onUndeliveredElement` handler in [Channel]. + */ +class ChannelUndeliveredElementFailureTest : TestBase() { + private val item = "LOST" + private val onCancelFail: (String) -> Unit = { throw TestException(it) } + private val shouldBeUnhandled: List<(Throwable) -> Boolean> = listOf({ it.isElementCancelException() }) + + private fun Throwable.isElementCancelException() = + this is UndeliveredElementException && cause is TestException && cause!!.message == item + + @Test + fun testSendCancelledFail() = runTest(unhandled = shouldBeUnhandled) { + val channel = Channel(onUndeliveredElement = onCancelFail) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + channel.send(item) + expectUnreached() + } + job.cancel() + } + + @Test + fun testSendSelectCancelledFail() = runTest(unhandled = shouldBeUnhandled) { + val channel = Channel(onUndeliveredElement = onCancelFail) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + select { + channel.onSend(item) { + expectUnreached() + } + } + } + job.cancel() + } + + @Test + fun testReceiveCancelledFail() = runTest(unhandled = shouldBeUnhandled) { + val channel = Channel(onUndeliveredElement = onCancelFail) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + channel.receive() + expectUnreached() // will be cancelled before it dispatches + } + channel.send(item) + job.cancel() + } + + @Test + fun testReceiveSelectCancelledFail() = runTest(unhandled = shouldBeUnhandled) { + val channel = Channel(onUndeliveredElement = onCancelFail) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + select { + channel.onReceive { + expectUnreached() + } + } + expectUnreached() // will be cancelled before it dispatches + } + channel.send(item) + job.cancel() + } + + @Test + fun testReceiveCatchingCancelledFail() = runTest(unhandled = shouldBeUnhandled) { + val channel = Channel(onUndeliveredElement = onCancelFail) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + channel.receiveCatching() + expectUnreached() // will be cancelled before it dispatches + } + channel.send(item) + job.cancel() + } + + @Test + fun testReceiveOrClosedSelectCancelledFail() = runTest(unhandled = shouldBeUnhandled) { + val channel = Channel(onUndeliveredElement = onCancelFail) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + select { + channel.onReceiveCatching { + expectUnreached() + } + } + expectUnreached() // will be cancelled before it dispatches + } + channel.send(item) + job.cancel() + } + + @Test + fun testHasNextCancelledFail() = runTest(unhandled = shouldBeUnhandled) { + val channel = Channel(onUndeliveredElement = onCancelFail) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + channel.iterator().hasNext() + expectUnreached() // will be cancelled before it dispatches + } + channel.send(item) + job.cancel() + } + + @Test + fun testChannelCancelledFail() = runTest(expected = { it.isElementCancelException() }) { + val channel = Channel(1, onUndeliveredElement = onCancelFail) + channel.send(item) + channel.cancel() + expectUnreached() + } + + @Test + fun testFailedHandlerInClosedConflatedChannel() = runTest(expected = { it is UndeliveredElementException }) { + val conflated = Channel(Channel.CONFLATED, onUndeliveredElement = { + finish(2) + throw TestException() + }) + expect(1) + conflated.close(IndexOutOfBoundsException()) + conflated.send(3) + } + + @Test + fun testFailedHandlerInClosedBufferedChannel() = runTest(expected = { it is UndeliveredElementException }) { + val conflated = Channel(3, onUndeliveredElement = { + finish(2) + throw TestException() + }) + expect(1) + conflated.close(IndexOutOfBoundsException()) + conflated.send(3) + } + + @Test + fun testSendDropOldestInvokeHandlerBuffered() = runTest(expected = { it is UndeliveredElementException }) { + val channel = Channel(1, BufferOverflow.DROP_OLDEST, onUndeliveredElement = { + finish(2) + throw TestException() + }) + + channel.send(42) + expect(1) + channel.send(12) + } + + @Test + fun testSendDropLatestInvokeHandlerBuffered() = runTest(expected = { it is UndeliveredElementException }) { + val channel = Channel(2, BufferOverflow.DROP_LATEST, onUndeliveredElement = { + finish(2) + throw TestException() + }) + + channel.send(42) + channel.send(12) + expect(1) + channel.send(12) + expectUnreached() + } + + @Test + fun testSendDropOldestInvokeHandlerConflated() = runTest(expected = { it is UndeliveredElementException }) { + val channel = Channel(Channel.CONFLATED, onUndeliveredElement = { + finish(2) + throw TestException() + }) + channel.send(42) + expect(1) + channel.send(42) + expectUnreached() + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementTest.kt new file mode 100644 index 0000000000..4be7c310d4 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ChannelUndeliveredElementTest.kt @@ -0,0 +1,197 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlin.test.* + +class ChannelUndeliveredElementTest : TestBase() { + @Test + fun testSendSuccessfully() = runTest { + runAllKindsTest { kind -> + val channel = kind.create { it.cancel() } + val res = Resource("OK") + launch { + channel.send(res) + } + val ok = channel.receive() + assertEquals("OK", ok.value) + assertFalse(res.isCancelled) // was not cancelled + channel.close() + assertFalse(res.isCancelled) // still was not cancelled + } + } + + @Test + fun testRendezvousSendCancelled() = runTest { + val channel = Channel { it.cancel() } + val res = Resource("OK") + val sender = launch(start = CoroutineStart.UNDISPATCHED) { + assertFailsWith { + channel.send(res) // suspends & get cancelled + } + } + sender.cancelAndJoin() + assertTrue(res.isCancelled) + } + + @Test + fun testBufferedSendCancelled() = runTest { + val channel = Channel(1) { it.cancel() } + val resA = Resource("A") + val resB = Resource("B") + val sender = launch(start = CoroutineStart.UNDISPATCHED) { + channel.send(resA) // goes to buffer + assertFailsWith { + channel.send(resB) // suspends & get cancelled + } + } + sender.cancelAndJoin() + assertFalse(resA.isCancelled) // it is in buffer, not cancelled + assertTrue(resB.isCancelled) // send was cancelled + channel.cancel() // now cancel the channel + assertTrue(resA.isCancelled) // now cancelled in buffer + } + + @Test + fun testUnlimitedChannelCancelled() = runTest { + val channel = Channel(Channel.UNLIMITED) { it.cancel() } + val resA = Resource("A") + val resB = Resource("B") + channel.send(resA) // goes to buffer + channel.send(resB) // goes to buffer + assertFalse(resA.isCancelled) // it is in buffer, not cancelled + assertFalse(resB.isCancelled) // it is in buffer, not cancelled + channel.cancel() // now cancel the channel + assertTrue(resA.isCancelled) // now cancelled in buffer + assertTrue(resB.isCancelled) // now cancelled in buffer + } + + @Test + fun testConflatedResourceCancelled() = runTest { + val channel = Channel(Channel.CONFLATED) { it.cancel() } + val resA = Resource("A") + val resB = Resource("B") + channel.send(resA) + assertFalse(resA.isCancelled) + assertFalse(resB.isCancelled) + channel.send(resB) + assertTrue(resA.isCancelled) // it was conflated (lost) and thus cancelled + assertFalse(resB.isCancelled) + channel.close() + assertFalse(resB.isCancelled) // not cancelled yet, can be still read by receiver + channel.cancel() + assertTrue(resB.isCancelled) // now it is cancelled + } + + @Test + fun testSendToClosedChannel() = runTest { + runAllKindsTest { kind -> + val channel = kind.create { it.cancel() } + channel.close() // immediately close channel + val res = Resource("OK") + assertFailsWith { + channel.send(res) // send fails to closed channel, resource was not delivered + } + assertTrue(res.isCancelled) + } + } + + private suspend fun runAllKindsTest(test: suspend CoroutineScope.(TestChannelKind) -> Unit) { + for (kind in TestChannelKind.values()) { + if (kind.viaBroadcast) continue // does not support onUndeliveredElement + try { + withContext(Job()) { + test(kind) + } + } catch(e: Throwable) { + error("$kind: $e", e) + } + } + } + + private class Resource(val value: String) { + private val _cancelled = atomic(false) + + val isCancelled: Boolean + get() = _cancelled.value + + fun cancel() { + check(!_cancelled.getAndSet(true)) { "Already cancelled" } + } + } + + @Test + fun testHandlerIsNotInvoked() = runTest { // #2826 + val channel = Channel { + expectUnreached() + } + + expect(1) + launch { + expect(2) + channel.receive() + } + channel.send(Unit) + finish(3) + } + + @Test + fun testChannelBufferOverflow() = runTest { + testBufferOverflowStrategy(listOf(1, 2), BufferOverflow.DROP_OLDEST) + testBufferOverflowStrategy(listOf(3), BufferOverflow.DROP_LATEST) + } + + private suspend fun testBufferOverflowStrategy(expectedDroppedElements: List, strategy: BufferOverflow) { + val list = ArrayList() + val channel = Channel( + capacity = 2, + onBufferOverflow = strategy, + onUndeliveredElement = { value -> list.add(value) } + ) + + channel.send(1) + channel.send(2) + + channel.send(3) + channel.trySend(4).onFailure { expectUnreached() } + assertEquals(expectedDroppedElements, list) + } + + + @Test + fun testTrySendDoesNotInvokeHandlerOnClosedConflatedChannel() = runTest { + val conflated = Channel(Channel.CONFLATED, onUndeliveredElement = { + expectUnreached() + }) + conflated.close(IndexOutOfBoundsException()) + conflated.trySend(3) + } + + @Test + fun testTrySendDoesNotInvokeHandlerOnClosedChannel() = runTest { + val conflated = Channel(3, onUndeliveredElement = { + expectUnreached() + }) + conflated.close(IndexOutOfBoundsException()) + repeat(10) { + conflated.trySend(3) + } + } + + @Test + fun testTrySendDoesNotInvokeHandler() { + for (capacity in 0..2) { + testTrySendDoesNotInvokeHandler(capacity) + } + } + + private fun testTrySendDoesNotInvokeHandler(capacity: Int) { + val channel = Channel(capacity, BufferOverflow.DROP_LATEST, onUndeliveredElement = { + expectUnreached() + }) + repeat(10) { + channel.trySend(3) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ChannelsTest.kt b/kotlinx-coroutines-core/common/test/channels/ChannelsTest.kt new file mode 100644 index 0000000000..235609c804 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ChannelsTest.kt @@ -0,0 +1,112 @@ +@file:Suppress("DEPRECATION") + +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.math.* +import kotlin.test.* + +class ChannelsTest: TestBase() { + private val testList = listOf(1, 2, 3) + + @Test + fun testIterableAsReceiveChannel() = runTest { + assertEquals(testList, testList.asReceiveChannel().toList()) + } + + @Test + fun testCloseWithMultipleSuspendedReceivers() = runTest { + // Once the channel is closed, the waiting + // requests should be cancelled in the order + // they were suspended in the channel. + val channel = Channel() + launch { + try { + expect(2) + channel.receive() + expectUnreached() + } catch (e: ClosedReceiveChannelException) { + expect(5) + } + } + + launch { + try { + expect(3) + channel.receive() + expectUnreached() + } catch (e: ClosedReceiveChannelException) { + expect(6) + } + } + + expect(1) + yield() + expect(4) + channel.close() + yield() + finish(7) + } + + @Test + fun testCloseWithMultipleSuspendedSenders() = runTest { + // Once the channel is closed, the waiting + // requests should be cancelled in the order + // they were suspended in the channel. + val channel = Channel() + launch { + try { + expect(2) + channel.send(42) + expectUnreached() + } catch (e: CancellationException) { + expect(5) + } + } + + launch { + try { + expect(3) + channel.send(42) + expectUnreached() + } catch (e: CancellationException) { + expect(6) + } + } + + expect(1) + yield() + expect(4) + channel.cancel() + yield() + finish(7) + } + + @Test + fun testEmptyList() = runTest { + assertTrue(emptyList().asReceiveChannel().toList().isEmpty()) + } + + @Test + fun testToList() = runTest { + assertEquals(testList, testList.asReceiveChannel().toList()) + + } + + @Test + fun testToListOnFailedChannel() = runTest { + val channel = Channel() + channel.close(TestException()) + assertFailsWith { + channel.toList() + } + } + + private fun Iterable.asReceiveChannel(context: CoroutineContext = Dispatchers.Unconfined): ReceiveChannel = + GlobalScope.produce(context) { + for (element in this@asReceiveChannel) + send(element) + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ConflatedBroadcastChannelTest.kt b/kotlinx-coroutines-core/common/test/channels/ConflatedBroadcastChannelTest.kt new file mode 100644 index 0000000000..9c534e0055 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ConflatedBroadcastChannelTest.kt @@ -0,0 +1,126 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +@Suppress("DEPRECATION_ERROR") +class ConflatedBroadcastChannelTest : TestBase() { + + @Test + fun testConcurrentModification() = runTest { + val channel = ConflatedBroadcastChannel() + val s1 = channel.openSubscription() + val s2 = channel.openSubscription() + + val job1 = launch(Dispatchers.Unconfined, CoroutineStart.UNDISPATCHED) { + expect(1) + s1.receive() + s1.cancel() + } + + val job2 = launch(Dispatchers.Unconfined, CoroutineStart.UNDISPATCHED) { + expect(2) + s2.receive() + } + + expect(3) + channel.send(1) + joinAll(job1, job2) + finish(4) + } + + @Test + fun testBasicScenario() = runTest { + expect(1) + val broadcast = ConflatedBroadcastChannel() + assertIs(exceptionFrom { broadcast.value }) + assertNull(broadcast.valueOrNull) + + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + val sub = broadcast.openSubscription() + assertNull(sub.tryReceive().getOrNull()) + expect(3) + assertEquals("one", sub.receive()) // suspends + expect(6) + assertEquals("two", sub.receive()) // suspends + expect(12) + sub.cancel() + expect(13) + } + + expect(4) + broadcast.send("one") // does not suspend + assertEquals("one", broadcast.value) + assertEquals("one", broadcast.valueOrNull) + expect(5) + yield() // to receiver + expect(7) + launch(start = CoroutineStart.UNDISPATCHED) { + expect(8) + val sub = broadcast.openSubscription() + assertEquals("one", sub.receive()) // does not suspend + expect(9) + assertEquals("two", sub.receive()) // suspends + expect(14) + assertEquals("three", sub.receive()) // suspends + expect(17) + assertNull(sub.receiveCatching().getOrNull()) // suspends until closed + expect(20) + sub.cancel() + expect(21) + } + + expect(10) + broadcast.send("two") // does not suspend + assertEquals("two", broadcast.value) + assertEquals("two", broadcast.valueOrNull) + expect(11) + yield() // to both receivers + expect(15) + broadcast.send("three") // does not suspend + assertEquals("three", broadcast.value) + assertEquals("three", broadcast.valueOrNull) + expect(16) + yield() // to second receiver + expect(18) + broadcast.close() + assertIs(exceptionFrom { broadcast.value }) + assertNull(broadcast.valueOrNull) + expect(19) + yield() // to second receiver + assertIs(exceptionFrom { broadcast.send("four") }) + finish(22) + } + + @Test + fun testInitialValueAndReceiveClosed() = runTest { + expect(1) + val broadcast = ConflatedBroadcastChannel(1) + assertEquals(1, broadcast.value) + assertEquals(1, broadcast.valueOrNull) + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + val sub = broadcast.openSubscription() + assertEquals(1, sub.receive()) + expect(3) + assertIs(exceptionFrom { sub.receive() }) // suspends + expect(6) + } + expect(4) + broadcast.close() + expect(5) + yield() // to child + finish(7) + } + + private inline fun exceptionFrom(block: () -> Unit): Throwable? { + return try { + block() + null + } catch (e: Throwable) { + e + } + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ConflatedChannelTest.kt b/kotlinx-coroutines-core/common/test/channels/ConflatedChannelTest.kt new file mode 100644 index 0000000000..e3e52dfc6f --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ConflatedChannelTest.kt @@ -0,0 +1,92 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +open class ConflatedChannelTest : TestBase() { + protected open fun createConflatedChannel() = + Channel(Channel.CONFLATED) + + @Test + fun testBasicConflationOfferTryReceive() { + val q = createConflatedChannel() + assertNull(q.tryReceive().getOrNull()) + assertTrue(q.trySend(1).isSuccess) + assertTrue(q.trySend(2).isSuccess) + assertTrue(q.trySend(3).isSuccess) + assertEquals(3, q.tryReceive().getOrNull()) + assertNull(q.tryReceive().getOrNull()) + } + + @Test + fun testConflatedSend() = runTest { + val q = createConflatedChannel() + q.send(1) + q.send(2) // shall conflated previously sent + assertEquals(2, q.receiveCatching().getOrNull()) + } + + @Test + fun testConflatedClose() = runTest { + val q = createConflatedChannel() + q.send(1) + q.close() // shall become closed but do not conflate last sent item yet + assertTrue(q.isClosedForSend) + assertFalse(q.isClosedForReceive) + assertEquals(1, q.receive()) + // not it is closed for receive, too + assertTrue(q.isClosedForSend) + assertTrue(q.isClosedForReceive) + assertNull(q.receiveCatching().getOrNull()) + } + + @Test + fun testConflationSendReceive() = runTest { + val q = createConflatedChannel() + expect(1) + launch { // receiver coroutine + expect(4) + assertEquals(2, q.receive()) + expect(5) + assertEquals(3, q.receive()) // this receive suspends + expect(8) + assertEquals(6, q.receive()) // last conflated value + expect(9) + } + expect(2) + q.send(1) + q.send(2) // shall conflate + expect(3) + yield() // to receiver + expect(6) + q.send(3) // send to the waiting receiver + q.send(4) // buffer + q.send(5) // conflate + q.send(6) // conflate again + expect(7) + yield() // to receiver + finish(10) + } + + @Test + fun testConsumeAll() = runTest { + val q = createConflatedChannel() + expect(1) + for (i in 1..10) { + q.send(i) // stores as last + } + q.cancel() + check(q.isClosedForSend) + check(q.isClosedForReceive) + assertFailsWith { q.receiveCatching().getOrThrow() } + finish(2) + } + + @Test + fun testCancelWithCause() = runTest({ it is TestCancellationException }) { + val channel = createConflatedChannel() + channel.cancel(TestCancellationException()) + channel.receive() + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ConsumeTest.kt b/kotlinx-coroutines-core/common/test/channels/ConsumeTest.kt new file mode 100644 index 0000000000..57d9b5b721 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ConsumeTest.kt @@ -0,0 +1,146 @@ +@file:OptIn(DelicateCoroutinesApi::class) +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class ConsumeTest: TestBase() { + + /** Check that [ReceiveChannel.consume] does not suffer from KT-58685 */ + @Test + fun testConsumeJsMiscompilation() = runTest { + val channel = Channel() + assertFailsWith { + try { + channel.consume { null } ?: throw IndexOutOfBoundsException() // should throw… + } catch (e: Exception) { + throw e // …but instead fails here + } + } + } + + /** Checks that [ReceiveChannel.consume] closes the channel when the block executes successfully. */ + @Test + fun testConsumeClosesOnSuccess() = runTest { + val channel = Channel() + channel.consume { } + assertTrue(channel.isClosedForReceive) + } + + /** Checks that [ReceiveChannel.consume] closes the channel when the block executes successfully. */ + @Test + fun testConsumeClosesOnFailure() = runTest { + val channel = Channel() + try { + channel.consume { throw TestException() } + } catch (e: TestException) { + // Expected + } + assertTrue(channel.isClosedForReceive) + } + + /** Checks that [ReceiveChannel.consume] closes the channel when the block does an early return. */ + @Test + fun testConsumeClosesOnEarlyReturn() = runTest { + val channel = Channel() + fun f() { + try { + channel.consume { return } + } catch (e: TestException) { + // Expected + } + } + f() + assertTrue(channel.isClosedForReceive) + } + + /** Checks that [ReceiveChannel.consume] closes the channel when the block executes successfully. */ + @Test + fun testConsumeEachClosesOnSuccess() = runTest { + val channel = Channel(Channel.UNLIMITED) + launch { channel.close() } + channel.consumeEach { fail("unreached") } + assertTrue(channel.isClosedForReceive) + } + + /** Checks that [ReceiveChannel.consume] closes the channel when the block executes successfully. */ + @Test + fun testConsumeEachClosesOnFailure() = runTest { + val channel = Channel(Channel.UNLIMITED) + channel.send(Unit) + try { + channel.consumeEach { throw TestException() } + } catch (e: TestException) { + // Expected + } + assertTrue(channel.isClosedForReceive) + } + + /** Checks that [ReceiveChannel.consume] closes the channel when the block does an early return. */ + @Test + fun testConsumeEachClosesOnEarlyReturn() = runTest { + val channel = Channel(Channel.UNLIMITED) + channel.send(Unit) + suspend fun f() { + channel.consumeEach { + return@f + } + } + f() + assertTrue(channel.isClosedForReceive) + } + + /** Checks that [ReceiveChannel.consumeEach] reacts to cancellation, but processes the elements that are + * readily available in the buffer. */ + @Test + fun testConsumeEachExitsOnCancellation() = runTest { + val undeliveredElements = mutableListOf() + val channel = Channel(2, onUndeliveredElement = { + undeliveredElements.add(it) + }) + launch { + // These two elements will be sent and put into the buffer: + channel.send(0) + channel.send(1) + // This element will not fit into the buffer, so `send` suspends: + channel.send(2) + // At this point, the consumer's `launch` is cancelled. + yield() // Allow the cancellation handler of the consumer to run. + // Try to send a new element, which will fail at this point: + channel.send(3) + fail("unreached") + } + launch { + channel.consumeEach { + cancel() + assertTrue(it in 0..2) + } + }.join() + assertTrue(channel.isClosedForReceive) + assertEquals(listOf(3), undeliveredElements) + } + + @Test + fun testConsumeEachThrowingOnChannelClosing() = runTest { + val channel = Channel() + channel.close(TestException()) + assertFailsWith { + channel.consumeEach { fail("unreached") } + } + } + + /** Check that [BroadcastChannel.consume] does not suffer from KT-58685 */ + @Suppress("DEPRECATION", "DEPRECATION_ERROR") + @Test + fun testBroadcastChannelConsumeJsMiscompilation() = runTest { + val channel = BroadcastChannel(1) + assertFailsWith { + try { + channel.consume { null } ?: throw IndexOutOfBoundsException() // should throw… + } catch (e: Exception) { + throw e // …but instead fails here + } + } + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ProduceConsumeTest.kt b/kotlinx-coroutines-core/common/test/channels/ProduceConsumeTest.kt new file mode 100644 index 0000000000..1ae5c8cd69 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ProduceConsumeTest.kt @@ -0,0 +1,56 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.test.* + +class ProduceConsumeTest : TestBase() { + + @Test + fun testRendezvous() = runTest { + testProducer(1) + } + + @Test + fun testSmallBuffer() = runTest { + testProducer(1) + } + + @Test + fun testMediumBuffer() = runTest { + testProducer(10) + } + + @Test + fun testLargeMediumBuffer() = runTest { + testProducer(1000) + } + + @Test + fun testUnlimited() = runTest { + testProducer(Channel.UNLIMITED) + } + + private suspend fun testProducer(producerCapacity: Int) { + testProducer(1, producerCapacity) + testProducer(10, producerCapacity) + testProducer(100, producerCapacity) + } + + private suspend fun testProducer(messages: Int, producerCapacity: Int) { + var sentAll = false + val producer = GlobalScope.produce(coroutineContext, capacity = producerCapacity) { + for (i in 1..messages) { + send(i) + } + sentAll = true + } + var consumed = 0 + for (x in producer) { + consumed++ + } + assertTrue(sentAll) + assertEquals(messages, consumed) + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/ProduceTest.kt b/kotlinx-coroutines-core/common/test/channels/ProduceTest.kt new file mode 100644 index 0000000000..c8e9667fb2 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/ProduceTest.kt @@ -0,0 +1,293 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.coroutines.* +import kotlin.test.* + +class ProduceTest : TestBase() { + @Test + fun testBasic() = runTest { + val c = produce { + expect(2) + send(1) + expect(3) + send(2) + expect(6) + } + expect(1) + check(c.receive() == 1) + expect(4) + check(c.receive() == 2) + expect(5) + assertNull(c.receiveCatching().getOrNull()) + finish(7) + } + + @Test + fun testCancelWithoutCause() = runTest { + val c = produce(NonCancellable) { + expect(2) + send(1) + expect(3) + try { + send(2) // will get cancelled + expectUnreached() + } catch (e: Throwable) { + expect(7) + check(e is CancellationException) + throw e + } + expectUnreached() + } + expect(1) + check(c.receive() == 1) + expect(4) + c.cancel() + expect(5) + assertFailsWith { c.receiveCatching().getOrThrow() } + expect(6) + yield() // to produce + finish(8) + } + + @Test + fun testCancelWithCause() = runTest { + val c = produce(NonCancellable) { + expect(2) + send(1) + expect(3) + try { + send(2) // will get cancelled + expectUnreached() + } catch (e: Throwable) { + expect(6) + check(e is TestCancellationException) + throw e + } + expectUnreached() + } + expect(1) + check(c.receive() == 1) + expect(4) + c.cancel(TestCancellationException()) + try { + c.receive() + expectUnreached() + } catch (e: TestCancellationException) { + expect(5) + } + yield() // to produce + finish(7) + } + + @Test + fun testCancelOnCompletionUnconfined() = runTest { + cancelOnCompletion(Dispatchers.Unconfined) + } + + @Test + fun testCancelOnCompletion() = runTest { + cancelOnCompletion(coroutineContext) + } + + @Test + fun testCancelWhenTheChannelIsClosed() = runTest { + val channel = produce { + send(1) + close() + expect(2) + launch { + expect(3) + hang { expect(5) } + } + } + + expect(1) + channel.receive() + yield() + expect(4) + channel.cancel() + (channel as Job).join() + finish(6) + } + + @Test + fun testAwaitCloseOnlyAllowedOnce() = runTest { + expect(1) + val c = produce { + try { + awaitClose() + } catch (e: CancellationException) { + assertFailsWith { + awaitClose() + } + finish(2) + throw e + } + } + yield() // let the `produce` procedure run + c.cancel() + } + + @Test + fun testInvokeOnCloseWithAwaitClose() = runTest { + expect(1) + produce { + invokeOnClose { } + assertFailsWith { + awaitClose() + } + finish(2) + } + } + + @Test + fun testAwaitConsumerCancellation() = runTest { + val parent = Job() + val channel = produce(parent) { + expect(2) + awaitClose { expect(4) } + } + expect(1) + yield() + expect(3) + channel.cancel() + parent.complete() + parent.join() + finish(5) + } + + @Test + fun testAwaitProducerCancellation() = runTest { + val parent = Job() + produce(parent) { + expect(2) + launch { + expect(3) + this@produce.cancel() + } + awaitClose { expect(4) } + } + expect(1) + parent.complete() + parent.join() + finish(5) + } + + @Test + fun testAwaitParentCancellation() = runTest { + val parent = Job() + produce(parent) { + expect(2) + awaitClose { expect(4) } + } + expect(1) + yield() + expect(3) + parent.cancelAndJoin() + finish(5) + } + + @Test + fun testAwaitIllegalState() = runTest { + val channel = produce { } + assertFailsWith { (channel as ProducerScope<*>).awaitClose() } + callbackFlow { + expect(1) + launch { + expect(2) + assertFailsWith { + awaitClose { expectUnreached() } + expectUnreached() + } + } + close() + }.collect() + finish(3) + } + + @Test + fun testUncaughtExceptionsInProduce() = runTest( + unhandled = listOf({ it is TestException }) + ) { + val c = produce { + launch(SupervisorJob()) { + throw TestException() + }.join() + send(3) + } + assertEquals(3, c.receive()) + } + + @Test + fun testCancellingProduceCoroutineButNotChannel() = runTest { + val c = produce(Job(), capacity = Channel.UNLIMITED) { + launch { throw TestException() } + try { + yield() + } finally { + repeat(10) { trySend(it) } + } + } + repeat(10) { assertEquals(it, c.receive()) } + } + + @Test + fun testReceivingValuesAfterFailingTheCoroutine() = runTest { + val produceJob = Job() + val c = produce(produceJob, capacity = Channel.UNLIMITED) { + repeat(5) { send(it) } + throw TestException() + } + produceJob.join() + assertTrue(produceJob.isCancelled) + repeat(5) { assertEquals(it, c.receive()) } + assertFailsWith { c.receive() } + } + + @Test + fun testSilentKillerInProduce() = runTest { + val parentScope = CoroutineScope(SupervisorJob() + Dispatchers.Default) + val channel = parentScope.produce(capacity = Channel.UNLIMITED) { + repeat(5) { + send(it) + } + parentScope.cancel() + // suspending after this point would fail, but sending succeeds + send(-1) + } + launch { + for (c in channel) { + println(c) // 0, 1, 2, 3, 4, -1 + } // throws an exception after reaching -1 + fail("unreached") + } + } + + @Test + fun testProduceWithInvalidCapacity() = runTest { + assertFailsWith { + produce(capacity = -3) { } + } + } + + private suspend fun cancelOnCompletion(coroutineContext: CoroutineContext) = CoroutineScope(coroutineContext).apply { + val source = Channel() + expect(1) + val produced = produce(coroutineContext, onCompletion = { source.cancelConsumed(it) }) { + expect(2) + source.receive() + } + + yield() + expect(3) + produced.cancel() + try { + source.receive() + } catch (e: CancellationException) { + finish(4) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/RendezvousChannelTest.kt b/kotlinx-coroutines-core/common/test/channels/RendezvousChannelTest.kt new file mode 100644 index 0000000000..00aa1925d2 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/RendezvousChannelTest.kt @@ -0,0 +1,296 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class RendezvousChannelTest : TestBase() { + @Test + fun testSimple() = runTest { + val q = Channel(Channel.RENDEZVOUS) + check(q.isEmpty) + expect(1) + val sender = launch { + expect(4) + q.send(1) // suspend -- the first to come to rendezvous + expect(7) + q.send(2) // does not suspend -- receiver is there + expect(8) + } + expect(2) + val receiver = launch { + expect(5) + check(q.receive() == 1) // does not suspend -- sender was there + expect(6) + check(q.receive() == 2) // suspends + expect(9) + } + expect(3) + sender.join() + receiver.join() + check(q.isEmpty) + finish(10) + } + + @Test + fun testClosedReceiveCatching() = runTest { + val q = Channel(Channel.RENDEZVOUS) + check(q.isEmpty && !q.isClosedForSend && !q.isClosedForReceive) + expect(1) + launch { + expect(3) + assertEquals(42, q.receiveCatching().getOrNull()) + expect(4) + assertNull(q.receiveCatching().getOrNull()) + expect(6) + } + expect(2) + q.send(42) + expect(5) + q.close() + check(!q.isEmpty && q.isClosedForSend && q.isClosedForReceive) + yield() + check(!q.isEmpty && q.isClosedForSend && q.isClosedForReceive) + finish(7) + } + + @Test + fun testClosedExceptions() = runTest { + val q = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(4) + try { q.receive() } + catch (e: ClosedReceiveChannelException) { + expect(5) + } + } + expect(2) + q.close() + expect(3) + yield() + expect(6) + try { q.send(42) } + catch (e: ClosedSendChannelException) { + finish(7) + } + } + + @Test + fun testTrySendTryReceive() = runTest { + val q = Channel(Channel.RENDEZVOUS) + assertFalse(q.trySend(1).isSuccess) + expect(1) + launch { + expect(3) + assertNull(q.tryReceive().getOrNull()) + expect(4) + assertEquals(2, q.receive()) + expect(7) + assertNull(q.tryReceive().getOrNull()) + yield() + expect(9) + assertEquals(3, q.tryReceive().getOrNull()) + expect(10) + } + expect(2) + yield() + expect(5) + assertTrue(q.trySend(2).isSuccess) + expect(6) + yield() + expect(8) + q.send(3) + finish(11) + } + + @Test + fun testIteratorClosed() = runTest { + val q = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(3) + q.close() + expect(4) + } + expect(2) + for (x in q) { + expectUnreached() + } + finish(5) + } + + @Test + fun testIteratorOne() = runTest { + val q = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(3) + q.send(1) + expect(4) + q.close() + expect(5) + } + expect(2) + for (x in q) { + expect(6) + assertEquals(1, x) + } + finish(7) + } + + @Test + fun testIteratorOneWithYield() = runTest { + val q = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(3) + q.send(1) // will suspend + expect(6) + q.close() + expect(7) + } + expect(2) + yield() // yield to sender coroutine right before starting for loop + expect(4) + for (x in q) { + expect(5) + assertEquals(1, x) + } + finish(8) + } + + @Test + fun testIteratorTwo() = runTest { + val q = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(3) + q.send(1) + expect(4) + q.send(2) + expect(7) + q.close() + expect(8) + } + expect(2) + for (x in q) { + when (x) { + 1 -> expect(5) + 2 -> expect(6) + else -> expectUnreached() + } + } + finish(9) + } + + @Test + fun testIteratorTwoWithYield() = runTest { + val q = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(3) + q.send(1) // will suspend + expect(6) + q.send(2) + expect(7) + q.close() + expect(8) + } + expect(2) + yield() // yield to sender coroutine right before starting for loop + expect(4) + for (x in q) { + when (x) { + 1 -> expect(5) + 2 -> expect(9) + else -> expectUnreached() + } + } + finish(10) + } + + @Test + fun testSuspendSendOnClosedChannel() = runTest { + val q = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(4) + q.send(42) // suspend + expect(11) + } + expect(2) + launch { + expect(5) + q.close() + expect(6) + } + expect(3) + yield() // to sender + expect(7) + yield() // try to resume sender (it will not resume despite the close!) + expect(8) + assertEquals(42, q.receiveCatching().getOrNull()) + expect(9) + assertNull(q.receiveCatching().getOrNull()) + expect(10) + yield() // to sender, it was resumed! + finish(12) + } + + @Test + fun testProduceBadClass() = runTest { + val bad = BadClass() + val c = produce { + expect(1) + send(bad) + } + assertSame(c.receive(), bad) + finish(2) + } + + @Test + fun testConsumeAll() = runTest { + val q = Channel(Channel.RENDEZVOUS) + for (i in 1..10) { + launch(start = CoroutineStart.UNDISPATCHED) { + expect(i) + q.send(i) // suspends + expectUnreached() // will get cancelled by cancel + } + } + expect(11) + q.cancel() + check(q.isClosedForSend) + check(q.isClosedForReceive) + assertFailsWith { q.receiveCatching().getOrThrow() } + finish(12) + } + + @Test + fun testCancelWithCause() = runTest({ it is TestCancellationException }) { + val channel = Channel(Channel.RENDEZVOUS) + channel.cancel(TestCancellationException()) + channel.receiveCatching().getOrThrow() + } + + /** Tests that [BufferOverflow.DROP_OLDEST] takes precedence over [Channel.RENDEZVOUS]. */ + @Test + fun testDropOldest() = runTest { + val channel = Channel(Channel.RENDEZVOUS, onBufferOverflow = BufferOverflow.DROP_OLDEST) + channel.send(1) + channel.send(2) + channel.send(3) + assertEquals(3, channel.receive()) + } + + /** Tests that [BufferOverflow.DROP_LATEST] takes precedence over [Channel.RENDEZVOUS]. */ + @Test + fun testDropLatest() = runTest { + val channel = Channel(Channel.RENDEZVOUS, onBufferOverflow = BufferOverflow.DROP_LATEST) + channel.send(1) + channel.send(2) + channel.send(3) + assertEquals(1, channel.receive()) + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/SendReceiveStressTest.kt b/kotlinx-coroutines-core/common/test/channels/SendReceiveStressTest.kt new file mode 100644 index 0000000000..5fa22b5494 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/SendReceiveStressTest.kt @@ -0,0 +1,46 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class SendReceiveStressTest : TestBase() { + + // Emulate parametrized by hand :( + + @Test + fun testBufferedChannel() = runTest { + testStress(Channel(2)) + } + + @Test + fun testUnlimitedChannel() = runTest { + testStress(Channel(Channel.UNLIMITED)) + } + + @Test + fun testRendezvousChannel() = runTest { + testStress(Channel(Channel.RENDEZVOUS)) + } + + private suspend fun testStress(channel: Channel) = coroutineScope { + val n = 100 // Do not increase, otherwise node.js will fail with timeout :( + val sender = launch { + for (i in 1..n) { + channel.send(i) + } + expect(2) + } + val receiver = launch { + for (i in 1..n) { + val next = channel.receive() + check(next == i) + } + expect(3) + } + expect(1) + sender.join() + receiver.join() + finish(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/channels/TestBroadcastChannelKind.kt b/kotlinx-coroutines-core/common/test/channels/TestBroadcastChannelKind.kt new file mode 100644 index 0000000000..693f1f18f9 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/TestBroadcastChannelKind.kt @@ -0,0 +1,22 @@ +package kotlinx.coroutines.channels + +@Suppress("DEPRECATION_ERROR") +enum class TestBroadcastChannelKind { + ARRAY_1 { + override fun create(): BroadcastChannel = BroadcastChannel(1) + override fun toString(): String = "BufferedBroadcastChannel(1)" + }, + ARRAY_10 { + override fun create(): BroadcastChannel = BroadcastChannel(10) + override fun toString(): String = "BufferedBroadcastChannel(10)" + }, + CONFLATED { + override fun create(): BroadcastChannel = ConflatedBroadcastChannel() + override fun toString(): String = "ConflatedBroadcastChannel" + override val isConflated: Boolean get() = true + } + ; + + abstract fun create(): BroadcastChannel + open val isConflated: Boolean get() = false +} diff --git a/kotlinx-coroutines-core/common/test/channels/TestChannelKind.kt b/kotlinx-coroutines-core/common/test/channels/TestChannelKind.kt new file mode 100644 index 0000000000..605c746649 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/TestChannelKind.kt @@ -0,0 +1,56 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* + +enum class TestChannelKind( + val capacity: Int, + private val description: String, + val viaBroadcast: Boolean = false +) { + RENDEZVOUS(0, "RendezvousChannel"), + BUFFERED_1(1, "BufferedChannel(1)"), + BUFFERED_2(2, "BufferedChannel(2)"), + BUFFERED_10(10, "BufferedChannel(10)"), + UNLIMITED(Channel.UNLIMITED, "UnlimitedChannel"), + CONFLATED(Channel.CONFLATED, "ConflatedChannel"), + BUFFERED_1_BROADCAST(1, "BufferedBroadcastChannel(1)", viaBroadcast = true), + BUFFERED_10_BROADCAST(10, "BufferedBroadcastChannel(10)", viaBroadcast = true), + CONFLATED_BROADCAST(Channel.CONFLATED, "ConflatedBroadcastChannel", viaBroadcast = true) + ; + + fun create(onUndeliveredElement: ((T) -> Unit)? = null): Channel = when { + viaBroadcast && onUndeliveredElement != null -> error("Broadcast channels to do not support onUndeliveredElement") + viaBroadcast -> @Suppress("DEPRECATION_ERROR") ChannelViaBroadcast(BroadcastChannel(capacity)) + else -> Channel(capacity, onUndeliveredElement = onUndeliveredElement) + } + + val isConflated get() = capacity == Channel.CONFLATED + override fun toString(): String = description +} + +internal class ChannelViaBroadcast( + @Suppress("DEPRECATION_ERROR") + private val broadcast: BroadcastChannel +): Channel, SendChannel by broadcast { + val sub = broadcast.openSubscription() + + override val isClosedForReceive: Boolean get() = sub.isClosedForReceive + override val isEmpty: Boolean get() = sub.isEmpty + + override suspend fun receive(): E = sub.receive() + override suspend fun receiveCatching(): ChannelResult = sub.receiveCatching() + override fun iterator(): ChannelIterator = sub.iterator() + override fun tryReceive(): ChannelResult = sub.tryReceive() + + override fun cancel(cause: CancellationException?) = broadcast.cancel(cause) + + // implementing hidden method anyway, so can cast to an internal class + @Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.2.0, binary compatibility with versions <= 1.1.x") + override fun cancel(cause: Throwable?): Boolean = error("unsupported") + + override val onReceive: SelectClause1 + get() = sub.onReceive + override val onReceiveCatching: SelectClause1> + get() = sub.onReceiveCatching +} diff --git a/kotlinx-coroutines-core/common/test/channels/UnlimitedChannelTest.kt b/kotlinx-coroutines-core/common/test/channels/UnlimitedChannelTest.kt new file mode 100644 index 0000000000..08bcc20336 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/channels/UnlimitedChannelTest.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class UnlimitedChannelTest : TestBase() { + @Test + fun testBasic() = runTest { + val c = Channel(Channel.UNLIMITED) + c.send(1) + assertTrue(c.trySend(2).isSuccess) + c.send(3) + check(c.close()) + check(!c.close()) + assertEquals(1, c.receive()) + assertEquals(2, c.tryReceive().getOrNull()) + assertEquals(3, c.receiveCatching().getOrNull()) + assertNull(c.receiveCatching().getOrNull()) + } + + @Test + fun testConsumeAll() = runTest { + val q = Channel(Channel.UNLIMITED) + for (i in 1..10) { + q.send(i) // buffers + } + q.cancel() + check(q.isClosedForSend) + check(q.isClosedForReceive) + assertFailsWith { q.receive() } + } + + @Test + fun testCancelWithCause() = runTest({ it is TestCancellationException }) { + val channel = Channel(Channel.UNLIMITED) + channel.cancel(TestCancellationException()) + channel.receive() + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/BuildersTest.kt b/kotlinx-coroutines-core/common/test/flow/BuildersTest.kt new file mode 100644 index 0000000000..6f2eda4d9f --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/BuildersTest.kt @@ -0,0 +1,39 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class BuildersTest : TestBase() { + + @Test + fun testSuspendLambdaAsFlow() = runTest { + val lambda = suspend { 42 } + assertEquals(42, lambda.asFlow().single()) + } + + @Test + fun testRangeAsFlow() = runTest { + assertEquals((0..9).toList(), (0..9).asFlow().toList()) + assertEquals(emptyList(), (0..-1).asFlow().toList()) + + assertEquals((0L..9L).toList(), (0L..9L).asFlow().toList()) + assertEquals(emptyList(), (0L..-1L).asFlow().toList()) + } + + @Test + fun testArrayAsFlow() = runTest { + assertEquals((0..9).toList(), IntArray(10) { it }.asFlow().toList()) + assertEquals(emptyList(), intArrayOf().asFlow().toList()) + + assertEquals((0L..9L).toList(), LongArray(10) { it.toLong() }.asFlow().toList()) + assertEquals(emptyList(), longArrayOf().asFlow().toList()) + } + + @Test + fun testSequence() = runTest { + val expected = (0..9).toList() + assertEquals(expected, expected.iterator().asFlow().toList()) + assertEquals(expected, expected.asIterable().asFlow().toList()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/FlowInvariantsTest.kt b/kotlinx-coroutines-core/common/test/flow/FlowInvariantsTest.kt new file mode 100644 index 0000000000..06eedb96ed --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/FlowInvariantsTest.kt @@ -0,0 +1,277 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.testing.flow.* +import kotlin.coroutines.* +import kotlin.reflect.* +import kotlin.test.* + +class FlowInvariantsTest : TestBase() { + + private fun runParametrizedTest( + expectedException: KClass? = null, + testBody: suspend (flowFactory: (suspend FlowCollector.() -> Unit) -> Flow) -> Unit + ) = runTest { + val r1 = runCatching { testBody { flow(it) } }.exceptionOrNull() + check(r1, expectedException) + reset() + + val r2 = runCatching { testBody { abstractFlow(it) } }.exceptionOrNull() + check(r2, expectedException) + } + + private fun abstractFlow(block: suspend FlowCollector.() -> Unit): Flow = object : AbstractFlow() { + override suspend fun collectSafely(collector: FlowCollector) { + collector.block() + } + } + + private fun check(exception: Throwable?, expectedException: KClass?) { + if (expectedException != null && exception == null) fail("Expected $expectedException, but test completed successfully") + if (expectedException != null && exception != null) assertTrue(expectedException.isInstance(exception)) + if (expectedException == null && exception != null) throw exception + } + + @Test + fun testWithContextContract() = runParametrizedTest(IllegalStateException::class) { flow -> + flow { + withContext(NonCancellable) { + emit(1) + } + }.collect { + expectUnreached() + } + } + + @Test + fun testWithDispatcherContractViolated() = runParametrizedTest(IllegalStateException::class) { flow -> + flow { + withContext(NamedDispatchers("foo")) { + emit(1) + } + }.collect { + expectUnreached() + } + } + + @Test + fun testWithNameContractViolated() = runParametrizedTest(IllegalStateException::class) { flow -> + flow { + withContext(CoroutineName("foo")) { + emit(1) + } + }.collect { + expectUnreached() + } + } + + @Test + fun testWithContextDoesNotChangeExecution() = runTest { + val flow = flow { + emit(NamedDispatchers.name()) + }.flowOn(NamedDispatchers("original")) + + var result = "unknown" + withContext(NamedDispatchers("misc")) { + flow + .flowOn(NamedDispatchers("upstream")) + .launchIn(this + NamedDispatchers("consumer")) { + onEach { + result = it + } + }.join() + } + assertEquals("original", result) + } + + @Test + fun testScopedJob() = runParametrizedTest(IllegalStateException::class) { flow -> + flow { emit(1) }.buffer(EmptyCoroutineContext, flow).collect { + expect(1) + } + finish(2) + } + + @Test + fun testScopedJobWithViolation() = runParametrizedTest(IllegalStateException::class) { flow -> + flow { emit(1) }.buffer(Dispatchers.Unconfined, flow).collect { + expect(1) + } + finish(2) + } + + @Test + fun testMergeViolation() = runParametrizedTest { flow -> + fun Flow.merge(other: Flow): Flow = flow { + coroutineScope { + launch { + collect { value -> emit(value) } + } + other.collect { value -> emit(value) } + } + } + + fun Flow.trickyMerge(other: Flow): Flow = flow { + coroutineScope { + launch { + collect { value -> + coroutineScope { emit(value) } + } + } + other.collect { value -> emit(value) } + } + } + + val flowInstance = flowOf(1) + assertFailsWith { flowInstance.merge(flowInstance).toList() } + assertFailsWith { flowInstance.trickyMerge(flowInstance).toList() } + } + + @Test + fun testNoMergeViolation() = runTest { + fun Flow.merge(other: Flow): Flow = channelFlow { + launch { + collect { value -> send(value) } + } + other.collect { value -> send(value) } + } + + fun Flow.trickyMerge(other: Flow): Flow = channelFlow { + coroutineScope { + launch { + collect { value -> + coroutineScope { send(value) } + } + } + other.collect { value -> send(value) } + } + } + + val flow = flowOf(1) + assertEquals(listOf(1, 1), flow.merge(flow).toList()) + assertEquals(listOf(1, 1), flow.trickyMerge(flow).toList()) + } + + @Test + fun testScopedCoroutineNoViolation() = runParametrizedTest { flow -> + fun Flow.buffer(): Flow = flow { + coroutineScope { + val channel = produce { + collect { + send(it) + } + } + channel.consumeEach { + emit(it) + } + } + } + assertEquals(listOf(1, 1), flowOf(1, 1).buffer().toList()) + } + + private fun Flow.buffer(coroutineContext: CoroutineContext, flow: (suspend FlowCollector.() -> Unit) -> Flow): Flow = flow { + coroutineScope { + val channel = Channel() + launch { + collect { value -> + channel.send(value) + } + channel.close() + } + + launch(coroutineContext) { + for (i in channel) { + emit(i) + } + } + } + } + + @Test + fun testEmptyCoroutineContextMap() = runTest { + emptyContextTest { + map { + expect(it) + it + 1 + } + } + } + + @Test + fun testEmptyCoroutineContextTransform() = runTest { + emptyContextTest { + transform { + expect(it) + emit(it + 1) + } + } + } + + @Test + fun testEmptyCoroutineContextTransformWhile() = runTest { + emptyContextTest { + transformWhile { + expect(it) + emit(it + 1) + true + } + } + } + + @Test + fun testEmptyCoroutineContextViolationTransform() = runTest { + try { + emptyContextTest { + transform { + expect(it) + withContext(Dispatchers.Unconfined) { + emit(it + 1) + } + } + } + expectUnreached() + } catch (e: IllegalStateException) { + assertTrue(e.message!!.contains("Flow invariant is violated"), "But had: ${e.message}") + finish(2) + } + } + + @Test + fun testEmptyCoroutineContextViolationTransformWhile() = runTest { + try { + emptyContextTest { + transformWhile { + expect(it) + withContext(Dispatchers.Unconfined) { + emit(it + 1) + } + true + } + } + expectUnreached() + } catch (e: IllegalStateException) { + assertTrue(e.message!!.contains("Flow invariant is violated")) + finish(2) + } + } + + private suspend fun emptyContextTest(block: Flow.() -> Flow) { + suspend fun collector(): Int { + var result: Int = -1 + channelFlow { + send(1) + }.block() + .collect { + expect(it) + result = it + } + return result + } + + val result = withEmptyContext { collector() } + assertEquals(2, result) + finish(3) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/IdFlowTest.kt b/kotlinx-coroutines-core/common/test/flow/IdFlowTest.kt new file mode 100644 index 0000000000..be3be66780 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/IdFlowTest.kt @@ -0,0 +1,61 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +// See https://github.com/Kotlin/kotlinx.coroutines/issues/1128 +class IdFlowTest : TestBase() { + @Test + fun testCancelInCollect() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + flow { + expect(2) + emit(1) + expect(3) + hang { finish(6) } + }.idScoped().collect { value -> + expect(4) + assertEquals(1, value) + kotlin.coroutines.coroutineContext.cancel() + expect(5) + } + expectUnreached() + } + + @Test + fun testCancelInFlow() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + flow { + expect(2) + emit(1) + kotlin.coroutines.coroutineContext.cancel() + expect(3) + }.idScoped().collect { value -> + finish(4) + assertEquals(1, value) + } + expectUnreached() + } +} + +/** + * This flow should be "identity" function with respect to cancellation. + */ +private fun Flow.idScoped(): Flow = flow { + coroutineScope { + val channel = produce { + collect { send(it) } + } + channel.consumeEach { + emit(it) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/NamedDispatchers.kt b/kotlinx-coroutines-core/common/test/flow/NamedDispatchers.kt new file mode 100644 index 0000000000..754b3087ba --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/NamedDispatchers.kt @@ -0,0 +1,62 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* + +/** + * Test dispatchers that emulate multiplatform context tracking. + */ +public object NamedDispatchers { + + private val stack = ArrayStack() + + public fun name(): String = stack.peek() ?: error("No names on stack") + + public fun nameOr(defaultValue: String): String = stack.peek() ?: defaultValue + + public operator fun invoke(name: String) = named(name) + + private fun named(name: String): CoroutineDispatcher = object : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + stack.push(name) + try { + block.run() + } finally { + val last = stack.pop() ?: error("No names on stack") + require(last == name) { "Inconsistent stack: expected $name, but had $last" } + } + } + } +} + +private class ArrayStack { + private var elements = arrayOfNulls(16) + private var head = 0 + + public fun push(value: String) { + if (elements.size == head - 1) ensureCapacity() + elements[head++] = value + } + + public fun peek(): String? = elements.getOrNull(head - 1) + + public fun pop(): String? { + if (head == 0) return null + return elements[--head] + } + + private fun ensureCapacity() { + val currentSize = elements.size + val newCapacity = currentSize shl 1 + val newElements = arrayOfNulls(newCapacity) + elements.copyInto( + destination = newElements, + startIndex = head + ) + elements.copyInto( + destination = newElements, + destinationOffset = elements.size - head, + endIndex = head + ) + elements = newElements + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/SafeFlowTest.kt b/kotlinx-coroutines-core/common/test/flow/SafeFlowTest.kt new file mode 100644 index 0000000000..26132ab0f5 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/SafeFlowTest.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class SafeFlowTest : TestBase() { + + @Test + fun testEmissionsFromDifferentStateMachine() = runTest { + val result = flow { + emit1(1) + emit2(2) + }.onEach { yield() }.toList() + assertEquals(listOf(1, 2), result) + finish(3) + } + + private suspend fun FlowCollector.emit1(expect: Int) { + emit(expect) + expect(expect) + } + + private suspend fun FlowCollector.emit2(expect: Int) { + emit(expect) + expect(expect) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/VirtualTime.kt b/kotlinx-coroutines-core/common/test/flow/VirtualTime.kt new file mode 100644 index 0000000000..771768e008 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/VirtualTime.kt @@ -0,0 +1,99 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.jvm.* + +internal class VirtualTimeDispatcher(enclosingScope: CoroutineScope) : CoroutineDispatcher(), Delay { + private val originalDispatcher = enclosingScope.coroutineContext[ContinuationInterceptor] as CoroutineDispatcher + private val heap = ArrayList() // TODO use MPP heap/ordered set implementation (commonize ThreadSafeHeap) + + var currentTime = 0L + private set + + init { + /* + * Launch "event-loop-owning" task on start of the virtual time event loop. + * It ensures the progress of the enclosing event-loop and polls the timed queue + * when the enclosing event loop is empty, emulating virtual time. + */ + enclosingScope.launch { + while (true) { + val delayNanos = ThreadLocalEventLoop.currentOrNull()?.processNextEvent() + ?: error("Event loop is missing, virtual time source works only as part of event loop") + if (delayNanos <= 0) continue + if (delayNanos > 0 && delayNanos != Long.MAX_VALUE) { + if (usesSharedEventLoop) { + val targetTime = currentTime + delayNanos + while (currentTime < targetTime) { + val nextTask = heap.minByOrNull { it.deadline } ?: break + if (nextTask.deadline > targetTime) break + heap.remove(nextTask) + currentTime = nextTask.deadline + nextTask.run() + } + currentTime = maxOf(currentTime, targetTime) + } else { + error("Unexpected external delay: $delayNanos") + } + } + val nextTask = heap.minByOrNull { it.deadline } ?: return@launch + heap.remove(nextTask) + currentTime = nextTask.deadline + nextTask.run() + } + } + } + + private inner class TimedTask( + private val runnable: Runnable, + @JvmField val deadline: Long + ) : DisposableHandle, Runnable by runnable { + + override fun dispose() { + heap.remove(this) + } + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + originalDispatcher.dispatch(context, block) + } + + override fun isDispatchNeeded(context: CoroutineContext): Boolean = originalDispatcher.isDispatchNeeded(context) + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { + val task = TimedTask(block, deadline(timeMillis)) + heap += task + return task + } + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val task = TimedTask(Runnable { with(continuation) { resumeUndispatched(Unit) } }, deadline(timeMillis)) + heap += task + continuation.invokeOnCancellation { task.dispose() } + } + + private fun deadline(timeMillis: Long) = + if (timeMillis == Long.MAX_VALUE) Long.MAX_VALUE else currentTime + timeMillis +} + +/** + * Runs a test ([TestBase.runTest]) with a virtual time source. + * This runner has the following constraints: + * 1) It works only in the event-loop environment and it is relying on it. + * None of the coroutines should be launched in any dispatcher different from a current + * 2) Regular tasks always dominate delayed ones. It means that + * `launch { while(true) yield() }` will block the progress of the delayed tasks + * 3) [TestBase.finish] should always be invoked. + * Given all the constraints into account, it is easy to mess up with a test and actually + * return from [withVirtualTime] before the test is executed completely. + * To decrease the probability of such error, additional `finish` constraint is added. + */ +public fun TestBase.withVirtualTime(block: suspend CoroutineScope.() -> Unit) = runTest { + withContext(Dispatchers.Unconfined) { + // Create a platform-independent event loop + val dispatcher = VirtualTimeDispatcher(this) + withContext(dispatcher) { block() } + checkFinishCall(allowNotUsingExpect = false) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/channels/ChannelBuildersFlowTest.kt b/kotlinx-coroutines-core/common/test/flow/channels/ChannelBuildersFlowTest.kt new file mode 100644 index 0000000000..7a51a8d7b6 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/channels/ChannelBuildersFlowTest.kt @@ -0,0 +1,137 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class ChannelBuildersFlowTest : TestBase() { + @Test + fun testChannelConsumeAsFlow() = runTest { + val channel = produce { + repeat(10) { + send(it + 1) + } + } + val flow = channel.consumeAsFlow() + assertEquals(55, flow.sum()) + assertFailsWith { flow.collect() } + } + + @Test + fun testChannelReceiveAsFlow() = runTest { + val channel = produce { + repeat(10) { + send(it + 1) + } + } + val flow = channel.receiveAsFlow() + assertEquals(55, flow.sum()) + assertEquals(emptyList(), flow.toList()) + } + + @Test + fun testConsumeAsFlowCancellation() = runTest { + val channel = produce(NonCancellable) { // otherwise failure will cancel scope as well + repeat(10) { + send(it + 1) + } + throw TestException() + } + val flow = channel.consumeAsFlow() + assertEquals(15, flow.take(5).sum()) + // the channel should have been canceled, even though took only 5 elements + assertTrue(channel.isClosedForReceive) + assertFailsWith { flow.collect() } + } + + @Test + fun testReceiveAsFlowCancellation() = runTest { + val channel = produce(NonCancellable) { // otherwise failure will cancel scope as well + repeat(10) { + send(it + 1) + } + throw TestException() + } + val flow = channel.receiveAsFlow() + assertEquals(15, flow.take(5).sum()) // sum of first 5 + assertEquals(40, flow.take(5).sum()) // sum the rest 5 + assertFailsWith { flow.sum() } // exception in the rest + } + + @Test + fun testConsumeAsFlowException() = runTest { + val channel = produce(NonCancellable) { // otherwise failure will cancel scope as well + repeat(10) { + send(it + 1) + } + throw TestException() + } + val flow = channel.consumeAsFlow() + assertFailsWith { flow.sum() } + assertFailsWith { flow.collect() } + } + + @Test + fun testReceiveAsFlowException() = runTest { + val channel = produce(NonCancellable) { // otherwise failure will cancel scope as well + repeat(10) { + send(it + 1) + } + throw TestException() + } + val flow = channel.receiveAsFlow() + assertFailsWith { flow.sum() } + assertFailsWith { flow.collect() } // repeated collection -- same exception + } + + @Test + fun testConsumeAsFlowProduceFusing() = runTest { + val channel = produce { send("OK") } + val flow = channel.consumeAsFlow() + assertSame(channel, flow.produceIn(this)) + assertFailsWith { flow.produceIn(this) } + channel.cancel() + } + + @Test + fun testReceiveAsFlowProduceFusing() = runTest { + val channel = produce { send("OK") } + val flow = channel.receiveAsFlow() + assertSame(channel, flow.produceIn(this)) + assertSame(channel, flow.produceIn(this)) // can use produce multiple times + channel.cancel() + } + + @Test + fun testConsumeAsFlowProduceBuffered() = runTest { + expect(1) + val channel = produce { + expect(3) + (1..10).forEach { send(it) } + expect(4) // produces everything because of buffering + } + val flow = channel.consumeAsFlow().buffer() // request buffering + expect(2) // producer is not running yet + val result = flow.produceIn(this) + // run the flow pipeline until it consumes everything into buffer + while (!channel.isClosedForReceive) yield() + expect(5) // produced had done running (buffered stuff) + assertNotSame(channel, result) + assertFailsWith { flow.produceIn(this) } + // check that we received everything + assertEquals((1..10).toList(), result.toList()) + finish(6) + } + + @Test + fun testProduceInAtomicity() = runTest { + val flow = flowOf(1).onCompletion { expect(2) } + val scope = CoroutineScope(wrapperDispatcher()) + flow.produceIn(scope) + expect(1) + scope.cancel() + scope.coroutineContext[Job]?.join() + finish(3) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/channels/ChannelFlowTest.kt b/kotlinx-coroutines-core/common/test/flow/channels/ChannelFlowTest.kt new file mode 100644 index 0000000000..6e5846125d --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/channels/ChannelFlowTest.kt @@ -0,0 +1,207 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class ChannelFlowTest : TestBase() { + @Test + fun testRegular() = runTest { + val flow = channelFlow { + assertTrue(trySend(1).isSuccess) + assertTrue(trySend(2).isSuccess) + assertTrue(trySend(3).isSuccess) + } + assertEquals(listOf(1, 2, 3), flow.toList()) + } + + @Test + fun testBuffer() = runTest { + val flow = channelFlow { + assertTrue(trySend(1).isSuccess) + assertTrue(trySend(2).isSuccess) + assertFalse(trySend(3).isSuccess) + }.buffer(1) + assertEquals(listOf(1, 2), flow.toList()) + } + + @Test + fun testConflated() = runTest { + val flow = channelFlow { + assertTrue(trySend(1).isSuccess) + assertTrue(trySend(2).isSuccess) + assertTrue(trySend(3).isSuccess) + assertTrue(trySend(4).isSuccess) + }.buffer(Channel.CONFLATED) + assertEquals(listOf(1, 4), flow.toList()) // two elements in the middle got conflated + } + + @Test + fun testFailureCancelsChannel() = runTest { + val flow = channelFlow { + trySend(1) + invokeOnClose { + expect(2) + } + }.onEach { throw TestException() } + + expect(1) + assertFailsWith(flow) + finish(3) + } + + @Test + fun testFailureInSourceCancelsConsumer() = runTest { + val flow = channelFlow { + expect(2) + throw TestException() + }.onEach { expectUnreached() } + + expect(1) + assertFailsWith(flow) + finish(3) + } + + @Test + fun testScopedCancellation() = runTest { + val flow = channelFlow { + expect(2) + launch(start = CoroutineStart.ATOMIC) { + hang { expect(3) } + } + throw TestException() + }.onEach { expectUnreached() } + + expect(1) + assertFailsWith(flow) + finish(4) + } + + @Test + fun testMergeOneCoroutineWithCancellation() = runTest { + val flow = flowOf(1, 2, 3) + val f = flow.mergeOneCoroutine(flow).take(2) + assertEquals(listOf(1, 1), f.toList()) + } + + @Test + fun testMergeTwoCoroutinesWithCancellation() = runTest { + val flow = flowOf(1, 2, 3) + val f = flow.mergeTwoCoroutines(flow).take(2) + assertEquals(listOf(1, 1), f.toList()) + } + + private fun Flow.mergeTwoCoroutines(other: Flow): Flow = channelFlow { + launch { + collect { send(it); yield() } + } + launch { + other.collect { send(it) } + } + } + + private fun Flow.mergeOneCoroutine(other: Flow): Flow = channelFlow { + launch { + collect { send(it); yield() } + } + + other.collect { send(it); yield() } + } + + @Test + @Ignore // #1374 + fun testBufferWithTimeout() = runTest { + fun Flow.bufferWithTimeout(): Flow = channelFlow { + expect(2) + launch { + expect(3) + hang { + expect(5) + } + } + launch { + expect(4) + collect { + withTimeout(-1) { + send(it) + } + expectUnreached() + } + expectUnreached() + } + } + + val flow = flowOf(1, 2, 3).bufferWithTimeout() + expect(1) + assertFailsWith(flow) + finish(6) + } + + @Test + fun testChildCancellation() = runTest { + channelFlow { + val job = launch { + expect(2) + hang { expect(4) } + } + expect(1) + yield() + expect(3) + job.cancelAndJoin() + send(5) + + }.collect { + expect(it) + } + + finish(6) + } + + @Test + fun testClosedPrematurely() = runTest(unhandled = listOf({ e -> e is ClosedSendChannelException })) { + val outerScope = this + val flow = channelFlow { + // ~ callback-based API, no children + outerScope.launch(Job()) { + expect(2) + send(1) + expectUnreached() + } + expect(1) + } + assertEquals(emptyList(), flow.toList()) + finish(3) + } + + @Test + fun testNotClosedPrematurely() = runTest { + val outerScope = this + val flow = channelFlow { + // ~ callback-based API + outerScope.launch(Job()) { + expect(2) + send(1) + close() + } + expect(1) + awaitClose() + } + + assertEquals(listOf(1), flow.toList()) + finish(3) + } + + @Test + fun testCancelledOnCompletion() = runTest { + val myFlow = callbackFlow { + expect(2) + close() + hang { expect(3) } + } + + expect(1) + myFlow.collect() + finish(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/channels/FlowCallbackTest.kt b/kotlinx-coroutines-core/common/test/flow/channels/FlowCallbackTest.kt new file mode 100644 index 0000000000..2b553a5e64 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/channels/FlowCallbackTest.kt @@ -0,0 +1,54 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class FlowCallbackTest : TestBase() { + @Test + fun testClosedPrematurely() = runTest { + val outerScope = this + val flow = callbackFlow { + // ~ callback-based API + outerScope.launch(Job()) { + expect(2) + try { + send(1) + expectUnreached() + } catch (e: IllegalStateException) { + expect(3) + assertTrue(e.message!!.contains("awaitClose")) + } + } + expect(1) + } + try { + flow.collect() + } catch (e: IllegalStateException) { + expect(4) + assertTrue(e.message!!.contains("awaitClose")) + } + finish(5) + } + + @Test + fun testNotClosedPrematurely() = runTest { + val outerScope = this + val flow = callbackFlow { + // ~ callback-based API + outerScope.launch(Job()) { + expect(2) + send(1) + close() + } + expect(1) + awaitClose() + } + + assertEquals(listOf(1), flow.toList()) + finish(3) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/internal/FlowScopeTest.kt b/kotlinx-coroutines-core/common/test/flow/internal/FlowScopeTest.kt new file mode 100644 index 0000000000..aeaa3c19e5 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/internal/FlowScopeTest.kt @@ -0,0 +1,74 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class FlowScopeTest : TestBase() { + + @Test + fun testCancellation() = runTest { + assertFailsWith { + flowScope { + expect(1) + val child = launch { + expect(3) + hang { expect(5) } + } + expect(2) + yield() + expect(4) + child.cancel() + } + } + finish(6) + } + + @Test + fun testCancellationWithChildCancelled() = runTest { + flowScope { + expect(1) + val child = launch { + expect(3) + hang { expect(5) } + } + expect(2) + yield() + expect(4) + child.cancel(ChildCancelledException()) + } + finish(6) + } + + @Test + fun testCancellationWithSuspensionPoint() = runTest { + assertFailsWith { + flowScope { + expect(1) + val child = launch { + expect(3) + hang { expect(6) } + } + expect(2) + yield() + expect(4) + child.cancel() + hang { expect(5) } + } + } + finish(7) + } + + @Test + fun testNestedScopes() = runTest { + assertFailsWith { + flowScope { + flowScope { + launch { + throw CancellationException("") + } + } + } + } + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/BooleanTerminationTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/BooleanTerminationTest.kt new file mode 100644 index 0000000000..3087c78f67 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/BooleanTerminationTest.kt @@ -0,0 +1,106 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class BooleanTerminationTest : TestBase() { + @Test + fun testAnyNominal() = runTest { + val flow = flow { + emit(1) + emit(2) + } + + assertTrue(flow.any { it > 0 }) + assertTrue(flow.any { it % 2 == 0 }) + assertFalse(flow.any { it > 5 }) + } + + @Test + fun testAnyEmpty() = runTest { + assertFalse(emptyFlow().any { it > 0 }) + } + + @Test + fun testAnyInfinite() = runTest { + assertTrue(flow { while (true) { emit(5) } }.any { it == 5 }) + } + + @Test + fun testAnyShortCircuit() = runTest { + assertTrue(flow { + emit(1) + emit(2) + expectUnreached() + }.any { + it == 2 + }) + } + + @Test + fun testAllNominal() = runTest { + val flow = flow { + emit(1) + emit(2) + } + + assertTrue(flow.all { it > 0 }) + assertFalse(flow.all { it % 2 == 0 }) + assertFalse(flow.all { it > 5 }) + } + + @Test + fun testAllEmpty() = runTest { + assertTrue(emptyFlow().all { it > 0 }) + } + + @Test + fun testAllInfinite() = runTest { + assertFalse(flow { while (true) { emit(5) } }.all { it == 0 }) + } + + @Test + fun testAllShortCircuit() = runTest { + assertFalse(flow { + emit(1) + emit(2) + expectUnreached() + }.all { + it <= 1 + }) + } + + @Test + fun testNoneNominal() = runTest { + val flow = flow { + emit(1) + emit(2) + } + + assertFalse(flow.none { it > 0 }) + assertFalse(flow.none { it % 2 == 0 }) + assertTrue(flow.none { it > 5 }) + } + + @Test + fun testNoneEmpty() = runTest { + assertTrue(emptyFlow().none { it > 0 }) + } + + @Test + fun testNoneInfinite() = runTest { + assertFalse(flow { while (true) { emit(5) } }.none { it == 5 }) + } + + @Test + fun testNoneShortCircuit() = runTest { + assertFalse(flow { + emit(1) + emit(2) + expectUnreached() + }.none { + it == 2 + }) + } + +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/BufferConflationTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/BufferConflationTest.kt new file mode 100644 index 0000000000..789a7132a2 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/BufferConflationTest.kt @@ -0,0 +1,143 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +/** + * A _behavioral_ test for conflation options that can be configured by the [buffer] operator to test that it is + * implemented properly and that adjacent [buffer] calls are fused properly. +*/ +class BufferConflationTest : TestBase() { + private val n = 100 // number of elements to emit for test + + private fun checkConflate( + capacity: Int, + onBufferOverflow: BufferOverflow = BufferOverflow.DROP_OLDEST, + op: suspend Flow.() -> Flow + ) = runTest { + expect(1) + // emit all and conflate, then collect first & last + val expectedList = when (onBufferOverflow) { + BufferOverflow.DROP_OLDEST -> listOf(0) + (n - capacity until n).toList() // first item & capacity last ones + BufferOverflow.DROP_LATEST -> (0..capacity).toList() // first & capacity following ones + else -> error("cannot happen") + } + flow { + repeat(n) { i -> + expect(i + 2) + emit(i) + } + } + .op() + .collect { i -> + val j = expectedList.indexOf(i) + expect(n + 2 + j) + } + finish(n + 2 + expectedList.size) + } + + @Test + fun testConflate() = + checkConflate(1) { + conflate() + } + + @Test + fun testBufferConflated() = + checkConflate(1) { + buffer(Channel.CONFLATED) + } + + @Test + fun testBufferDropOldest() = + checkConflate(1) { + buffer(onBufferOverflow = BufferOverflow.DROP_OLDEST) + } + + @Test + fun testBuffer0DropOldest() = + checkConflate(1) { + buffer(0, onBufferOverflow = BufferOverflow.DROP_OLDEST) + } + + @Test + fun testBuffer1DropOldest() = + checkConflate(1) { + buffer(1, onBufferOverflow = BufferOverflow.DROP_OLDEST) + } + + @Test + fun testBuffer10DropOldest() = + checkConflate(10) { + buffer(10, onBufferOverflow = BufferOverflow.DROP_OLDEST) + } + + @Test + fun testConflateOverridesBuffer() = + checkConflate(1) { + buffer(42).conflate() + } + + @Test // conflate().conflate() should work like a single conflate + fun testDoubleConflate() = + checkConflate(1) { + conflate().conflate() + } + + @Test + fun testConflateBuffer10Combine() = + checkConflate(10) { + conflate().buffer(10) + } + + @Test + fun testBufferDropLatest() = + checkConflate(1, BufferOverflow.DROP_LATEST) { + buffer(onBufferOverflow = BufferOverflow.DROP_LATEST) + } + + @Test + fun testBuffer0DropLatest() = + checkConflate(1, BufferOverflow.DROP_LATEST) { + buffer(0, onBufferOverflow = BufferOverflow.DROP_LATEST) + } + + @Test + fun testBuffer1DropLatest() = + checkConflate(1, BufferOverflow.DROP_LATEST) { + buffer(1, onBufferOverflow = BufferOverflow.DROP_LATEST) + } + + @Test // overrides previous buffer + fun testBufferDropLatestOverrideBuffer() = + checkConflate(1, BufferOverflow.DROP_LATEST) { + buffer(42).buffer(onBufferOverflow = BufferOverflow.DROP_LATEST) + } + + @Test // overrides previous conflate + fun testBufferDropLatestOverrideConflate() = + checkConflate(1, BufferOverflow.DROP_LATEST) { + conflate().buffer(onBufferOverflow = BufferOverflow.DROP_LATEST) + } + + @Test + fun testBufferDropLatestBuffer7Combine() = + checkConflate(7, BufferOverflow.DROP_LATEST) { + buffer(onBufferOverflow = BufferOverflow.DROP_LATEST).buffer(7) + } + + @Test + fun testConflateOverrideBufferDropLatest() = + checkConflate(1) { + buffer(onBufferOverflow = BufferOverflow.DROP_LATEST).conflate() + } + + @Test + fun testBuffer3DropOldestOverrideBuffer8DropLatest() = + checkConflate(3, BufferOverflow.DROP_OLDEST) { + buffer(8, onBufferOverflow = BufferOverflow.DROP_LATEST) + .buffer(3, BufferOverflow.DROP_OLDEST) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/flow/operators/BufferTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/BufferTest.kt new file mode 100644 index 0000000000..c8407901a1 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/BufferTest.kt @@ -0,0 +1,194 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.math.* +import kotlin.test.* + +/** + * A _behavioral_ test for buffering that is introduced by the [buffer] operator to test that it is + * implemented properly and that adjacent [buffer] calls are fused properly. + */ +class BufferTest : TestBase() { + private val n = 200 // number of elements to emit for test + private val defaultBufferSize = 64 // expected default buffer size (per docs) + + // Use capacity == -1 to check case of "no buffer" + private fun checkBuffer(capacity: Int, op: suspend Flow.() -> Flow) = runTest { + expect(1) + /* + Channels perform full rendezvous. Sender does not suspend when there is a suspended receiver and vice-versa. + Thus, perceived batch size is +2 from capacity. + */ + val batchSize = capacity + 2 + flow { + repeat(n) { i -> + val batchNo = i / batchSize + val batchIdx = i % batchSize + expect(batchNo * batchSize * 2 + batchIdx + 2) + emit(i) + } + } + .op() // insert user-defined operator + .collect { i -> + val batchNo = i / batchSize + val batchIdx = i % batchSize + // last batch might have smaller size + val k = min((batchNo + 1) * batchSize, n) - batchNo * batchSize + expect(batchNo * batchSize * 2 + k + batchIdx + 2) + } + finish(2 * n + 2) + } + + @Test + // capacity == -1 to checkBuffer means "no buffer" -- emits / collects are sequentially ordered + fun testBaseline() = + checkBuffer(-1) { this } + + @Test + fun testBufferDefault() = + checkBuffer(defaultBufferSize) { + buffer() + } + + @Test + fun testBufferRendezvous() = + checkBuffer(0) { + buffer(0) + } + + @Test + fun testBuffer1() = + checkBuffer(1) { + buffer(1) + } + + @Test + fun testBuffer2() = + checkBuffer(2) { + buffer(2) + } + + @Test + fun testBuffer3() = + checkBuffer(3) { + buffer(3) + } + + @Test + fun testBuffer00Fused() = + checkBuffer(0) { + buffer(0).buffer(0) + } + + @Test + fun testBuffer01Fused() = + checkBuffer(1) { + buffer(0).buffer(1) + } + + @Test + fun testBuffer11Fused() = + checkBuffer(2) { + buffer(1).buffer(1) + } + + @Test + fun testBuffer111Fused() = + checkBuffer(3) { + buffer(1).buffer(1).buffer(1) + } + + @Test + fun testBuffer123Fused() = + checkBuffer(6) { + buffer(1).buffer(2).buffer(3) + } + + @Test // multiple calls to buffer() create one channel of default size + fun testBufferDefaultTwiceFused() = + checkBuffer(defaultBufferSize) { + buffer().buffer() + } + + @Test // explicit buffer takes precedence of default buffer on fuse + fun testBufferDefaultBufferFused() = + checkBuffer(7) { + buffer().buffer(7) + } + + @Test // explicit buffer takes precedence of default buffer on fuse + fun testBufferBufferDefaultFused() = + checkBuffer(8) { + buffer(8).buffer() + } + + @Test // flowOn operator does not use buffer when dispatches does not change + fun testFlowOnNameNoBuffer() = + checkBuffer(-1) { + flowOn(CoroutineName("Name")) + } + + @Test // flowOn operator uses default buffer size when dispatcher changes + fun testFlowOnDispatcherBufferDefault() = + checkBuffer(defaultBufferSize) { + flowOn(wrapperDispatcher()) + } + + @Test // flowOn(...).buffer(n) sets explicit buffer size to n + fun testFlowOnDispatcherBufferFused() = + checkBuffer(5) { + flowOn(wrapperDispatcher()).buffer(5) + } + + @Test // buffer(n).flowOn(...) sets explicit buffer size to n + fun testBufferFlowOnDispatcherFused() = + checkBuffer(6) { + buffer(6).flowOn(wrapperDispatcher()) + } + + @Test // flowOn(...).buffer(n) sets explicit buffer size to n + fun testFlowOnNameBufferFused() = + checkBuffer(7) { + flowOn(CoroutineName("Name")).buffer(7) + } + + @Test // buffer(n).flowOn(...) sets explicit buffer size to n + fun testBufferFlowOnNameFused() = + checkBuffer(8) { + buffer(8).flowOn(CoroutineName("Name")) + } + + @Test // multiple flowOn/buffer all fused together + fun testBufferFlowOnMultipleFused() = + checkBuffer(12) { + flowOn(wrapperDispatcher()).buffer(3) + .flowOn(CoroutineName("Name")).buffer(4) + .flowOn(wrapperDispatcher()).buffer(5) + } + + @Test + fun testCancellation() = runTest { + val result = flow { + emit(1) + emit(2) + emit(3) + expectUnreached() + emit(4) + }.buffer(0) + .take(2) + .toList() + assertEquals(listOf(1, 2), result) + } + + @Test + fun testFailsOnIllegalArguments() { + val flow = emptyFlow() + assertFailsWith { flow.buffer(capacity = -3) } + assertFailsWith { flow.buffer(capacity = Int.MIN_VALUE) } + assertFailsWith { flow.buffer(capacity = Channel.CONFLATED, onBufferOverflow = BufferOverflow.DROP_LATEST) } + assertFailsWith { flow.buffer(capacity = Channel.CONFLATED, onBufferOverflow = BufferOverflow.DROP_OLDEST) } + } +} + diff --git a/kotlinx-coroutines-core/common/test/flow/operators/CancellableTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/CancellableTest.kt new file mode 100644 index 0000000000..9086dab445 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/CancellableTest.kt @@ -0,0 +1,34 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class CancellableTest : TestBase() { + + @Test + fun testCancellable() = runTest { + var sum = 0 + val flow = (0..1000).asFlow() + .onEach { + if (it != 0) currentCoroutineContext().cancel() + sum += it + } + + flow.launchIn(this).join() + assertEquals(500500, sum) + + sum = 0 + flow.cancellable().launchIn(this).join() + assertEquals(1, sum) + } + + @Test + fun testFastPath() { + val flow = listOf(1).asFlow() + assertNotSame(flow, flow.cancellable()) + + val cancellableFlow = flow { emit(42) } + assertSame(cancellableFlow, cancellableFlow.cancellable()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/CatchTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/CatchTest.kt new file mode 100644 index 0000000000..420470a508 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/CatchTest.kt @@ -0,0 +1,201 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.test.* + +class CatchTest : TestBase() { + @Test + fun testCatchEmit() = runTest { + val flow = flow { + emit(1) + throw TestException() + } + + assertEquals(42, flow.catch { emit(41) }.sum()) + assertFailsWith(flow) + } + + @Test + fun testCatchEmitExceptionFromDownstream() = runTest { + var executed = 0 + val flow = flow { + emit(1) + }.catch { emit(42) }.map { + ++executed + throw TestException() + } + + assertFailsWith(flow) + assertEquals(1, executed) + } + + @Test + fun testCatchEmitAll() = runTest { + val flow = flow { + emit(1) + throw TestException() + }.catch { emitAll(flowOf(2)) } + + assertEquals(3, flow.sum()) + } + + @Test + fun testCatchEmitAllExceptionFromDownstream() = runTest { + var executed = 0 + val flow = flow { + emit(1) + }.catch { emitAll(flowOf(1, 2, 3)) }.map { + ++executed + throw TestException() + } + + assertFailsWith(flow) + assertEquals(1, executed) + } + + @Test + fun testWithTimeoutCatch() = runTest { + val flow = flow { + withTimeout(1) { + hang { expect(1) } + } + expectUnreached() + }.catch { emit(1) } + + assertEquals(1, flow.single()) + finish(2) + } + + @Test + fun testCancellationFromUpstreamCatch() = runTest { + val flow = flow { + hang { } + }.catch { expectUnreached() } + + val job = launch { + expect(1) + flow.collect { } + } + + yield() + expect(2) + job.cancelAndJoin() + finish(3) + } + + @Test + fun testCatchContext() = runTest { + expect(1) + val flow = flow { + expect(2) + emit("OK") + expect(3) + throw TestException() + } + val d0 = coroutineContext[ContinuationInterceptor] as CoroutineContext + val d1 = wrapperDispatcher(coroutineContext) + val d2 = wrapperDispatcher(coroutineContext) + flow + .catch { e -> + expect(4) + assertIs(e) + assertEquals("A", kotlin.coroutines.coroutineContext[CoroutineName]?.name) + assertSame(d1, kotlin.coroutines.coroutineContext[ContinuationInterceptor] as CoroutineContext) + throw e // rethrow downstream + } + .flowOn(CoroutineName("A")) + .catch { e -> + expect(5) + assertIs(e) + assertEquals("B", kotlin.coroutines.coroutineContext[CoroutineName]?.name) + assertSame(d1, kotlin.coroutines.coroutineContext[ContinuationInterceptor] as CoroutineContext) + throw e // rethrow downstream + } + .flowOn(CoroutineName("B")) + .catch { e -> + expect(6) + assertIs(e) + assertSame(d1, kotlin.coroutines.coroutineContext[ContinuationInterceptor] as CoroutineContext) + throw e // rethrow downstream + } + .flowOn(d1) + .catch { e -> + expect(7) + assertIs(e) + assertSame(d2, kotlin.coroutines.coroutineContext[ContinuationInterceptor] as CoroutineContext) + throw e // rethrow downstream + } + .flowOn(d2) + // flowOn with a different dispatcher introduces asynchrony so that all exceptions in the + // upstream flows are handled before they go downstream + .onEach { + expectUnreached() // already cancelled + } + .catch { e -> + expect(8) + assertIs(e) + assertSame(d0, kotlin.coroutines.coroutineContext[ContinuationInterceptor] as CoroutineContext) + } + .collect() + finish(9) + } + + @Test + fun testUpstreamExceptionConcurrentWithDownstream() = runTest { + val flow = flow { + try { + expect(1) + emit(1) + } finally { + expect(3) + throw TestException() + } + }.catch { expectUnreached() }.onEach { + expect(2) + throw TestException2() + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testUpstreamExceptionConcurrentWithDownstreamCancellation() = runTest { + val flow = flow { + try { + expect(1) + emit(1) + } finally { + expect(3) + throw TestException() + } + }.catch { expectUnreached() }.onEach { + expect(2) + throw CancellationException("") + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testUpstreamCancellationIsIgnoredWhenDownstreamFails() = runTest { + val flow = flow { + try { + expect(1) + emit(1) + } finally { + expect(3) + throw CancellationException("") + } + }.catch { expectUnreached() }.onEach { + expect(2) + throw TestException("") + } + + assertFailsWith(flow) + finish(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/ChunkedTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/ChunkedTest.kt new file mode 100644 index 0000000000..e901c8c4cd --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/ChunkedTest.kt @@ -0,0 +1,88 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.testing.* +import kotlin.test.* + +@OptIn(ExperimentalCoroutinesApi::class) +class ChunkedTest : TestBase() { + + @Test + fun testChunked() = runTest { + doTest(flowOf(1, 2, 3, 4, 5), 2, listOf(listOf(1, 2), listOf(3, 4), listOf(5))) + doTest(flowOf(1, 2, 3, 4, 5), 3, listOf(listOf(1, 2, 3), listOf(4, 5))) + doTest(flowOf(1, 2, 3, 4), 2, listOf(listOf(1, 2), listOf(3, 4))) + doTest(flowOf(1), 3, listOf(listOf(1))) + } + + private suspend fun doTest(flow: Flow, chunkSize: Int, expected: List>) { + assertEquals(expected, flow.chunked(chunkSize).toList()) + assertEquals(flow.toList().chunked(chunkSize), flow.chunked(chunkSize).toList()) + } + + @Test + fun testEmpty() = runTest { + doTest(emptyFlow(), 1, emptyList()) + doTest(emptyFlow(), 2, emptyList()) + } + + @Test + fun testChunkedCancelled() = runTest { + val result = flow { + expect(1) + emit(1) + emit(2) + expect(2) + }.chunked(1).buffer().take(1).toList() + assertEquals(listOf(listOf(1)), result) + finish(3) + } + + @Test + fun testChunkedCancelledWithSuspension() = runTest { + val result = flow { + expect(1) + emit(1) + yield() + expectUnreached() + emit(2) + }.chunked(1).buffer().take(1).toList() + assertEquals(listOf(listOf(1)), result) + finish(2) + } + + @Test + fun testChunkedDoesNotIgnoreCancellation() = runTest { + expect(1) + val result = flow { + coroutineScope { + launch { + hang { expect(2) } + } + yield() + emit(1) + emit(2) + } + }.chunked(1).take(1).toList() + assertEquals(listOf(listOf(1)), result) + finish(3) + } + + @Test + fun testIae() { + assertFailsWith { emptyFlow().chunked(-1) } + assertFailsWith { emptyFlow().chunked(0) } + assertFailsWith { emptyFlow().chunked(Int.MIN_VALUE) } + assertFailsWith { emptyFlow().chunked(Int.MIN_VALUE + 1) } + } + + @Test + fun testSample() = runTest { + val result = flowOf("a", "b", "c", "d", "e") + .chunked(2) + .map { it.joinToString(separator = "") } + .toList() + assertEquals(listOf("ab", "cd", "e"), result) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/CombineParametersTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/CombineParametersTest.kt new file mode 100644 index 0000000000..35ddba78d6 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/CombineParametersTest.kt @@ -0,0 +1,187 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class CombineParametersTest : TestBase() { + + @Test + fun testThreeParameters() = runTest { + val flow = combine(flowOf("1"), flowOf(2), flowOf(null)) { a, b, c -> a + b + c } + assertEquals("12null", flow.single()) + + val flow2 = combineTransform(flowOf("1"), flowOf(2), flowOf(null)) { a, b, c -> emit(a + b + c) } + assertEquals("12null", flow2.single()) + } + + @Test + fun testThreeParametersTransform() = runTest { + val flow = combineTransform(flowOf("1"), flowOf(2), flowOf(null)) { a, b, c -> emit(a + b + c) } + assertEquals("12null", flow.single()) + } + + @Test + fun testFourParameters() = runTest { + val flow = combine(flowOf("1"), flowOf(2), flowOf("3"), flowOf(null)) { a, b, c, d -> a + b + c + d } + assertEquals("123null", flow.single()) + } + + @Test + fun testFourParametersTransform() = runTest { + val flow = combineTransform(flowOf("1"), flowOf(2), flowOf("3"), flowOf(null)) { a, b, c, d -> + emit(a + b + c + d) + } + assertEquals("123null", flow.single()) + } + + @Test + fun testFiveParameters() = runTest { + val flow = combine(flowOf("1"), flowOf(2), flowOf("3"), flowOf(4.toByte()), flowOf(null)) { a, b, c, d, e -> + a + b + c + d + e + } + assertEquals("1234null", flow.single()) + } + + @Test + fun testFiveParametersTransform() = runTest { + val flow = + combineTransform(flowOf("1"), flowOf(2), flowOf("3"), flowOf(4.toByte()), flowOf(null)) { a, b, c, d, e -> + emit(a + b + c + d + e) + } + assertEquals("1234null", flow.single()) + } + + @Test + fun testNonMatchingTypes() = runTest { + val flow = combine(flowOf(1), flowOf("2")) { args: Array -> + args[0]?.toString() + args[1]?.toString() + } + assertEquals("12", flow.single()) + } + + @Test + fun testNonMatchingTypesIterable() = runTest { + val flow = combine(listOf(flowOf(1), flowOf("2"))) { args: Array -> + args[0]?.toString() + args[1]?.toString() + } + assertEquals("12", flow.single()) + } + + @Test + fun testVararg() = runTest { + val flow = combine( + flowOf("1"), + flowOf(2), + flowOf("3"), + flowOf(4.toByte()), + flowOf("5"), + flowOf(null) + ) { arr -> arr.joinToString("") } + assertEquals("12345null", flow.single()) + } + + @Test + fun testVarargTransform() = runTest { + val flow = combineTransform( + flowOf("1"), + flowOf(2), + flowOf("3"), + flowOf(4.toByte()), + flowOf("5"), + flowOf(null) + ) { arr -> emit(arr.joinToString("")) } + assertEquals("12345null", flow.single()) + } + + @Test + fun testSingleVararg() = runTest { + val list = combine(flowOf(1, 2, 3)) { args: Array -> args[0] }.toList() + assertEquals(listOf(1, 2, 3), list) + } + + @Test + fun testSingleVarargTransform() = runTest { + val list = combineTransform(flowOf(1, 2, 3)) { args: Array -> emit(args[0]) }.toList() + assertEquals(listOf(1, 2, 3), list) + } + + @Test + fun testReified() = runTest { + val value = combine(flowOf(1), flowOf(2)) { args: Array -> + assertIs>(args) + args[0] + args[1] + }.single() + assertEquals(3, value) + } + + @Test + fun testReifiedTransform() = runTest { + val value = combineTransform(flowOf(1), flowOf(2)) { args: Array -> + assertIs>(args) + emit(args[0] + args[1]) + }.single() + assertEquals(3, value) + } + + @Test + fun testTransformEmptyIterable() = runTest { + val value = combineTransform(emptyList()) { args: Array -> + emit(args[0] + args[1]) + }.singleOrNull() + assertNull(value) + } + + @Test + fun testTransformEmptyVararg() = runTest { + val value = combineTransform { args: Array -> + emit(args[0] + args[1]) + }.singleOrNull() + assertNull(value) + } + + @Test + fun testEmptyIterable() = runTest { + val value = combine(emptyList()) { args: Array -> + args[0] + args[1] + }.singleOrNull() + assertNull(value) + } + + @Test + fun testEmptyVararg() = runTest { + val value = combine { args: Array -> + args[0] + args[1] + }.singleOrNull() + assertNull(value) + } + + @Test + fun testFairnessInVariousConfigurations() = runTest { + // Test various configurations + for (flowsCount in 2..5) { + for (flowSize in 1..5) { + val flows = List(flowsCount) { (1..flowSize).asFlow() } + val combined = combine(flows) { it.joinToString(separator = "") }.toList() + val expected = List(flowSize) { (it + 1).toString().repeat(flowsCount) } + assertEquals(expected, combined, "Count: $flowsCount, size: $flowSize") + } + } + } + + @Test + fun testEpochOverflow() = runTest { + val flow = (0..1023).asFlow() + val result = flow.combine(flow) { a, b -> a + b }.toList() + assertEquals(List(1024) { it * 2 } , result) + } + + @Test + fun testArrayType() = runTest { + val arr = flowOf(1) + combine(listOf(arr, arr)) { + println(it[0]) + it[0] + }.toList().also { println(it) } + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/CombineTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/CombineTest.kt new file mode 100644 index 0000000000..a69e39f553 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/CombineTest.kt @@ -0,0 +1,312 @@ +@file:Suppress("UNCHECKED_CAST") +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* +import kotlinx.coroutines.flow.combine as combineOriginal +import kotlinx.coroutines.flow.combineTransform as combineTransformOriginal + +abstract class CombineTestBase : TestBase() { + + abstract fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow + + @Test + fun testCombineLatest() = runTest { + val flow = flowOf("a", "b", "c") + val flow2 = flowOf(1, 2, 3) + val list = flow.combineLatest(flow2, String::plus).toList() + assertEquals(listOf("a1", "b2", "c3"), list) + } + + @Test + fun testNulls() = runTest { + val flow = flowOf("a", null, null) + val flow2 = flowOf(1, 2, 3) + val list = flow.combineLatest(flow2, String?::plus).toList() + assertEquals(listOf("a1", "null2", "null3"), list) + } + + @Test + fun testNullsOther() = runTest { + val flow = flowOf("a", "b", "c") + val flow2 = flowOf(null, 2, null) + val list = flow.combineLatest(flow2, String::plus).toList() + assertEquals(listOf("anull", "b2", "cnull"), list) + } + + @Test + fun testEmptyFlow() = runTest { + val flow = emptyFlow().combineLatest(emptyFlow(), String::plus) + assertNull(flow.singleOrNull()) + } + + @Test + fun testFirstIsEmpty() = runTest { + val f1 = emptyFlow() + val f2 = flowOf(1) + assertEquals(emptyList(), f1.combineLatest(f2, String::plus).toList()) + } + + @Test + fun testSecondIsEmpty() = runTest { + val f1 = flowOf("a") + val f2 = emptyFlow() + assertEquals(emptyList(), f1.combineLatest(f2, String::plus).toList()) + } + + @Test + fun testPreservingOrder() = runTest { + val f1 = flow { + expect(1) + emit("a") + expect(3) + emit("b") + emit("c") + expect(4) + } + + val f2 = flow { + expect(2) + emit(1) + yield() + yield() + expect(5) + emit(2) + expect(6) + yield() + expect(7) + emit(3) + } + + val result = f1.combineLatest(f2, String::plus).toList() + assertEquals(listOf("a1", "b1", "c1", "c2", "c3"), result) + finish(8) + } + + @Test + fun testPreservingOrderReversed() = runTest { + val f1 = flow { + expect(1) + emit("a") + expect(3) + emit("b") + emit("c") + expect(4) + } + + val f2 = flow { + yield() // One more yield because now this flow starts first + expect(2) + emit(1) + yield() + yield() + expect(5) + emit(2) + expect(6) + yield() + expect(7) + emit(3) + } + + val result = f2.combineLatest(f1) { i, j -> j + i }.toList() + assertEquals(listOf("a1", "b1", "c1", "c2", "c3"), result) + finish(8) + } + + @Test + fun testContextIsIsolated() = runTest { + val f1 = flow { + emit("a") + assertEquals("first", NamedDispatchers.name()) + expect(1) + }.flowOn(NamedDispatchers("first")).onEach { + assertEquals("nested", NamedDispatchers.name()) + expect(2) + }.flowOn(NamedDispatchers("nested")) + + val f2 = flow { + emit(1) + assertEquals("second", NamedDispatchers.name()) + expect(3) + }.flowOn(NamedDispatchers("second")) + .onEach { + assertEquals("onEach", NamedDispatchers.name()) + expect(4) + }.flowOn(NamedDispatchers("onEach")) + + val value = withContext(NamedDispatchers("main")) { + f1.combineLatest(f2) { i, j -> + assertEquals("main", NamedDispatchers.name()) + expect(5) + i + j + }.single() + } + + assertEquals("a1", value) + finish(6) + } + + @Test + fun testErrorInDownstreamCancelsUpstream() = runTest { + val f1 = flow { + emit("a") + hang { + expect(2) + } + }.flowOn(NamedDispatchers("first")) + + val f2 = flow { + emit(1) + hang { + expect(3) + } + }.flowOn(NamedDispatchers("second")) + + val flow = f1.combineLatest(f2) { i, j -> + assertEquals("combine", NamedDispatchers.name()) + expect(1) + i + j + }.flowOn(NamedDispatchers("combine")).onEach { + throw TestException() + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testErrorCancelsSibling() = runTest { + val f1 = flow { + emit("a") + hang { + expect(1) + } + }.flowOn(NamedDispatchers("first")) + + val f2 = flow { + emit(1) + throw TestException() + }.flowOn(NamedDispatchers("second")) + + val flow = f1.combineLatest(f2) { _, _ -> 1 } + assertFailsWith(flow) + finish(2) + } + + @Test + fun testCancellationExceptionUpstream() = runTest { + val f1 = flow { + expect(1) + emit(1) + throw CancellationException("") + } + val f2 = flow { + emit(1) + expectUnreached() + } + + val flow = f1.combineLatest(f2) { _, _ -> 1 }.onEach { expect(2) } + assertFailsWith(flow) + finish(3) + } + + @Test + fun testCancellationExceptionDownstream() = runTest { + val f1 = flow { + emit(1) + expect(2) + hang { expect(5) } + } + val f2 = flow { + emit(1) + expect(3) + hang { expect(6) } + } + + val flow = f1.combineLatest(f2) { _, _ -> 1 }.onEach { + expect(1) + yield() + expect(4) + throw CancellationException("") + } + assertFailsWith(flow) + finish(7) + } + + @Test + fun testCancelledCombine() = runTest( + expected = { it is CancellationException } + ) { + coroutineScope { + val flow = flow { + emit(Unit) // emit + } + cancel() // cancel the scope + flow.combineLatest(flow) { _, _ -> }.collect { + // should not be reached, because cancelled before it runs + expectUnreached() + } + } + } +} + +class CombineTest : CombineTestBase() { + override fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow = combineOriginal(other, transform) +} + +class CombineOverloadTest : CombineTestBase() { + override fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow = combineOriginal(this, other, transform) +} + +class CombineTransformTest : CombineTestBase() { + override fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow = combineTransformOriginal(other) { a, b -> + emit(transform(a, b)) + } +} +// Array null-out is an additional test for our array elimination optimization + +class CombineVarargAdapterTest : CombineTestBase() { + override fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow = + combineOriginal(this, other) { args: Array -> + transform(args[0] as T1, args[1] as T2).also { + args[0] = null + args[1] = null + } + } +} + +class CombineIterableTest : CombineTestBase() { + override fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow = + combineOriginal(listOf(this, other)) { args -> + transform(args[0] as T1, args[1] as T2).also { + args[0] = null + args[1] = null + } + } +} + +class CombineTransformAdapterTest : CombineTestBase() { + override fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow = + combineTransformOriginal(flow = this, flow2 = other) { a1, a2 -> emit(transform(a1, a2)) } +} + +class CombineTransformVarargAdapterTest : CombineTestBase() { + override fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow = + combineTransformOriginal(this, other) { args: Array -> + emit(transform(args[0] as T1, args[1] as T2)) // Mess up with array + args[0] = null + args[1] = null + } +} + +class CombineTransformIterableTest : CombineTestBase() { + override fun Flow.combineLatest(other: Flow, transform: suspend (T1, T2) -> R): Flow = + combineTransformOriginal(listOf(this, other)) { args -> + emit(transform(args[0] as T1, args[1] as T2)) + // Mess up with array + args[0] = null + args[1] = null + } +} + diff --git a/kotlinx-coroutines-core/common/test/flow/operators/ConflateTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/ConflateTest.kt new file mode 100644 index 0000000000..7b3878c10b --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/ConflateTest.kt @@ -0,0 +1,23 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class ConflateTest : TestBase() { + @Test // from example + fun testExample() = withVirtualTime { + expect(1) + val flow = flow { + for (i in 1..30) { + delay(100) + emit(i) + } + } + val result = flow.conflate().onEach { + delay(1000) + }.toList() + assertEquals(listOf(1, 10, 20, 30), result) + finish(2) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/flow/operators/DebounceTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/DebounceTest.kt new file mode 100644 index 0000000000..76270d0dfe --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/DebounceTest.kt @@ -0,0 +1,321 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* +import kotlin.time.Duration.Companion.milliseconds + +class DebounceTest : TestBase() { + @Test + fun testBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(1500) + emit("B") + delay(500) + emit("C") + delay(250) + emit("D") + delay(2000) + emit("E") + expect(4) + } + + expect(2) + val result = flow.debounce(1000).toList() + assertEquals(listOf("A", "D", "E"), result) + finish(5) + } + + @Test + fun testSingleNull() = runTest { + val flow = flowOf(null).debounce(Long.MAX_VALUE) + assertNull(flow.single()) + } + + @Test + fun testBasicWithNulls() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(1500) + emit("B") + delay(500) + emit("C") + delay(250) + emit(null) + delay(2000) + emit(null) + expect(4) + } + + expect(2) + val result = flow.debounce(1000).toList() + assertEquals(listOf("A", null, null), result) + finish(5) + } + + @Test + fun testEmpty() = runTest { + val flow = emptyFlow().debounce(Long.MAX_VALUE) + assertNull(flow.singleOrNull()) + } + + @Test + fun testScalar() = withVirtualTime { + val flow = flowOf(1, 2, 3).debounce(1000) + assertEquals(3, flow.single()) + finish(1) + } + + @Test + fun testPace() = withVirtualTime { + val flow = flow { + expect(1) + repeat(10) { + emit(-it) + delay(99) + } + + repeat(10) { + emit(it) + delay(101) + } + expect(2) + }.debounce(100) + + assertEquals((0..9).toList(), flow.toList()) + finish(3) + } + + @Test + fun testUpstreamError()= testUpstreamError(TimeoutCancellationException("")) + + @Test + fun testUpstreamErrorCancellation() = testUpstreamError(TimeoutCancellationException("")) + + private inline fun testUpstreamError(cause: T) = runTest { + val latch = Channel() + val flow = flow { + expect(1) + emit(1) + expect(2) + latch.receive() + throw cause + }.debounce(1).map { + latch.send(Unit) + hang { expect(3) } + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testUpstreamErrorIsolatedContext() = runTest { + val latch = Channel() + val flow = flow { + assertEquals("upstream", NamedDispatchers.name()) + expect(1) + emit(1) + expect(2) + latch.receive() + throw TestException() + }.flowOn(NamedDispatchers("upstream")).debounce(1).map { + latch.send(Unit) + hang { expect(3) } + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testUpstreamErrorDebounceNotTriggered() = runTest { + val flow = flow { + expect(1) + emit(1) + expect(2) + throw TestException() + }.debounce(Long.MAX_VALUE).map { + expectUnreached() + } + assertFailsWith(flow) + finish(3) + } + + @Test + fun testUpstreamErrorDebounceNotTriggeredInIsolatedContext() = runTest { + val flow = flow { + expect(1) + emit(1) + expect(2) + throw TestException() + }.flowOn(NamedDispatchers("source")).debounce(Long.MAX_VALUE).map { + expectUnreached() + } + assertFailsWith(flow) + finish(3) + } + + @Test + fun testDownstreamError() = runTest { + val flow = flow { + expect(1) + emit(1) + hang { expect(3) } + }.debounce(100).map { + expect(2) + yield() + throw TestException() + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testDownstreamErrorIsolatedContext() = runTest { + val flow = flow { + assertEquals("upstream", NamedDispatchers.name()) + expect(1) + emit(1) + hang { expect(3) } + }.flowOn(NamedDispatchers("upstream")).debounce(100).map { + expect(2) + yield() + throw TestException() + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testDurationBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(1500.milliseconds) + emit("B") + delay(500.milliseconds) + emit("C") + delay(250.milliseconds) + emit("D") + delay(2000.milliseconds) + emit("E") + expect(4) + } + + expect(2) + val result = flow.debounce(1000.milliseconds).toList() + assertEquals(listOf("A", "D", "E"), result) + finish(5) + } + + @Test + fun testDebounceSelectorBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit(1) + delay(90) + emit(2) + delay(90) + emit(3) + delay(1010) + emit(4) + delay(1010) + emit(5) + expect(4) + } + + expect(2) + val result = flow.debounce { + if (it == 1) { + 0 + } else { + 1000 + } + }.toList() + + assertEquals(listOf(1, 3, 4, 5), result) + finish(5) + } + + @Test + fun testZeroDebounceTime() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + emit("B") + emit("C") + expect(4) + } + + expect(2) + val result = flow.debounce(0).toList() + + assertEquals(listOf("A", "B", "C"), result) + finish(5) + } + + @Test + fun testZeroDebounceTimeSelector() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + emit("B") + expect(4) + } + + expect(2) + val result = flow.debounce { 0 }.toList() + + assertEquals(listOf("A", "B"), result) + finish(5) + } + + @Test + fun testDebounceDurationSelectorBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(1500.milliseconds) + emit("B") + delay(500.milliseconds) + emit("C") + delay(250.milliseconds) + emit("D") + delay(2000.milliseconds) + emit("E") + expect(4) + } + + expect(2) + val result = flow.debounce { + if (it == "C") { + 0.milliseconds + } else { + 1000.milliseconds + } + }.toList() + + assertEquals(listOf("A", "C", "D", "E"), result) + finish(5) + } + + @Test + fun testFailsWithIllegalArgument() { + val flow = emptyFlow() + assertFailsWith { flow.debounce(-1) } + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/DistinctUntilChangedTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/DistinctUntilChangedTest.kt new file mode 100644 index 0000000000..874361bb0a --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/DistinctUntilChangedTest.kt @@ -0,0 +1,123 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class DistinctUntilChangedTest : TestBase() { + + private class Box(val i: Int) + + @Test + fun testDistinctUntilChanged() = runTest { + val flow = flowOf(1, 1, 2, 2, 1).distinctUntilChanged() + assertEquals(4, flow.sum()) + } + + @Test + fun testDistinctUntilChangedKeySelector() = runTest { + val flow = flow { + emit(Box(1)) + emit(Box(1)) + emit(Box(2)) + emit(Box(1)) + } + + val sum1 = flow.distinctUntilChanged().map { it.i }.sum() + val sum2 = flow.distinctUntilChangedBy(Box::i).map { it.i }.sum() + assertEquals(5, sum1) + assertEquals(4, sum2) + } + + @Test + fun testDistinctUntilChangedAreEquivalent() = runTest { + val flow = flow { + emit(Box(1)) + emit(Box(1)) + emit(Box(2)) + emit(Box(1)) + } + + val sum1 = flow.distinctUntilChanged().map { it.i }.sum() + val sum2 = flow.distinctUntilChanged { old, new -> old.i == new.i }.map { it.i }.sum() + assertEquals(5, sum1) + assertEquals(4, sum2) + } + + @Test + fun testDistinctUntilChangedAreEquivalentSingleValue() = runTest { + val flow = flowOf(1) + val values = flow.distinctUntilChanged { _, _ -> fail("Expected not to compare single value.") }.toList() + assertEquals(listOf(1), values) + } + + @Test + fun testThrowingKeySelector() = runTest { + val flow = flow { + coroutineScope { + launch(start = CoroutineStart.ATOMIC) { + hang { expect(3) } + } + expect(2) + emit(1) + } + }.distinctUntilChangedBy { throw TestException() } + + expect(1) + assertFailsWith(flow) + finish(4) + } + + @Test + fun testThrowingAreEquivalent() = runTest { + val flow = flow { + coroutineScope { + launch(start = CoroutineStart.ATOMIC) { + hang { expect(3) } + } + expect(2) + emit(1) + emit(2) + } + }.distinctUntilChanged { _, _ -> throw TestException() } + + expect(1) + assertFailsWith(flow) + finish(4) + } + + @Test + fun testDistinctUntilChangedNull() = runTest { + val flow = flowOf(null, 1, null, null).distinctUntilChanged() + assertEquals(listOf(null, 1, null), flow.toList()) + } + + @Test + fun testRepeatedDistinctFusionDefault() = testRepeatedDistinctFusion { + distinctUntilChanged() + } + + // A separate variable is needed for K/N that does not optimize non-captured lambdas (yet) + private val areEquivalentTestFun: (old: Int, new: Int) -> Boolean = { old, new -> old == new } + + @Test + fun testRepeatedDistinctFusionAreEquivalent() = testRepeatedDistinctFusion { + distinctUntilChanged(areEquivalentTestFun) + } + + // A separate variable is needed for K/N that does not optimize non-captured lambdas (yet) + private val keySelectorTestFun: (Int) -> Int = { it % 2 } + + @Test + fun testRepeatedDistinctFusionByKey() = testRepeatedDistinctFusion { + distinctUntilChangedBy(keySelectorTestFun) + } + + private fun testRepeatedDistinctFusion(op: Flow.() -> Flow) = runTest { + val flow = (1..10).asFlow() + val d1 = flow.op() + assertNotSame(flow, d1) + val d2 = d1.op() + assertSame(d1, d2) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/DropTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/DropTest.kt new file mode 100644 index 0000000000..50f70b9977 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/DropTest.kt @@ -0,0 +1,57 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class DropTest : TestBase() { + @Test + fun testDrop() = runTest { + val flow = flow { + emit(1) + emit(2) + emit(3) + } + + assertEquals(5, flow.drop(1).sum()) + assertEquals(0, flow.drop(Int.MAX_VALUE).sum()) + assertNull(flow.drop(Int.MAX_VALUE).singleOrNull()) + assertEquals(3, flow.drop(1).take(2).drop(1).single()) + } + + @Test + fun testEmptyFlow() = runTest { + assertEquals(0, flowOf().drop(1).sum()) + } + + @Test + fun testNegativeCount() { + assertFailsWith { + emptyFlow().drop(-1) + } + } + + @Test + fun testErrorCancelsUpstream() = runTest { + val flow = flow { + coroutineScope { + launch(start = CoroutineStart.ATOMIC) { + hang { expect(5) } + } + expect(2) + emit(1) + expect(3) + emit(2) + expectUnreached() + } + }.drop(1) + .map { + expect(4) + throw TestException() + }.catch { emit(42) } + + expect(1) + assertEquals(42, flow.single()) + finish(6) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/DropWhileTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/DropWhileTest.kt new file mode 100644 index 0000000000..a811935f94 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/DropWhileTest.kt @@ -0,0 +1,48 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class DropWhileTest : TestBase() { + @Test + fun testDropWhile() = runTest { + val flow = flow { + emit(1) + emit(2) + emit(3) + } + + assertEquals(6, flow.dropWhile { false }.sum()) + assertNull(flow.dropWhile { true }.singleOrNull()) + assertEquals(5, flow.dropWhile { it < 2 }.sum()) + assertEquals(1, flow.take(1).dropWhile { it > 1 }.single()) + } + + @Test + fun testEmptyFlow() = runTest { + assertEquals(0, flowOf().dropWhile { true }.sum()) + assertEquals(0, flowOf().dropWhile { false }.sum()) + } + + @Test + fun testErrorCancelsUpstream() = runTest { + val flow = flow { + coroutineScope { + launch(start = CoroutineStart.ATOMIC) { + hang { expect(4) } + } + expect(2) + emit(1) + expectUnreached() + } + }.dropWhile { + expect(3) + throw TestException() + } + + expect(1) + assertFailsWith(flow) + finish(5) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FilterTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FilterTest.kt new file mode 100644 index 0000000000..a68cdf4280 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FilterTest.kt @@ -0,0 +1,78 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class FilterTest : TestBase() { + @Test + fun testFilter() = runTest { + val flow = flowOf(1, 2) + assertEquals(2, flow.filter { it % 2 == 0 }.sum()) + assertEquals(3, flow.filter { true }.sum()) + assertEquals(0, flow.filter { false }.sum()) + } + + @Test + fun testEmptyFlow() = runTest { + val sum = emptyFlow().filter { true }.sum() + assertEquals(0, sum) + } + + @Test + fun testErrorCancelsUpstream() = runTest { + var cancelled = false + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang {cancelled = true} + } + emit(1) + } + }.filter { + latch.receive() + throw TestException() + }.catch { emit(42) } + + assertEquals(42, flow.single()) + assertTrue(cancelled) + } + + + @Test + fun testFilterNot() = runTest { + val flow = flowOf(1, 2) + assertEquals(0, flow.filterNot { true }.sum()) + assertEquals(3, flow.filterNot { false }.sum()) + } + + @Test + fun testEmptyFlowFilterNot() = runTest { + val sum = emptyFlow().filterNot { true }.sum() + assertEquals(0, sum) + } + + @Test + fun testErrorCancelsUpstreamwFilterNot() = runTest { + var cancelled = false + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang {cancelled = true} + } + emit(1) + } + }.filterNot { + latch.receive() + throw TestException() + }.catch { emit(42) } + + assertEquals(42, flow.single()) + assertTrue(cancelled) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FilterTrivialTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FilterTrivialTest.kt new file mode 100644 index 0000000000..71a7a73647 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FilterTrivialTest.kt @@ -0,0 +1,74 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class FilterTrivialTest : TestBase() { + + @Test + fun testFilterNotNull() = runTest { + val flow = flowOf(1, 2, null) + assertEquals(3, flow.filterNotNull().sum()) + } + + @Test + fun testEmptyFlowNotNull() = runTest { + val sum = emptyFlow().filterNotNull().sum() + assertEquals(0, sum) + } + + @Test + fun testFilterIsInstance() = runTest { + val flow = flowOf("value", 2.0) + assertEquals(2.0, flow.filterIsInstance().single()) + assertEquals("value", flow.filterIsInstance().single()) + } + + @Test + fun testParametrizedFilterIsInstance() = runTest { + val flow = flowOf("value", 2.0) + assertEquals(2.0, flow.filterIsInstance(Double::class).single()) + assertEquals("value", flow.filterIsInstance(String::class).single()) + } + + @Test + fun testSubtypesFilterIsInstance() = runTest { + open class Super + class Sub : Super() + + val flow = flowOf(Super(), Super(), Super(), Sub(), Sub(), Sub()) + assertEquals(6, flow.filterIsInstance().count()) + assertEquals(3, flow.filterIsInstance().count()) + } + + @Test + fun testSubtypesParametrizedFilterIsInstance() = runTest { + open class Super + class Sub : Super() + + val flow = flowOf(Super(), Super(), Super(), Sub(), Sub(), Sub()) + assertEquals(6, flow.filterIsInstance(Super::class).count()) + assertEquals(3, flow.filterIsInstance(Sub::class).count()) + } + + @Test + fun testFilterIsInstanceNullable() = runTest { + val flow = flowOf(1, 2, null) + assertEquals(2, flow.filterIsInstance().count()) + assertEquals(3, flow.filterIsInstance().count()) + } + + @Test + fun testEmptyFlowIsInstance() = runTest { + val sum = emptyFlow().filterIsInstance().sum() + assertEquals(0, sum) + } + + @Test + fun testEmptyFlowParametrizedIsInstance() = runTest { + val sum = emptyFlow().filterIsInstance(Int::class).sum() + assertEquals(0, sum) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlatMapBaseTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapBaseTest.kt new file mode 100644 index 0000000000..6e518572e2 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapBaseTest.kt @@ -0,0 +1,88 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +abstract class FlatMapBaseTest : TestBase() { + + abstract fun Flow.flatMap(mapper: suspend (T) -> Flow): Flow + + @Test + fun testFlatMap() = runTest { + val n = 100 + val sum = (1..100).asFlow() + .flatMap { value -> + // 1 + (1 + 2) + (1 + 2 + 3) + ... (1 + .. + n) + flow { + repeat(value) { + emit(it + 1) + } + } + }.sum() + + assertEquals(n * (n + 1) * (n + 2) / 6, sum) + } + + @Test + fun testSingle() = runTest { + val flow = flow { + repeat(100) { + emit(it) + } + }.flatMap { value -> + if (value == 99) flowOf(42) + else flowOf() + } + + val value = flow.single() + assertEquals(42, value) + } + + @Test + fun testNulls() = runTest { + val list = flowOf(1, null, 2).flatMap { + flowOf(1, null, null, 2) + }.toList() + + assertEquals(List(3) { listOf(1, null, null, 2)}.flatten(), list) + } + + @Test + fun testContext() = runTest { + val captured = ArrayList() + val flow = flowOf(1) + .flowOn(NamedDispatchers("irrelevant")) + .flatMap { + captured += NamedDispatchers.name() + flow { + captured += NamedDispatchers.name() + emit(it) + } + } + + flow.flowOn(NamedDispatchers("1")).sum() + flow.flowOn(NamedDispatchers("2")).sum() + assertEquals(listOf("1", "1", "2", "2"), captured) + } + + @Test + fun testIsolatedContext() = runTest { + val flow = flowOf(1) + .flowOn(NamedDispatchers("irrelevant")) + .flatMap { + flow { + assertEquals("inner", NamedDispatchers.name()) + emit(it) + } + }.flowOn(NamedDispatchers("inner")) + .flatMap { + flow { + assertEquals("outer", NamedDispatchers.name()) + emit(it) + } + }.flowOn(NamedDispatchers("outer")) + + assertEquals(1, flow.singleOrNull()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlatMapConcatTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapConcatTest.kt new file mode 100644 index 0000000000..59ea47ce05 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapConcatTest.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlin.test.* + +class FlatMapConcatTest : FlatMapBaseTest() { + + override fun Flow.flatMap(mapper: suspend (T) -> Flow): Flow = flatMapConcat(transform = mapper) + + @Test + fun testFlatMapConcurrency() = runTest { + var concurrentRequests = 0 + val flow = (1..100).asFlow().flatMapConcat { value -> + flow { + ++concurrentRequests + emit(value) + delay(Long.MAX_VALUE) + } + } + + val consumer = launch { + flow.collect { value -> + expect(value) + } + } + + repeat(4) { + yield() + } + + assertEquals(1, concurrentRequests) + consumer.cancelAndJoin() + finish(2) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlatMapLatestTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapLatestTest.kt new file mode 100644 index 0000000000..b98e97d442 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapLatestTest.kt @@ -0,0 +1,134 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class FlatMapLatestTest : TestBase() { + + @Test + fun testFlatMapLatest() = runTest { + val flow = flowOf(1, 2, 3).flatMapLatest { value -> + flowOf(value, value + 1) + } + assertEquals(listOf(1, 2, 2, 3, 3, 4), flow.toList()) + } + + @Test + fun testEmission() = runTest { + val list = flow { + repeat(5) { + emit(it) + } + }.flatMapLatest { flowOf(it) }.toList() + assertEquals(listOf(0, 1, 2, 3, 4), list) + } + + @Test + fun testSwitchIntuitiveBehaviour() = runTest { + val flow = flowOf(1, 2, 3, 4, 5) + flow.flatMapLatest { + flow { + expect(it) + emit(it) + yield() // Explicit cancellation check + if (it != 5) expectUnreached() + else expect(6) + } + }.collect() + finish(7) + } + + @Test + fun testSwitchRendevouzBuffer() = runTest { + val flow = flowOf(1, 2, 3, 4, 5) + flow.flatMapLatest { + flow { + emit(it) + // Reach here every uneven element because of channel's unfairness + expect(it) + } + }.buffer(0).onEach { expect(it + 1) } + .collect() + finish(7) + } + + @Test + fun testHangFlows() = runTest { + val flow = listOf(1, 2, 3, 4).asFlow() + val result = flow.flatMapLatest { value -> + flow { + if (value != 4) hang { expect(value) } + emit(42) + } + }.toList() + + assertEquals(listOf(42), result) + finish(4) + } + + @Test + fun testEmptyFlow() = runTest { + assertNull(emptyFlow().flatMapLatest { flowOf(1) }.singleOrNull()) + } + + @Test + fun testFailureInTransform() = runTest { + val flow = flowOf(1, 2).flatMapLatest { value -> + flow { + if (value == 1) { + emit(1) + hang { expect(1) } + } else { + expect(2) + throw TestException() + } + } + } + assertFailsWith(flow) + finish(3) + } + + @Test + fun testFailureDownstream() = runTest { + val flow = flowOf(1).flatMapLatest { value -> + flow { + expect(1) + emit(value) + expect(2) + hang { expect(4) } + } + }.flowOn(NamedDispatchers("downstream")).onEach { + expect(3) + throw TestException() + } + assertFailsWith(flow) + finish(5) + } + + @Test + fun testFailureUpstream() = runTest { + val flow = flow { + expect(1) + emit(1) + yield() + expect(3) + throw TestException() + }.flatMapLatest { + flow { + expect(2) + hang { + expect(4) + } + } + } + assertFailsWith(flow) + finish(5) + } + + @Test + fun testTake() = runTest { + val flow = flowOf(1, 2, 3, 4, 5).flatMapLatest { flowOf(it) } + assertEquals(listOf(1), flow.take(1).toList()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlatMapMergeBaseTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapMergeBaseTest.kt new file mode 100644 index 0000000000..0cd1e0ffac --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapMergeBaseTest.kt @@ -0,0 +1,90 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.testing.* +import kotlin.test.* +import kotlin.test.assertFailsWith + +abstract class FlatMapMergeBaseTest : FlatMapBaseTest() { + @Test + fun testFailureCancellation() = runTest { + val flow = flow { + expect(2) + emit(1) + expect(3) + emit(2) + expect(4) + }.flatMap { + if (it == 1) flow { + hang { expect(6) } + } else flow { + expect(5) + throw TestException() + } + } + + expect(1) + assertFailsWith { flow.singleOrNull() } + finish(7) + } + + @Test + fun testConcurrentFailure() = runTest { + val latch = Channel() + val flow = flow { + expect(2) + emit(1) + expect(3) + emit(2) + }.flatMap { + if (it == 1) flow { + expect(5) + latch.send(Unit) + hang { + expect(7) + throw TestException2() + + } + } else { + expect(4) + latch.receive() + expect(6) + throw TestException() + } + } + + expect(1) + assertFailsWith(flow) + finish(8) + } + + @Test + fun testFailureInMapOperationCancellation() = runTest { + val latch = Channel() + val flow = flow { + expect(2) + emit(1) + expect(3) + emit(2) + expectUnreached() + }.flatMap { + if (it == 1) flow { + expect(5) + latch.send(Unit) + hang { expect(7) } + } else { + expect(4) + latch.receive() + expect(6) + throw TestException() + } + } + + expect(1) + assertFailsWith { flow.count() } + finish(8) + } + + @Test + abstract fun testFlatMapConcurrency(): TestResult +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlatMapMergeFastPathTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapMergeFastPathTest.kt new file mode 100644 index 0000000000..c8a41247fb --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapMergeFastPathTest.kt @@ -0,0 +1,85 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import kotlin.test.* +import kotlin.test.assertFailsWith + +class FlatMapMergeFastPathTest : FlatMapMergeBaseTest() { + + override fun Flow.flatMap(mapper: suspend (T) -> Flow): Flow = flatMapMerge(transform = mapper).buffer(64) + + @Test + override fun testFlatMapConcurrency() = runTest { + var concurrentRequests = 0 + val flow = (1..100).asFlow().flatMapMerge(concurrency = 2) { value -> + flow { + ++concurrentRequests + emit(value) + delay(Long.MAX_VALUE) + } + }.buffer(64) + + val consumer = launch { + flow.collect { value -> + expect(value) + } + } + + repeat(4) { + yield() + } + + assertEquals(2, concurrentRequests) + consumer.cancelAndJoin() + finish(3) + } + + @Test + fun testCancellationExceptionDownstream() = runTest { + val flow = flowOf(1, 2, 3).flatMapMerge { + flow { + emit(it) + throw CancellationException("") + } + }.buffer(64) + + assertEquals(listOf(1, 2, 3), flow.toList()) + } + + @Test + fun testCancellationExceptionUpstream() = runTest { + val flow = flow { + expect(1) + emit(1) + expect(2) + yield() + throw CancellationException("") + }.flatMapMerge { + flow { + expect(3) + emit(it) + hang { expect(4) } + } + }.buffer(64) + + assertFailsWith(flow) + finish(5) + } + + @Test + fun testCancellation() = runTest { + val result = flow { + emit(1) + emit(2) + emit(3) + emit(4) + expectUnreached() // Cancelled by take + emit(5) + }.flatMapMerge(2) { v -> flow { emit(v) } } + .buffer(64) + .take(2) + .toList() + assertEquals(listOf(1, 2), result) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlatMapMergeTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapMergeTest.kt new file mode 100644 index 0000000000..e3b4ba2e49 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlatMapMergeTest.kt @@ -0,0 +1,113 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class FlatMapMergeTest : FlatMapMergeBaseTest() { + + override fun Flow.flatMap(mapper: suspend (T) -> Flow): Flow = flatMapMerge(transform = mapper) + + @Test + override fun testFlatMapConcurrency() = runTest { + var concurrentRequests = 0 + val flow = (1..100).asFlow().flatMapMerge(concurrency = 2) { value -> + flow { + ++concurrentRequests + emit(value) + delay(Long.MAX_VALUE) + } + } + + val consumer = launch { + flow.collect { value -> + expect(value) + } + } + + repeat(4) { + yield() + } + + assertEquals(2, concurrentRequests) + consumer.cancelAndJoin() + finish(3) + } + + @Test + fun testAtomicStart() = runTest { + try { + coroutineScope { + val job = coroutineContext[Job]!! + val flow = flow { + expect(3) + emit(1) + } + .onCompletion { expect(5) } + .flatMapMerge { + expect(4) + flowOf(it).onCompletion { expectUnreached() } } + .onCompletion { expect(6) } + + launch { + expect(1) + flow.collect() + } + launch { + expect(2) + yield() + job.cancel() + } + } + } catch (e: CancellationException) { + finish(7) + } + } + + @Test + fun testCancellationExceptionDownstream() = runTest { + val flow = flowOf(1, 2, 3).flatMapMerge { + flow { + emit(it) + throw CancellationException("") + } + } + + assertEquals(listOf(1, 2, 3), flow.toList()) + } + + @Test + fun testCancellationExceptionUpstream() = runTest { + val flow = flow { + expect(1) + emit(1) + expect(2) + yield() + throw CancellationException("") + }.flatMapMerge { + flow { + expect(3) + emit(it) + hang { expect(4) } + } + } + + assertFailsWith(flow) + finish(5) + } + + @Test + fun testCancellation() = runTest { + val result = flow { + emit(1) + emit(2) + emit(3) + emit(4) + expectUnreached() // Cancelled by take + emit(5) + }.flatMapMerge(2) { v -> flow { emit(v) } } + .take(2) + .toList() + assertEquals(listOf(1, 2), result) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlattenConcatTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlattenConcatTest.kt new file mode 100644 index 0000000000..31741b44d5 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlattenConcatTest.kt @@ -0,0 +1,49 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class FlattenConcatTest : FlatMapBaseTest() { + + override fun Flow.flatMap(mapper: suspend (T) -> Flow): Flow = map(mapper).flattenConcat() + + @Test + fun testFlatMapConcurrency() = runTest { + var concurrentRequests = 0 + val flow = (1..100).asFlow().map { value -> + flow { + ++concurrentRequests + emit(value) + delay(Long.MAX_VALUE) + } + }.flattenConcat() + + val consumer = launch { + flow.collect { value -> + expect(value) + } + } + + repeat(4) { + yield() + } + + assertEquals(1, concurrentRequests) + consumer.cancelAndJoin() + finish(2) + } + + @Test + fun testCancellation() = runTest { + val flow = flow { + repeat(5) { + emit(flow { + if (it == 2) throw CancellationException("") + emit(1) + }) + } + } + assertFailsWith(flow.flattenConcat()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlattenMergeTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlattenMergeTest.kt new file mode 100644 index 0000000000..ee3804f382 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlattenMergeTest.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.* +import kotlin.test.* + +class FlattenMergeTest : FlatMapMergeBaseTest() { + + override fun Flow.flatMap(mapper: suspend (T) -> Flow): Flow = map(mapper).flattenMerge() + + @Test + override fun testFlatMapConcurrency() = runTest { + var concurrentRequests = 0 + val flow = (1..100).asFlow().map { value -> + flow { + ++concurrentRequests + emit(value) + delay(Long.MAX_VALUE) + } + }.flattenMerge(concurrency = 2) + + val consumer = launch { + flow.collect { value -> + expect(value) + } + } + + repeat(4) { + yield() + } + + assertEquals(2, concurrentRequests) + consumer.cancelAndJoin() + finish(3) + } + + @Test + fun testContextPreservationAcrossFlows() = runTest { + val result = flow { + flowOf(1, 2).flatMapMerge { + flow { + yield() + emit(it) + } + }.collect { + emit(it) + } + }.toList() + assertEquals(listOf(1, 2), result) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlowContextOptimizationsTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlowContextOptimizationsTest.kt new file mode 100644 index 0000000000..32022c4c14 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlowContextOptimizationsTest.kt @@ -0,0 +1,113 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.test.* +import kotlin.coroutines.coroutineContext as currentContext + +class FlowContextOptimizationsTest : TestBase() { + @Test + fun testBaseline() = runTest { + val flowDispatcher = wrapperDispatcher(currentContext) + val collectContext = currentContext + flow { + assertSame(flowDispatcher, currentContext[ContinuationInterceptor] as CoroutineContext) + expect(1) + emit(1) + expect(2) + emit(2) + expect(3) + } + .flowOn(flowDispatcher) + .collect { value -> + assertEquals(collectContext.minusKey(Job), currentContext.minusKey(Job)) + if (value == 1) expect(4) + else expect(5) + } + + finish(6) + } + + @Test + fun testFusedSameContext() = runTest { + flow { + expect(1) + emit(1) + expect(3) + emit(2) + expect(5) + } + .flowOn(currentContext.minusKey(Job)) + .collect { value -> + if (value == 1) expect(2) + else expect(4) + } + finish(6) + } + + @Test + fun testFusedSameContextWithIntermediateOperators() = runTest { + flow { + expect(1) + emit(1) + expect(3) + emit(2) + expect(5) + } + .flowOn(currentContext.minusKey(Job)) + .map { it } + .flowOn(currentContext.minusKey(Job)) + .collect { value -> + if (value == 1) expect(2) + else expect(4) + } + finish(6) + } + + @Test + fun testFusedSameDispatcher() = runTest { + flow { + assertEquals("Name", currentContext[CoroutineName]?.name) + expect(1) + emit(1) + expect(3) + emit(2) + expect(5) + } + .flowOn(CoroutineName("Name")) + .collect { value -> + assertNull(currentContext[CoroutineName]?.name) + if (value == 1) expect(2) + else expect(4) + } + finish(6) + } + + @Test + fun testFusedManySameDispatcher() = runTest { + flow { + assertEquals("Name1", currentContext[CoroutineName]?.name) + assertEquals("OK", currentContext[CustomContextElement]?.str) + expect(1) + emit(1) + expect(3) + emit(2) + expect(5) + } + .flowOn(CoroutineName("Name1")) // the first one works + .flowOn(CoroutineName("Name2")) + .flowOn(CoroutineName("Name3") + CustomContextElement("OK")) // but this is not lost + .collect { value -> + assertNull(currentContext[CoroutineName]?.name) + assertNull(currentContext[CustomContextElement]?.str) + if (value == 1) expect(2) + else expect(4) + } + finish(6) + } + + data class CustomContextElement(val str: String) : AbstractCoroutineContextElement(Key) { + companion object Key : CoroutineContext.Key + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/FlowOnTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/FlowOnTest.kt new file mode 100644 index 0000000000..bb0fabb867 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/FlowOnTest.kt @@ -0,0 +1,358 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.testing.flow.* +import kotlin.test.* + +class FlowOnTest : TestBase() { + + @Test + fun testFlowOn() = runTest { + val source = Source(42) + val consumer = Consumer(42) + + val flow = source::produce.asFlow() + flow.flowOn(NamedDispatchers("ctx1")).launchIn(this) { + onEach { consumer.consume(it) } + }.join() + + assertEquals("ctx1", source.contextName) + assertEquals("main", consumer.contextName) + + flow.flowOn(NamedDispatchers("ctx2")).launchIn(this) { + onEach { consumer.consume(it) } + }.join() + + assertEquals("ctx2", source.contextName) + assertEquals("main", consumer.contextName) + } + + @Test + fun testFlowOnAndOperators() = runTest { + val source = Source(42) + val consumer = Consumer(42) + val captured = ArrayList() + val mapper: suspend (Int) -> Int = { + captured += NamedDispatchers.nameOr("main") + it + } + + val flow = source::produce.asFlow() + flow.map(mapper) + .flowOn(NamedDispatchers("ctx1")) + .map(mapper) + .flowOn(NamedDispatchers("ctx2")) + .map(mapper) + .launchIn(this) { + onEach { consumer.consume(it) } + }.join() + + assertEquals(listOf("ctx1", "ctx2", "main"), captured) + assertEquals("ctx1", source.contextName) + assertEquals("main", consumer.contextName) + } + + @Test + public fun testFlowOnThrowingSource() = runTest { + val flow = flow { + expect(1) + emit(NamedDispatchers.name()) + expect(3) + throw TestException() + }.map { + expect(2) + assertEquals("throwing", it) + it + }.flowOn(NamedDispatchers("throwing")) + + assertFailsWith { flow.single() } + ensureActive() + finish(4) + } + + @Test + public fun testFlowOnThrowingOperator() = runTest { + val flow = flow { + expect(1) + emit(NamedDispatchers.name()) + expectUnreached() + }.map { + expect(2) + assertEquals("throwing", it) + throw TestException() + }.flowOn(NamedDispatchers("throwing")) + + assertFailsWith(flow) + ensureActive() + finish(3) + } + + @Test + public fun testFlowOnDownstreamOperator() = runTest() { + val flow = flow { + expect(2) + emit(NamedDispatchers.name()) + hang { expect(5) } + delay(Long.MAX_VALUE) + }.map { + expect(3) + it + }.flowOn(NamedDispatchers("throwing")) + .map { + expect(4); + throw TestException() + } + + expect(1) + assertFailsWith { flow.single() } + ensureActive() + finish(6) + } + + @Test + public fun testFlowOnThrowingConsumer() = runTest { + val flow = flow { + expect(2) + emit(NamedDispatchers.name()) + hang { expect(4) } + } + + expect(1) + flow.flowOn(NamedDispatchers("...")).launchIn(this + NamedDispatchers("launch")) { + onEach { + expect(3) + throw TestException() + } + catch { expect(5) } + }.join() + + ensureActive() + finish(6) + } + + @Test + fun testFlowOnWithJob() = runTest({ it is IllegalArgumentException }) { + flow { + emit(1) + }.flowOn(NamedDispatchers("foo") + Job()) + } + + @Test + fun testFlowOnCancellation() = runTest { + val latch = Channel() + expect(1) + val job = launch(NamedDispatchers("launch")) { + flow { + expect(2) + latch.send(Unit) + expect(3) + hang { + assertEquals("cancelled", NamedDispatchers.name()) + expect(5) + } + }.flowOn(NamedDispatchers("cancelled")).single() + } + + latch.receive() + expect(4) + job.cancel() + job.join() + ensureActive() + finish(6) + } + + @Test + fun testFlowOnCancellationHappensBefore() = runTest { + launch { + try { + flow { + expect(1) + val flowJob = kotlin.coroutines.coroutineContext[Job]!! + launch { + expect(2) + flowJob.cancel() + } + hang { expect(3) } + }.flowOn(NamedDispatchers("upstream")).single() + } catch (e: CancellationException) { + expect(4) + } + }.join() + ensureActive() + finish(5) + } + + @Test + fun testIndependentOperatorContext() = runTest { + val value = flow { + assertEquals("base", NamedDispatchers.nameOr("main")) + expect(1) + emit(-239) + }.map { + assertEquals("base", NamedDispatchers.nameOr("main")) + expect(2) + it + }.flowOn(NamedDispatchers("base")) + .map { + assertEquals("main", NamedDispatchers.nameOr("main")) + expect(3) + it + }.single() + + assertEquals(-239, value) + finish(4) + } + + @Test + fun testMultipleFlowOn() = runTest { + flow { + assertEquals("ctx1", NamedDispatchers.nameOr("main")) + expect(1) + emit(1) + }.map { + assertEquals("ctx1", NamedDispatchers.nameOr("main")) + expect(2) + }.flowOn(NamedDispatchers("ctx1")) + .map { + assertEquals("ctx2", NamedDispatchers.nameOr("main")) + expect(3) + }.flowOn(NamedDispatchers("ctx2")) + .map { + assertEquals("ctx3", NamedDispatchers.nameOr("main")) + expect(4) + }.flowOn(NamedDispatchers("ctx3")) + .map { + assertEquals("main", NamedDispatchers.nameOr("main")) + expect(5) + } + .single() + + finish(6) + } + + @Test + fun testTimeoutExceptionUpstream() = runTest { + val flow = flow { + emit(1) + yield() + withTimeout(-1) {} + emit(42) + }.flowOn(NamedDispatchers("foo")).onEach { + expect(1) + } + assertFailsWith(flow) + finish(2) + } + + @Test + fun testTimeoutExceptionDownstream() = runTest { + val flow = flow { + emit(1) + hang { expect(2) } + }.flowOn(NamedDispatchers("foo")).onEach { + expect(1) + withTimeout(-1) {} + } + assertFailsWith(flow) + finish(3) + } + + @Test + fun testCancellation() = runTest { + val result = flow { + emit(1) + emit(2) + emit(3) + expectUnreached() + emit(4) + }.flowOn(wrapperDispatcher()) + .buffer(0) + .take(2) + .toList() + assertEquals(listOf(1, 2), result) + } + + @Test + fun testAtomicStart() = runTest { + try { + coroutineScope { + val job = coroutineContext[Job]!! + val flow = flow { + expect(3) + emit(1) + } + .onCompletion { expect(4) } + .flowOn(wrapperDispatcher()) + .onCompletion { expect(5) } + + launch { + expect(1) + flow.collect() + } + launch { + expect(2) + job.cancel() + } + } + } catch (e: CancellationException) { + finish(6) + } + } + + @Test + fun testException() = runTest { + val flow = flow { + emit(314) + delay(Long.MAX_VALUE) + }.flowOn(NamedDispatchers("upstream")) + .map { + throw TestException() + } + + assertFailsWith { flow.single() } + assertFailsWith(flow) + ensureActive() + } + + @Test + fun testIllegalArgumentException() { + val flow = emptyFlow() + assertFailsWith { flow.flowOn(Job()) } + } + + private inner class Source(private val value: Int) { + public var contextName: String = "unknown" + + fun produce(): Int { + contextName = NamedDispatchers.nameOr("main") + return value + } + } + + private inner class Consumer(private val expected: Int) { + public var contextName: String = "unknown" + + fun consume(value: Int) { + contextName = NamedDispatchers.nameOr("main") + assertEquals(expected, value) + } + } + + @Test + fun testCancelledFlowOn() = runTest { + assertFailsWith { + coroutineScope { + val scope = this + flow { + emit(Unit) // emit to buffer + scope.cancel() // now cancel outer scope + }.flowOn(wrapperDispatcher()).collect { + // should not be reached, because cancelled before it runs + expectUnreached() + } + } + } + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/IndexedTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/IndexedTest.kt new file mode 100644 index 0000000000..420bfc8f79 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/IndexedTest.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class IndexedTest : TestBase() { + + @Test + fun testWithIndex() = runTest { + val flow = flowOf(3, 2, 1).withIndex() + assertEquals(listOf(IndexedValue(0, 3), IndexedValue(1, 2), IndexedValue(2, 1)), flow.toList()) + } + + @Test + fun testWithIndexEmpty() = runTest { + val flow = emptyFlow().withIndex() + assertEquals(emptyList(), flow.toList()) + } + + @Test + fun testCollectIndexed() = runTest { + val result = ArrayList>() + flowOf(3L, 2L, 1L).collectIndexed { index, value -> + result.add(IndexedValue(index, value)) + } + assertEquals(listOf(IndexedValue(0, 3L), IndexedValue(1, 2L), IndexedValue(2, 1L)), result) + } + + @Test + fun testCollectIndexedEmptyFlow() = runTest { + val flow = flow { + expect(1) + } + + flow.collectIndexed { _, _ -> + expectUnreached() + } + + finish(2) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/LintTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/LintTest.kt new file mode 100644 index 0000000000..7fc2282dab --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/LintTest.kt @@ -0,0 +1,27 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class LintTest: TestBase() { + /** + * Tests that using [SharedFlow.toList] and similar functions by passing a mutable collection does add values + * to the provided collection. + */ + @Test + fun testSharedFlowToCollection() = runTest { + val sharedFlow = MutableSharedFlow() + val list = mutableListOf() + val set = mutableSetOf() + val jobs = listOf(suspend { sharedFlow.toList(list) }, { sharedFlow.toSet(set) }).map { + launch(Dispatchers.Unconfined) { it() } + } + repeat(10) { + sharedFlow.emit(it) + } + jobs.forEach { it.cancelAndJoin() } + assertEquals((0..9).toList(), list) + assertEquals((0..9).toSet(), set) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/MapNotNullTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/MapNotNullTest.kt new file mode 100644 index 0000000000..f2aa3cca98 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/MapNotNullTest.kt @@ -0,0 +1,47 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class MapNotNullTest : TestBase() { + @Test + fun testMap() = runTest { + val flow = flow { + emit(1) + emit(null) + emit(2) + } + + val result = flow.mapNotNull { it }.sum() + assertEquals(3, result) + } + + @Test + fun testEmptyFlow() = runTest { + val sum = emptyFlow().mapNotNull { expectUnreached(); it }.sum() + assertEquals(0, sum) + } + + @Test + fun testErrorCancelsUpstream() = runTest { + var cancelled = false + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang { cancelled = true } + } + emit(1) + } + }.mapNotNull { + latch.receive() + throw TestException() + }.catch { emit(42) } + + assertEquals(42, flow.single()) + assertTrue(cancelled) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/MapTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/MapTest.kt new file mode 100644 index 0000000000..11d1d4da0d --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/MapTest.kt @@ -0,0 +1,47 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class MapTest : TestBase() { + @Test + fun testMap() = runTest { + val flow = flow { + emit(1) + emit(2) + } + + val result = flow.map { it + 1 }.sum() + assertEquals(5, result) + } + + @Test + fun testEmptyFlow() = runTest { + val sum = emptyFlow().map { expectUnreached(); it }.sum() + assertEquals(0, sum) + } + + @Test + fun testErrorCancelsUpstream() = runTest { + var cancelled = false + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang { cancelled = true } + } + emit(1) + expectUnreached() + } + }.map { + latch.receive() + throw TestException() + }.catch { emit(42) } + + assertEquals(42, flow.single()) + assertTrue(cancelled) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/MergeTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/MergeTest.kt new file mode 100644 index 0000000000..694db2511c --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/MergeTest.kt @@ -0,0 +1,123 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* +import kotlinx.coroutines.flow.merge as originalMerge + +abstract class MergeTest : TestBase() { + + abstract fun Iterable>.merge(): Flow + + @Test + fun testMerge() = runTest { + val n = 100 + val sum = (1..n).map { flowOf(it) } + .merge() + .sum() + + assertEquals(n * (n + 1) / 2, sum) + } + + @Test + fun testSingle() = runTest { + val flow = listOf(flowOf(), flowOf(42), flowOf()).merge() + val value = flow.single() + assertEquals(42, value) + } + + @Test + fun testNulls() = runTest { + val list = listOf(flowOf(1), flowOf(null), flowOf(2)).merge().toList() + assertEquals(listOf(1, null, 2), list) + } + + @Test + fun testContext() = runTest { + val flow = flow { + emit(NamedDispatchers.name()) + }.flowOn(NamedDispatchers("source")) + + val result = listOf(flow).merge().flowOn(NamedDispatchers("irrelevant")).toList() + assertEquals(listOf("source"), result) + } + + @Test + fun testOneSourceCancelled() = runTest { + val flow = flow { + expect(1) + emit(1) + expect(2) + yield() + throw CancellationException("") + } + + val otherFlow = flow { + repeat(5) { + emit(1) + yield() + } + + expect(3) + } + + val result = listOf(flow, otherFlow).merge().toList() + assertEquals(MutableList(6) { 1 }, result) + finish(4) + } + + @Test + fun testOneSourceCancelledNonFused() = runTest { + val flow = flow { + expect(1) + emit(1) + expect(2) + yield() + throw CancellationException("") + } + + val otherFlow = flow { + repeat(5) { + emit(1) + yield() + } + + expect(3) + } + + val result = listOf(flow, otherFlow).nonFuseableMerge().toList() + assertEquals(MutableList(6) { 1 }, result) + finish(4) + } + + private fun Iterable>.nonFuseableMerge(): Flow { + return channelFlow { + forEach { flow -> + launch { + flow.collect { send(it) } + } + } + } + } + + @Test + fun testIsolatedContext() = runTest { + val flow = flow { + emit(NamedDispatchers.name()) + } + + val result = listOf(flow.flowOn(NamedDispatchers("1")), flow.flowOn(NamedDispatchers("2"))) + .merge() + .flowOn(NamedDispatchers("irrelevant")) + .toList() + assertEquals(listOf("1", "2"), result) + } +} + +class IterableMergeTest : MergeTest() { + override fun Iterable>.merge(): Flow = originalMerge() +} + +class VarargMergeTest : MergeTest() { + override fun Iterable>.merge(): Flow = originalMerge(*toList().toTypedArray()) +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/OnCompletionTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/OnCompletionTest.kt new file mode 100644 index 0000000000..14ee53164d --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/OnCompletionTest.kt @@ -0,0 +1,397 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.internal.* +import kotlin.test.* + +class OnCompletionTest : TestBase() { + + @Test + fun testOnCompletion() = runTest { + flow { + expect(1) + emit(2) + expect(4) + }.onEach { + expect(2) + }.onCompletion { + assertNull(it) + expect(5) + }.onEach { + expect(3) + }.collect() + finish(6) + } + + @Test + fun testOnCompletionWithException() = runTest { + flowOf(1).onEach { + expect(1) + throw TestException() + }.onCompletion { + assertIs(it) + expect(2) + }.catch { + assertIs(it) + expect(3) + }.collect() + finish(4) + } + + @Test + fun testOnCompletionWithExceptionDownstream() = runTest { + flow { + expect(1) + emit(2) + }.onEach { + expect(2) + }.onCompletion { + assertIs(it) // flow fails because of this exception + expect(4) + }.onEach { + expect(3) + throw TestException() + }.catch { + assertIs(it) + expect(5) + }.collect() + finish(6) + } + + @Test + fun testMultipleOnCompletions() = runTest { + flowOf(1).onCompletion { + assertIs(it) + expect(2) + }.onEach { + expect(1) + throw TestException() + }.onCompletion { + assertIs(it) + expect(3) + }.catch { + assertIs(it) + expect(4) + }.collect() + finish(5) + } + + @Test + fun testExceptionFromOnCompletion() = runTest { + flowOf(1).onEach { + expect(1) + throw TestException() + }.onCompletion { + expect(2) + throw TestException2() + }.catch { + assertIs(it) + expect(3) + }.collect() + finish(4) + } + + @Test + fun testContextPreservation() = runTest { + flowOf(1).onCompletion { + assertEquals("OK", NamedDispatchers.name()) + assertNull(it) + expect(1) + }.flowOn(NamedDispatchers("OK")) + .onEach { + expect(2) + assertEquals("default", NamedDispatchers.nameOr("default")) + throw TestException() + } + .catch { + assertIs(it) + expect(3) + }.collect() + finish(4) + } + + @Test + fun testEmitExample() = runTest { + val flow = flowOf("a", "b", "c") + .onCompletion() { emit("Done") } + assertEquals(listOf("a", "b", "c", "Done"), flow.toList()) + } + + sealed class TestData { + data class Value(val i: Int) : TestData() + data class Done(val e: Throwable?) : TestData() { + override fun equals(other: Any?): Boolean = + other is Done && other.e?.message == e?.message + } + } + + @Test + fun testCrashedEmit() = runTest { + expect(1) + val collected = ArrayList() + assertFailsWith { + (1..10).asFlow() + .map { TestData.Value(it) } + .onEach { value -> + value as TestData.Value + expect(value.i + 1) + if (value.i == 6) throw TestException("OK") + yield() + } + .onCompletion { e -> + expect(8) + assertIs(e) + emit(TestData.Done(e)) // will fail + }.collect { + collected += it + } + } + val expected: List = (1..5).map { TestData.Value(it) } + assertEquals(expected, collected) + finish(9) + } + + @Test + fun testCancelledEmit() = runTest { + expect(1) + val collected = ArrayList() + assertFailsWith { + coroutineScope { + (1..10).asFlow() + .map { TestData.Value(it) } + .onEach { value -> + value as TestData.Value + expect(value.i + 1) + if (value.i == 6) coroutineContext.cancel() + yield() + } + .onCompletion { e -> + expect(8) + assertIs(e) + try { + emit(TestData.Done(e)) + expectUnreached() + } finally { + expect(9) + } + }.collect { + collected += it + } + } + } + val expected = (1..5).map { TestData.Value(it) } + assertEquals(expected, collected) + finish(10) + } + + @Test + fun testFailedEmit() = runTest { + val cause = TestException() + assertFailsWith { + flow { + expect(1) + emit(TestData.Value(2)) + expectUnreached() + }.onCompletion { + assertSame(cause, it) // flow failed because of the exception in downstream + expect(3) + try { + emit(TestData.Done(it)) + expectUnreached() + } catch (e: TestException) { + assertSame(cause, e) + finish(4) + } + }.collect { + expect((it as TestData.Value).i) + throw cause + } + } + } + + @Test + fun testFirst() = runTest { + val value = flowOf(239).onCompletion { + assertNotNull(it) // the flow did not complete normally + expect(1) + try { + emit(42) + expectUnreached() + } catch (e: Throwable) { + assertTrue { e is AbortFlowException } + } + }.first() + assertEquals(239, value) + finish(2) + } + + @Test + fun testSingle() = runTest { + assertFailsWith { + flowOf(239).onCompletion { + assertNull(it) + expect(1) + try { + emit(42) + expectUnreached() + } catch (e: Throwable) { + // Second emit -- failure + assertTrue { e is IllegalArgumentException } + throw e + } + }.single() + expectUnreached() + } + finish(2) + } + + @Test + fun testEmptySingleInterference() = runTest { + val value = emptyFlow().onCompletion { + assertNull(it) + expect(1) + emit(42) + }.single() + assertEquals(42, value) + finish(2) + } + + @Test + fun testTransparencyViolation() = runTest { + val flow = emptyFlow().onCompletion { + expect(2) + coroutineScope { + launch { + try { + emit(1) + } catch (e: IllegalStateException) { + expect(3) + } + } + } + } + expect(1) + assertNull(flow.singleOrNull()) + finish(4) + } + + @Test + fun testTakeOnCompletion() = runTest { + // even though it uses "take" from the outside it completes normally + val flow = (1..10).asFlow().take(5) + val result = flow.onCompletion { cause -> + assertNull(cause) + emit(-1) + }.toList() + val expected = (1..5).toList() + (-1) + assertEquals(expected, result) + } + + @Test + fun testCancelledEmitAllFlow() = runTest { + // emitAll does not call 'collect' on onCompletion collector + // if the target flow is empty + flowOf(1, 2, 3) + .onCompletion { emitAll(MutableSharedFlow()) } + .take(1) + .collect() + } + + @Test + fun testCancelledEmitAllChannel() = runTest { + // emitAll does not call 'collect' on onCompletion collector + // if the target channel is empty + flowOf(1, 2, 3) + .onCompletion { emitAll(Channel()) } + .take(1) + .collect() + } + + /** + * Tests that the operators that are used to limit the flow (like [take] and [zip]) faithfully propagate the + * cancellation exception to the original owner. + */ + @Test + fun testOnCompletionBetweenLimitingOperators() = runTest { + // `zip` doesn't eat the exception thrown by `take`: + flowOf(1, 2, 3) + .zip(flowOf(4, 5)) { a, b -> a + b } + .onCompletion { + expect(2) + assertNotNull(it) + } + .take(1) + .collect { + expect(1) + } + + // `take` doesn't eat the exception thrown by `zip`: + flowOf(1, 2, 3) + .take(2) + .onCompletion { + expect(4) + assertNotNull(it) + } + .zip(flowOf(4)) { a, b -> a + b } + .collect { + expect(3) + } + + // `take` doesn't eat the exception thrown by `first`: + flowOf(1, 2, 3) + .take(2) + .onCompletion { + expect(5) + assertNotNull(it) + } + .first() + + // `zip` doesn't eat the exception thrown by `first`: + flowOf(1, 2, 3) + .zip(flowOf(4, 5)) { a, b -> a + b } + .onCompletion { + expect(6) + assertNotNull(it) + } + .first() + + // `take` doesn't eat the exception thrown by another `take`: + flowOf(1, 2, 3) + .take(2) + .onCompletion { + expect(8) + assertNotNull(it) + } + .take(1) + .collect { + expect(7) + } + + // `zip` doesn't eat the exception thrown by another `zip`: + flowOf(1, 2, 3) + .zip(flowOf(4, 5)) { a, b -> a + b } + .onCompletion { + expect(10) + assertNotNull(it) + } + .zip(flowOf(6)) { a, b -> a + b } + .collect { + expect(9) + } + + finish(11) + } + + /** + * Tests that emitting new elements after completion doesn't overwrite the old elements. + */ + @Test + fun testEmittingElementsAfterCancellation() = runTest { + assertEquals(1, flowOf(1, 2, 3) + .take(100) + .onCompletion { emit(4) } + .first()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/OnEachTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/OnEachTest.kt new file mode 100644 index 0000000000..9698b28fcc --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/OnEachTest.kt @@ -0,0 +1,47 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class OnEachTest : TestBase() { + @Test + fun testOnEach() = runTest { + val flow = flow { + emit(1) + emit(2) + } + + val result = flow.onEach { expect(it) }.sum() + assertEquals(3, result) + finish(3) + } + + @Test + fun testEmptyFlow() = runTest { + val value = emptyFlow().onEach { fail() }.singleOrNull() + assertNull(value) + } + + @Test + fun testErrorCancelsUpstream() = runTest { + var cancelled = false + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang { cancelled = true } + } + emit(1) + } + }.onEach { + latch.receive() + throw TestException() + }.catch { emit(42) } + + assertEquals(42, flow.single()) + assertTrue(cancelled) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/OnEmptyTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/OnEmptyTest.kt new file mode 100644 index 0000000000..c493818ea5 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/OnEmptyTest.kt @@ -0,0 +1,77 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class OnEmptyTest : TestBase() { + + @Test + fun testOnEmptyInvoked() = runTest { + val flow = emptyFlow().onEmpty { emit(1) } + assertEquals(1, flow.single()) + } + + @Test + fun testOnEmptyNotInvoked() = runTest { + val flow = flowOf(1).onEmpty { emit(2) } + assertEquals(1, flow.single()) + } + + @Test + fun testOnEmptyNotInvokedOnError() = runTest { + val flow = flow { + throw TestException() + }.onEmpty { expectUnreached() } + assertFailsWith(flow) + } + + @Test + fun testOnEmptyNotInvokedOnCancellation() = runTest { + val flow = flow { + expect(2) + hang { expect(4) } + }.onEmpty { expectUnreached() } + + expect(1) + val job = flow.onEach { expectUnreached() }.launchIn(this) + yield() + expect(3) + job.cancelAndJoin() + finish(5) + } + + @Test + fun testOnEmptyCancellation() = runTest { + val flow = emptyFlow().onEmpty { + expect(2) + hang { expect(4) } + emit(1) + } + expect(1) + val job = flow.onEach { expectUnreached() }.launchIn(this) + yield() + expect(3) + job.cancelAndJoin() + finish(5) + } + + @Test + fun testTransparencyViolation() = runTest { + val flow = emptyFlow().onEmpty { + expect(2) + coroutineScope { + launch { + try { + emit(1) + } catch (e: IllegalStateException) { + expect(3) + } + } + } + } + expect(1) + assertNull(flow.singleOrNull()) + finish(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/OnStartTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/OnStartTest.kt new file mode 100644 index 0000000000..15fc4b5249 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/OnStartTest.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class OnStartTest : TestBase() { + @Test + fun testEmitExample() = runTest { + val flow = flowOf("a", "b", "c") + .onStart { emit("Begin") } + assertEquals(listOf("Begin", "a", "b", "c"), flow.toList()) + } + + @Test + fun testTransparencyViolation() = runTest { + val flow = emptyFlow().onStart { + expect(2) + coroutineScope { + launch { + try { + emit(1) + } catch (e: IllegalStateException) { + expect(3) + } + } + } + } + expect(1) + assertNull(flow.singleOrNull()) + finish(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/RetryTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/RetryTest.kt new file mode 100644 index 0000000000..41377cba5d --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/RetryTest.kt @@ -0,0 +1,161 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class RetryTest : TestBase() { + @Test + fun testRetryWhen() = runTest { + expect(1) + val flow = flow { + emit(1) + throw TestException() + } + val sum = flow.retryWhen { cause, attempt -> + assertIs(cause) + expect(2 + attempt.toInt()) + attempt < 3 + }.catch { cause -> + expect(6) + assertIs(cause) + }.sum() + assertEquals(4, sum) + finish(7) + } + + @Test + fun testRetry() = runTest { + var counter = 0 + val flow = flow { + emit(1) + if (++counter < 4) throw TestException() + } + + assertEquals(4, flow.retry(4).sum()) + counter = 0 + assertFailsWith(flow) + counter = 0 + assertFailsWith(flow.retry(2)) + } + + @Test + fun testRetryPredicate() = runTest { + var counter = 0 + val flow = flow { + emit(1); + if (++counter == 1) throw TestException() + } + + assertEquals(2, flow.retry(1) { it is TestException }.sum()) + counter = 0 + assertFailsWith(flow.retry(1) { it !is TestException }) + } + + @Test + fun testRetryExceptionFromDownstream() = runTest { + var executed = 0 + val flow = flow { + emit(1) + }.retry(42).map { + ++executed + throw TestException() + } + + assertFailsWith(flow) + assertEquals(1, executed) + } + + @Test + fun testWithTimeoutRetried() = runTest { + var state = 0 + val flow = flow { + if (state++ == 0) { + expect(1) + withTimeout(1) { + hang { expect(2) } + } + expectUnreached() + } + expect(3) + emit(1) + }.retry(1) + + assertEquals(1, flow.single()) + finish(4) + } + + @Test + fun testCancellationFromUpstreamIsNotRetried() = runTest { + val flow = flow { + hang { } + }.retry() + + val job = launch { + expect(1) + flow.collect { } + } + + yield() + expect(2) + job.cancelAndJoin() + finish(3) + } + + @Test + fun testUpstreamExceptionConcurrentWithDownstream() = runTest { + val flow = flow { + try { + expect(1) + emit(1) + } finally { + expect(3) + throw TestException() + } + }.retry { expectUnreached(); true }.onEach { + expect(2) + throw TestException2() + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testUpstreamExceptionConcurrentWithDownstreamCancellation() = runTest { + val flow = flow { + try { + expect(1) + emit(1) + } finally { + expect(3) + throw TestException() + } + }.retry { expectUnreached(); true }.onEach { + expect(2) + throw CancellationException("") + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testUpstreamCancellationIsIgnoredWhenDownstreamFails() = runTest { + val flow = flow { + try { + expect(1) + emit(1) + } finally { + expect(3) + throw CancellationException("") + } + }.retry { expectUnreached(); true }.onEach { + expect(2) + throw TestException("") + } + + assertFailsWith(flow) + finish(4) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/SampleTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/SampleTest.kt new file mode 100644 index 0000000000..70c9d245b1 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/SampleTest.kt @@ -0,0 +1,301 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds + +class SampleTest : TestBase() { + @Test + public fun testBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(1500) + emit("B") + delay(500) + emit("C") + delay(250) + emit("D") + delay(2000) + emit("E") + expect(4) + } + + expect(2) + val result = flow.sample(1000).toList() + assertEquals(listOf("A", "B", "D"), result) + finish(5) + } + + @Test + fun testDelayedFirst() = withVirtualTime { + val flow = flow { + delay(60) + emit(1) + delay(60) + expect(1) + }.sample(100) + assertEquals(1, flow.singleOrNull()) + finish(2) + } + + @Test + fun testBasic2() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit(1) + emit(2) + delay(501) + emit(3) + delay(100) + emit(4) + delay(100) + emit(5) + emit(6) + delay(301) + emit(7) + delay(501) + expect(4) + } + + expect(2) + val result = flow.sample(500).toList() + assertEquals(listOf(2, 6, 7), result) + finish(5) + } + + @Test + fun testFixedDelay() = withVirtualTime { + val flow = flow { + emit("A") + delay(150) + emit("B") + expect(1) + }.sample(100) + assertEquals("A", flow.single()) + finish(2) + } + + @Test + fun testSingleNull() = withVirtualTime { + val flow = flow { + emit(null) + delay(2) + expect(1) + }.sample(1) + assertNull(flow.single()) + finish(2) + } + + @Test + fun testBasicWithNulls() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(1500) + emit(null) + delay(500) + emit("C") + delay(250) + emit(null) + delay(2000) + emit("E") + expect(4) + } + + expect(2) + val result = flow.sample(1000).toList() + assertEquals(listOf("A", null, null), result) + finish(5) + } + + @Test + fun testEmpty() = runTest { + val flow = emptyFlow().sample(Long.MAX_VALUE) + assertNull(flow.singleOrNull()) + } + + @Test + fun testScalar() = runTest { + val flow = flowOf(1, 2, 3).sample(Long.MAX_VALUE) + assertNull(flow.singleOrNull()) + } + + @Test + // note that this test depends on the sampling strategy -- when sampling time starts on a quiescent flow that suddenly emits + fun testLongWait() = withVirtualTime { + expect(1) + val flow = flow { + expect(2) + emit("A") + delay(3500) // long delay -- multiple sampling intervals + emit("B") + delay(900) // crosses time = 4000 barrier + emit("C") + delay(3000) // long wait again + + } + val result = flow.sample(1000).toList() + assertEquals(listOf("A", "B", "C"), result) + finish(3) + } + + @Test + fun testPace() = withVirtualTime { + val flow = flow { + expect(1) + repeat(4) { + emit(-it) + delay(50) + } + + repeat(4) { + emit(it) + delay(100) + } + expect(2) + }.sample(100) + + assertEquals(listOf(-1, -3, 0, 1, 2, 3), flow.toList()) + finish(3) + } + + @Test + fun testUpstreamError() = testUpstreamError(TestException()) + + @Test + fun testUpstreamErrorCancellationException() = testUpstreamError(CancellationException("")) + + private inline fun testUpstreamError(cause: T) = runTest { + val latch = Channel() + val flow = flow { + expect(1) + emit(1) + expect(2) + latch.receive() + throw cause + }.sample(1).map { + latch.send(Unit) + hang { expect(3) } + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testUpstreamErrorIsolatedContext() = runTest { + val latch = Channel() + val flow = flow { + assertEquals("upstream", NamedDispatchers.name()) + expect(1) + emit(1) + expect(2) + latch.receive() + throw TestException() + }.flowOn(NamedDispatchers("upstream")).sample(1).map { + latch.send(Unit) + hang { expect(3) } + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testUpstreamErrorSampleNotTriggered() = runTest { + val flow = flow { + expect(1) + emit(1) + expect(2) + throw TestException() + }.sample(Long.MAX_VALUE).map { + expectUnreached() + } + assertFailsWith(flow) + finish(3) + } + + @Test + fun testUpstreamErrorSampleNotTriggeredInIsolatedContext() = runTest { + val flow = flow { + expect(1) + emit(1) + expect(2) + throw TestException() + }.flowOn(NamedDispatchers("unused")).sample(Long.MAX_VALUE).map { + expectUnreached() + } + + assertFailsWith(flow) + finish(3) + } + + @Test + fun testDownstreamError() = runTest { + val flow = flow { + expect(1) + emit(1) + hang { expect(3) } + }.sample(100).map { + expect(2) + yield() + throw TestException() + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testDownstreamErrorIsolatedContext() = runTest { + val flow = flow { + assertEquals("upstream", NamedDispatchers.name()) + expect(1) + emit(1) + hang { expect(3) } + }.flowOn(NamedDispatchers("upstream")).sample(100).map { + expect(2) + yield() + throw TestException() + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testDurationBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(1500.milliseconds) + emit("B") + delay(500.milliseconds) + emit("C") + delay(250.milliseconds) + emit("D") + delay(2000.milliseconds) + emit("E") + expect(4) + } + + expect(2) + val result = flow.sample(1000.milliseconds).toList() + assertEquals(listOf("A", "B", "D"), result) + finish(5) + } + + @Test + fun testFailsWithIllegalArgument() { + val flow = emptyFlow() + assertFailsWith { flow.debounce(-1) } + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/ScanTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/ScanTest.kt new file mode 100644 index 0000000000..7f19f97f24 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/ScanTest.kt @@ -0,0 +1,72 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class ScanTest : TestBase() { + @Test + fun testScan() = runTest { + val flow = flowOf(1, 2, 3, 4, 5) + val result = flow.runningReduce { acc, v -> acc + v }.toList() + assertEquals(listOf(1, 3, 6, 10, 15), result) + } + + @Test + fun testScanWithInitial() = runTest { + val flow = flowOf(1, 2, 3) + val result = flow.scan(emptyList()) { acc, value -> acc + value }.toList() + assertEquals(listOf(emptyList(), listOf(1), listOf(1, 2), listOf(1, 2, 3)), result) + } + + @Test + fun testFoldWithInitial() = runTest { + val flow = flowOf(1, 2, 3) + val result = flow.runningFold(emptyList()) { acc, value -> acc + value }.toList() + assertEquals(listOf(emptyList(), listOf(1), listOf(1, 2), listOf(1, 2, 3)), result) + } + + @Test + fun testNulls() = runTest { + val flow = flowOf(null, 2, null, null, null, 5) + val result = flow.runningReduce { acc, v -> if (v == null) acc else (if (acc == null) v else acc + v) }.toList() + assertEquals(listOf(null, 2, 2, 2, 2, 7), result) + } + + @Test + fun testEmptyFlow() = runTest { + val result = emptyFlow().runningReduce { _, _ -> 1 }.toList() + assertTrue(result.isEmpty()) + } + + @Test + fun testErrorCancelsUpstream() = runTest { + expect(1) + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang { expect(3) } + } + emit(1) + emit(2) + } + }.runningReduce { _, value -> + expect(value) // 2 + latch.receive() + throw TestException() + }.catch { /* ignore */ } + + assertEquals(1, flow.single()) + finish(4) + } + + private operator fun Collection.plus(element: T): List { + val result = ArrayList(size + 1) + result.addAll(this) + result.add(element) + return result + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/TakeTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/TakeTest.kt new file mode 100644 index 0000000000..3ecc2f4add --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/TakeTest.kt @@ -0,0 +1,144 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class TakeTest : TestBase() { + @Test + fun testTake() = runTest { + val flow = flow { + emit(1) + emit(2) + } + + assertEquals(3, flow.take(2).sum()) + assertEquals(3, flow.take(Int.MAX_VALUE).sum()) + assertEquals(1, flow.take(1).single()) + assertEquals(2, flow.drop(1).take(1).single()) + } + + @Test + fun testIllegalArgument() { + assertFailsWith { flowOf(1).take(0) } + assertFailsWith { flowOf(1).take(-1) } + } + + @Test + fun testTakeSuspending() = runTest { + val flow = flow { + emit(1) + yield() + emit(2) + yield() + } + + assertEquals(3, flow.take(2).sum()) + assertEquals(3, flow.take(Int.MAX_VALUE).sum()) + assertEquals(1, flow.take(1).single()) + assertEquals(2, flow.drop(1).take(1).single()) + } + + @Test + fun testEmptyFlow() = runTest { + val sum = emptyFlow().take(10).sum() + assertEquals(0, sum) + } + + @Test + fun testNonPositiveValues() { + val flow = flowOf(1) + assertFailsWith { + flow.take(-1) + } + + assertFailsWith { + flow.take(0) + } + } + + @Test + fun testCancelUpstream() = runTest { + var cancelled = false + val flow = flow { + coroutineScope { + launch(start = CoroutineStart.ATOMIC) { + hang { cancelled = true } + } + + emit(1) + } + } + + assertEquals(1, flow.take(1).single()) + assertTrue(cancelled) + } + + @Test + fun testErrorCancelsUpstream() = runTest { + var cancelled = false + val flow = flow { + coroutineScope { + launch(start = CoroutineStart.ATOMIC) { + hang { cancelled = true } + } + emit(1) + } + }.take(2) + .map { + throw TestException() + }.catch { emit(42) } + + assertEquals(42, flow.single()) + assertTrue(cancelled) + } + + @Test + fun takeWithRetries() = runTest { + val flow = flow { + expect(1) + emit(1) + expect(2) + emit(2) + + while (true) { + emit(42) + expectUnreached() + } + + }.retry(2) { + expectUnreached() + true + }.take(2) + + val sum = flow.sum() + assertEquals(3, sum) + finish(3) + } + + @Test + fun testNonIdempotentRetry() = runTest { + var count = 0 + flow { while (true) emit(1) } + .retry { count++ % 2 != 0 } + .take(1) + .collect { + expect(1) + } + finish(2) + } + + @Test + fun testNestedTake() = runTest { + val inner = flow { + emit(1) + expectUnreached() + }.take(1) + val outer = flow { + while(true) { + emitAll(inner) + } + } + assertEquals(listOf(1, 1, 1), outer.take(3).toList()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/TakeWhileTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/TakeWhileTest.kt new file mode 100644 index 0000000000..f097b777cd --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/TakeWhileTest.kt @@ -0,0 +1,64 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class TakeWhileTest : TestBase() { + + @Test + fun testTakeWhile() = runTest { + val flow = flow { + emit(1) + emit(2) + } + + assertEquals(3, flow.takeWhile { true }.sum()) + assertEquals(1, flow.takeWhile { it < 2 }.single()) + assertEquals(2, flow.drop(1).takeWhile { it < 3 }.single()) + assertNull(flow.drop(1).takeWhile { it < 2 }.singleOrNull()) + } + + @Test + fun testEmptyFlow() = runTest { + assertEquals(0, emptyFlow().takeWhile { true }.sum()) + assertEquals(0, emptyFlow().takeWhile { false }.sum()) + } + + @Test + fun testCancelUpstream() = runTest { + var cancelled = false + val flow = flow { + coroutineScope { + launch(start = CoroutineStart.ATOMIC) { + hang { cancelled = true } + } + + emit(1) + emit(2) + } + } + + assertEquals(1, flow.takeWhile { it < 2 }.single()) + assertTrue(cancelled) + } + + @Test + fun testErrorCancelsUpstream() = runTest { + var cancelled = false + val flow = flow { + coroutineScope { + launch(start = CoroutineStart.ATOMIC) { + hang { cancelled = true } + } + emit(1) + } + }.takeWhile { + throw TestException() + } + + assertFailsWith(flow) + assertTrue(cancelled) + assertEquals(42, flow.catch { emit(42) }.single()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/TimeoutTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/TimeoutTest.kt new file mode 100644 index 0000000000..6d3b8a8ff5 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/TimeoutTest.kt @@ -0,0 +1,258 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.internal.* +import kotlin.coroutines.* +import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.seconds + +class TimeoutTest : TestBase() { + @Test + fun testBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(100) + emit("B") + delay(100) + emit("C") + expect(4) + delay(400) + expectUnreached() + } + + expect(2) + val list = mutableListOf() + assertFailsWith(flow.timeout(300.milliseconds).onEach { list.add(it) }) + assertEquals(listOf("A", "B", "C"), list) + finish(5) + } + + @Test + fun testSingleNull() = withVirtualTime { + val flow = flow { + emit(null) + delay(1) + expect(1) + }.timeout(2.milliseconds) + assertNull(flow.single()) + finish(2) + } + + @Test + fun testBasicCustomAction() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(100) + emit("B") + delay(100) + emit("C") + expect(4) + delay(400) + expectUnreached() + } + + expect(2) + val list = mutableListOf() + flow.timeout(300.milliseconds).catch { if (it is TimeoutCancellationException) emit("-1") }.collect { list.add(it) } + assertEquals(listOf("A", "B", "C", "-1"), list) + finish(5) + } + + @Test + fun testDelayedFirst() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + delay(100) + emit(1) + expect(4) + }.timeout(250.milliseconds) + expect(2) + assertEquals(1, flow.singleOrNull()) + finish(5) + } + + @Test + fun testEmpty() = withVirtualTime { + val flow = emptyFlow().timeout(1.milliseconds) + assertNull(flow.singleOrNull()) + finish(1) + } + + @Test + fun testScalar() = withVirtualTime { + val flow = flowOf(1, 2, 3).timeout(1.milliseconds) + assertEquals(listOf(1, 2, 3), flow.toList()) + finish(1) + } + + @Test + fun testUpstreamError() = testUpstreamError(TestException()) + + @Test + fun testUpstreamErrorTimeoutException() = + testUpstreamError(TimeoutCancellationException("Timed out waiting for ${0} ms", Job())) + + @Test + fun testUpstreamErrorCancellationException() = testUpstreamError(CancellationException("")) + + private inline fun testUpstreamError(cause: T) = runTest { + try { + // Workaround for JS legacy bug + flow { + emit(1) + throw cause + }.timeout(1000.milliseconds).collect() + expectUnreached() + } catch (e: Throwable) { + assertTrue { e is T } + finish(1) + } + } + + @Test + fun testUpstreamExceptionsTakingPriority() = withVirtualTime { + val flow = flow { + expect(2) + withContext(NonCancellable) { + delay(2.milliseconds) + } + assertFalse(currentCoroutineContext().isActive) // cancelled already + expect(3) + throw TestException() + }.timeout(1.milliseconds) + expect(1) + assertFailsWith { + flow.collect { + expectUnreached() + } + } + finish(4) + } + + @Test + fun testDownstreamError() = runTest { + val flow = flow { + expect(1) + emit(1) + hang { expect(3) } + expectUnreached() + }.timeout(100.milliseconds).map { + expect(2) + yield() + throw TestException() + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testUpstreamTimeoutIsolatedContext() = withVirtualTime { + val flow = flow { + assertEquals("upstream", NamedDispatchers.name()) + expect(1) + emit(1) + expect(2) + delay(300) + expectUnreached() + }.flowOn(NamedDispatchers("upstream")).timeout(100.milliseconds) + + assertFailsWith(flow) + finish(3) + } + + @Test + fun testUpstreamTimeoutActionIsolatedContext() = withVirtualTime { + val flow = flow { + assertEquals("upstream", NamedDispatchers.name()) + expect(1) + emit(1) + expect(2) + delay(300) + expectUnreached() + }.flowOn(NamedDispatchers("upstream")).timeout(100.milliseconds).catch { + expect(3) + emit(2) + } + + assertEquals(listOf(1, 2), flow.toList()) + finish(4) + } + + @Test + fun testSharedFlowTimeout() = withVirtualTime { + // Workaround for JS legacy bug + try { + MutableSharedFlow().asSharedFlow().timeout(100.milliseconds).collect() + expectUnreached() + } catch (e: TimeoutCancellationException) { + finish(1) + } + } + + @Test + fun testSharedFlowCancelledNoTimeout() = runTest { + val mutableSharedFlow = MutableSharedFlow() + val list = arrayListOf() + + expect(1) + val consumerJob = launch { + expect(3) + mutableSharedFlow.asSharedFlow().timeout(100.milliseconds).collect { list.add(it) } + expectUnreached() + } + val producerJob = launch { + expect(4) + repeat(10) { + delay(50) + mutableSharedFlow.emit(it) + } + yield() + consumerJob.cancel() + expect(5) + } + + expect(2) + + producerJob.join() + consumerJob.join() + + assertEquals((0 until 10).toList(), list) + finish(6) + } + + @Test + fun testImmediateTimeout() { + testImmediateTimeout(Duration.ZERO) + reset() + testImmediateTimeout(-1.seconds) + } + + @Test + fun testClosing() = runTest { + assertFailsWith { + channelFlow { close(TestException()) } + .timeout(Duration.INFINITE) + .collect { + expectUnreached() + } + } + } + + private fun testImmediateTimeout(timeout: Duration) { + expect(1) + val flow = emptyFlow().timeout(timeout) + flow::collect.startCoroutine(NopCollector, Continuation(EmptyCoroutineContext) { + assertIs(it.exceptionOrNull()) + finish(2) + }) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/TransformLatestTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/TransformLatestTest.kt new file mode 100644 index 0000000000..e072eabd71 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/TransformLatestTest.kt @@ -0,0 +1,157 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class TransformLatestTest : TestBase() { + + @Test + fun testTransformLatest() = runTest { + val flow = flowOf(1, 2, 3).transformLatest { value -> + emit(value) + emit(value + 1) + } + assertEquals(listOf(1, 2, 2, 3, 3, 4), flow.toList()) + } + + @Test + fun testEmission() = runTest { + val list = flow { + repeat(5) { + emit(it) + } + }.transformLatest { + emit(it) + }.toList() + assertEquals(listOf(0, 1, 2, 3, 4), list) + } + + @Test + fun testSwitchIntuitiveBehaviour() = runTest { + val flow = flowOf(1, 2, 3, 4, 5) + flow.transformLatest { + expect(it) + emit(it) + yield() // Explicit cancellation check + if (it != 5) expectUnreached() + else expect(6) + }.collect() + finish(7) + } + + @Test + fun testSwitchRendezvousBuffer() = runTest { + val flow = flowOf(1, 2, 3, 4, 5) + flow.transformLatest { + emit(it) + // Reach here every uneven element because of channel's unfairness + expect(it) + }.buffer(0).onEach { expect(it + 1) }.collect() + finish(7) + } + + @Test + fun testSwitchBuffer() = runTest { + val flow = flowOf(1, 2, 3, 42, 4) + flow.transformLatest { + emit(it) + expect(it) + }.buffer(2).collect() + finish(5) + } + + @Test + fun testHangFlows() = runTest { + val flow = listOf(1, 2, 3, 4).asFlow() + val result = flow.transformLatest { value -> + if (value != 4) hang { expect(value) } + emit(42) + }.toList() + + assertEquals(listOf(42), result) + finish(4) + } + + @Test + fun testEmptyFlow() = runTest { + assertNull(emptyFlow().transformLatest { emit(1) }.singleOrNull()) + } + + @Test + fun testIsolatedContext() = runTest { + val flow = flow { + assertEquals("source", NamedDispatchers.name()) + expect(1) + emit(4) + expect(2) + emit(5) + expect(3) + }.flowOn(NamedDispatchers("source")).transformLatest { value -> + emitAll(flow { + assertEquals("switch$value", NamedDispatchers.name()) + expect(value) + emit(value) + }.flowOn(NamedDispatchers("switch$value"))) + }.onEach { + expect(it + 2) + assertEquals("main", NamedDispatchers.nameOr("main")) + } + assertEquals(2, flow.count()) + finish(8) + } + + @Test + fun testFailureInTransform() = runTest { + val flow = flowOf(1, 2).transformLatest { value -> + if (value == 1) { + emit(1) + hang { expect(1) } + } else { + expect(2) + throw TestException() + } + } + assertFailsWith(flow) + finish(3) + } + + @Test + fun testFailureDownstream() = runTest { + val flow = flowOf(1).transformLatest { value -> + expect(1) + emit(value) + expect(2) + hang { expect(4) } + }.flowOn(NamedDispatchers("downstream")).onEach { + expect(3) + throw TestException() + } + assertFailsWith(flow) + finish(5) + } + + @Test + fun testFailureUpstream() = runTest { + val flow = flow { + expect(1) + emit(1) + yield() + expect(3) + throw TestException() + }.transformLatest { + expect(2) + hang { + expect(4) + } + } + assertFailsWith(flow) + finish(5) + } + + @Test + fun testTake() = runTest { + val flow = flowOf(1, 2, 3, 4, 5).transformLatest { emit(it) } + assertEquals(listOf(1), flow.take(1).toList()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/operators/TransformTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/TransformTest.kt new file mode 100644 index 0000000000..b70460a052 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/TransformTest.kt @@ -0,0 +1,17 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class TransformTest : TestBase() { + @Test + fun testDoubleEmit() = runTest { + val flow = flowOf(1, 2, 3) + .transform { + emit(it) + emit(it) + } + assertEquals(listOf(1, 1, 2, 2, 3, 3), flow.toList()) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/flow/operators/TransformWhileTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/TransformWhileTest.kt new file mode 100644 index 0000000000..4da0d26ab8 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/TransformWhileTest.kt @@ -0,0 +1,67 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class TransformWhileTest : TestBase() { + @Test + fun testSimple() = runTest { + val flow = (0..10).asFlow() + val expected = listOf("A", "B", "C", "D") + val actual = flow.transformWhile { value -> + when(value) { + 0 -> { emit("A"); true } + 1 -> true + 2 -> { emit("B"); emit("C"); true } + 3 -> { emit("D"); false } + else -> { expectUnreached(); false } + } + }.toList() + assertEquals(expected, actual) + } + + @Test + fun testCancelUpstream() = runTest { + var cancelled = false + val flow = flow { + coroutineScope { + launch(start = CoroutineStart.ATOMIC) { + hang { cancelled = true } + } + emit(1) + emit(2) + emit(3) + } + } + val transformed = flow.transformWhile { + emit(it) + it < 2 + } + assertEquals(listOf(1, 2), transformed.toList()) + assertTrue(cancelled) + } + + @Test + fun testExample() = runTest { + val source = listOf( + DownloadProgress(0), + DownloadProgress(50), + DownloadProgress(100), + DownloadProgress(147) + ) + val expected = source.subList(0, 3) + val actual = source.asFlow().completeWhenDone().toList() + assertEquals(expected, actual) + } + + private fun Flow.completeWhenDone(): Flow = + transformWhile { progress -> + emit(progress) // always emit progress + !progress.isDone() // continue while download is not done + } + + private data class DownloadProgress(val percent: Int) { + fun isDone() = percent >= 100 + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/flow/operators/ZipTest.kt b/kotlinx-coroutines-core/common/test/flow/operators/ZipTest.kt new file mode 100644 index 0000000000..1f770dfe21 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/operators/ZipTest.kt @@ -0,0 +1,251 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class ZipTest : TestBase() { + + @Test + fun testZip() = runTest { + val f1 = flowOf("a", "b", "c") + val f2 = flowOf(1, 2, 3) + assertEquals(listOf("a1", "b2", "c3"), f1.zip(f2, String::plus).toList()) + } + + @Test + fun testUnevenZip() = runTest { + val f1 = flowOf("a", "b", "c", "d", "e") + val f2 = flowOf(1, 2, 3) + assertEquals(listOf("a1", "b2", "c3"), f1.zip(f2, String::plus).toList()) + assertEquals(listOf("a1", "b2", "c3"), f2.zip(f1) { i, j -> j + i }.toList()) + } + + @Test + fun testEmptyFlows() = runTest { + val f1 = emptyFlow() + val f2 = emptyFlow() + assertEquals(emptyList(), f1.zip(f2, String::plus).toList()) + } + + @Test + fun testEmpty() = runTest { + val f1 = emptyFlow() + val f2 = flowOf(1) + assertEquals(emptyList(), f1.zip(f2, String::plus).toList()) + } + + @Test + fun testEmptyOther() = runTest { + val f1 = flowOf("a") + val f2 = emptyFlow() + assertEquals(emptyList(), f1.zip(f2, String::plus).toList()) + } + + @Test + fun testNulls() = runTest { + val f1 = flowOf("a", null, null, "d") + val f2 = flowOf(1, 2, 3) + assertEquals(listOf("a1", "null2", "null3"), f1.zip(f2, String?::plus).toList()) + } + + @Test + fun testNullsOther() = runTest { + val f1 = flowOf("a", "b", "c") + val f2 = flowOf(1, null, null, 2) + assertEquals(listOf("a1", "bnull", "cnull"), f1.zip(f2, String::plus).toList()) + } + + @Test + fun testCancelWhenFlowIsDone() = runTest { + val f1 = flow { + emit("1") + emit("2") + } + + val f2 = flow { + emit("a") + emit("b") + expectUnreached() + } + assertEquals(listOf("1a", "2b"), f1.zip(f2, String::plus).toList()) + finish(1) + } + + @Test + fun testCancelWhenFlowIsDone2() = runTest { + val f1 = flow { + emit("1") + emit("2") + try { + emit("3") + expectUnreached() + } finally { + expect(1) + } + } + + val f2 = flowOf("a", "b") + assertEquals(listOf("1a", "2b"), f1.zip(f2, String::plus).toList()) + finish(2) + } + + @Test + fun testCancelWhenFlowIsDoneReversed() = runTest { + val f1 = flow { + emit("1") + emit("2") + hang { + expect(1) + } + } + + val f2 = flow { + emit("a") + emit("b") + yield() + } + + assertEquals(listOf("a1", "b2"), f2.zip(f1, String::plus).toList()) + finish(2) + } + + @Test + fun testContextIsIsolatedReversed() = runTest { + val f1 = flow { + emit("a") + assertEquals("first", NamedDispatchers.name()) + expect(3) + }.flowOn(NamedDispatchers("first")).onEach { + assertEquals("with", NamedDispatchers.name()) + expect(4) + }.flowOn(NamedDispatchers("with")) + + val f2 = flow { + emit(1) + assertEquals("second", NamedDispatchers.name()) + expect(1) + }.flowOn(NamedDispatchers("second")).onEach { + assertEquals("nested", NamedDispatchers.name()) + expect(2) + }.flowOn(NamedDispatchers("nested")) + + val value = withContext(NamedDispatchers("main")) { + f1.zip(f2) { i, j -> + assertEquals("main", NamedDispatchers.name()) + expect(5) + i + j + }.single() + } + + assertEquals("a1", value) + finish(6) + } + + @Test + fun testErrorInDownstreamCancelsUpstream() = runTest { + val f1 = flow { + emit("a") + hang { + expect(3) + } + }.flowOn(NamedDispatchers("first")) + + val f2 = flow { + emit(1) + hang { + expect(2) + } + }.flowOn(NamedDispatchers("second")) + + val flow = f1.zip(f2) { i, j -> + assertEquals("zip", NamedDispatchers.name()) + expect(1) + i + j + }.flowOn(NamedDispatchers("zip")).onEach { + throw TestException() + } + + assertFailsWith(flow) + finish(4) + } + + @Test + fun testErrorCancelsSibling() = runTest { + val f1 = flow { + emit("a") + hang { + expect(1) + } + }.flowOn(NamedDispatchers("first")) + + val f2 = flow { + emit(1) + throw TestException() + }.flowOn(NamedDispatchers("second")) + + val flow = f1.zip(f2) { _, _ -> 1 } + assertFailsWith(flow) + finish(2) + } + + @Test + fun testCancellationUpstream() = runTest { + val f1 = flow { + expect(1) + emit(1) + expect(5) + throw CancellationException("") + } + + val f2 = flow { + expect(2) + emit(1) + expect(3) + hang { expect(6) } + } + + val flow = f1.zip(f2) { _, _ -> 1 }.onEach { expect(4) } + assertFailsWith(flow) + finish(7) + } + + @Test + fun testCancellationDownstream() = runTest { + val f1 = flow { + expect(1) + emit(1) + expectUnreached() // Will throw CE + } + + val f2 = flow { + expect(2) + emit(1) + expect(3) + hang { expect(5) } + } + + val flow = f1.zip(f2, { _, _ -> 1 }).onEach { + expect(4) + yield() + throw CancellationException("") + } + assertFailsWith(flow) + finish(6) + } + + @Test + fun testCancellationOfCollector() = runTest { + val f1 = flow { + emit("1") + awaitCancellation() + } + + val f2 = flow { + emit("2") + yield() + } + + f1.zip(f2, String::plus).collect { } + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/ShareInBufferTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/ShareInBufferTest.kt new file mode 100644 index 0000000000..8871e31816 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/ShareInBufferTest.kt @@ -0,0 +1,95 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.math.* +import kotlin.test.* + +/** + * Similar to [BufferTest], but tests [shareIn] buffering and its fusion with [buffer] operators. + */ +class ShareInBufferTest : TestBase() { + private val n = 200 // number of elements to emit for test + private val defaultBufferSize = 64 // expected default buffer size (per docs) + + // Use capacity == -1 to check case of "no buffer" + private fun checkBuffer(capacity: Int, op: suspend Flow.(CoroutineScope) -> Flow) = runTest { + expect(1) + /* + Shared flows do not perform full rendezvous. On buffer overflow emitter always suspends until all + subscribers get the value and then resumes. Thus, perceived batch size is +1 from buffer capacity. + */ + val batchSize = capacity + 1 + val upstream = flow { + repeat(n) { i -> + val batchNo = i / batchSize + val batchIdx = i % batchSize + expect(batchNo * batchSize * 2 + batchIdx + 2) + emit(i) + } + emit(-1) // done + } + coroutineScope { + upstream + .op(this) + .takeWhile { i -> i >= 0 } // until done + .collect { i -> + val batchNo = i / batchSize + val batchIdx = i % batchSize + // last batch might have smaller size + val k = min((batchNo + 1) * batchSize, n) - batchNo * batchSize + expect(batchNo * batchSize * 2 + k + batchIdx + 2) + } + coroutineContext.cancelChildren() // cancels sharing + } + finish(2 * n + 2) + } + + @Test + fun testReplay0DefaultBuffer() = + checkBuffer(defaultBufferSize) { + shareIn(it, SharingStarted.Eagerly) + } + + @Test + fun testReplay1DefaultBuffer() = + checkBuffer(defaultBufferSize) { + shareIn(it, SharingStarted.Eagerly, 1) + } + + @Test // buffer is padded to default size as needed + fun testReplay10DefaultBuffer() = + checkBuffer(maxOf(10, defaultBufferSize)) { + shareIn(it, SharingStarted.Eagerly, 10) + } + + @Test // buffer is padded to default size as needed + fun testReplay100DefaultBuffer() = + checkBuffer( maxOf(100, defaultBufferSize)) { + shareIn(it, SharingStarted.Eagerly, 100) + } + + @Test + fun testDefaultBufferKeepsDefault() = + checkBuffer(defaultBufferSize) { + buffer().shareIn(it, SharingStarted.Eagerly) + } + + @Test + fun testOverrideDefaultBuffer0() = + checkBuffer(0) { + buffer(0).shareIn(it, SharingStarted.Eagerly) + } + + @Test + fun testOverrideDefaultBuffer10() = + checkBuffer(10) { + buffer(10).shareIn(it, SharingStarted.Eagerly) + } + + @Test // buffer and replay sizes add up + fun testBufferReplaySum() = + checkBuffer(41) { + buffer(10).buffer(20).shareIn(it, SharingStarted.Eagerly, 11) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/ShareInConflationTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/ShareInConflationTest.kt new file mode 100644 index 0000000000..73d14accec --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/ShareInConflationTest.kt @@ -0,0 +1,159 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +/** + * Similar to [ShareInBufferTest] and [BufferConflationTest], + * but tests [shareIn] and its fusion with [conflate] operator. + */ +class ShareInConflationTest : TestBase() { + private val n = 100 + + private fun checkConflation( + bufferCapacity: Int, + onBufferOverflow: BufferOverflow = BufferOverflow.DROP_OLDEST, + op: suspend Flow.(CoroutineScope) -> Flow + ) = runTest { + expect(1) + // emit all and conflate, then should collect bufferCapacity the latest ones + val done = Job() + flow { + repeat(n) { i -> + expect(i + 2) + emit(i) + } + done.join() // wait until done collection + emit(-1) // signal flow completion + } + .op(this) + .takeWhile { i -> i >= 0 } + .collect { i -> + val first = if (onBufferOverflow == BufferOverflow.DROP_LATEST) 0 else n - bufferCapacity + val last = first + bufferCapacity - 1 + if (i in first..last) { + expect(n + i - first + 2) + if (i == last) done.complete() // received the last one + } else { + error("Unexpected $i") + } + } + finish(n + bufferCapacity + 2) + } + + @Test + fun testConflateReplay1() = + checkConflation(1) { + conflate().shareIn(it, SharingStarted.Eagerly, 1) + } + + @Test // still looks like conflating the last value for the first subscriber (will not replay to others though) + fun testConflateReplay0() = + checkConflation(1) { + conflate().shareIn(it, SharingStarted.Eagerly, 0) + } + + @Test + fun testConflateReplay5() = + checkConflation(5) { + conflate().shareIn(it, SharingStarted.Eagerly, 5) + } + + @Test + fun testBufferDropOldestReplay1() = + checkConflation(1) { + buffer(onBufferOverflow = BufferOverflow.DROP_OLDEST).shareIn(it, SharingStarted.Eagerly, 1) + } + + @Test + fun testBufferDropOldestReplay0() = + checkConflation(1) { + buffer(onBufferOverflow = BufferOverflow.DROP_OLDEST).shareIn(it, SharingStarted.Eagerly, 0) + } + + @Test + fun testBufferDropOldestReplay10() = + checkConflation(10) { + buffer(onBufferOverflow = BufferOverflow.DROP_OLDEST).shareIn(it, SharingStarted.Eagerly, 10) + } + + @Test + fun testBuffer20DropOldestReplay0() = + checkConflation(20) { + buffer(20, onBufferOverflow = BufferOverflow.DROP_OLDEST).shareIn(it, SharingStarted.Eagerly, 0) + } + + @Test + fun testBuffer7DropOldestReplay11() = + checkConflation(18) { + buffer(7, onBufferOverflow = BufferOverflow.DROP_OLDEST).shareIn(it, SharingStarted.Eagerly, 11) + } + + @Test // a preceding buffer() gets overridden by conflate() + fun testBufferConflateOverride() = + checkConflation(1) { + buffer(23).conflate().shareIn(it, SharingStarted.Eagerly, 1) + } + + @Test // a preceding buffer() gets overridden by buffer(onBufferOverflow = BufferOverflow.DROP_OLDEST) + fun testBufferDropOldestOverride() = + checkConflation(1) { + buffer(23).buffer(onBufferOverflow = BufferOverflow.DROP_OLDEST).shareIn(it, SharingStarted.Eagerly, 1) + } + + @Test + fun testBufferDropLatestReplay0() = + checkConflation(1, BufferOverflow.DROP_LATEST) { + buffer(onBufferOverflow = BufferOverflow.DROP_LATEST).shareIn(it, SharingStarted.Eagerly, 0) + } + + @Test + fun testBufferDropLatestReplay1() = + checkConflation(1, BufferOverflow.DROP_LATEST) { + buffer(onBufferOverflow = BufferOverflow.DROP_LATEST).shareIn(it, SharingStarted.Eagerly, 1) + } + + @Test + fun testBufferDropLatestReplay10() = + checkConflation(10, BufferOverflow.DROP_LATEST) { + buffer(onBufferOverflow = BufferOverflow.DROP_LATEST).shareIn(it, SharingStarted.Eagerly, 10) + } + + @Test + fun testBuffer0DropLatestReplay0() = + checkConflation(1, BufferOverflow.DROP_LATEST) { + buffer(0, onBufferOverflow = BufferOverflow.DROP_LATEST).shareIn(it, SharingStarted.Eagerly, 0) + } + + @Test + fun testBuffer0DropLatestReplay1() = + checkConflation(1, BufferOverflow.DROP_LATEST) { + buffer(0, onBufferOverflow = BufferOverflow.DROP_LATEST).shareIn(it, SharingStarted.Eagerly, 1) + } + + @Test + fun testBuffer0DropLatestReplay10() = + checkConflation(10, BufferOverflow.DROP_LATEST) { + buffer(0, onBufferOverflow = BufferOverflow.DROP_LATEST).shareIn(it, SharingStarted.Eagerly, 10) + } + + @Test + fun testBuffer5DropLatestReplay0() = + checkConflation(5, BufferOverflow.DROP_LATEST) { + buffer(5, onBufferOverflow = BufferOverflow.DROP_LATEST).shareIn(it, SharingStarted.Eagerly, 0) + } + + @Test + fun testBuffer5DropLatestReplay10() = + checkConflation(15, BufferOverflow.DROP_LATEST) { + buffer(5, onBufferOverflow = BufferOverflow.DROP_LATEST).shareIn(it, SharingStarted.Eagerly, 10) + } + + @Test // a preceding buffer() gets overridden by buffer(onBufferOverflow = BufferOverflow.DROP_LATEST) + fun testBufferDropLatestOverride() = + checkConflation(1, BufferOverflow.DROP_LATEST) { + buffer(23).buffer(onBufferOverflow = BufferOverflow.DROP_LATEST).shareIn(it, SharingStarted.Eagerly, 0) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/ShareInFusionTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/ShareInFusionTest.kt new file mode 100644 index 0000000000..c7b4650827 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/ShareInFusionTest.kt @@ -0,0 +1,53 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class ShareInFusionTest : TestBase() { + /** + * Test perfect fusion for operators **after** [shareIn]. + */ + @Test + fun testOperatorFusion() = runTest { + val sh = emptyFlow().shareIn(this, SharingStarted.Eagerly) + assertTrue(sh !is MutableSharedFlow<*>) // cannot be cast to mutable shared flow!!! + assertSame(sh, (sh as Flow<*>).cancellable()) + assertSame(sh, (sh as Flow<*>).flowOn(Dispatchers.Default)) + assertSame(sh, sh.buffer(Channel.RENDEZVOUS)) + coroutineContext.cancelChildren() + } + + @Test + fun testFlowOnContextFusion() = runTest { + val flow = flow { + assertEquals("FlowCtx", currentCoroutineContext()[CoroutineName]?.name) + emit("OK") + }.flowOn(CoroutineName("FlowCtx")) + assertEquals("OK", flow.shareIn(this, SharingStarted.Eagerly, 1).first()) + coroutineContext.cancelChildren() + } + + /** + * Tests that `channelFlow { ... }.buffer(x)` works according to the [channelFlow] docs, and subsequent + * application of [shareIn] does not leak upstream. + */ + @Test + fun testChannelFlowBufferShareIn() = runTest { + expect(1) + val flow = channelFlow { + // send a batch of 10 elements using [offer] + for (i in 1..10) { + assertTrue(trySend(i).isSuccess) // offer must succeed, because buffer + } + send(0) // done + }.buffer(10) // request a buffer of 10 + // ^^^^^^^^^ buffer stays here + val shared = flow.shareIn(this, SharingStarted.Eagerly) + shared + .takeWhile { it > 0 } + .collect { i -> expect(i + 1) } + finish(12) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/ShareInTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/ShareInTest.kt new file mode 100644 index 0000000000..1df59b91fe --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/ShareInTest.kt @@ -0,0 +1,241 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class ShareInTest : TestBase() { + @Test + fun testReplay0Eager() = runTest { + expect(1) + val flow = flowOf("OK") + val shared = flow.shareIn(this, SharingStarted.Eagerly) + yield() // actually start sharing + // all subscribers miss "OK" + val jobs = List(10) { + shared.onEach { expectUnreached() }.launchIn(this) + } + yield() // ensure nothing is collected + jobs.forEach { it.cancel() } + finish(2) + } + + @Test + fun testReplay0Lazy() = testReplayZeroOrOne(0) + + @Test + fun testReplay1Lazy() = testReplayZeroOrOne(1) + + private fun testReplayZeroOrOne(replay: Int) = runTest { + expect(1) + val doneBarrier = Job() + val flow = flow { + expect(2) + emit("OK") + doneBarrier.join() + emit("DONE") + } + val sharingJob = Job() + val shared = flow.shareIn(this + sharingJob, started = SharingStarted.Lazily, replay = replay) + yield() // should not start sharing + // first subscriber gets "OK", other subscribers miss "OK" + val n = 10 + val replayOfs = replay * (n - 1) + val subscriberJobs = List(n) { index -> + val subscribedBarrier = Job() + val job = shared + .onSubscription { + subscribedBarrier.complete() + } + .onEach { value -> + when (value) { + "OK" -> { + expect(3 + index) + if (replay == 0) { // only the first subscriber collects "OK" without replay + assertEquals(0, index) + } + } + "DONE" -> { + expect(4 + index + replayOfs) + } + else -> expectUnreached() + } + } + .takeWhile { it != "DONE" } + .launchIn(this) + subscribedBarrier.join() // wait until the launched job subscribed before launching the next one + job + } + doneBarrier.complete() + subscriberJobs.joinAll() + expect(4 + n + replayOfs) + sharingJob.cancel() + finish(5 + n + replayOfs) + } + + @Test + fun testUpstreamCompleted() = + testUpstreamCompletedOrFailed(failed = false) + + @Test + fun testUpstreamFailed() = + testUpstreamCompletedOrFailed(failed = true) + + private fun testUpstreamCompletedOrFailed(failed: Boolean) = runTest { + val emitted = Job() + val terminate = Job() + val sharingJob = CompletableDeferred() + val upstream = flow { + emit("OK") + emitted.complete() + terminate.join() + if (failed) throw TestException() + } + val shared = upstream.shareIn(this + sharingJob, SharingStarted.Eagerly, 1) + assertEquals(emptyList(), shared.replayCache) + emitted.join() // should start sharing, emit & cache + assertEquals(listOf("OK"), shared.replayCache) + terminate.complete() + sharingJob.complete(Unit) + sharingJob.join() // should complete sharing + assertEquals(listOf("OK"), shared.replayCache) // cache is still there + if (failed) { + assertIs(sharingJob.getCompletionExceptionOrNull()) + } else { + assertNull(sharingJob.getCompletionExceptionOrNull()) + } + } + + @Test + fun testWhileSubscribedBasic() = + testWhileSubscribed(1, SharingStarted.WhileSubscribed()) + + @Test + fun testWhileSubscribedCustomAtLeast1() = + testWhileSubscribed(1, SharingStarted.WhileSubscribedAtLeast(1)) + + @Test + fun testWhileSubscribedCustomAtLeast2() = + testWhileSubscribed(2, SharingStarted.WhileSubscribedAtLeast(2)) + + @OptIn(ExperimentalStdlibApi::class) + private fun testWhileSubscribed(threshold: Int, started: SharingStarted) = runTest { + expect(1) + val flowState = FlowState() + val n = 3 // max number of subscribers + val log = Channel(2 * n) + + suspend fun checkStartTransition(subscribers: Int) { + when (subscribers) { + in 0 until threshold -> assertFalse(flowState.started) + threshold -> { + flowState.awaitStart() // must eventually start the flow + for (i in 1..threshold) { + assertEquals("sub$i: OK", log.receive()) // threshold subs must receive the values + } + } + in threshold + 1..n -> assertTrue(flowState.started) + } + } + + suspend fun checkStopTransition(subscribers: Int) { + when (subscribers) { + in threshold + 1..n -> assertTrue(flowState.started) + threshold - 1 -> flowState.awaitStop() // upstream flow must be eventually stopped + in 0..threshold - 2 -> assertFalse(flowState.started) // should have stopped already + } + } + + val flow = flow { + flowState.track { + emit("OK") + delay(Long.MAX_VALUE) // await forever, will get cancelled + } + } + + val shared = flow.shareIn(this, started) + repeat(5) { // repeat scenario a few times + yield() + assertFalse(flowState.started) // flow is not running even if we yield + // start 3 subscribers + val subs = ArrayList() + for (i in 1..n) { + subs += shared + .onEach { value -> // only the first threshold subscribers get the value + when (i) { + in 1..threshold -> log.trySend("sub$i: $value") + else -> expectUnreached() + } + } + .onCompletion { log.trySend("sub$i: completion") } + .launchIn(this) + checkStartTransition(i) + } + // now cancel all subscribers + for (i in 1..n) { + subs.removeFirst().cancel() // cancel subscriber + assertEquals("sub$i: completion", log.receive()) // subscriber shall shutdown + checkStopTransition(n - i) + } + } + coroutineContext.cancelChildren() // cancel sharing job + finish(2) + } + + @Suppress("TestFunctionName") + private fun SharingStarted.Companion.WhileSubscribedAtLeast(threshold: Int) = + SharingStarted { subscriptionCount -> + subscriptionCount.map { if (it >= threshold) SharingCommand.START else SharingCommand.STOP } + } + + private class FlowState { + private val timeLimit = 10000L + private val _started = MutableStateFlow(false) + val started: Boolean get() = _started.value + fun start() = check(_started.compareAndSet(expect = false, update = true)) + fun stop() = check(_started.compareAndSet(expect = true, update = false)) + suspend fun awaitStart() = withTimeout(timeLimit) { _started.first { it } } + suspend fun awaitStop() = withTimeout(timeLimit) { _started.first { !it } } + } + + private suspend fun FlowState.track(block: suspend () -> Unit) { + start() + try { + block() + } finally { + stop() + } + } + + @Test + fun testShouldStart() = runTest { + val flow = flow { + expect(2) + emit(1) + expect(3) + }.shareIn(this, SharingStarted.Lazily) + + expect(1) + flow.onSubscription { throw CancellationException("") } + .catch { e -> assertTrue { e is CancellationException } } + .collect() + yield() + finish(4) + } + + @Test + fun testShouldStartScalar() = runTest { + val j = Job() + val shared = flowOf(239).stateIn(this + j, SharingStarted.Lazily, 42) + assertEquals(42, shared.first()) + yield() + assertEquals(239, shared.first()) + j.cancel() + } + + @Test + fun testSubscriptionByFirstSuspensionInSharedFlow() = runTest { + testSubscriptionByFirstSuspensionInCollect(flowOf(1).stateIn(this@runTest), emit = { }) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/SharedFlowScenarioTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/SharedFlowScenarioTest.kt new file mode 100644 index 0000000000..f4417e109a --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/SharedFlowScenarioTest.kt @@ -0,0 +1,404 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.coroutines.* +import kotlin.test.* + +/** + * This test suit for [SharedFlow] has a dense framework that allows to test complex + * suspend/resume scenarios while keeping the code readable. Each test here is for + * one specific [SharedFlow] configuration, testing all the various corner cases in its + * behavior. + */ +class SharedFlowScenarioTest : TestBase() { + @Test + fun testReplay1Extra2() = + testSharedFlow(MutableSharedFlow(1, 2)) { + // total buffer size == 3 + expectReplayOf() + emitRightNow(1); expectReplayOf(1) + emitRightNow(2); expectReplayOf(2) + emitRightNow(3); expectReplayOf(3) + emitRightNow(4); expectReplayOf(4) // no prob - no subscribers + val a = subscribe("a"); collect(a, 4) + emitRightNow(5); expectReplayOf(5) + emitRightNow(6); expectReplayOf(6) + emitRightNow(7); expectReplayOf(7) + // suspend/collect sequentially + val e8 = emitSuspends(8); collect(a, 5); emitResumes(e8); expectReplayOf(8) + val e9 = emitSuspends(9); collect(a, 6); emitResumes(e9); expectReplayOf(9) + // buffer full, but parallel emitters can still suspend (queue up) + val e10 = emitSuspends(10) + val e11 = emitSuspends(11) + val e12 = emitSuspends(12) + collect(a, 7); emitResumes(e10); expectReplayOf(10) // buffer 8, 9 | 10 + collect(a, 8); emitResumes(e11); expectReplayOf(11) // buffer 9, 10 | 11 + sharedFlow.resetReplayCache(); expectReplayOf() // 9, 10 11 | no replay + collect(a, 9); emitResumes(e12); expectReplayOf(12) + collect(a, 10, 11, 12); expectReplayOf(12) // buffer empty | 12 + emitRightNow(13); expectReplayOf(13) + emitRightNow(14); expectReplayOf(14) + emitRightNow(15); expectReplayOf(15) // buffer 13, 14 | 15 + val e16 = emitSuspends(16) + val e17 = emitSuspends(17) + val e18 = emitSuspends(18) + cancel(e17); expectReplayOf(15) // cancel in the middle of three emits; buffer 13, 14 | 15 + collect(a, 13); emitResumes(e16); expectReplayOf(16) // buffer 14, 15, | 16 + collect(a, 14); emitResumes(e18); expectReplayOf(18) // buffer 15, 16 | 18 + val e19 = emitSuspends(19) + val e20 = emitSuspends(20) + val e21 = emitSuspends(21) + cancel(e21); expectReplayOf(18) // cancel last emit; buffer 15, 16, 18 + collect(a, 15); emitResumes(e19); expectReplayOf(19) // buffer 16, 18 | 19 + collect(a, 16); emitResumes(e20); expectReplayOf(20) // buffer 18, 19 | 20 + collect(a, 18, 19, 20); expectReplayOf(20) // buffer empty | 20 + emitRightNow(22); expectReplayOf(22) + emitRightNow(23); expectReplayOf(23) + emitRightNow(24); expectReplayOf(24) // buffer 22, 23 | 24 + val e25 = emitSuspends(25) + val e26 = emitSuspends(26) + val e27 = emitSuspends(27) + cancel(e25); expectReplayOf(24) // cancel first emit, buffer 22, 23 | 24 + sharedFlow.resetReplayCache(); expectReplayOf() // buffer 22, 23, 24 | no replay + val b = subscribe("b") // new subscriber + collect(a, 22); emitResumes(e26); expectReplayOf(26) // buffer 23, 24 | 26 + collect(b, 26) + collect(a, 23); emitResumes(e27); expectReplayOf(27) // buffer 24, 26 | 27 + collect(a, 24, 26, 27) // buffer empty | 27 + emitRightNow(28); expectReplayOf(28) + emitRightNow(29); expectReplayOf(29) // buffer 27, 28 | 29 + collect(a, 28, 29) // but b is slow + val e30 = emitSuspends(30) + val e31 = emitSuspends(31) + val e32 = emitSuspends(32) + val e33 = emitSuspends(33) + val e34 = emitSuspends(34) + val e35 = emitSuspends(35) + val e36 = emitSuspends(36) + val e37 = emitSuspends(37) + val e38 = emitSuspends(38) + val e39 = emitSuspends(39) + cancel(e31) // cancel emitter in queue + cancel(b) // cancel slow subscriber -> 3 emitters resume + emitResumes(e30); emitResumes(e32); emitResumes(e33); expectReplayOf(33) // buffer 30, 32 | 33 + val c = subscribe("c"); collect(c, 33) // replays + cancel(e34) + collect(a, 30); emitResumes(e35); expectReplayOf(35) // buffer 32, 33 | 35 + cancel(e37) + cancel(a); emitResumes(e36); emitResumes(e38); expectReplayOf(38) // buffer 35, 36 | 38 + collect(c, 35); emitResumes(e39); expectReplayOf(39) // buffer 36, 38 | 39 + collect(c, 36, 38, 39); expectReplayOf(39) + cancel(c); expectReplayOf(39) // replay stays + } + + @Test + fun testReplay1() = + testSharedFlow(MutableSharedFlow(1)) { + emitRightNow(0); expectReplayOf(0) + emitRightNow(1); expectReplayOf(1) + emitRightNow(2); expectReplayOf(2) + sharedFlow.resetReplayCache(); expectReplayOf() + sharedFlow.resetReplayCache(); expectReplayOf() + emitRightNow(3); expectReplayOf(3) + emitRightNow(4); expectReplayOf(4) + val a = subscribe("a"); collect(a, 4) + emitRightNow(5); expectReplayOf(5); collect(a, 5) + emitRightNow(6) + sharedFlow.resetReplayCache(); expectReplayOf() + sharedFlow.resetReplayCache(); expectReplayOf() + val e7 = emitSuspends(7) + val e8 = emitSuspends(8) + val e9 = emitSuspends(9) + collect(a, 6); emitResumes(e7); expectReplayOf(7) + sharedFlow.resetReplayCache(); expectReplayOf() + sharedFlow.resetReplayCache(); expectReplayOf() // buffer 7 | -- no replay, but still buffered + val b = subscribe("b") + collect(a, 7); emitResumes(e8); expectReplayOf(8) + collect(b, 8) // buffer | 8 -- a is slow + val e10 = emitSuspends(10) + val e11 = emitSuspends(11) + val e12 = emitSuspends(12) + cancel(e9) + collect(a, 8); emitResumes(e10); expectReplayOf(10) + collect(a, 10) // now b's slow + cancel(e11) + collect(b, 10); emitResumes(e12); expectReplayOf(12) + collect(a, 12) + collect(b, 12) + sharedFlow.resetReplayCache(); expectReplayOf() + sharedFlow.resetReplayCache(); expectReplayOf() // nothing is buffered -- both collectors up to date + emitRightNow(13); expectReplayOf(13) + collect(b, 13) // a is slow + val e14 = emitSuspends(14) + val e15 = emitSuspends(15) + val e16 = emitSuspends(16) + cancel(e14) + cancel(a); emitResumes(e15); expectReplayOf(15) // cancelling slow subscriber + collect(b, 15); emitResumes(e16); expectReplayOf(16) + collect(b, 16) + } + + @Test + fun testReplay2Extra2DropOldest() = + testSharedFlow(MutableSharedFlow(2, 2, BufferOverflow.DROP_OLDEST)) { + emitRightNow(0); expectReplayOf(0) + emitRightNow(1); expectReplayOf(0, 1) + emitRightNow(2); expectReplayOf(1, 2) + emitRightNow(3); expectReplayOf(2, 3) + emitRightNow(4); expectReplayOf(3, 4) + val a = subscribe("a") + collect(a, 3) + emitRightNow(5); expectReplayOf(4, 5) + emitRightNow(6); expectReplayOf(5, 6) + emitRightNow(7); expectReplayOf(6, 7) // buffer 4, 5 | 6, 7 + emitRightNow(8); expectReplayOf(7, 8) // buffer 5, 6 | 7, 8 + emitRightNow(9); expectReplayOf(8, 9) // buffer 6, 7 | 8, 9 + collect(a, 6, 7) + val b = subscribe("b") + collect(b, 8, 9) // buffer | 8, 9 + emitRightNow(10); expectReplayOf(9, 10) // buffer 8 | 9, 10 + collect(a, 8, 9, 10) // buffer | 9, 10, note "b" had not collected 10 yet + emitRightNow(11); expectReplayOf(10, 11) // buffer | 10, 11 + emitRightNow(12); expectReplayOf(11, 12) // buffer 10 | 11, 12 + emitRightNow(13); expectReplayOf(12, 13) // buffer 10, 11 | 12, 13 + emitRightNow(14); expectReplayOf(13, 14) // buffer 11, 12 | 13, 14, "b" missed 10 + collect(b, 11, 12, 13, 14) + sharedFlow.resetReplayCache(); expectReplayOf() // buffer 11, 12, 13, 14 | + sharedFlow.resetReplayCache(); expectReplayOf() + collect(a, 11, 12, 13, 14) + emitRightNow(15); expectReplayOf(15) + collect(a, 15) + collect(b, 15) + } + + @Test // https://github.com/Kotlin/kotlinx.coroutines/issues/2320 + fun testResumeFastSubscriberOnResumedEmitter() = + testSharedFlow(MutableSharedFlow(1)) { + // create two subscribers and start collecting + val s1 = subscribe("s1"); resumeCollecting(s1) + val s2 = subscribe("s2"); resumeCollecting(s2) + // now emit 0, make sure it is collected + emitRightNow(0); expectReplayOf(0) + awaitCollected(s1, 0) + awaitCollected(s2, 0) + // now emit 1, and only first subscriber continues and collects it + emitRightNow(1); expectReplayOf(1) + collect(s1, 1) + // now emit 2, it suspend (s2 is blocking it) + val e2 = emitSuspends(2) + resumeCollecting(s1) // resume, but does not collect (e2 is still queued) + collect(s2, 1) // resume + collect next --> resumes emitter, thus resumes s1 + awaitCollected(s1, 2) // <-- S1 collects value from the newly resumed emitter here !!! + emitResumes(e2); expectReplayOf(2) + // now emit 3, it suspends (s2 blocks it) + val e3 = emitSuspends(3) + collect(s2, 2) + emitResumes(e3); expectReplayOf(3) + } + + @Test + fun testSuspendedConcurrentEmitAndCancelSubscriberReplay1() = + testSharedFlow(MutableSharedFlow(1)) { + val a = subscribe("a"); + emitRightNow(0); expectReplayOf(0) + collect(a, 0) + emitRightNow(1); expectReplayOf(1) + val e2 = emitSuspends(2) // suspends until 1 is collected + val e3 = emitSuspends(3) // suspends until 1 is collected, too + cancel(a) // must resume emitters 2 & 3 + emitResumes(e2) + emitResumes(e3) + expectReplayOf(3) // but replay size is 1 so only 3 should be kept + // Note: originally, SharedFlow was in a broken state here with 3 elements in the buffer + val b = subscribe("b") + collect(b, 3) + emitRightNow(4); expectReplayOf(4) + collect(b, 4) + } + + @Test + fun testSuspendedConcurrentEmitAndCancelSubscriberReplay1ExtraBuffer1() = + testSharedFlow(MutableSharedFlow( replay = 1, extraBufferCapacity = 1)) { + val a = subscribe("a"); + emitRightNow(0); expectReplayOf(0) + collect(a, 0) + emitRightNow(1); expectReplayOf(1) + emitRightNow(2); expectReplayOf(2) + val e3 = emitSuspends(3) // suspends until 1 is collected + val e4 = emitSuspends(4) // suspends until 1 is collected, too + val e5 = emitSuspends(5) // suspends until 1 is collected, too + cancel(a) // must resume emitters 3, 4, 5 + emitResumes(e3) + emitResumes(e4) + emitResumes(e5) + expectReplayOf(5) + val b = subscribe("b") + collect(b, 5) + emitRightNow(6); expectReplayOf(6) + collect(b, 6) + } + + private fun testSharedFlow( + sharedFlow: MutableSharedFlow, + scenario: suspend ScenarioDsl.() -> Unit + ) = runTest { + var dsl: ScenarioDsl? = null + try { + coroutineScope { + dsl = ScenarioDsl(sharedFlow, coroutineContext) + dsl!!.scenario() + dsl!!.stop() + } + } catch (e: Throwable) { + dsl?.printLog() + throw e + } + } + + private data class TestJob(val job: Job, val name: String) { + override fun toString(): String = name + } + + private open class Action + private data class EmitResumes(val job: TestJob) : Action() + private data class Collected(val job: TestJob, val value: Any?) : Action() + private data class ResumeCollecting(val job: TestJob) : Action() + private data class Cancelled(val job: TestJob) : Action() + + @OptIn(ExperimentalStdlibApi::class) + private class ScenarioDsl( + val sharedFlow: MutableSharedFlow, + coroutineContext: CoroutineContext + ) { + private val log = ArrayList() + private val timeout = 10000L + private val scope = CoroutineScope(coroutineContext + Job()) + private val actions = HashSet() + private val actionWaiters = ArrayDeque>() + private var expectedReplay = emptyList() + + private fun checkReplay() { + assertEquals(expectedReplay, sharedFlow.replayCache) + } + + private fun wakeupWaiters() { + repeat(actionWaiters.size) { + actionWaiters.removeFirst().resume(Unit) + } + } + + private fun addAction(action: Action) { + actions.add(action) + wakeupWaiters() + } + + private suspend fun awaitAction(action: Action) { + withTimeoutOrNull(timeout) { + while (!actions.remove(action)) { + suspendCancellableCoroutine { actionWaiters.add(it) } + } + } ?: error("Timed out waiting for action: $action") + wakeupWaiters() + } + + private fun launchEmit(a: T): TestJob { + val name = "emit($a)" + val job = scope.launch(start = CoroutineStart.UNDISPATCHED) { + val job = TestJob(coroutineContext[Job]!!, name) + try { + log(name) + sharedFlow.emit(a) + log("$name resumes") + addAction(EmitResumes(job)) + } catch(e: CancellationException) { + log("$name cancelled") + addAction(Cancelled(job)) + } + } + return TestJob(job, name) + } + + fun expectReplayOf(vararg a: T) { + expectedReplay = a.toList() + checkReplay() + } + + fun emitRightNow(a: T) { + val job = launchEmit(a) + assertTrue(actions.remove(EmitResumes(job))) + } + + fun emitSuspends(a: T): TestJob { + val job = launchEmit(a) + assertFalse(EmitResumes(job) in actions) + checkReplay() + return job + } + + suspend fun emitResumes(job: TestJob) { + awaitAction(EmitResumes(job)) + } + + suspend fun cancel(job: TestJob) { + log("cancel(${job.name})") + job.job.cancel() + awaitAction(Cancelled(job)) + } + + fun subscribe(id: String): TestJob { + val name = "collect($id)" + val job = scope.launch(start = CoroutineStart.UNDISPATCHED) { + val job = TestJob(coroutineContext[Job]!!, name) + try { + awaitAction(ResumeCollecting(job)) + log("$name start") + sharedFlow.collect { value -> + log("$name -> $value") + addAction(Collected(job, value)) + awaitAction(ResumeCollecting(job)) + log("$name -> $value resumes") + } + error("$name completed") + } catch(e: CancellationException) { + log("$name cancelled") + addAction(Cancelled(job)) + } + } + return TestJob(job, name) + } + + // collect ~== resumeCollecting + awaitCollected (for each value) + suspend fun collect(job: TestJob, vararg a: T) { + for (value in a) { + checkReplay() // should not have changed + resumeCollecting(job) + awaitCollected(job, value) + } + } + + suspend fun resumeCollecting(job: TestJob) { + addAction(ResumeCollecting(job)) + } + + suspend fun awaitCollected(job: TestJob, value: T) { + awaitAction(Collected(job, value)) + } + + fun stop() { + log("--- stop") + scope.cancel() + } + + private fun log(text: String) { + log.add(text) + } + + fun printLog() { + println("--- The most recent log entries ---") + log.takeLast(30).forEach(::println) + println("--- That's it ---") + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/SharedFlowTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/SharedFlowTest.kt new file mode 100644 index 0000000000..eab4b79ab5 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/SharedFlowTest.kt @@ -0,0 +1,838 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.random.* +import kotlin.test.* + +/** + * This test suite contains some basic tests for [SharedFlow]. There are some scenarios here written + * using [expect] and they are not very readable. See [SharedFlowScenarioTest] for a better + * behavioral test-suit. + */ +class SharedFlowTest : TestBase() { + @Test + fun testRendezvousSharedFlowBasic() = runTest { + expect(1) + val sh = MutableSharedFlow() + assertTrue(sh.replayCache.isEmpty()) + assertEquals(0, sh.subscriptionCount.value) + sh.emit(1) // no suspend + assertTrue(sh.replayCache.isEmpty()) + assertEquals(0, sh.subscriptionCount.value) + expect(2) + // one collector + val job1 = launch(start = CoroutineStart.UNDISPATCHED) { + expect(3) + sh.collect { + when(it) { + 4 -> expect(5) + 6 -> expect(7) + 10 -> expect(11) + 13 -> expect(14) + else -> expectUnreached() + } + } + expectUnreached() // does not complete normally + } + expect(4) + assertEquals(1, sh.subscriptionCount.value) + sh.emit(4) + assertTrue(sh.replayCache.isEmpty()) + expect(6) + sh.emit(6) + expect(8) + // one more collector + val job2 = launch(start = CoroutineStart.UNDISPATCHED) { + expect(9) + sh.collect { + when(it) { + 10 -> expect(12) + 13 -> expect(15) + 17 -> expect(18) + null -> expect(20) + 21 -> expect(22) + else -> expectUnreached() + } + } + expectUnreached() // does not complete normally + } + expect(10) + assertEquals(2, sh.subscriptionCount.value) + sh.emit(10) // to both collectors now! + assertTrue(sh.replayCache.isEmpty()) + expect(13) + sh.emit(13) + expect(16) + job1.cancel() // cancel the first collector + yield() + assertEquals(1, sh.subscriptionCount.value) + expect(17) + sh.emit(17) // only to second collector + expect(19) + sh.emit(null) // emit null to the second collector + expect(21) + sh.emit(21) // non-null again + expect(23) + job2.cancel() // cancel the second collector + yield() + assertEquals(0, sh.subscriptionCount.value) + expect(24) + sh.emit(24) // does not go anywhere + assertEquals(0, sh.subscriptionCount.value) + assertTrue(sh.replayCache.isEmpty()) + finish(25) + } + + @Test + fun testRendezvousSharedFlowReset() = runTest { + expect(1) + val sh = MutableSharedFlow() + val barrier = Channel(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + sh.collect { + when (it) { + 3 -> { + expect(4) + barrier.receive() // hold on before collecting next one + } + 6 -> expect(10) + else -> expectUnreached() + } + } + expectUnreached() // does not complete normally + } + expect(3) + sh.emit(3) // rendezvous + expect(5) + assertFalse(sh.tryEmit(5)) // collector is not ready now + launch(start = CoroutineStart.UNDISPATCHED) { + expect(6) + sh.emit(6) // suspends + expect(12) + } + expect(7) + yield() // no wakeup -> all suspended + expect(8) + // now reset cache -> nothing happens, there is no cache + sh.resetReplayCache() + yield() + expect(9) + // now resume collector + barrier.send(Unit) + yield() // to collector + expect(11) + yield() // to emitter + expect(13) + assertFalse(sh.tryEmit(13)) // rendezvous does not work this way + job.cancel() + finish(14) + } + + @Test + fun testReplay1SharedFlowBasic() = runTest { + expect(1) + val sh = MutableSharedFlow(1) + assertTrue(sh.replayCache.isEmpty()) + assertEquals(0, sh.subscriptionCount.value) + sh.emit(1) // no suspend + assertEquals(listOf(1), sh.replayCache) + assertEquals(0, sh.subscriptionCount.value) + expect(2) + sh.emit(2) // no suspend + assertEquals(listOf(2), sh.replayCache) + expect(3) + // one collector + val job1 = launch(start = CoroutineStart.UNDISPATCHED) { + expect(4) + sh.collect { + when(it) { + 2 -> expect(5) // got it immediately from replay cache + 6 -> expect(8) + null -> expect(14) + 17 -> expect(18) + else -> expectUnreached() + } + } + expectUnreached() // does not complete normally + } + expect(6) + assertEquals(1, sh.subscriptionCount.value) + sh.emit(6) // does not suspend, but buffers + assertEquals(listOf(6), sh.replayCache) + expect(7) + yield() + expect(9) + // one more collector + val job2 = launch(start = CoroutineStart.UNDISPATCHED) { + expect(10) + sh.collect { + when(it) { + 6 -> expect(11) // from replay cache + null -> expect(15) + else -> expectUnreached() + } + } + expectUnreached() // does not complete normally + } + expect(12) + assertEquals(2, sh.subscriptionCount.value) + sh.emit(null) + expect(13) + assertEquals(listOf(null), sh.replayCache) + yield() + assertEquals(listOf(null), sh.replayCache) + expect(16) + job2.cancel() + yield() + assertEquals(1, sh.subscriptionCount.value) + expect(17) + sh.emit(17) + assertEquals(listOf(17), sh.replayCache) + yield() + expect(19) + job1.cancel() + yield() + assertEquals(0, sh.subscriptionCount.value) + assertEquals(listOf(17), sh.replayCache) + finish(20) + } + + @Test + fun testReplay1() = runTest { + expect(1) + val sh = MutableSharedFlow(1) + assertEquals(listOf(), sh.replayCache) + val barrier = Channel(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + sh.collect { + when (it) { + 3 -> { + expect(4) + barrier.receive() // collector waits + } + 5 -> expect(10) + 6 -> expect(11) + else -> expectUnreached() + } + } + expectUnreached() // does not complete normally + } + expect(3) + assertTrue(sh.tryEmit(3)) // buffered + assertEquals(listOf(3), sh.replayCache) + yield() // to collector + expect(5) + assertTrue(sh.tryEmit(5)) // buffered + assertEquals(listOf(5), sh.replayCache) + launch(start = CoroutineStart.UNDISPATCHED) { + expect(6) + sh.emit(6) // buffer full, suspended + expect(13) + } + expect(7) + assertEquals(listOf(5), sh.replayCache) + sh.resetReplayCache() // clear cache + assertEquals(listOf(), sh.replayCache) + expect(8) + yield() // emitter still suspended + expect(9) + assertEquals(listOf(), sh.replayCache) + assertFalse(sh.tryEmit(10)) // still no buffer space + assertEquals(listOf(), sh.replayCache) + barrier.send(Unit) // resume collector + yield() // to collector + expect(12) + yield() // to emitter, that should have resumed + expect(14) + job.cancel() + assertEquals(listOf(6), sh.replayCache) + finish(15) + } + + @Test + fun testReplay2Extra1() = runTest { + expect(1) + val sh = MutableSharedFlow( + replay = 2, + extraBufferCapacity = 1 + ) + assertEquals(listOf(), sh.replayCache) + assertTrue(sh.tryEmit(0)) + assertEquals(listOf(0), sh.replayCache) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + var cnt = 0 + sh.collect { + when (it) { + 0 -> when (cnt++) { + 0 -> expect(3) + 1 -> expect(14) + else -> expectUnreached() + } + 1 -> expect(6) + 2 -> expect(7) + 3 -> expect(8) + 4 -> expect(12) + 5 -> expect(13) + 16 -> expect(17) + else -> expectUnreached() + } + } + expectUnreached() // does not complete normally + } + expect(4) + assertTrue(sh.tryEmit(1)) // buffered + assertEquals(listOf(0, 1), sh.replayCache) + assertTrue(sh.tryEmit(2)) // buffered + assertEquals(listOf(1, 2), sh.replayCache) + assertTrue(sh.tryEmit(3)) // buffered (buffer size is 3) + assertEquals(listOf(2, 3), sh.replayCache) + expect(5) + yield() // to collector + expect(9) + assertEquals(listOf(2, 3), sh.replayCache) + assertTrue(sh.tryEmit(4)) // can buffer now + assertEquals(listOf(3, 4), sh.replayCache) + assertTrue(sh.tryEmit(5)) // can buffer now + assertEquals(listOf(4, 5), sh.replayCache) + assertTrue(sh.tryEmit(0)) // can buffer one more, let it be zero again + assertEquals(listOf(5, 0), sh.replayCache) + expect(10) + assertFalse(sh.tryEmit(10)) // cannot buffer anymore! + sh.resetReplayCache() // replay cache + assertEquals(listOf(), sh.replayCache) // empty + assertFalse(sh.tryEmit(0)) // still cannot buffer anymore (reset does not help) + assertEquals(listOf(), sh.replayCache) // empty + expect(11) + yield() // resume collector, will get next values + expect(15) + sh.resetReplayCache() // reset again, nothing happens + assertEquals(listOf(), sh.replayCache) // empty + yield() // collector gets nothing -- no change + expect(16) + assertTrue(sh.tryEmit(16)) + assertEquals(listOf(16), sh.replayCache) + yield() // gets it + expect(18) + job.cancel() + finish(19) + } + + @Test + fun testBufferNoReplayCancelWhileBuffering() = runTest { + val n = 123 + val sh = MutableSharedFlow(replay = 0, extraBufferCapacity = n) + repeat(3) { + val m = n / 2 // collect half, then suspend + val barrier = Channel(1) + val collectorJob = sh + .onSubscription { + barrier.send(1) + } + .onEach { value -> + if (value == m) { + barrier.send(2) + delay(Long.MAX_VALUE) + } + } + .launchIn(this) + assertEquals(1, barrier.receive()) // make sure it subscribes + launch(start = CoroutineStart.UNDISPATCHED) { + for (i in 0 until n + m) sh.emit(i) // these emits should go Ok + barrier.send(3) + sh.emit(n + 4) // this emit will suspend on buffer overflow + barrier.send(4) + } + assertEquals(2, barrier.receive()) // wait until m collected + assertEquals(3, barrier.receive()) // wait until all are emitted + collectorJob.cancel() // cancelling collector job must clear buffer and resume emitter + assertEquals(4, barrier.receive()) // verify that emitter resumes + } + } + + @Test + fun testRepeatedResetWithReplay() = runTest { + val n = 10 + val sh = MutableSharedFlow(n) + var i = 0 + repeat(3) { + // collector is slow + val collector = sh.onEach { delay(Long.MAX_VALUE) }.launchIn(this) + val emitter = launch { + repeat(3 * n) { sh.emit(i); i++ } + } + repeat(3) { yield() } // enough to run it to suspension + assertEquals((i - n until i).toList(), sh.replayCache) + sh.resetReplayCache() + assertEquals(emptyList(), sh.replayCache) + repeat(3) { yield() } // enough to run it to suspension + assertEquals(emptyList(), sh.replayCache) // still blocked + collector.cancel() + emitter.cancel() + repeat(3) { yield() } // enough to run it to suspension + } + } + + @Test + fun testSynchronousSharedFlowEmitterCancel() = runTest { + expect(1) + val sh = MutableSharedFlow() + val barrier1 = Job() + val barrier2 = Job() + val barrier3 = Job() + val collector1 = sh.onEach { + when (it) { + 1 -> expect(3) + 2 -> { + expect(6) + barrier2.complete() + } + 3 -> { + expect(9) + barrier3.complete() + } + else -> expectUnreached() + } + }.launchIn(this) + val collector2 = sh.onEach { + when (it) { + 1 -> { + expect(4) + barrier1.complete() + delay(Long.MAX_VALUE) + } + else -> expectUnreached() + } + }.launchIn(this) + repeat(2) { yield() } // launch both subscribers + val emitter = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + sh.emit(1) + barrier1.join() + expect(5) + sh.emit(2) // suspends because of slow collector2 + expectUnreached() // will be cancelled + } + barrier2.join() // wait + expect(7) + // Now cancel the emitter! + emitter.cancel() + yield() + // Cancel slow collector + collector2.cancel() + yield() + // emit to fast collector1 + expect(8) + sh.emit(3) + barrier3.join() + expect(10) + // cancel it, too + collector1.cancel() + finish(11) + } + + @Test + fun testDifferentBufferedFlowCapacities() = runTest { + if (isBoundByJsTestTimeout) return@runTest // Too slow for JS, bounded by 2 sec. default JS timeout + for (replay in 0..10) { + for (extraBufferCapacity in 0..5) { + if (replay == 0 && extraBufferCapacity == 0) continue // test only buffered shared flows + try { + val sh = MutableSharedFlow(replay, extraBufferCapacity) + // repeat the whole test a few times to make sure it works correctly when slots are reused + repeat(3) { + testBufferedFlow(sh, replay) + } + } catch (e: Throwable) { + error("Failed for replay=$replay, extraBufferCapacity=$extraBufferCapacity", e) + } + } + } + } + + private suspend fun testBufferedFlow(sh: MutableSharedFlow, replay: Int) = withContext(Job()) { + reset() + expect(1) + val n = 100 // initially emitted to fill buffer + for (i in 1..n) assertTrue(sh.tryEmit(i)) + // initial expected replayCache + val rcStart = n - replay + 1 + val rcRange = rcStart..n + val rcSize = n - rcStart + 1 + assertEquals(rcRange.toList(), sh.replayCache) + // create collectors + val m = 10 // collectors created + var ofs = 0 + val k = 42 // emissions to collectors + val ecRange = n + 1..n + k + val jobs = List(m) { jobIndex -> + launch(start = CoroutineStart.UNDISPATCHED) { + sh.collect { i -> + when (i) { + in rcRange -> expect(2 + i - rcStart + jobIndex * rcSize) + in ecRange -> expect(2 + ofs + jobIndex) + else -> expectUnreached() + } + } + expectUnreached() // does not complete normally + } + } + ofs = rcSize * m + 2 + expect(ofs) + // emit to all k times + for (p in ecRange) { + sh.emit(p) + expect(1 + ofs) // buffered, no suspend + yield() + ofs += 2 + m + expect(ofs) + } + assertEquals(ecRange.toList().takeLast(replay), sh.replayCache) + // cancel all collectors + jobs.forEach { it.cancel() } + yield() + // replay cache is still there + assertEquals(ecRange.toList().takeLast(replay), sh.replayCache) + finish(1 + ofs) + } + + @Test + fun testDropLatest() = testDropLatestOrOldest(BufferOverflow.DROP_LATEST) + + @Test + fun testDropOldest() = testDropLatestOrOldest(BufferOverflow.DROP_OLDEST) + + private fun testDropLatestOrOldest(bufferOverflow: BufferOverflow) = runTest { + reset() + expect(1) + val sh = MutableSharedFlow(1, onBufferOverflow = bufferOverflow) + sh.emit(1) + sh.emit(2) + // always keeps last w/o collectors + assertEquals(listOf(2), sh.replayCache) + assertEquals(0, sh.subscriptionCount.value) + // one collector + val valueAfterOverflow = when (bufferOverflow) { + BufferOverflow.DROP_OLDEST -> 5 + BufferOverflow.DROP_LATEST -> 4 + else -> error("not supported in this test: $bufferOverflow") + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + sh.collect { + when(it) { + 2 -> { // replayed + expect(3) + yield() // and suspends, busy waiting + } + valueAfterOverflow -> expect(7) + 8 -> expect(9) + else -> expectUnreached() + } + } + expectUnreached() // does not complete normally + } + expect(4) + assertEquals(1, sh.subscriptionCount.value) + assertEquals(listOf(2), sh.replayCache) + sh.emit(4) // buffering, collector is busy + assertEquals(listOf(4), sh.replayCache) + expect(5) + sh.emit(5) // Buffer overflow here, will not suspend + assertEquals(listOf(valueAfterOverflow), sh.replayCache) + expect(6) + yield() // to the job + expect(8) + sh.emit(8) // not busy now + assertEquals(listOf(8), sh.replayCache) // buffered + yield() // to process + expect(10) + job.cancel() // cancel the job + yield() + assertEquals(0, sh.subscriptionCount.value) + finish(11) + } + + @Test + public fun testOnSubscription() = runTest { + expect(1) + val sh = MutableSharedFlow() + fun share(s: String) { launch(start = CoroutineStart.UNDISPATCHED) { sh.emit(s) } } + sh + .onSubscription { + emit("collector->A") + share("share->A") + } + .onSubscription { + emit("collector->B") + share("share->B") + } + .onStart { + emit("collector->C") + share("share->C") // get's lost, no subscribers yet + } + .onStart { + emit("collector->D") + share("share->D") // get's lost, no subscribers yet + } + .onEach { + when (it) { + "collector->D" -> expect(2) + "collector->C" -> expect(3) + "collector->A" -> expect(4) + "collector->B" -> expect(5) + "share->A" -> expect(6) + "share->B" -> { + expect(7) + currentCoroutineContext().cancel() + } + else -> expectUnreached() + } + } + .launchIn(this) + .join() + finish(8) + } + + @Test + @Suppress("DEPRECATION") // 'catch' + fun onSubscriptionThrows() = runTest { + expect(1) + val sh = MutableSharedFlow(1) + sh.tryEmit("OK") // buffer a string + assertEquals(listOf("OK"), sh.replayCache) + sh + .onSubscription { + expect(2) + throw TestException() + } + .catch { e -> + assertIs(e) + expect(3) + } + .collect { + // onSubscription throw before replay is emitted, so no value is collected if it throws + expectUnreached() + } + assertEquals(0, sh.subscriptionCount.value) + finish(4) + } + + @Test + fun testBigReplayManySubscribers() = testManySubscribers(true) + + @Test + fun testBigBufferManySubscribers() = testManySubscribers(false) + + private fun testManySubscribers(replay: Boolean) = runTest { + val n = 100 + val rnd = Random(replay.hashCode()) + val sh = MutableSharedFlow( + replay = if (replay) n else 0, + extraBufferCapacity = if (replay) 0 else n + ) + val subs = ArrayList() + for (i in 1..n) { + sh.emit(i) + val subBarrier = Channel() + val subJob = SubJob() + subs += subJob + // will receive all starting from replay or from new emissions only + subJob.lastReceived = if (replay) 0 else i + subJob.job = sh + .onSubscription { + subBarrier.send(Unit) // signal subscribed + } + .onEach { value -> + assertEquals(subJob.lastReceived + 1, value) + subJob.lastReceived = value + } + .launchIn(this) + subBarrier.receive() // wait until subscribed + // must have also receive all from the replay buffer directly after being subscribed + assertEquals(subJob.lastReceived, i) + // 50% of time cancel one subscriber + if (i % 2 == 0) { + val victim = subs.removeAt(rnd.nextInt(subs.size)) + yield() // make sure victim processed all emissions + assertEquals(victim.lastReceived, i) + victim.job.cancel() + } + } + yield() // make sure the last emission is processed + for (subJob in subs) { + assertEquals(subJob.lastReceived, n) + subJob.job.cancel() + } + } + + private class SubJob { + lateinit var job: Job + var lastReceived = 0 + } + + @Test + fun testStateFlowModel() = runTest { + if (isBoundByJsTestTimeout) return@runTest // Too slow for JS, bounded by 2 sec. default JS timeout + val stateFlow = MutableStateFlow(null) + val expect = modelLog(stateFlow) + val sharedFlow = MutableSharedFlow( + replay = 1, + onBufferOverflow = BufferOverflow.DROP_OLDEST + ) + sharedFlow.tryEmit(null) // initial value + val actual = modelLog(sharedFlow) { distinctUntilChanged() } + for (i in 0 until minOf(expect.size, actual.size)) { + if (actual[i] != expect[i]) { + for (j in maxOf(0, i - 10)..i) println("Actual log item #$j: ${actual[j]}") + assertEquals(expect[i], actual[i], "Log item #$i") + } + } + assertEquals(expect.size, actual.size) + } + + private suspend fun modelLog( + sh: MutableSharedFlow, + op: Flow.() -> Flow = { this } + ): List = coroutineScope { + val rnd = Random(1) + val result = ArrayList() + val job = launch { + sh.op().collect { value -> + result.add("Collect: $value") + repeat(rnd.nextInt(0..2)) { + result.add("Collect: yield") + yield() + } + } + } + repeat(1000) { + val value = if (rnd.nextBoolean()) null else rnd.nextData() + if (rnd.nextInt(20) == 0) { + result.add("resetReplayCache & emit: $value") + if (sh !is StateFlow<*>) sh.resetReplayCache() + assertTrue(sh.tryEmit(value)) + } else { + result.add("Emit: $value") + sh.emit(value) + } + repeat(rnd.nextInt(0..2)) { + result.add("Emit: yield") + yield() + } + } + result.add("main: cancel") + job.cancel() + result.add("main: yield") + yield() + result.add("main: join") + job.join() + result + } + + data class Data(val x: Int) + private val dataCache = (1..5).associateWith { Data(it) } + + // Note that we test proper null support here, too + private fun Random.nextData(): Data? { + val x = nextInt(0..5) + if (x == 0) return null + // randomly reuse ref or create a new instance + return if(nextBoolean()) dataCache[x] else Data(x) + } + + @Test + fun testOperatorFusion() { + val sh = MutableSharedFlow() + assertSame(sh, (sh as Flow<*>).cancellable()) + assertSame(sh, (sh as Flow<*>).flowOn(Dispatchers.Default)) + assertSame(sh, sh.buffer(Channel.RENDEZVOUS)) + } + + @Test + fun testIllegalArgumentException() { + assertFailsWith { MutableSharedFlow(-1) } + assertFailsWith { MutableSharedFlow(0, extraBufferCapacity = -1) } + assertFailsWith { MutableSharedFlow(0, onBufferOverflow = BufferOverflow.DROP_LATEST) } + assertFailsWith { MutableSharedFlow(0, onBufferOverflow = BufferOverflow.DROP_OLDEST) } + } + + @Test + public fun testReplayCancellability() = testCancellability(fromReplay = true) + + @Test + public fun testEmitCancellability() = testCancellability(fromReplay = false) + + private fun testCancellability(fromReplay: Boolean) = runTest { + expect(1) + val sh = MutableSharedFlow(5) + fun emitTestData() { + for (i in 1..5) assertTrue(sh.tryEmit(i)) + } + if (fromReplay) emitTestData() // fill in replay first + var subscribed = true + val job = sh + .onSubscription { subscribed = true } + .onEach { i -> + when (i) { + 1 -> expect(2) + 2 -> expect(3) + 3 -> { + expect(4) + currentCoroutineContext().cancel() + } + else -> expectUnreached() // shall check for cancellation + } + } + .launchIn(this) + yield() + assertTrue(subscribed) // yielding in enough + if (!fromReplay) emitTestData() // emit after subscription + job.join() + finish(5) + } + + @Test + fun testSubscriptionCount() = runTest { + val flow = MutableSharedFlow() + fun startSubscriber() = launch(start = CoroutineStart.UNDISPATCHED) { flow.collect() } + + assertEquals(0, flow.subscriptionCount.first()) + + val j1 = startSubscriber() + assertEquals(1, flow.subscriptionCount.first()) + + val j2 = startSubscriber() + assertEquals(2, flow.subscriptionCount.first()) + + j1.cancelAndJoin() + assertEquals(1, flow.subscriptionCount.first()) + + j2.cancelAndJoin() + assertEquals(0, flow.subscriptionCount.first()) + } + + @Test + fun testSubscriptionByFirstSuspensionInSharedFlow() = runTest { + testSubscriptionByFirstSuspensionInCollect(MutableSharedFlow()) { emit(it) } + } +} + +/** + * Check that, by the time [SharedFlow.collect] suspends for the first time, its subscription is already active. + */ +inline fun> CoroutineScope.testSubscriptionByFirstSuspensionInCollect(flow: T, emit: T.(Int) -> Unit) { + var received = 0 + val job = launch(start = CoroutineStart.UNDISPATCHED) { + flow.collect { + received = it + } + } + flow.emit(1) + assertEquals(1, received) + job.cancel() +} diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/SharingStartedTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/SharingStartedTest.kt new file mode 100644 index 0000000000..09450a1c50 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/SharingStartedTest.kt @@ -0,0 +1,180 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.test.* + +/** + * Functional tests for [SharingStarted] using [withVirtualTime] and a DSL to describe + * testing scenarios and expected behavior for different implementations. + */ +class SharingStartedTest : TestBase() { + @Test + fun testEagerly() = + testSharingStarted(SharingStarted.Eagerly, SharingCommand.START) { + subscriptions(1) + rampUpAndDown() + subscriptions(0) + delay(100) + } + + @Test + fun testLazily() = + testSharingStarted(SharingStarted.Lazily) { + subscriptions(1, SharingCommand.START) + rampUpAndDown() + subscriptions(0) + } + + @Test + fun testWhileSubscribed() = + testSharingStarted(SharingStarted.WhileSubscribed()) { + subscriptions(1, SharingCommand.START) + rampUpAndDown() + subscriptions(0, SharingCommand.STOP) + delay(100) + } + + @Test + fun testWhileSubscribedExpireImmediately() = + testSharingStarted(SharingStarted.WhileSubscribed(replayExpirationMillis = 0)) { + subscriptions(1, SharingCommand.START) + rampUpAndDown() + subscriptions(0, SharingCommand.STOP_AND_RESET_REPLAY_CACHE) + delay(100) + } + + @Test + fun testWhileSubscribedWithTimeout() = + testSharingStarted(SharingStarted.WhileSubscribed(stopTimeoutMillis = 100)) { + subscriptions(1, SharingCommand.START) + rampUpAndDown() + subscriptions(0) + delay(50) // don't give it time to stop + subscriptions(1) // resubscribe again + rampUpAndDown() + subscriptions(0) + afterTime(100, SharingCommand.STOP) + delay(100) + } + + @Test + fun testWhileSubscribedExpiration() = + testSharingStarted(SharingStarted.WhileSubscribed(replayExpirationMillis = 200)) { + subscriptions(1, SharingCommand.START) + rampUpAndDown() + subscriptions(0, SharingCommand.STOP) + delay(150) // don't give it time to reset cache + subscriptions(1, SharingCommand.START) + rampUpAndDown() + subscriptions(0, SharingCommand.STOP) + afterTime(200, SharingCommand.STOP_AND_RESET_REPLAY_CACHE) + } + + @Test + fun testWhileSubscribedStopAndExpiration() = + testSharingStarted(SharingStarted.WhileSubscribed(stopTimeoutMillis = 400, replayExpirationMillis = 300)) { + subscriptions(1, SharingCommand.START) + rampUpAndDown() + subscriptions(0) + delay(350) // don't give it time to stop + subscriptions(1) + rampUpAndDown() + subscriptions(0) + afterTime(400, SharingCommand.STOP) + delay(250) // don't give it time to reset cache + subscriptions(1, SharingCommand.START) + rampUpAndDown() + subscriptions(0) + afterTime(400, SharingCommand.STOP) + afterTime(300, SharingCommand.STOP_AND_RESET_REPLAY_CACHE) + delay(100) + } + + private suspend fun SharingStartedDsl.rampUpAndDown() { + for (i in 2..10) { + delay(100) + subscriptions(i) + } + delay(1000) + for (i in 9 downTo 1) { + subscriptions(i) + delay(100) + } + } + + private fun testSharingStarted( + started: SharingStarted, + initialCommand: SharingCommand? = null, + scenario: suspend SharingStartedDsl.() -> Unit + ) = withVirtualTime { + expect(1) + val dsl = SharingStartedDsl(started, initialCommand, coroutineContext) + dsl.launch() + // repeat every scenario 3 times + repeat(3) { + dsl.scenario() + delay(1000) + } + dsl.stop() + finish(2) + } + + private class SharingStartedDsl( + val started: SharingStarted, + initialCommand: SharingCommand?, + coroutineContext: CoroutineContext + ) { + val subscriptionCount = MutableStateFlow(0) + var previousCommand: SharingCommand? = null + var expectedCommand: SharingCommand? = initialCommand + var expectedTime = 0L + + val dispatcher = coroutineContext[ContinuationInterceptor] as VirtualTimeDispatcher + val scope = CoroutineScope(coroutineContext + Job()) + + suspend fun launch() { + started + .command(subscriptionCount.asStateFlow()) + .onEach { checkCommand(it) } + .launchIn(scope) + letItRun() + } + + fun checkCommand(command: SharingCommand) { + assertTrue(command != previousCommand) + previousCommand = command + assertEquals(expectedCommand, command) + assertEquals(expectedTime, dispatcher.currentTime) + } + + suspend fun subscriptions(count: Int, command: SharingCommand? = null) { + expectedTime = dispatcher.currentTime + subscriptionCount.value = count + if (command != null) { + afterTime(0, command) + } else { + letItRun() + } + } + + suspend fun afterTime(time: Long = 0, command: SharingCommand) { + expectedCommand = command + val remaining = (time - 1).coerceAtLeast(0) // previous letItRun delayed 1ms + expectedTime += remaining + delay(remaining) + letItRun() + } + + private suspend fun letItRun() { + delay(1) + assertEquals(expectedCommand, previousCommand) // make sure expected command was emitted + expectedTime++ // make one more time tick we've delayed + } + + fun stop() { + scope.cancel() + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/SharingStartedWhileSubscribedTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/SharingStartedWhileSubscribedTest.kt new file mode 100644 index 0000000000..c8199f51b2 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/SharingStartedWhileSubscribedTest.kt @@ -0,0 +1,76 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.seconds + +class SharingStartedWhileSubscribedTest : TestBase() { + @Test // make sure equals works properly, or otherwise other tests don't make sense + fun testEqualsAndHashcode() { + val params = listOf(0L, 1L, 10L, 100L, 213L, Long.MAX_VALUE) + // HashMap will simultaneously test equals, hashcode and their consistency + val map = HashMap>() + for (i in params) { + for (j in params) { + map[SharingStarted.WhileSubscribed(i, j)] = i to j + } + } + for (i in params) { + for (j in params) { + assertEquals(i to j, map[SharingStarted.WhileSubscribed(i, j)]) + } + } + } + + @Test + fun testDurationParams() { + assertEquals(SharingStarted.WhileSubscribed(0), SharingStarted.WhileSubscribed(Duration.ZERO)) + assertEquals(SharingStarted.WhileSubscribed(10), SharingStarted.WhileSubscribed(10.milliseconds)) + assertEquals(SharingStarted.WhileSubscribed(1000), SharingStarted.WhileSubscribed(1.seconds)) + assertEquals(SharingStarted.WhileSubscribed(Long.MAX_VALUE), SharingStarted.WhileSubscribed(Duration.INFINITE)) + assertEquals(SharingStarted.WhileSubscribed(replayExpirationMillis = 0), SharingStarted.WhileSubscribed(replayExpiration = Duration.ZERO)) + assertEquals(SharingStarted.WhileSubscribed(replayExpirationMillis = 3), SharingStarted.WhileSubscribed( + replayExpiration = 3.milliseconds + )) + assertEquals(SharingStarted.WhileSubscribed(replayExpirationMillis = 7000), + SharingStarted.WhileSubscribed(replayExpiration = 7.seconds)) + assertEquals(SharingStarted.WhileSubscribed(replayExpirationMillis = Long.MAX_VALUE), SharingStarted.WhileSubscribed(replayExpiration = Duration.INFINITE)) + } + + @Test + fun testShouldRestart() = runTest { + var started = 0 + val flow = flow { + expect(1 + ++started) + emit(1) + hang { } + }.shareIn(this, SharingStarted.WhileSubscribed(100 /* ms */)) + + expect(1) + flow.first() + delay(200) + flow.first() + finish(4) + coroutineContext.job.cancelChildren() + } + + @Test + fun testImmediateUnsubscribe() = runTest { + val flow = flow { + expect(2) + emit(1) + hang { finish(4) } + }.shareIn(this, SharingStarted.WhileSubscribed(400, 0 /* ms */), 1) + + expect(1) + repeat(5) { + flow.first() + delay(100) + } + expect(3) + coroutineContext.job.cancelChildren() + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/StateFlowTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/StateFlowTest.kt new file mode 100644 index 0000000000..f8c1a835be --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/StateFlowTest.kt @@ -0,0 +1,186 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class StateFlowTest : TestBase() { + @Test + fun testNormalAndNull() = runTest { + expect(1) + val state = MutableStateFlow(0) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + assertFailsWith { + state.collect { value -> + when (value) { + 0 -> expect(3) + 1 -> expect(5) + null -> expect(8) + 2 -> expect(10) + else -> expectUnreached() + } + } + } + expect(12) + } + expect(4) // collector is waiting + state.value = 1 // fire in the hole! + assertEquals(1, state.value) + yield() + expect(6) + state.value = 1 // same value, nothing happens + yield() + expect(7) + state.value = null // null value + assertNull(state.value) + yield() + expect(9) + state.value = 2 // another value + assertEquals(2, state.value) + yield() + expect(11) + job.cancel() + yield() + finish(13) + } + + @Test + fun testEqualsConflation() = runTest { + expect(1) + val state = MutableStateFlow(Data(0)) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + assertFailsWith { + state.collect { value -> + when (value.i) { + 0 -> expect(3) // initial value + 2 -> expect(5) + 4 -> expect(7) + else -> error("Unexpected $value") + } + } + } + expect(9) + } + state.value = Data(1) // conflated + state.value = Data(0) // equals to last emitted + yield() // no repeat zero + state.value = Data(3) // conflated + state.value = Data(2) // delivered + expect(4) + yield() + state.value = Data(2) // equals to last one, dropped + yield() + state.value = Data(5) // conflated + state.value = Data(4) // delivered + expect(6) + yield() + expect(8) + job.cancel() + yield() + finish(10) + } + + data class Data(val i: Int) + + @Test + fun testDataModel() = runTest { + val s = CounterModel() + launch { + val sum = s.counter.take(11).sum() + assertEquals(55, sum) + } + repeat(10) { + yield() + s.inc() + } + } + + class CounterModel { + // private data flow + private val _counter = MutableStateFlow(0) + + // publicly exposed as a flow + val counter: StateFlow get() = _counter + + fun inc() { + _counter.value++ + } + } + + @Test + public fun testOnSubscriptionWithException() = runTest { + expect(1) + val state = MutableStateFlow("A") + state + .onSubscription { + emit("collector->A") + state.value = "A" + } + .onSubscription { + emit("collector->B") + state.value = "B" + throw TestException() + } + .onStart { + emit("collector->C") + state.value = "C" + } + .onStart { + emit("collector->D") + state.value = "D" + } + .onEach { + when (it) { + "collector->D" -> expect(2) + "collector->C" -> expect(3) + "collector->A" -> expect(4) + "collector->B" -> expect(5) + else -> expectUnreached() + } + } + .catch { e -> + assertIs(e) + expect(6) + } + .launchIn(this) + .join() + assertEquals(0, state.subscriptionCount.value) + finish(7) + } + + @Test + fun testOperatorFusion() { + val state = MutableStateFlow(String) + assertSame(state, (state as Flow<*>).cancellable()) + assertSame(state, (state as Flow<*>).distinctUntilChanged()) + assertSame(state, (state as Flow<*>).flowOn(Dispatchers.Default)) + assertSame(state, (state as Flow<*>).conflate()) + assertSame(state, state.buffer(Channel.CONFLATED)) + assertSame(state, state.buffer(Channel.RENDEZVOUS)) + } + + @Test + fun testResetUnsupported() { + val state = MutableStateFlow(42) + assertFailsWith { state.resetReplayCache() } + assertEquals(42, state.value) + assertEquals(listOf(42), state.replayCache) + } + + @Test + fun testUpdate() = runTest { + val state = MutableStateFlow(0) + state.update { it + 2 } + assertEquals(2, state.value) + state.update { it + 3 } + assertEquals(5, state.value) + } + + @Test + fun testSubscriptionByFirstSuspensionInStateFlow() = runTest { + testSubscriptionByFirstSuspensionInCollect(MutableStateFlow(0)) { value = it; yield() } + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/sharing/StateInTest.kt b/kotlinx-coroutines-core/common/test/flow/sharing/StateInTest.kt new file mode 100644 index 0000000000..bb2139c4a8 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/sharing/StateInTest.kt @@ -0,0 +1,111 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.coroutines.* +import kotlin.test.* + +/** + * It is mostly covered by [ShareInTest], this just add state-specific checks. + */ +class StateInTest : TestBase() { + @Test + fun testOperatorFusion() = runTest { + val state = flowOf("OK").stateIn(this) + assertTrue(state !is MutableStateFlow<*>) // cannot be cast to mutable state flow + assertSame(state, (state as Flow<*>).cancellable()) + assertSame(state, (state as Flow<*>).distinctUntilChanged()) + assertSame(state, (state as Flow<*>).flowOn(Dispatchers.Default)) + assertSame(state, (state as Flow<*>).conflate()) + assertSame(state, state.buffer(Channel.CONFLATED)) + assertSame(state, state.buffer(Channel.RENDEZVOUS)) + assertSame(state, state.buffer(onBufferOverflow = BufferOverflow.DROP_OLDEST)) + assertSame(state, state.buffer(0, onBufferOverflow = BufferOverflow.DROP_OLDEST)) + assertSame(state, state.buffer(1, onBufferOverflow = BufferOverflow.DROP_OLDEST)) + coroutineContext.cancelChildren() + } + + @Test + fun testUpstreamCompletedNoInitialValue() = + testUpstreamCompletedOrFailedReset(failed = false, withInitialValue = false) + + @Test + fun testUpstreamFailedNoInitialValue() = + testUpstreamCompletedOrFailedReset(failed = true, withInitialValue = false) + + @Test + fun testUpstreamCompletedWithInitialValue() = + testUpstreamCompletedOrFailedReset(failed = false, withInitialValue = true) + + @Test + fun testUpstreamFailedWithInitialValue() = + testUpstreamCompletedOrFailedReset(failed = true, withInitialValue = true) + + private fun testUpstreamCompletedOrFailedReset(failed: Boolean, withInitialValue: Boolean) = runTest { + val emitted = Job() + val terminate = Job() + val sharingJob = CompletableDeferred() + val upstream = flow { + emit("OK") + emitted.complete() + terminate.join() + if (failed) throw TestException() + } + val scope = this + sharingJob + val shared: StateFlow + if (withInitialValue) { + shared = upstream.stateIn(scope, SharingStarted.Eagerly, null) + assertEquals(null, shared.value) + } else { + shared = upstream.stateIn(scope) + assertEquals("OK", shared.value) // waited until upstream emitted + } + emitted.join() // should start sharing, emit & cache + assertEquals("OK", shared.value) + terminate.complete() + sharingJob.complete(Unit) + sharingJob.join() // should complete sharing + assertEquals("OK", shared.value) // value is still there + if (failed) { + assertIs(sharingJob.getCompletionExceptionOrNull()) + } else { + assertNull(sharingJob.getCompletionExceptionOrNull()) + } + } + + @Test + fun testUpstreamFailedImmediatelyWithInitialValue() = runTest { + val ceh = CoroutineExceptionHandler { _, _ -> expect(2) } + val flow = flow { + expect(1) + throw TestException() + } + assertFailsWith { flow.stateIn(CoroutineScope(currentCoroutineContext() + Job() + ceh)) } + finish(3) + } + + @Test + fun testSubscriptionByFirstSuspensionInStateFlow() = runTest { + testSubscriptionByFirstSuspensionInCollect(flowOf(1).stateIn(this@runTest)) { } + } + + @Test + fun testRethrowsCEOnCancelledScope() = runTest { + val cancelledScope = CoroutineScope(EmptyCoroutineContext).apply { cancel("CancelMessageToken") } + val flow = flowOf(1, 2, 3) + assertFailsWith("CancelMessageToken") { + flow.stateIn(cancelledScope) + } + } + + @Test + fun testThrowsNoSuchElementExceptionOnEmptyFlow() = runTest { + val flow = emptyFlow() + assertFailsWith { + flow.stateIn(this) + } + // Ensure that the collecting scope is not cancelled by the NoSuchElementException + assertEquals(true, coroutineContext[Job]?.isActive) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/terminal/CollectLatestTest.kt b/kotlinx-coroutines-core/common/test/flow/terminal/CollectLatestTest.kt new file mode 100644 index 0000000000..2cecf8c43f --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/terminal/CollectLatestTest.kt @@ -0,0 +1,53 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class CollectLatestTest : TestBase() { + @Test + fun testNoSuspension() = runTest { + flowOf(1, 2, 3).collectLatest { + expect(it) + } + finish(4) + } + + @Test + fun testSuspension() = runTest { + flowOf(1, 2, 3).collectLatest { + yield() + expect(1) + } + finish(2) + } + + @Test + fun testUpstreamErrorSuspension() = runTest({it is TestException}) { + try { + flow { + emit(1) + throw TestException() + }.collectLatest { expect(1) } + expectUnreached() + } finally { + finish(2) + } + } + + @Test + fun testDownstreamError() = runTest({it is TestException}) { + try { + flow { + emit(1) + hang { expect(1) } + }.collectLatest { + throw TestException() + } + expectUnreached() + } finally { + finish(2) + } + + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/flow/terminal/CountTest.kt b/kotlinx-coroutines-core/common/test/flow/terminal/CountTest.kt new file mode 100644 index 0000000000..167fdec02c --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/terminal/CountTest.kt @@ -0,0 +1,44 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class CountTest : TestBase() { + @Test + fun testCount() = runTest { + val flow = flowOf(239, 240) + assertEquals(2, flow.count()) + assertEquals(2, flow.count { true }) + assertEquals(1, flow.count { it % 2 == 0}) + assertEquals(0, flow.count { false }) + } + + @Test + fun testNoValues() = runTest { + assertEquals(0, flowOf().count()) + assertEquals(0, flowOf().count { false }) + assertEquals(0, flowOf().count { true }) + } + + @Test + fun testException() = runTest { + val flow = flow { + throw TestException() + } + + assertFailsWith { flow.count() } + assertFailsWith { flow.count { false } } + } + + @Test + fun testExceptionAfterValue() = runTest { + val flow = flow { + emit(1) + throw TestException() + } + + assertFailsWith { flow.count() } + assertFailsWith { flow.count { false } } + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/terminal/FirstTest.kt b/kotlinx-coroutines-core/common/test/flow/terminal/FirstTest.kt new file mode 100644 index 0000000000..cdb36bd8f8 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/terminal/FirstTest.kt @@ -0,0 +1,195 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CoroutineStart.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.internal.* +import kotlin.test.* +import kotlin.time.* + +class FirstTest : TestBase() { + @Test + fun testFirst() = runTest { + val flow = flowOf(1, 2, 3) + assertEquals(1, flow.first()) + } + + @Test + fun testNulls() = runTest { + val flow = flowOf(null, 1) + assertNull(flow.first()) + assertNull(flow.first { it == null }) + assertEquals(1, flow.first { it != null }) + } + + @Test + fun testFirstWithPredicate() = runTest { + val flow = flowOf(1, 2, 3) + assertEquals(1, flow.first { it > 0 }) + assertEquals(2, flow.first { it > 1 }) + assertFailsWith { flow.first { it > 3 } } + } + + @Test + fun testFirstCancellation() = runTest { + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang { expect(1) } + } + emit(1) + emit(2) + } + } + + + val result = flow.first { + latch.receive() + true + } + assertEquals(1, result) + finish(2) + } + + @Test + fun testEmptyFlow() = runTest { + assertFailsWith { emptyFlow().first() } + assertFailsWith { emptyFlow().first { true } } + } + + @Test + fun testErrorCancelsUpstream() = runTest { + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang { expect(1) } + } + emit(1) + } + } + + assertFailsWith { + flow.first { + latch.receive() + throw TestException() + } + } + + assertEquals(1, flow.first()) + finish(2) + } + + @Test + fun testFirstOrNull() = runTest { + val flow = flowOf(1, 2, 3) + assertEquals(1, flow.firstOrNull()) + } + + @Test + fun testFirstOrNullWithPredicate() = runTest { + val flow = flowOf(1, 2, 3) + assertEquals(1, flow.firstOrNull { it > 0 }) + assertEquals(2, flow.firstOrNull { it > 1 }) + assertNull(flow.firstOrNull { it > 3 }) + } + + @Test + fun testFirstOrNullCancellation() = runTest { + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang { expect(1) } + } + emit(1) + emit(2) + } + } + + + val result = flow.firstOrNull { + latch.receive() + true + } + assertEquals(1, result) + finish(2) + } + + @Test + fun testFirstOrNullWithEmptyFlow() = runTest { + assertNull(emptyFlow().firstOrNull()) + assertNull(emptyFlow().firstOrNull { true }) + } + + @Test + fun testFirstOrNullWithNullElement() = runTest { + assertNull(flowOf(null).firstOrNull()) + assertNull(flowOf(null).firstOrNull { true }) + } + + @Test + fun testFirstOrNullWhenErrorCancelsUpstream() = runTest { + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + hang { expect(1) } + } + emit(1) + } + } + + assertFailsWith { + flow.firstOrNull { + latch.receive() + throw TestException() + } + } + + assertEquals(1, flow.firstOrNull()) + finish(2) + } + + @Test + fun testBadClass() = runTest { + val instance = BadClass() + val flow = flowOf(instance) + assertSame(instance, flow.first()) + assertSame(instance, flow.firstOrNull()) + assertSame(instance, flow.first { true }) + assertSame(instance, flow.firstOrNull { true }) + } + + @Test + fun testAbortFlowException() = runTest { + val flow = flow { + throw AbortFlowException(NopCollector) // Emulate cancellation + } + + assertFailsWith { flow.first() } + } + + @Test + fun testFirstThrowOnCancellation() = runTest { + val job = launch(start = UNDISPATCHED) { + flow { + try { + emit(Unit) + } finally { + runCatching { yield() } + finish(2) + } + }.first() + expectUnreached() + } + expect(1) + job.cancel() + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/flow/terminal/FoldTest.kt b/kotlinx-coroutines-core/common/test/flow/terminal/FoldTest.kt new file mode 100644 index 0000000000..350d0751aa --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/terminal/FoldTest.kt @@ -0,0 +1,51 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class FoldTest : TestBase() { + @Test + fun testFold() = runTest { + val flow = flow { + emit(1) + emit(2) + } + + val result = flow.fold(3) { value, acc -> value + acc } + assertEquals(6, result) + } + + @Test + fun testEmptyFold() = runTest { + val flow = flowOf() + assertEquals(42, flow.fold(42) { value, acc -> value + acc }) + } + + @Test + fun testErrorCancelsUpstream() = runTest { + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + expect(3) + hang { expect(5) } + } + expect(2) + emit(1) + } + } + + expect(1) + assertFailsWith { + flow.fold(42) { _, _ -> + latch.receive() + expect(4) + throw TestException() + } + } + finish(6) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/terminal/LastTest.kt b/kotlinx-coroutines-core/common/test/flow/terminal/LastTest.kt new file mode 100644 index 0000000000..b424b2e6de --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/terminal/LastTest.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class LastTest : TestBase() { + @Test + fun testLast() = runTest { + val flow = flowOf(1, 2, 3) + assertEquals(3, flow.last()) + assertEquals(3, flow.lastOrNull()) + } + + @Test + fun testNulls() = runTest { + val flow = flowOf(1, null) + assertNull(flow.last()) + assertNull(flow.lastOrNull()) + } + + @Test + fun testNullsLastOrNull() = runTest { + val flow = flowOf(null, 1) + assertEquals(1, flow.lastOrNull()) + } + + @Test + fun testEmptyFlow() = runTest { + assertFailsWith { emptyFlow().last() } + assertNull(emptyFlow().lastOrNull()) + } + + @Test + fun testBadClass() = runTest { + val instance = BadClass() + val flow = flowOf(instance) + assertSame(instance, flow.last()) + assertSame(instance, flow.lastOrNull()) + + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/terminal/LaunchInTest.kt b/kotlinx-coroutines-core/common/test/flow/terminal/LaunchInTest.kt new file mode 100644 index 0000000000..e2b01f635f --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/terminal/LaunchInTest.kt @@ -0,0 +1,53 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class LaunchInTest : TestBase() { + + @Test + fun testLaunchIn() = runTest { + val flow = flow { + expect(1) + emit(1) + throw TestException() + }.onEach { + assertEquals(1, it) + expect(2) + }.onCompletion { + assertIs(it) + expect(3) + }.catch { + assertTrue { it is TestException } + expect(4) + } + + flow.launchIn(this).join() + finish(5) + } + + @Test + fun testDispatcher() = runTest { + flow { + assertEquals("flow", NamedDispatchers.name()) + emit(1) + expect(1) + }.launchIn(this + NamedDispatchers("flow")).join() + finish(2) + } + + @Test + fun testUnhandledError() = runTest(expected = { it is TestException }) { + flow { + emit(1) + expect(1) + }.catch { + expectUnreached() + }.onCompletion { + finish(2) + throw TestException() + }.launchIn(this) + } + +} diff --git a/kotlinx-coroutines-core/common/test/flow/terminal/ReduceTest.kt b/kotlinx-coroutines-core/common/test/flow/terminal/ReduceTest.kt new file mode 100644 index 0000000000..ee70063cc6 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/terminal/ReduceTest.kt @@ -0,0 +1,71 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class ReduceTest : TestBase() { + @Test + fun testReduce() = runTest { + val flow = flow { + emit(1) + emit(2) + } + + val result = flow.reduce { value, acc -> value + acc } + assertEquals(3, result) + } + + @Test + fun testEmptyReduce() = runTest { + val flow = emptyFlow() + assertFailsWith { flow.reduce { acc, value -> value + acc } } + } + + @Test + fun testNullableReduce() = runTest { + val flow = flowOf(1, null, null, 2) + var invocations = 0 + val sum = flow.reduce { _, value -> + ++invocations + value + } + assertEquals(2, sum) + assertEquals(3, invocations) + } + + @Test + fun testReduceNulls() = runTest { + assertNull(flowOf(null).reduce { _, value -> value }) + assertNull(flowOf(null, null).reduce { _, value -> value }) + assertFailsWith { flowOf().reduce { _, value -> value } } + } + + @Test + fun testErrorCancelsUpstream() = runTest { + val latch = Channel() + val flow = flow { + coroutineScope { + launch { + latch.send(Unit) + expect(3) + hang { expect(5) } + } + expect(2) + emit(1) + emit(2) + } + } + + expect(1) + assertFailsWith { + flow.reduce { _, _ -> + latch.receive() + expect(4) + throw TestException() + } + } + finish(6) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/terminal/SingleTest.kt b/kotlinx-coroutines-core/common/test/flow/terminal/SingleTest.kt new file mode 100644 index 0000000000..b1741bb0b5 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/terminal/SingleTest.kt @@ -0,0 +1,91 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class SingleTest : TestBase() { + + @Test + fun testSingle() = runTest { + val flow = flow { + emit(239L) + } + + assertEquals(239L, flow.single()) + assertEquals(239L, flow.singleOrNull()) + } + + @Test + fun testMultipleValues() = runTest { + val flow = flow { + emit(239L) + emit(240L) + } + assertFailsWith { flow.single() } + assertNull(flow.singleOrNull()) + } + + @Test + fun testNoValues() = runTest { + assertFailsWith { flow {}.single() } + assertNull(flow {}.singleOrNull()) + } + + @Test + fun testException() = runTest { + val flow = flow { + throw TestException() + } + + assertFailsWith { flow.single() } + assertFailsWith { flow.singleOrNull() } + } + + @Test + fun testExceptionAfterValue() = runTest { + val flow = flow { + emit(1) + throw TestException() + } + + assertFailsWith { flow.single() } + assertFailsWith { flow.singleOrNull() } + } + + @Test + fun testNullableSingle() = runTest { + assertEquals(1, flowOf(1).single()) + assertNull(flowOf(null).single()) + assertFailsWith { flowOf().single() } + + assertEquals(1, flowOf(1).singleOrNull()) + assertNull(flowOf(null).singleOrNull()) + assertNull(flowOf().singleOrNull()) + } + + @Test + fun testBadClass() = runTest { + val instance = BadClass() + val flow = flowOf(instance) + assertSame(instance, flow.single()) + assertSame(instance, flow.singleOrNull()) + + val flow2 = flow { + emit(BadClass()) + emit(BadClass()) + } + assertFailsWith { flow2.single() } + } + + @Test + fun testSingleNoWait() = runTest { + val flow = flow { + emit(1) + emit(2) + awaitCancellation() + } + + assertNull(flow.singleOrNull()) + } +} diff --git a/kotlinx-coroutines-core/common/test/flow/terminal/ToCollectionTest.kt b/kotlinx-coroutines-core/common/test/flow/terminal/ToCollectionTest.kt new file mode 100644 index 0000000000..a5c5fe653f --- /dev/null +++ b/kotlinx-coroutines-core/common/test/flow/terminal/ToCollectionTest.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class ToCollectionTest : TestBase() { + + private val flow = flow { + repeat(10) { + emit(42) + } + } + + private val emptyFlow = flowOf() + + @Test + fun testToList() = runTest { + assertEquals(List(10) { 42 }, flow.toList()) + assertEquals(emptyList(), emptyFlow.toList()) + } + + @Test + fun testToSet() = runTest { + assertEquals(setOf(42), flow.toSet()) + assertEquals(emptySet(), emptyFlow.toSet()) + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectBiasTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectBiasTest.kt new file mode 100644 index 0000000000..42cdf94552 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectBiasTest.kt @@ -0,0 +1,41 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class SelectBiasTest : TestBase() { + val n = 10_000 + + @Test + fun testBiased() = runTest { + val d0 = async { 0 } + val d1 = async { 1 } + val counter = IntArray(2) + repeat(n) { + val selected = select { + d0.onAwait { 0 } + d1.onAwait { 1 } + } + counter[selected]++ + } + assertEquals(n, counter[0]) + assertEquals(0, counter[1]) + } + + @Test + fun testUnbiased() = runTest { + val d0 = async { 0 } + val d1 = async { 1 } + val counter = IntArray(2) + repeat(n) { + val selected = selectUnbiased { + d0.onAwait { 0 } + d1.onAwait { 1 } + } + counter[selected]++ + } + assertTrue(counter[0] >= n / 4) + assertTrue(counter[1] >= n / 4) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/selects/SelectBufferedChannelTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectBufferedChannelTest.kt new file mode 100644 index 0000000000..3abee423e0 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectBufferedChannelTest.kt @@ -0,0 +1,414 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class SelectBufferedChannelTest : TestBase() { + + @Test + fun testSelectSendSuccess() = runTest { + expect(1) + val channel = Channel(1) + launch { + expect(2) + assertEquals("OK", channel.receive()) + finish(6) + } + yield() // to launched coroutine + expect(3) + select { + channel.onSend("OK") { + expect(4) + } + } + expect(5) + } + + @Test + fun testSelectSendSuccessWithDefault() = runTest { + expect(1) + val channel = Channel(1) + launch { + expect(2) + assertEquals("OK", channel.receive()) + finish(6) + } + yield() // to launched coroutine + expect(3) + select { + channel.onSend("OK") { + expect(4) + } + default { + expectUnreached() + } + } + expect(5) + } + + @Test + fun testSelectSendReceiveBuf() = runTest { + expect(1) + val channel = Channel(1) + select { + channel.onSend("OK") { + expect(2) + } + } + expect(3) + select { + channel.onReceive { v -> + expect(4) + assertEquals("OK", v) + } + } + finish(5) + } + + @Test + fun testSelectSendWait() = runTest { + expect(1) + val channel = Channel(1) + launch { + expect(4) + assertEquals("BUF", channel.receive()) + expect(5) + assertEquals("OK", channel.receive()) + expect(6) + } + expect(2) + channel.send("BUF") + expect(3) + select { + channel.onSend("OK") { + expect(7) + } + } + finish(8) + } + + @Test + fun testSelectReceiveSuccess() = runTest { + expect(1) + val channel = Channel(1) + channel.send("OK") + expect(2) + select { + channel.onReceive { v -> + expect(3) + assertEquals("OK", v) + } + } + finish(4) + } + + @Test + fun testSelectReceiveSuccessWithDefault() = runTest { + expect(1) + val channel = Channel(1) + channel.send("OK") + expect(2) + select { + channel.onReceive { v -> + expect(3) + assertEquals("OK", v) + } + default { + expectUnreached() + } + } + finish(4) + } + + @Test + fun testSelectReceiveWaitWithDefault() = runTest { + expect(1) + val channel = Channel(1) + select { + channel.onReceive { + expectUnreached() + } + default { + expect(2) + } + } + expect(3) + channel.send("BUF") + expect(4) + // make sure second send blocks (select above is over) + launch { + expect(6) + channel.send("CHK") + finish(10) + } + expect(5) + yield() + expect(7) + assertEquals("BUF", channel.receive()) + expect(8) + assertEquals("CHK", channel.receive()) + expect(9) + } + + @Test + fun testSelectReceiveWait() = runTest { + expect(1) + val channel = Channel(1) + launch { + expect(3) + channel.send("OK") + expect(4) + } + expect(2) + select { + channel.onReceive { v -> + expect(5) + assertEquals("OK", v) + } + } + finish(6) + } + + @Test + fun testSelectReceiveClosed() = runTest({it is ClosedReceiveChannelException}) { + expect(1) + val channel = Channel(1) + channel.close() + finish(2) + select { + channel.onReceive { + expectUnreached() + } + } + expectUnreached() + } + + @Test + fun testSelectReceiveWaitClosed() = runTest({it is ClosedReceiveChannelException}) { + expect(1) + val channel = Channel(1) + launch { + expect(3) + channel.close() + finish(4) + } + expect(2) + select { + channel.onReceive { + expectUnreached() + } + } + expectUnreached() + } + + @Test + fun testSelectSendResourceCleanup() = runTest { + val channel = Channel(1) + val n = 1000 + expect(1) + channel.send(-1) // fill the buffer, so all subsequent sends cannot proceed + repeat(n) { i -> + select { + channel.onSend(i) { expectUnreached() } + default { expect(i + 2) } + } + } + finish(n + 2) + } + + @Test + fun testSelectReceiveResourceCleanup() = runTest { + val channel = Channel(1) + val n = 1000 + expect(1) + repeat(n) { i -> + select { + channel.onReceive { expectUnreached() } + default { expect(i + 2) } + } + } + finish(n + 2) + } + + @Test + fun testSelectReceiveDispatchNonSuspending() = runTest { + val channel = Channel(1) + expect(1) + channel.send(42) + expect(2) + launch { + expect(4) + select { + channel.onReceive { v -> + expect(5) + assertEquals(42, v) + expect(6) + } + } + expect(7) // returns from select without further dispatch + } + expect(3) + yield() // to launched + finish(8) + } + + @Test + fun testSelectReceiveDispatchNonSuspending2() = runTest { + val channel = Channel(1) + expect(1) + channel.send(42) + expect(2) + launch { + expect(4) + select { + channel.onReceive { v -> + expect(5) + assertEquals(42, v) + expect(6) + yield() // back to main + expect(8) + } + } + expect(9) // returns from select without further dispatch + } + expect(3) + yield() // to launched + expect(7) + yield() // again + finish(10) + } + + @Test + fun testSelectReceiveOrClosedWaitClosed() = runTest { + expect(1) + val channel = Channel(1) + launch { + expect(3) + channel.close() + expect(4) + } + expect(2) + select { + channel.onReceiveCatching { + expect(5) + assertTrue(it.isClosed) + assertNull(it.exceptionOrNull()) + } + } + + finish(6) + } + + @Test + fun testSelectReceiveOrClosedWaitClosedWithCause() = runTest { + expect(1) + val channel = Channel(1) + launch { + expect(3) + channel.close(TestException()) + expect(4) + } + expect(2) + select { + channel.onReceiveCatching { + expect(5) + assertTrue(it.isClosed) + assertIs(it.exceptionOrNull()) + } + } + + finish(6) + } + + @Test + fun testSelectReceiveCatching() = runTest { + val c = Channel(1) + val iterations = 10 + expect(1) + val job = launch { + repeat(iterations) { + select { + c.onReceiveCatching { v -> + expect(4 + it * 2) + assertEquals(it, v.getOrNull()) + } + } + } + } + + expect(2) + repeat(iterations) { + expect(3 + it * 2) + c.send(it) + yield() + } + + job.join() + finish(3 + iterations * 2) + } + + @Test + fun testSelectReceiveOrClosedDispatch() = runTest { + val c = Channel(1) + expect(1) + launch { + expect(3) + val res = select { + c.onReceiveCatching { v -> + expect(6) + assertEquals(42, v.getOrNull()) + yield() // back to main + expect(8) + "OK" + } + } + expect(9) + assertEquals("OK", res) + } + expect(2) + yield() // to launch + expect(4) + c.send(42) // do not suspend + expect(5) + yield() // to receive + expect(7) + yield() // again + finish(10) + } + + // only for debugging + internal fun SelectBuilder.default(block: suspend () -> R) = onTimeout(0, block) + + @Test + fun testSelectReceiveOrClosedForClosedChannel() = runTest { + val channel = Channel(1) + channel.close() + expect(1) + select { + expect(2) + channel.onReceiveCatching { + assertTrue(it.isClosed) + assertNull(it.exceptionOrNull()) + finish(3) + } + } + } + + @Test + fun testSelectReceiveOrClosedForClosedChannelWithValue() = runTest { + val channel = Channel(1) + channel.send(42) + channel.close() + expect(1) + select { + expect(2) + channel.onReceiveCatching { + assertFalse(it.isClosed) + assertEquals(42, it.getOrNull()) + finish(3) + } + } + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectDeferredTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectDeferredTest.kt new file mode 100644 index 0000000000..5b977d992b --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectDeferredTest.kt @@ -0,0 +1,188 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* +import kotlin.time.Duration.Companion.seconds + +class SelectDeferredTest : TestBase() { + @Test + fun testSimpleReturnsImmediately() = runTest { + expect(1) + val d1 = async { + expect(3) + 42 + } + expect(2) + val res = select { + d1.onAwait { v -> + expect(4) + assertEquals(42, v) + "OK" + } + } + expect(5) + assertEquals("OK", res) + finish(6) + } + + @Test + fun testSimpleWithYield() = runTest { + expect(1) + val d1 = async { + expect(3) + 42 + } + launch { + expect(4) + yield() // back to main + expect(6) + } + expect(2) + val res = select { + d1.onAwait { v -> + expect(5) + assertEquals(42, v) + yield() // to launch + expect(7) + "OK" + } + } + finish(8) + assertEquals("OK", res) + } + + @Test + fun testSelectIncompleteLazy() = runTest { + expect(1) + val d1 = async(start = CoroutineStart.LAZY) { + expect(5) + 42 + } + launch { + expect(3) + val res = select { + d1.onAwait { v -> + expect(7) + assertEquals(42, v) + "OK" + } + } + expect(8) + assertEquals("OK", res) + } + expect(2) + yield() // to launch + expect(4) + yield() // to started async + expect(6) + yield() // to triggered select + finish(9) + } + + @Test + fun testSelectTwo() = runTest { + expect(1) + val d1 = async { + expect(3) + yield() // to the other deffered + expect(5) + yield() // to fired select + expect(7) + "d1" + } + val d2 = async { + expect(4) + "d2" // returns result + } + expect(2) + val res = select { + d1.onAwait { + expectUnreached() + "FAIL" + } + d2.onAwait { v2 -> + expect(6) + assertEquals("d2", v2) + yield() // to first deferred + expect(8) + "OK" + } + } + assertEquals("OK", res) + finish(9) + } + + /** + * Tests that completing a [Deferred] with an exception will cause the [select] that uses [Deferred.onAwait] + * to throw the same exception. + */ + @Test + fun testSelectFailure() = runTest { + val d = CompletableDeferred() + d.completeExceptionally(TestException()) + val d2 = CompletableDeferred(42) + assertFailsWith { + select { + d.onAwait { expectUnreached() } + d2.onAwait { 4 } + } + } + } + + @Test + fun testSelectCancel() = runTest( + expected = { it is CancellationException } + ) { + expect(1) + val d = CompletableDeferred() + launch { + finish(3) + d.cancel() // will cancel after select starts + } + expect(2) + select { + d.onAwait { + expectUnreached() // will not select + } + } + expectUnreached() + } + + @Test + fun testSelectIncomplete() = runTest { + val deferred = async { Wrapper("OK") } + val result = select { + assertFalse(deferred.isCompleted) + assertTrue(deferred.isActive) + deferred.onAwait { + it + } + } + + assertEquals("OK", result.value) + } + + @Test + fun testSelectIncompleteFastPath() = runTest { + val deferred = async(Dispatchers.Unconfined) { Wrapper("OK") } + val result = select { + assertTrue(deferred.isCompleted) + assertFalse(deferred.isActive) + deferred.onAwait { + it + } + } + + assertEquals("OK", result.value) + } + + private class Wrapper(val value: String) : Incomplete { + override val isActive: Boolean + get() = error("") + override val list: NodeList? + get() = error("") + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectJobTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectJobTest.kt new file mode 100644 index 0000000000..6af517c0d8 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectJobTest.kt @@ -0,0 +1,64 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class SelectJobTest : TestBase() { + @Test + fun testSelectCompleted() = runTest { + expect(1) + launch { // makes sure we don't yield to it earlier + finish(4) // after main exits + } + val job = Job() + job.cancel() + select { + job.onJoin { + expect(2) + } + } + expect(3) + // will wait for the first coroutine + } + + @Test + fun testSelectIncomplete() = runTest { + expect(1) + val job = Job() + launch { // makes sure we don't yield to it earlier + expect(3) + val res = select { + job.onJoin { + expect(6) + "OK" + } + } + expect(7) + assertEquals("OK", res) + } + expect(2) + yield() + expect(4) + job.cancel() + expect(5) + yield() + finish(8) + } + + @Test + fun testSelectLazy() = runTest { + expect(1) + val job = launch(start = CoroutineStart.LAZY) { + expect(2) + } + val res = select { + job.onJoin { + expect(3) + "OK" + } + } + finish(4) + assertEquals("OK", res) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/selects/SelectLoopTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectLoopTest.kt new file mode 100644 index 0000000000..dd83d4afd8 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectLoopTest.kt @@ -0,0 +1,40 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class SelectLoopTest : TestBase() { + // https://github.com/Kotlin/kotlinx.coroutines/issues/1130 + @Test + fun testChannelSelectLoop() = runTest( + expected = { it is TestException } + ) { + expect(1) + val channel = Channel() + val job = launch { + expect(2) + channel.send(Unit) + expect(3) + throw TestException() + } + try { + while (true) { + select { + channel.onReceiveCatching { + expectUnreached() + } + job.onJoin { + expectUnreached() + } + } + } + } catch (e: CancellationException) { + // select will get cancelled because of the failure of job + finish(4) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectMutexTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectMutexTest.kt new file mode 100644 index 0000000000..db3361f3be --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectMutexTest.kt @@ -0,0 +1,53 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import kotlin.test.* + +class SelectMutexTest : TestBase() { + @Test + fun testSelectLock() = runTest { + val mutex = Mutex() + expect(1) + launch { // ensure that it is not scheduled earlier than needed + finish(4) // after main exits + } + val res = select { + mutex.onLock { + assertTrue(mutex.isLocked) + expect(2) + "OK" + } + } + assertEquals("OK", res) + expect(3) + // will wait for the first coroutine + } + + @Test + fun testSelectLockWait() = runTest { + val mutex = Mutex(true) // locked + expect(1) + launch { + expect(3) + val res = select { + // will suspended + mutex.onLock { + assertTrue(mutex.isLocked) + expect(6) + "OK" + } + } + assertEquals("OK", res) + expect(7) + } + expect(2) + yield() // to launched coroutine + expect(4) + mutex.unlock() + expect(5) + yield() // to resumed select + finish(8) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/selects/SelectOldTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectOldTest.kt new file mode 100644 index 0000000000..b500c9ff5f --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectOldTest.kt @@ -0,0 +1,149 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class SelectOldTest : TestBase() { + @Test + fun testSelectCompleted() = runTest { + expect(1) + launch { // makes sure we don't yield to it earlier + finish(4) // after main exits + } + val job = Job() + job.cancel() + selectOld { + job.onJoin { + expect(2) + } + } + expect(3) + // will wait for the first coroutine + } + + @Test + fun testSelectUnbiasedCompleted() = runTest { + expect(1) + launch { // makes sure we don't yield to it earlier + finish(4) // after main exits + } + val job = Job() + job.cancel() + selectUnbiasedOld { + job.onJoin { + expect(2) + } + } + expect(3) + // will wait for the first coroutine + } + + @Test + fun testSelectIncomplete() = runTest { + expect(1) + val job = Job() + launch { // makes sure we don't yield to it earlier + expect(3) + val res = selectOld { + job.onJoin { + expect(6) + "OK" + } + } + expect(7) + assertEquals("OK", res) + } + expect(2) + yield() + expect(4) + job.cancel() + expect(5) + yield() + finish(8) + } + + @Test + fun testSelectUnbiasedIncomplete() = runTest { + expect(1) + val job = Job() + launch { // makes sure we don't yield to it earlier + expect(3) + val res = selectUnbiasedOld { + job.onJoin { + expect(6) + "OK" + } + } + expect(7) + assertEquals("OK", res) + } + expect(2) + yield() + expect(4) + job.cancel() + expect(5) + yield() + finish(8) + } + + @Test + fun testSelectUnbiasedComplete() = runTest { + expect(1) + val job = Job() + job.complete() + expect(2) + val res = selectUnbiasedOld { + job.onJoin { + expect(3) + "OK" + } + } + assertEquals("OK", res) + finish(4) + } + + @Test + fun testSelectUnbiasedThrows() = runTest { + try { + select { + expect(1) + throw TestException() + } + } catch (e: TestException) { + finish(2) + } + } + + @Test + fun testSelectLazy() = runTest { + expect(1) + val job = launch(start = CoroutineStart.LAZY) { + expect(2) + } + val res = selectOld { + job.onJoin { + expect(3) + "OK" + } + } + finish(4) + assertEquals("OK", res) + } + + @Test + fun testSelectUnbiasedLazy() = runTest { + expect(1) + val job = launch(start = CoroutineStart.LAZY) { + expect(2) + } + val res = selectUnbiasedOld { + job.onJoin { + expect(3) + "OK" + } + } + finish(4) + assertEquals("OK", res) + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectRendezvousChannelTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectRendezvousChannelTest.kt new file mode 100644 index 0000000000..ad9ec556a8 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectRendezvousChannelTest.kt @@ -0,0 +1,476 @@ +@file:Suppress("NAMED_ARGUMENTS_NOT_ALLOWED") // KT-21913 + +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class SelectRendezvousChannelTest : TestBase() { + + @Test + fun testSelectSendSuccess() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + launch { + expect(2) + assertEquals("OK", channel.receive()) + finish(6) + } + yield() // to launched coroutine + expect(3) + select { + channel.onSend("OK") { + expect(4) + } + } + expect(5) + } + + @Test + fun testSelectSendSuccessWithDefault() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + launch { + expect(2) + assertEquals("OK", channel.receive()) + finish(6) + } + yield() // to launched coroutine + expect(3) + select { + channel.onSend("OK") { + expect(4) + } + default { + expectUnreached() + } + } + expect(5) + } + + @Test + fun testSelectSendWaitWithDefault() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + select { + channel.onSend("OK") { + expectUnreached() + } + default { + expect(2) + } + } + expect(3) + // make sure receive blocks (select above is over) + launch { + expect(5) + assertEquals("CHK", channel.receive()) + finish(8) + } + expect(4) + yield() + expect(6) + channel.send("CHK") + expect(7) + } + + @Test + fun testSelectSendWait() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + launch { + expect(3) + assertEquals("OK", channel.receive()) + expect(4) + } + expect(2) + select { + channel.onSend("OK") { + expect(5) + } + } + finish(6) + } + + @Test + fun testSelectReceiveSuccess() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + launch { + expect(2) + channel.send("OK") + finish(6) + } + yield() // to launched coroutine + expect(3) + select { + channel.onReceive { v -> + expect(4) + assertEquals("OK", v) + } + } + expect(5) + } + + @Test + fun testSelectReceiveSuccessWithDefault() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + launch { + expect(2) + channel.send("OK") + finish(6) + } + yield() // to launched coroutine + expect(3) + select { + channel.onReceive { v -> + expect(4) + assertEquals("OK", v) + } + default { + expectUnreached() + } + } + expect(5) + } + + @Test + fun testSelectReceiveWaitWithDefault() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + select { + channel.onReceive { + expectUnreached() + } + default { + expect(2) + } + } + expect(3) + // make sure send blocks (select above is over) + launch { + expect(5) + channel.send("CHK") + finish(8) + } + expect(4) + yield() + expect(6) + assertEquals("CHK", channel.receive()) + expect(7) + } + + @Test + fun testSelectReceiveWait() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + launch { + expect(3) + channel.send("OK") + expect(4) + } + expect(2) + select { + channel.onReceive { v -> + expect(5) + assertEquals("OK", v) + } + } + finish(6) + } + + @Test + fun testSelectReceiveClosed() = runTest(expected = { it is ClosedReceiveChannelException }) { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + channel.close() + finish(2) + select { + channel.onReceive { + expectUnreached() + } + } + expectUnreached() + } + + @Test + fun testSelectReceiveWaitClosed() = runTest(expected = {it is ClosedReceiveChannelException}) { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + launch { + expect(3) + channel.close() + finish(4) + } + expect(2) + select { + channel.onReceive { + expectUnreached() + } + } + expectUnreached() + } + + @Test + fun testSelectSendResourceCleanup() = runTest { + val channel = Channel(Channel.RENDEZVOUS) + val n = 1_000 + expect(1) + repeat(n) { i -> + select { + channel.onSend(i) { expectUnreached() } + default { expect(i + 2) } + } + } + finish(n + 2) + } + + @Test + fun testSelectReceiveResourceCleanup() = runTest { + val channel = Channel(Channel.RENDEZVOUS) + val n = 1_000 + expect(1) + repeat(n) { i -> + select { + channel.onReceive { expectUnreached() } + default { expect(i + 2) } + } + } + finish(n + 2) + } + + @Test + fun testSelectAtomicFailure() = runTest { + val c1 = Channel(Channel.RENDEZVOUS) + val c2 = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(3) + val res = select { + c1.onReceive { v1 -> + expect(4) + assertEquals(42, v1) + yield() // back to main + expect(7) + "OK" + } + c2.onReceive { + "FAIL" + } + } + expect(8) + assertEquals("OK", res) + } + expect(2) + c1.send(42) // send to coroutine, suspends + expect(5) + c2.close() // makes sure that selected expression does not fail! + expect(6) + yield() // back + finish(9) + } + + @Test + fun testSelectWaitDispatch() = runTest { + val c = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(3) + val res = select { + c.onReceive { v -> + expect(6) + assertEquals(42, v) + yield() // back to main + expect(8) + "OK" + } + } + expect(9) + assertEquals("OK", res) + } + expect(2) + yield() // to launch + expect(4) + c.send(42) // do not suspend + expect(5) + yield() // to receive + expect(7) + yield() // again + finish(10) + } + + @Test + fun testSelectReceiveCatchingWaitClosed() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + launch { + expect(3) + channel.close() + expect(4) + } + expect(2) + select { + channel.onReceiveCatching { + expect(5) + assertTrue(it.isClosed) + assertNull(it.exceptionOrNull()) + } + } + + finish(6) + } + + @Test + fun testSelectReceiveCatchingWaitClosedWithCause() = runTest { + expect(1) + val channel = Channel(Channel.RENDEZVOUS) + launch { + expect(3) + channel.close(TestException()) + expect(4) + } + expect(2) + select { + channel.onReceiveCatching { + expect(5) + assertTrue(it.isClosed) + assertIs(it.exceptionOrNull()) + } + } + + finish(6) + } + + @Test + fun testSelectReceiveCatchingForClosedChannel() = runTest { + val channel = Channel() + channel.close() + expect(1) + select { + expect(2) + channel.onReceiveCatching { + assertTrue(it.isClosed) + assertNull(it.exceptionOrNull()) + finish(3) + } + } + } + + @Test + fun testSelectReceiveCatching() = runTest { + val channel = Channel(Channel.RENDEZVOUS) + val iterations = 10 + expect(1) + val job = launch { + repeat(iterations) { + select { + channel.onReceiveCatching { v -> + expect(4 + it * 2) + assertEquals(it, v.getOrThrow()) + } + } + } + } + + expect(2) + repeat(iterations) { + expect(3 + it * 2) + channel.send(it) + yield() + } + + job.join() + finish(3 + iterations * 2) + } + + @Test + fun testSelectReceiveCatchingDispatch() = runTest { + val c = Channel(Channel.RENDEZVOUS) + expect(1) + launch { + expect(3) + val res = select { + c.onReceiveCatching { v -> + expect(6) + assertEquals(42, v.getOrThrow()) + yield() // back to main + expect(8) + "OK" + } + } + expect(9) + assertEquals("OK", res) + } + expect(2) + yield() // to launch + expect(4) + c.send(42) // do not suspend + expect(5) + yield() // to receive + expect(7) + yield() // again + finish(10) + } + + @Test + fun testSelectSendWhenClosed() = runTest { + expect(1) + val c = Channel(Channel.RENDEZVOUS) + launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + c.send(1) // enqueue sender + finish(4) + } + c.close() // then close + assertFailsWith { + // select sender should fail + expect(3) + select { + c.onSend(2) { + expectUnreached() + } + } + } + assertEquals(1, c.receive()) + } + + // only for debugging + internal fun SelectBuilder.default(block: suspend () -> R) = onTimeout(0, block) + + @Test + fun testSelectSendAndReceive() = runTest { + val c = Channel() + assertFailsWith { + select { + c.onSend(1) { expectUnreached() } + c.onReceive { expectUnreached() } + } + } + checkNotBroken(c) + } + + @Test + fun testSelectReceiveAndSend() = runTest { + val c = Channel() + assertFailsWith { + select { + c.onReceive { expectUnreached() } + c.onSend(1) { expectUnreached() } + } + } + checkNotBroken(c) + } + + // makes sure the channel is not broken + private suspend fun checkNotBroken(c: Channel) { + coroutineScope { + launch { + c.send(42) + } + assertEquals(42, c.receive()) + } + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectTimeoutDurationTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectTimeoutDurationTest.kt new file mode 100644 index 0000000000..f78e1c39f3 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectTimeoutDurationTest.kt @@ -0,0 +1,89 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.seconds + +class SelectTimeoutDurationTest : TestBase() { + @Test + fun testBasic() = runTest { + expect(1) + val result = select { + onTimeout(1000.milliseconds) { + expectUnreached() + "FAIL" + } + onTimeout(100.milliseconds) { + expect(2) + "OK" + } + onTimeout(500.milliseconds) { + expectUnreached() + "FAIL" + } + } + assertEquals("OK", result) + finish(3) + } + + @Test + fun testZeroTimeout() = runTest { + expect(1) + val result = select { + onTimeout(1.seconds) { + expectUnreached() + "FAIL" + } + onTimeout(Duration.ZERO) { + expect(2) + "OK" + } + } + assertEquals("OK", result) + finish(3) + } + + @Test + fun testNegativeTimeout() = runTest { + expect(1) + val result = select { + onTimeout(1.seconds) { + expectUnreached() + "FAIL" + } + onTimeout(-10.milliseconds) { + expect(2) + "OK" + } + } + assertEquals("OK", result) + finish(3) + } + + @Test + fun testUnbiasedNegativeTimeout() = runTest { + val counters = intArrayOf(0, 0, 0) + val iterations =10_000 + for (i in 0..iterations) { + val result = selectUnbiased { + onTimeout((-1).seconds) { + 0 + } + onTimeout(Duration.ZERO) { + 1 + } + onTimeout(1.seconds) { + expectUnreached() + 2 + } + } + ++counters[result] + } + assertEquals(0, counters[2]) + assertTrue { counters[0] > iterations / 4 } + assertTrue { counters[1] > iterations / 4 } + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectTimeoutTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectTimeoutTest.kt new file mode 100644 index 0000000000..f3abfe4b11 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectTimeoutTest.kt @@ -0,0 +1,86 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class SelectTimeoutTest : TestBase() { + @Test + fun testBasic() = runTest { + expect(1) + val result = select { + onTimeout(1000) { + expectUnreached() + "FAIL" + } + onTimeout(100) { + expect(2) + "OK" + } + onTimeout(500) { + expectUnreached() + "FAIL" + } + } + assertEquals("OK", result) + finish(3) + } + + @Test + fun testZeroTimeout() = runTest { + expect(1) + val result = select { + onTimeout(1000) { + expectUnreached() + "FAIL" + } + onTimeout(0) { + expect(2) + "OK" + } + } + assertEquals("OK", result) + finish(3) + } + + @Test + fun testNegativeTimeout() = runTest { + expect(1) + val result = select { + onTimeout(1000) { + expectUnreached() + "FAIL" + } + onTimeout(-10) { + expect(2) + "OK" + } + } + assertEquals("OK", result) + finish(3) + } + + @Test + fun testUnbiasedNegativeTimeout() = runTest { + val counters = intArrayOf(0, 0, 0) + val iterations =10_000 + for (i in 0..iterations) { + val result = selectUnbiased { + onTimeout(-1000) { + 0 + } + onTimeout(0) { + 1 + } + onTimeout(1000) { + expectUnreached() + 2 + } + } + ++counters[result] + } + assertEquals(0, counters[2]) + assertTrue { counters[0] > iterations / 4 } + assertTrue { counters[1] > iterations / 4 } + } +} diff --git a/kotlinx-coroutines-core/common/test/selects/SelectUnlimitedChannelTest.kt b/kotlinx-coroutines-core/common/test/selects/SelectUnlimitedChannelTest.kt new file mode 100644 index 0000000000..e460bd707c --- /dev/null +++ b/kotlinx-coroutines-core/common/test/selects/SelectUnlimitedChannelTest.kt @@ -0,0 +1,26 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class SelectUnlimitedChannelTest : TestBase() { + @Test + fun testSelectSendWhenClosed() = runTest { + expect(1) + val c = Channel(Channel.UNLIMITED) + c.send(1) // enqueue buffered element + c.close() // then close + assertFailsWith { + // select sender should fail + expect(2) + select { + c.onSend(2) { + expectUnreached() + } + } + } + finish(3) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/common/test/sync/MutexTest.kt b/kotlinx-coroutines-core/common/test/sync/MutexTest.kt new file mode 100644 index 0000000000..c2d555f32b --- /dev/null +++ b/kotlinx-coroutines-core/common/test/sync/MutexTest.kt @@ -0,0 +1,200 @@ +package kotlinx.coroutines.sync + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import kotlin.test.* + +class MutexTest : TestBase() { + @Test + fun testSimple() = runTest { + val mutex = Mutex() + expect(1) + launch { + expect(4) + mutex.lock() // suspends + expect(7) // now got lock + mutex.unlock() + expect(8) + } + expect(2) + mutex.lock() // locked + expect(3) + yield() // yield to child + expect(5) + mutex.unlock() + expect(6) + yield() // now child has lock + finish(9) + } + + @Test + fun tryLockTest() { + val mutex = Mutex() + assertFalse(mutex.isLocked) + assertTrue(mutex.tryLock()) + assertTrue(mutex.isLocked) + assertFalse(mutex.tryLock()) + assertTrue(mutex.isLocked) + mutex.unlock() + assertFalse(mutex.isLocked) + assertTrue(mutex.tryLock()) + assertTrue(mutex.isLocked) + assertFalse(mutex.tryLock()) + assertTrue(mutex.isLocked) + mutex.unlock() + assertFalse(mutex.isLocked) + } + + @Test + fun withLockTest() = runTest { + val mutex = Mutex() + assertFalse(mutex.isLocked) + mutex.withLock { + assertTrue(mutex.isLocked) + } + assertFalse(mutex.isLocked) + } + + @Test + fun testWithLockFailureUnlocksTheMutex() = runTest { + val mutex = Mutex() + assertFalse(mutex.isLocked) + try { + mutex.withLock { + expect(1) + assertTrue(mutex.isLocked) + throw TestException() + } + } catch (e: TestException) { + expect(2) + } + assertFalse(mutex.isLocked) + finish(3) + } + + @Test + fun withLockOnEarlyReturnTest() = runTest { + val mutex = Mutex() + assertFalse(mutex.isLocked) + suspend fun f() { + mutex.withLock { + assertTrue(mutex.isLocked) + return@f + } + } + f() + assertFalse(mutex.isLocked) + } + + @Test + fun testUnconfinedStackOverflow() { + val waiters = 10000 + val mutex = Mutex(true) + var done = 0 + repeat(waiters) { + GlobalScope.launch(Dispatchers.Unconfined) { // a lot of unconfined waiters + mutex.withLock { + done++ + } + } + } + mutex.unlock() // should not produce StackOverflowError + assertEquals(waiters, done) + } + + @Test + fun holdLock() = runTest { + val mutex = Mutex() + val firstOwner = Any() + val secondOwner = Any() + + // no lock + assertFalse(mutex.holdsLock(firstOwner)) + assertFalse(mutex.holdsLock(secondOwner)) + + // owner firstOwner + mutex.lock(firstOwner) + val secondLockJob = launch { + mutex.lock(secondOwner) + } + + assertTrue(mutex.holdsLock(firstOwner)) + assertFalse(mutex.holdsLock(secondOwner)) + + // owner secondOwner + mutex.unlock(firstOwner) + secondLockJob.join() + + assertFalse(mutex.holdsLock(firstOwner)) + assertTrue(mutex.holdsLock(secondOwner)) + + mutex.unlock(secondOwner) + + // no lock + assertFalse(mutex.holdsLock(firstOwner)) + assertFalse(mutex.holdsLock(secondOwner)) + } + + @Test + fun testUnlockWithNullOwner() { + val owner = Any() + val mutex = Mutex() + assertTrue(mutex.tryLock(owner)) + assertFailsWith { mutex.unlock(Any()) } + mutex.unlock(null) + assertFalse(mutex.holdsLock(owner)) + assertFalse(mutex.isLocked) + } + + @Test + fun testUnlockWithoutOwnerWithLockedQueue() = runTest { + val owner = Any() + val owner2 = Any() + val mutex = Mutex() + assertTrue(mutex.tryLock(owner)) + expect(1) + launch { + expect(2) + mutex.lock(owner2) + } + yield() + expect(3) + assertFailsWith { mutex.unlock(owner2) } + mutex.unlock() + assertFalse(mutex.holdsLock(owner)) + assertTrue(mutex.holdsLock(owner2)) + finish(4) + } + + @Test + fun testIllegalStateInvariant() = runTest { + val mutex = Mutex() + val owner = Any() + assertTrue(mutex.tryLock(owner)) + assertFailsWith { mutex.tryLock(owner) } + assertFailsWith { mutex.lock(owner) } + assertFailsWith { select { mutex.onLock(owner) {} } } + } + + @Test + fun testWithLockJsMiscompilation() = runTest { + // This is a reproducer for KT-58685 + // On Kotlin/JS IR, the compiler miscompiles calls to 'unlock' in an inlined finally + // This is visible on the withLock function + // Until the compiler bug is fixed, this test case checks that we do not suffer from it + val mutex = Mutex() + assertFailsWith { + try { + mutex.withLock { null } ?: throw IndexOutOfBoundsException() // should throw… + } catch (e: Exception) { + throw e // …but instead fails here + } + } + } + + @Test + fun testMutexIsNotSemaphore() { + assertIsNot(Mutex()) + } +} diff --git a/kotlinx-coroutines-core/common/test/sync/SemaphoreTest.kt b/kotlinx-coroutines-core/common/test/sync/SemaphoreTest.kt new file mode 100644 index 0000000000..89f066c2b8 --- /dev/null +++ b/kotlinx-coroutines-core/common/test/sync/SemaphoreTest.kt @@ -0,0 +1,214 @@ +package kotlinx.coroutines.sync + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class SemaphoreTest : TestBase() { + + @Test + fun testSimple() = runTest { + val semaphore = Semaphore(2) + launch { + expect(3) + semaphore.release() + expect(4) + } + expect(1) + semaphore.acquire() + semaphore.acquire() + expect(2) + semaphore.acquire() + finish(5) + } + + @Test + fun testSimpleAsMutex() = runTest { + val semaphore = Semaphore(1) + expect(1) + launch { + expect(4) + semaphore.acquire() // suspends + expect(7) // now got lock + semaphore.release() + expect(8) + } + expect(2) + semaphore.acquire() // locked + expect(3) + yield() // yield to child + expect(5) + semaphore.release() + expect(6) + yield() // now child has lock + finish(9) + } + + @Test + fun tryAcquireTest() = runTest { + val semaphore = Semaphore(2) + assertTrue(semaphore.tryAcquire()) + assertTrue(semaphore.tryAcquire()) + assertFalse(semaphore.tryAcquire()) + assertEquals(0, semaphore.availablePermits) + semaphore.release() + assertEquals(1, semaphore.availablePermits) + assertTrue(semaphore.tryAcquire()) + assertEquals(0, semaphore.availablePermits) + } + + @Test + fun withSemaphoreTest() = runTest { + val semaphore = Semaphore(1) + assertEquals(1, semaphore.availablePermits) + semaphore.withPermit { + assertEquals(0, semaphore.availablePermits) + } + assertEquals(1, semaphore.availablePermits) + } + + @Test + fun withSemaphoreOnFailureTest() = runTest { + val semaphore = Semaphore(1) + assertEquals(1, semaphore.availablePermits) + try { + semaphore.withPermit { + assertEquals(0, semaphore.availablePermits) + throw TestException() + } + } catch (e: TestException) { + // Expected + } + assertEquals(1, semaphore.availablePermits) + } + + @Test + fun withSemaphoreOnEarlyReturnTest() = runTest { + val semaphore = Semaphore(1) + assertEquals(1, semaphore.availablePermits) + suspend fun f() { + semaphore.withPermit { + assertEquals(0, semaphore.availablePermits) + return@f + } + } + f() + assertEquals(1, semaphore.availablePermits) + } + + @Test + fun fairnessTest() = runTest { + val semaphore = Semaphore(1) + semaphore.acquire() + launch(coroutineContext) { + // first to acquire + expect(2) + semaphore.acquire() // suspend + expect(6) + } + launch(coroutineContext) { + // second to acquire + expect(3) + semaphore.acquire() // suspend + expect(9) + } + expect(1) + yield() + expect(4) + semaphore.release() + expect(5) + yield() + expect(7) + semaphore.release() + expect(8) + yield() + finish(10) + } + + @Test + fun testCancellationReturnsPermitBack() = runTest { + val semaphore = Semaphore(1) + semaphore.acquire() + assertEquals(0, semaphore.availablePermits) + val job = launch { + assertFalse(semaphore.tryAcquire()) + semaphore.acquire() + } + yield() + job.cancelAndJoin() + assertEquals(0, semaphore.availablePermits) + semaphore.release() + assertEquals(1, semaphore.availablePermits) + } + + @Test + fun testCancellationDoesNotResumeWaitingAcquirers() = runTest { + val semaphore = Semaphore(1) + semaphore.acquire() + val job1 = launch { // 1st job in the waiting queue + expect(2) + semaphore.acquire() + expectUnreached() + } + val job2 = launch { // 2nd job in the waiting queue + expect(3) + semaphore.acquire() + expectUnreached() + } + expect(1) + yield() + expect(4) + job2.cancel() + yield() + expect(5) + job1.cancel() + finish(6) + } + + @Test + fun testAcquiredPermits() = runTest { + val semaphore = Semaphore(5, acquiredPermits = 4) + assertEquals(semaphore.availablePermits, 1) + semaphore.acquire() + assertEquals(semaphore.availablePermits, 0) + assertFalse(semaphore.tryAcquire()) + semaphore.release() + assertEquals(semaphore.availablePermits, 1) + assertTrue(semaphore.tryAcquire()) + } + + @Test + fun testReleaseAcquiredPermits() = runTest { + val semaphore = Semaphore(5, acquiredPermits = 4) + assertEquals(semaphore.availablePermits, 1) + repeat(4) { semaphore.release() } + assertEquals(5, semaphore.availablePermits) + assertFailsWith { semaphore.release() } + repeat(5) { assertTrue(semaphore.tryAcquire()) } + assertFalse(semaphore.tryAcquire()) + } + + @Test + fun testIllegalArguments() { + assertFailsWith { Semaphore(-1, 0) } + assertFailsWith { Semaphore(0, 0) } + assertFailsWith { Semaphore(1, -1) } + assertFailsWith { Semaphore(1, 2) } + } + + @Test + fun testWithPermitJsMiscompilation() = runTest { + // This is a reproducer for KT-58685 + // On Kotlin/JS IR, the compiler miscompiles calls to 'release' in an inlined finally + // This is visible on the withPermit function + // Until the compiler bug is fixed, this test case checks that we do not suffer from it + val semaphore = Semaphore(1) + assertFailsWith { + try { + semaphore.withPermit { null } ?: throw IndexOutOfBoundsException() // should throw… + } catch (e: Exception) { + throw e // …but instead fails here + } + } + } +} diff --git a/kotlinx-coroutines-core/concurrent/src/Builders.concurrent.kt b/kotlinx-coroutines-core/concurrent/src/Builders.concurrent.kt new file mode 100644 index 0000000000..7c0581b9d9 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/src/Builders.concurrent.kt @@ -0,0 +1,24 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* + +/** + * Runs a new coroutine and **blocks** the current thread until its completion. + * + * It is designed to bridge regular blocking code to libraries that are written in suspending style, to be used in + * `main` functions and in tests. + * + * Calling [runBlocking] from a suspend function is redundant. + * For example, the following code is incorrect: + * ``` + * suspend fun loadConfiguration() { + * // DO NOT DO THIS: + * val data = runBlocking { // <- redundant and blocks the thread, do not do that + * fetchConfigurationData() // suspending function + * } + * ``` + * + * Here, instead of releasing the thread on which `loadConfiguration` runs if `fetchConfigurationData` suspends, it will + * block, potentially leading to thread starvation issues. + */ +public expect fun runBlocking(context: CoroutineContext = EmptyCoroutineContext, block: suspend CoroutineScope.() -> T): T diff --git a/kotlinx-coroutines-core/concurrent/src/Dispatchers.kt b/kotlinx-coroutines-core/concurrent/src/Dispatchers.kt new file mode 100644 index 0000000000..d18efdc35f --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/src/Dispatchers.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines + +/** + * The [CoroutineDispatcher] that is designed for offloading blocking IO tasks to a shared pool of threads. + * Additional threads in this pool are created on demand. + * Default IO pool size is `64`; on JVM it can be configured using JVM-specific mechanisms, + * please refer to `Dispatchers.IO` documentation on JVM platform. + * + * ### Elasticity for limited parallelism + * + * `Dispatchers.IO` has a unique property of elasticity: its views + * obtained with [CoroutineDispatcher.limitedParallelism] are + * not restricted by the `Dispatchers.IO` parallelism. Conceptually, there is + * a dispatcher backed by an unlimited pool of threads, and both `Dispatchers.IO` + * and views of `Dispatchers.IO` are actually views of that dispatcher. In practice + * this means that, despite not abiding by `Dispatchers.IO`'s parallelism + * restrictions, its views share threads and resources with it. + * + * In the following example + * ``` + * // 100 threads for MySQL connection + * val myMysqlDbDispatcher = Dispatchers.IO.limitedParallelism(100) + * // 60 threads for MongoDB connection + * val myMongoDbDispatcher = Dispatchers.IO.limitedParallelism(60) + * ``` + * the system may have up to `64 + 100 + 60` threads dedicated to blocking tasks during peak loads, + * but during its steady state there is only a small number of threads shared + * among `Dispatchers.IO`, `myMysqlDbDispatcher` and `myMongoDbDispatcher` + * + * It is recommended to replace manually created thread-backed executors with `Dispatchers.IO.limitedParallelism` instead: + * ``` + * // Requires manual closing, allocates resources for all threads + * val databasePoolDispatcher = newFixedThreadPoolContext(128) + * + * // Provides the same number of threads as a resource but shares and caches them internally + * val databasePoolDispatcher = Dispatchers.IO.limitedParallelism(128) + * ``` + */ +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +public expect val Dispatchers.IO: CoroutineDispatcher + + diff --git a/kotlinx-coroutines-core/concurrent/src/MultithreadedDispatchers.common.kt b/kotlinx-coroutines-core/concurrent/src/MultithreadedDispatchers.common.kt new file mode 100644 index 0000000000..b85c878cba --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/src/MultithreadedDispatchers.common.kt @@ -0,0 +1,65 @@ +@file:JvmMultifileClass +@file:JvmName("ThreadPoolDispatcherKt") +package kotlinx.coroutines + +import kotlin.jvm.* + +/** + * Creates a coroutine execution context using a single thread with built-in [yield] support. + * **NOTE: The resulting [CloseableCoroutineDispatcher] owns native resources (its thread). + * Resources are reclaimed by [CloseableCoroutineDispatcher.close].** + * + * If the resulting dispatcher is [closed][CloseableCoroutineDispatcher.close] and + * attempt to submit a task is made, then: + * - On the JVM, the [Job] of the affected task is [cancelled][Job.cancel] and the task is submitted to the + * [Dispatchers.IO], so that the affected coroutine can clean up its resources and promptly complete. + * - On Native, the attempt to submit a task throws an exception. + * + * This is a **delicate** API. The result of this method is a closeable resource with the + * associated native resources (threads or native workers). It should not be allocated in place, + * should be closed at the end of its lifecycle, and has non-trivial memory and CPU footprint. + * If you do not need a separate thread pool, but only have to limit effective parallelism of the dispatcher, + * it is recommended to use [`Dispatchers.IO.limitedParallelism(1)`][CoroutineDispatcher.limitedParallelism] + * or [`Dispatchers.Default.limitedParallelism(1)`][CoroutineDispatcher.limitedParallelism] instead. + * + * If you need a completely separate thread pool with scheduling policy that is based on the standard + * JDK executors, use the following expression: + * `Executors.newSingleThreadExecutor().asCoroutineDispatcher()`. + * See `Executor.asCoroutineDispatcher` for details. + * + * @param name the base name of the created thread. + */ +@ExperimentalCoroutinesApi +@DelicateCoroutinesApi +public fun newSingleThreadContext(name: String): CloseableCoroutineDispatcher = + newFixedThreadPoolContext(1, name) + +/** + * Creates a coroutine execution context with the fixed-size thread-pool and built-in [yield] support. + * **NOTE: The resulting [CoroutineDispatcher] owns native resources (its threads). + * Resources are reclaimed by [CloseableCoroutineDispatcher.close].** + * + * If the resulting dispatcher is [closed][CloseableCoroutineDispatcher.close] and + * attempt to submit a continuation task is made, + * - On the JVM, the [Job] of the affected task is [cancelled][Job.cancel] and the task is submitted to the + * [Dispatchers.IO], so that the affected coroutine can clean up its resources and promptly complete. + * - On Native, the attempt to submit a task throws an exception. + * + * This is a **delicate** API. The result of this method is a closeable resource with the + * associated native resources (threads or native workers). It should not be allocated in place, + * should be closed at the end of its lifecycle, and has non-trivial memory and CPU footprint. + * If you do not need a separate thread pool, but only have to limit effective parallelism of the dispatcher, + * it is recommended to use [`Dispatchers.IO.limitedParallelism(nThreads)`][CoroutineDispatcher.limitedParallelism] + * or [`Dispatchers.Default.limitedParallelism(nThreads)`][CoroutineDispatcher.limitedParallelism] instead. + * + * If you need a completely separate thread pool with scheduling policy that is based on the standard + * JDK executors, use the following expression: + * `Executors.newFixedThreadPool().asCoroutineDispatcher()`. + * See `Executor.asCoroutineDispatcher` for details. + * + * @param nThreads the number of threads. + * @param name the base name of the created threads. + */ +@ExperimentalCoroutinesApi +@DelicateCoroutinesApi +public expect fun newFixedThreadPoolContext(nThreads: Int, name: String): CloseableCoroutineDispatcher diff --git a/kotlinx-coroutines-core/concurrent/src/channels/Channels.kt b/kotlinx-coroutines-core/concurrent/src/channels/Channels.kt new file mode 100644 index 0000000000..c157677d22 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/src/channels/Channels.kt @@ -0,0 +1,60 @@ +@file:JvmMultifileClass +@file:JvmName("ChannelsKt") + +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlin.jvm.* + +/** + * Adds [element] to this channel, **blocking** the caller while this channel is full, + * and returning either [successful][ChannelResult.isSuccess] result when the element was added, or + * failed result representing closed channel with a corresponding exception. + * + * This is a way to call [Channel.send] method in a safe manner inside a blocking code using [runBlocking] and catching, + * so this function should not be used from coroutine. + * + * Example of usage: + * + * ``` + * // From callback API + * channel.trySendBlocking(element) + * .onSuccess { /* request next element or debug log */ } + * .onFailure { t: Throwable? -> /* throw or log */ } + * ``` + * + * For this operation it is guaranteed that [failure][ChannelResult.failed] always contains an exception in it. + * + * @throws `InterruptedException` on JVM if the current thread is interrupted during the blocking send operation. + */ +public fun SendChannel.trySendBlocking(element: E): ChannelResult { + /* + * Sent successfully -- bail out. + * But failure may indicate either that the channel is full or that + * it is close. Go to slow path on failure to simplify the successful path and + * to materialize default exception. + */ + trySend(element).onSuccess { return ChannelResult.success(Unit) } + return runBlocking { + val r = runCatching { send(element) } + if (r.isSuccess) ChannelResult.success(Unit) + else ChannelResult.closed(r.exceptionOrNull()) + } +} + +/** @suppress */ +@Deprecated( + level = DeprecationLevel.HIDDEN, + message = "Deprecated in the favour of 'trySendBlocking'. " + + "Consider handling the result of 'trySendBlocking' explicitly and rethrow exception if necessary", + replaceWith = ReplaceWith("trySendBlocking(element)") +) // WARNING in 1.5.0, ERROR in 1.6.0 +public fun SendChannel.sendBlocking(element: E) { + // fast path + if (trySend(element).isSuccess) + return + // slow path + runBlocking { + send(element) + } +} diff --git a/kotlinx-coroutines-core/concurrent/src/internal/LockFreeLinkedList.kt b/kotlinx-coroutines-core/concurrent/src/internal/LockFreeLinkedList.kt new file mode 100644 index 0000000000..2e6ce5ae4a --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/src/internal/LockFreeLinkedList.kt @@ -0,0 +1,287 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlin.jvm.* + +private typealias Node = LockFreeLinkedListNode + +/** + * Doubly-linked concurrent list node with remove support. + * Based on paper + * ["Lock-Free and Practical Doubly Linked List-Based Deques Using Single-Word Compare-and-Swap"](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.140.4693&rep=rep1&type=pdf) + * by Sundell and Tsigas with considerable changes. + * + * The core idea of the algorithm is to maintain a doubly-linked list with an ever-present sentinel node (it is + * never removed) that serves both as a list head and tail and to linearize all operations (both insert and remove) on + * the update of the next pointer. Removed nodes have their next pointer marked with [Removed] class. + * + * Important notes: + * - There are no operations to add items to left side of the list, only to the end (right side), because we cannot + * efficiently linearize them with atomic multi-step head-removal operations. In short, + * support for [describeRemoveFirst] operation precludes ability to add items at the beginning. + * - Previous pointers are not marked for removal. We don't support linearizable backwards traversal. + * - Remove-helping logic is simplified and consolidated in [correctPrev] method. + * + * @suppress **This is unstable API and it is subject to change.** + */ +@Suppress("LeakingThis") +@InternalCoroutinesApi +public actual open class LockFreeLinkedListNode { + private val _next = atomic(this) // Node | Removed | OpDescriptor + private val _prev = atomic(this) // Node to the left (cannot be marked as removed) + private val _removedRef = atomic(null) // lazily cached removed ref to this + + private fun removed(): Removed = + _removedRef.value ?: Removed(this).also { _removedRef.lazySet(it) } + + public actual open val isRemoved: Boolean get() = next is Removed + + // LINEARIZABLE. Returns Node | Removed + public val next: Any get() = _next.value + + // LINEARIZABLE. Returns next non-removed Node + public actual val nextNode: Node get() = + next.let { (it as? Removed)?.ref ?: it as Node } // unwraps the `next` node + + // LINEARIZABLE WHEN THIS NODE IS NOT REMOVED: + // Returns prev non-removed Node, makes sure prev is correct (prev.next === this) + // NOTE: if this node is removed, then returns non-removed previous node without applying + // prev.next correction, which does not provide linearizable backwards iteration, but can be used to + // resume forward iteration when current node was removed. + public actual val prevNode: Node + get() = correctPrev() ?: findPrevNonRemoved(_prev.value) + + private tailrec fun findPrevNonRemoved(current: Node): Node { + if (!current.isRemoved) return current + return findPrevNonRemoved(current._prev.value) + } + + // ------ addOneIfEmpty ------ + + public actual fun addOneIfEmpty(node: Node): Boolean { + node._prev.lazySet(this) + node._next.lazySet(this) + while (true) { + val next = next + if (next !== this) return false // this is not an empty list! + if (_next.compareAndSet(this, node)) { + // added successfully (linearized add) -- fixup the list + node.finishAdd(this) + return true + } + } + } + + // ------ addLastXXX ------ + + /** + * Adds last item to this list. Returns `false` if the list is closed. + */ + public actual fun addLast(node: Node, permissionsBitmask: Int): Boolean { + while (true) { // lock-free loop on prev.next + val currentPrev = prevNode + return when { + currentPrev is ListClosed -> + currentPrev.forbiddenElementsBitmask and permissionsBitmask == 0 && + currentPrev.addLast(node, permissionsBitmask) + currentPrev.addNext(node, this) -> true + else -> continue + } + } + } + + /** + * Forbids adding new items to this list. + */ + public actual fun close(forbiddenElementsBit: Int) { + addLast(ListClosed(forbiddenElementsBit), forbiddenElementsBit) + } + + /** + * Given: + * ``` + * +-----------------------+ + * this | node V next + * +---+---+ +---+---+ +---+---+ + * ... <-- | P | N | | P | N | | P | N | --> .... + * +---+---+ +---+---+ +---+---+ + * ^ | + * +-----------------------+ + * ``` + * Produces: + * ``` + * this node next + * +---+---+ +---+---+ +---+---+ + * ... <-- | P | N | ==> | P | N | --> | P | N | --> .... + * +---+---+ +---+---+ +---+---+ + * ^ | ^ | + * +---------+ +---------+ + * ``` + * Where `==>` denotes linearization point. + * Returns `false` if `next` was not following `this` node. + */ + @PublishedApi + internal fun addNext(node: Node, next: Node): Boolean { + node._prev.lazySet(this) + node._next.lazySet(next) + if (!_next.compareAndSet(next, node)) return false + // added successfully (linearized add) -- fixup the list + node.finishAdd(next) + return true + } + + // ------ removeXXX ------ + + /** + * Removes this node from the list. Returns `true` when removed successfully, or `false` if the node was already + * removed or if it was not added to any list in the first place. + * + * **Note**: Invocation of this operation does not guarantee that remove was actually complete if result was `false`. + * In particular, invoking [nextNode].[prevNode] might still return this node even though it is "already removed". + */ + public actual open fun remove(): Boolean = + removeOrNext() == null + + // returns null if removed successfully or next node if this node is already removed + @PublishedApi + internal fun removeOrNext(): Node? { + while (true) { // lock-free loop on next + val next = this.next + if (next is Removed) return next.ref // was already removed -- don't try to help (original thread will take care) + if (next === this) return next // was not even added + val removed = (next as Node).removed() + if (_next.compareAndSet(next, removed)) { + // was removed successfully (linearized remove) -- fixup the list + next.correctPrev() + return null + } + } + } + + // This is Harris's RDCSS (Restricted Double-Compare Single Swap) operation + // It inserts "op" descriptor of when "op" status is still undecided (rolls back otherwise) + + + // ------ other helpers ------ + + /** + * Given: + * ``` + * + * prev this next + * +---+---+ +---+---+ +---+---+ + * ... <-- | P | N | --> | P | N | --> | P | N | --> .... + * +---+---+ +---+---+ +---+---+ + * ^ ^ | | + * | +---------+ | + * +-------------------------+ + * ``` + * Produces: + * ``` + * prev this next + * +---+---+ +---+---+ +---+---+ + * ... <-- | P | N | --> | P | N | --> | P | N | --> .... + * +---+---+ +---+---+ +---+---+ + * ^ | ^ | + * +---------+ +---------+ + * ``` + */ + private fun finishAdd(next: Node) { + next._prev.loop { nextPrev -> + if (this.next !== next) return // this or next was removed or another node added, remover/adder fixes up links + if (next._prev.compareAndSet(nextPrev, this)) { + // This newly added node could have been removed, and the above CAS would have added it physically again. + // Let us double-check for this situation and correct if needed + if (isRemoved) next.correctPrev() + return + } + } + } + + /** + * Returns the corrected value of the previous node while also correcting the `prev` pointer + * (so that `this.prev.next === this`) and helps complete node removals to the left ot this node. + * + * It returns `null` in two special cases: + * + * - When this node is removed. In this case there is no need to waste time on corrections, because + * remover of this node will ultimately call [correctPrev] on the next node and that will fix all + * the links from this node, too. + */ + private tailrec fun correctPrev(): Node? { + val oldPrev = _prev.value + var prev: Node = oldPrev + var last: Node? = null // will be set so that last.next === prev + while (true) { // move the left until first non-removed node + val prevNext: Any = prev._next.value + when { + // fast path to find quickly find prev node when everything is properly linked + prevNext === this -> { + if (oldPrev === prev) return prev // nothing to update -- all is fine, prev found + // otherwise need to update prev + if (!this._prev.compareAndSet(oldPrev, prev)) { + // Note: retry from scratch on failure to update prev + return correctPrev() + } + return prev // return the correct prev + } + // slow path when we need to help remove operations + this.isRemoved -> return null // nothing to do, this node was removed, bail out asap to save time + prevNext is Removed -> { + if (last !== null) { + // newly added (prev) node is already removed, correct last.next around it + if (!last._next.compareAndSet(prev, prevNext.ref)) { + return correctPrev() // retry from scratch on failure to update next + } + prev = last + last = null + } else { + prev = prev._prev.value + } + } + else -> { // prevNext is a regular node, but not this -- help delete + last = prev + prev = prevNext as Node + } + } + } + } + + internal fun validateNode(prev: Node, next: Node) { + assert { prev === this._prev.value } + assert { next === this._next.value } + } + + override fun toString(): String = "${this::classSimpleName}@${this.hexAddress}" +} + +private class Removed(@JvmField val ref: Node) { + override fun toString(): String = "Removed[$ref]" +} + +/** + * Head (sentinel) item of the linked list that is never removed. + * + * @suppress **This is unstable API and it is subject to change.** + */ +public actual open class LockFreeLinkedListHead : LockFreeLinkedListNode() { + /** + * Iterates over all elements in this list of a specified type. + */ + public actual inline fun forEach(block: (Node) -> Unit) { + var cur: Node = next as Node + while (cur != this) { + block(cur) + cur = cur.nextNode + } + } + + // just a defensive programming -- makes sure that list head sentinel is never removed + public actual final override fun remove(): Nothing = error("head cannot be removed") + + // optimization: because head is never removed, we don't have to read _next.value to check these: + override val isRemoved: Boolean get() = false +} + +private class ListClosed(@JvmField val forbiddenElementsBitmask: Int): LockFreeLinkedListNode() diff --git a/kotlinx-coroutines-core/concurrent/src/internal/OnDemandAllocatingPool.kt b/kotlinx-coroutines-core/concurrent/src/internal/OnDemandAllocatingPool.kt new file mode 100644 index 0000000000..8c21159224 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/src/internal/OnDemandAllocatingPool.kt @@ -0,0 +1,102 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* + +/** + * A thread-safe resource pool. + * + * [maxCapacity] is the maximum amount of elements. + * [create] is the function that creates a new element. + * + * This is only used in the Native implementation, + * but is part of the `concurrent` source set in order to test it on the JVM. + */ +internal class OnDemandAllocatingPool( + private val maxCapacity: Int, + private val create: (Int) -> T +) { + /** + * Number of existing elements + isClosed flag in the highest bit. + * Once the flag is set, the value is guaranteed not to change anymore. + */ + private val controlState = atomic(0) + private val elements = atomicArrayOfNulls(maxCapacity) + + /** + * Returns the number of elements that need to be cleaned up due to the pool being closed. + */ + @Suppress("NOTHING_TO_INLINE") + private inline fun tryForbidNewElements(): Int { + controlState.loop { + if (it.isClosed()) return 0 // already closed + if (controlState.compareAndSet(it, it or IS_CLOSED_MASK)) return it + } + } + + @Suppress("NOTHING_TO_INLINE") + private inline fun Int.isClosed(): Boolean = this and IS_CLOSED_MASK != 0 + + /** + * Request that a new element is created. + * + * Returns `false` if the pool is closed. + * + * Note that it will still return `true` even if an element was not created due to reaching [maxCapacity]. + * + * Rethrows the exceptions thrown from [create]. In this case, this operation has no effect. + */ + fun allocate(): Boolean { + controlState.loop { ctl -> + if (ctl.isClosed()) return false + if (ctl >= maxCapacity) return true + if (controlState.compareAndSet(ctl, ctl + 1)) { + elements[ctl].value = create(ctl) + return true + } + } + } + + /** + * Close the pool. + * + * This will prevent any new elements from being created. + * All the elements present in the pool will be returned. + * + * The function is thread-safe. + * + * [close] can be called multiple times, but only a single call will return a non-empty list. + * This is due to the elements being cleaned out from the pool on the first invocation to avoid memory leaks, + * and no new elements being created after. + */ + fun close(): List { + val elementsExisting = tryForbidNewElements() + return (0 until elementsExisting).map { i -> + // we wait for the element to be created, because we know that eventually it is going to be there + loop { + val element = elements[i].getAndSet(null) + if (element != null) { + return@map element + } + } + } + } + + // for tests + internal fun stateRepresentation(): String { + val ctl = controlState.value + val elementsStr = (0 until (ctl and IS_CLOSED_MASK.inv())).map { elements[it].value }.toString() + val closedStr = if (ctl.isClosed()) "[closed]" else "" + return elementsStr + closedStr + } + + override fun toString(): String = "OnDemandAllocatingPool(${stateRepresentation()})" +} + +// KT-25023 +private inline fun loop(block: () -> Unit): Nothing { + while (true) { + block() + } +} + +private const val IS_CLOSED_MASK = 1 shl 31 diff --git a/kotlinx-coroutines-core/concurrent/test/AbstractDispatcherConcurrencyTest.kt b/kotlinx-coroutines-core/concurrent/test/AbstractDispatcherConcurrencyTest.kt new file mode 100644 index 0000000000..b7648eb600 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/AbstractDispatcherConcurrencyTest.kt @@ -0,0 +1,52 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + + +abstract class AbstractDispatcherConcurrencyTest : TestBase() { + + public abstract val dispatcher: CoroutineDispatcher + + @Test + fun testLaunchAndJoin() = runTest { + expect(1) + var capturedMutableState = 0 + val job = GlobalScope.launch(dispatcher) { + ++capturedMutableState + expect(2) + } + runBlocking { job.join() } + assertEquals(1, capturedMutableState) + finish(3) + } + + @Test + fun testDispatcherHasOwnThreads() = runTest { + val channel = Channel() + GlobalScope.launch(dispatcher) { + channel.send(42) + } + + var result = ChannelResult.failure() + while (!result.isSuccess) { + result = channel.tryReceive() + // Block the thread, wait + } + // Delivery was successful, let's check it + assertEquals(42, result.getOrThrow()) + } + + @Test + fun testDelayInDispatcher() = runTest { + expect(1) + val job = GlobalScope.launch(dispatcher) { + expect(2) + delay(100) + expect(3) + } + runBlocking { job.join() } + finish(4) + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/AtomicCancellationTest.kt b/kotlinx-coroutines-core/concurrent/test/AtomicCancellationTest.kt new file mode 100644 index 0000000000..d43c307b65 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/AtomicCancellationTest.kt @@ -0,0 +1,143 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import kotlin.test.* + +class AtomicCancellationTest : TestBase() { + @Test + fun testSendCancellable() = runBlocking { + expect(1) + val channel = Channel() + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + channel.send(42) // suspends + expectUnreached() // should NOT execute because of cancellation + } + expect(3) + assertEquals(42, channel.receive()) // will schedule sender for further execution + job.cancel() // cancel the job next + yield() // now yield + finish(4) + } + + @Suppress("UNUSED_VARIABLE") + @Test + fun testSelectSendCancellable() = runBlocking { + expect(1) + val channel = Channel() + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + val result = select { // suspends + channel.onSend(42) { + expect(4) + "OK" + } + } + expectUnreached() // should NOT execute because of cancellation + } + expect(3) + assertEquals(42, channel.receive()) // will schedule sender for further execution + job.cancel() // cancel the job next + yield() // now yield + finish(4) + } + + @Test + fun testReceiveCancellable() = runBlocking { + expect(1) + val channel = Channel() + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + assertEquals(42, channel.receive()) // suspends + expectUnreached() // should NOT execute because of cancellation + } + expect(3) + channel.send(42) // will schedule receiver for further execution + job.cancel() // cancel the job next + yield() // now yield + finish(4) + } + + @Test + fun testSelectReceiveCancellable() = runBlocking { + expect(1) + val channel = Channel() + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + val result = select { // suspends + channel.onReceive { + assertEquals(42, it) + expect(4) + "OK" + } + } + expectUnreached() // should NOT execute because of cancellation + } + expect(3) + channel.send(42) // will schedule receiver for further execution + job.cancel() // cancel the job next + yield() // now yield + finish(4) + } + + @Test + fun testSelectDeferredAwaitCancellable() = runBlocking { + expect(1) + val deferred = async { // deferred, not yet complete + expect(4) + "OK" + } + assertEquals(false, deferred.isCompleted) + var job: Job? = null + launch { // will cancel job as soon as deferred completes + expect(5) + assertEquals(true, deferred.isCompleted) + job!!.cancel() + } + job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + select { // suspends + deferred.onAwait { expectUnreached() } + } + expectUnreached() // will not execute -- cancelled while dispatched + } finally { + finish(7) // but will execute finally blocks + } + } + expect(3) // continues to execute when the job suspends + yield() // to deferred & canceller + expect(6) + } + + @Test + fun testSelectJobJoinCancellable() = runBlocking { + expect(1) + val jobToJoin = launch { // not yet complete + expect(4) + } + assertEquals(false, jobToJoin.isCompleted) + var job: Job? = null + launch { // will cancel job as soon as jobToJoin completes + expect(5) + assertEquals(true, jobToJoin.isCompleted) + job!!.cancel() + } + job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + select { // suspends + jobToJoin.onJoin { expectUnreached() } + } + expectUnreached() // will not execute -- cancelled while dispatched + } finally { + finish(7) // but will execute finally blocks + } + } + expect(3) // continues to execute when the job suspends + yield() // to jobToJoin & canceller + expect(6) + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/CommonThreadLocalTest.kt b/kotlinx-coroutines-core/concurrent/test/CommonThreadLocalTest.kt new file mode 100644 index 0000000000..b692e2d002 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/CommonThreadLocalTest.kt @@ -0,0 +1,72 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.exceptions.* +import kotlinx.coroutines.internal.* +import kotlin.test.* + +class CommonThreadLocalTest: TestBase() { + + /** + * Tests the basic functionality of [commonThreadLocal]: storing a separate value for each thread. + */ + @Test + fun testThreadLocalBeingThreadLocal() = runTest { + val threadLocal = commonThreadLocal(Symbol("Test1")) + newSingleThreadContext("").use { + threadLocal.set(10) + assertEquals(10, threadLocal.get()) + val job1 = launch(it) { + threadLocal.set(20) + assertEquals(20, threadLocal.get()) + } + assertEquals(10, threadLocal.get()) + job1.join() + val job2 = launch(it) { + assertEquals(20, threadLocal.get()) + } + job2.join() + } + } + + /** + * Tests using [commonThreadLocal] with a nullable type. + */ + @Test + fun testThreadLocalWithNullableType() = runTest { + val threadLocal = commonThreadLocal(Symbol("Test2")) + newSingleThreadContext("").use { + assertNull(threadLocal.get()) + threadLocal.set(10) + assertEquals(10, threadLocal.get()) + val job1 = launch(it) { + assertNull(threadLocal.get()) + threadLocal.set(20) + assertEquals(20, threadLocal.get()) + } + assertEquals(10, threadLocal.get()) + job1.join() + threadLocal.set(null) + assertNull(threadLocal.get()) + val job2 = launch(it) { + assertEquals(20, threadLocal.get()) + threadLocal.set(null) + assertNull(threadLocal.get()) + } + job2.join() + } + } + + /** + * Tests that several instances of [commonThreadLocal] with different names don't affect each other. + */ + @Test + fun testThreadLocalsWithDifferentNamesNotInterfering() { + val value1 = commonThreadLocal(Symbol("Test3a")) + val value2 = commonThreadLocal(Symbol("Test3b")) + value1.set(5) + value2.set(6) + assertEquals(5, value1.get()) + assertEquals(6, value2.get()) + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/ConcurrentExceptionsStressTest.kt b/kotlinx-coroutines-core/concurrent/test/ConcurrentExceptionsStressTest.kt new file mode 100644 index 0000000000..a3fa590c44 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/ConcurrentExceptionsStressTest.kt @@ -0,0 +1,64 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.exceptions.* +import kotlinx.coroutines.internal.* +import kotlin.test.* + +class ConcurrentExceptionsStressTest : TestBase() { + private val nWorkers = 4 + private val nRepeat = 1000 * stressTestMultiplier + + private var workers: Array = emptyArray() + + @AfterTest + fun tearDown() { + workers.forEach { + it.close() + } + } + + @Test + fun testStress() = runTest { + workers = Array(nWorkers) { index -> + newSingleThreadContext("JobExceptionsStressTest-$index") + } + + repeat(nRepeat) { + testOnce() + } + } + + @Suppress("SuspendFunctionOnCoroutineScope") // workaround native inline fun stacktraces + private suspend fun CoroutineScope.testOnce() { + val deferred = async(NonCancellable) { + repeat(nWorkers) { index -> + // Always launch a coroutine even if parent job was already cancelled (atomic start) + launch(workers[index], start = CoroutineStart.ATOMIC) { + randomWait() + throw StressException(index) + } + } + } + deferred.join() + assertTrue(deferred.isCancelled) + val completionException = deferred.getCompletionExceptionOrNull() + val cause = completionException as? StressException + ?: unexpectedException("completion", completionException) + val suppressed = cause.suppressedExceptions + val indices = listOf(cause.index) + suppressed.mapIndexed { index, e -> + (e as? StressException)?.index ?: unexpectedException("suppressed $index", e) + } + repeat(nWorkers) { index -> + assertTrue(index in indices, "Exception $index is missing: $indices") + } + assertEquals(nWorkers, indices.size, "Duplicated exceptions in list: $indices") + } + + private fun unexpectedException(msg: String, e: Throwable?): Nothing { + throw IllegalStateException("Unexpected $msg exception", e) + } + + private class StressException(val index: Int) : Throwable() +} + diff --git a/kotlinx-coroutines-core/concurrent/test/ConcurrentTestUtilities.common.kt b/kotlinx-coroutines-core/concurrent/test/ConcurrentTestUtilities.common.kt new file mode 100644 index 0000000000..8cc7f939bf --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/ConcurrentTestUtilities.common.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.* +import kotlin.concurrent.Volatile +import kotlin.random.* + +fun randomWait() { + val n = Random.nextInt(1000) + if (n < 500) return // no wait 50% of time + repeat(n) { + BlackHole.sink *= 3 + } + // use the BlackHole value somehow, so even if the compiler gets smarter, it won't remove the object + val sinkValue = if (BlackHole.sink > 16) 1 else 0 + if (n + sinkValue > 900) yieldThread() +} + +private object BlackHole { + @Volatile + var sink = 1 +} + +expect inline fun yieldThread() + +expect fun currentThreadName(): String + +inline fun CloseableCoroutineDispatcher.use(block: (CloseableCoroutineDispatcher) -> Unit) { + try { + block(this) + } finally { + close() + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/DefaultDispatchersConcurrencyTest.kt b/kotlinx-coroutines-core/concurrent/test/DefaultDispatchersConcurrencyTest.kt new file mode 100644 index 0000000000..c231bb178e --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/DefaultDispatchersConcurrencyTest.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines + +class DefaultDispatcherConcurrencyTest : AbstractDispatcherConcurrencyTest() { + override val dispatcher: CoroutineDispatcher = Dispatchers.Default +} + +class IoDispatcherConcurrencyTest : AbstractDispatcherConcurrencyTest() { + override val dispatcher: CoroutineDispatcher = Dispatchers.IO +} diff --git a/kotlinx-coroutines-core/concurrent/test/JobStructuredJoinStressTest.kt b/kotlinx-coroutines-core/concurrent/test/JobStructuredJoinStressTest.kt new file mode 100644 index 0000000000..4abb495836 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/JobStructuredJoinStressTest.kt @@ -0,0 +1,61 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +/** + * Test a race between job failure and join. + * + * See [#1123](https://github.com/Kotlin/kotlinx.coroutines/issues/1123). + */ +class JobStructuredJoinStressTest : TestBase() { + private val nRepeats = 10_000 * stressTestMultiplier + + @Test + fun testStressRegularJoin() = runTest { + stress(Job::join) + } + + @Test + fun testStressSuspendCancellable() = runTest { + stress { job -> + suspendCancellableCoroutine { cont -> + job.invokeOnCompletion { cont.resume(Unit) } + } + } + } + + @Test + fun testStressSuspendCancellableReusable() = runTest { + stress { job -> + suspendCancellableCoroutineReusable { cont -> + job.invokeOnCompletion { cont.resume(Unit) } + } + } + } + + private fun stress(join: suspend (Job) -> Unit) { + expect(1) + repeat(nRepeats) { index -> + assertFailsWith { + runBlocking { + // launch in background + val job = launch(Dispatchers.Default) { + throw TestException("OK") // crash + } + try { + join(job) + error("Should not complete successfully") + } catch (e: CancellationException) { + // must always crash with cancellation exception + expect(2 + index) + } catch (e: Throwable) { + error("Unexpected exception", e) + } + } + } + } + finish(2 + nRepeats) + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/LimitedParallelismConcurrentTest.kt b/kotlinx-coroutines-core/concurrent/test/LimitedParallelismConcurrentTest.kt new file mode 100644 index 0000000000..5985aaa137 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/LimitedParallelismConcurrentTest.kt @@ -0,0 +1,92 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.exceptions.* +import kotlin.coroutines.* +import kotlin.test.* + +class LimitedParallelismConcurrentTest : TestBase() { + + private val targetParallelism = 4 + private val iterations = 100_000 + private val parallelism = atomic(0) + + private fun checkParallelism() { + val value = parallelism.incrementAndGet() + randomWait() + assertTrue { value <= targetParallelism } + parallelism.decrementAndGet() + } + + @Test + fun testLimitedExecutor() = runTest { + val executor = newFixedThreadPoolContext(targetParallelism, "test") + val view = executor.limitedParallelism(targetParallelism) + doStress { + repeat(iterations) { + launch(view) { + checkParallelism() + } + } + } + executor.close() + } + + private suspend inline fun doStress(crossinline block: suspend CoroutineScope.() -> Unit) { + repeat(stressTestMultiplier) { + coroutineScope { + block() + } + } + } + + @Test + fun testTaskFairness() = runTest { + val executor = newSingleThreadContext("test") + val view = executor.limitedParallelism(1) + val view2 = executor.limitedParallelism(1) + val j1 = launch(view) { + while (true) { + yield() + } + } + val j2 = launch(view2) { j1.cancel() } + joinAll(j1, j2) + executor.close() + } + + /** + * Tests that, when no tasks are present, the limited dispatcher does not dispatch any tasks. + * This is important for the case when a dispatcher is closeable and the [CoroutineDispatcher.limitedParallelism] + * machinery could trigger a dispatch after the dispatcher is closed. + */ + @Test + fun testNotDoingDispatchesWhenNoTasksArePresent() = runTest { + class NaggingDispatcher: CoroutineDispatcher() { + private val closed = atomic(false) + override fun dispatch(context: CoroutineContext, block: Runnable) { + if (closed.value) + fail("Dispatcher was closed, but still dispatched a task") + Dispatchers.Default.dispatch(context, block) + } + fun close() { + closed.value = true + } + } + repeat(stressTestMultiplier * 500_000) { + val dispatcher = NaggingDispatcher() + val view = dispatcher.limitedParallelism(1) + val deferred = CompletableDeferred() + val job = launch(view) { + deferred.await() + } + launch(Dispatchers.Default) { + deferred.complete(Unit) + } + job.join() + dispatcher.close() + } + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/MultithreadedDispatcherStressTest.kt b/kotlinx-coroutines-core/concurrent/test/MultithreadedDispatcherStressTest.kt new file mode 100644 index 0000000000..9445637078 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/MultithreadedDispatcherStressTest.kt @@ -0,0 +1,31 @@ +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlin.coroutines.* +import kotlin.test.* + +class MultithreadedDispatcherStressTest { + private val shared = atomic(0) + + /** + * Tests that [newFixedThreadPoolContext] will not drop tasks when closed. + */ + @Test + fun testClosingNotDroppingTasks() { + repeat(7) { + shared.value = 0 + val nThreads = it + 1 + val dispatcher = newFixedThreadPoolContext(nThreads, "testMultiThreadedContext") + repeat(1_000) { + dispatcher.dispatch(EmptyCoroutineContext, Runnable { + shared.incrementAndGet() + }) + } + dispatcher.close() + while (shared.value < 1_000) { + // spin. + // the test will hang here if the dispatcher drops tasks. + } + } + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/RunBlockingTest.kt b/kotlinx-coroutines-core/concurrent/test/RunBlockingTest.kt new file mode 100644 index 0000000000..43f7976ffa --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/RunBlockingTest.kt @@ -0,0 +1,197 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.exceptions.* +import kotlin.coroutines.* +import kotlin.test.* +import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.seconds + +class RunBlockingTest : TestBase() { + + @Test + fun testWithTimeoutBusyWait() = runTest { + val value = withTimeoutOrNull(10) { + while (isActive) { + // Busy wait + } + "value" + } + + assertEquals("value", value) + } + + @Test + fun testPrivateEventLoop() { + expect(1) + runBlocking { + expect(2) + assertIs(coroutineContext[ContinuationInterceptor]) + yield() // is supported! + expect(3) + } + finish(4) + } + + @Test + fun testOuterEventLoop() { + expect(1) + runBlocking { + expect(2) + val outerEventLoop = coroutineContext[ContinuationInterceptor] as EventLoop + runBlocking(coroutineContext) { + expect(3) + // still same event loop + assertSame(coroutineContext[ContinuationInterceptor], outerEventLoop) + yield() // still works + expect(4) + } + expect(5) + } + finish(6) + } + + @Test + fun testOtherDispatcher() = runTest { + expect(1) + val name = "RunBlockingTest.testOtherDispatcher" + val thread = newSingleThreadContext(name) + runBlocking(thread) { + expect(2) + assertSame(coroutineContext[ContinuationInterceptor], thread) + assertTrue(currentThreadName().contains(name)) + yield() // should work + expect(3) + } + finish(4) + thread.close() + } + + @Test + fun testCancellation() = runTest { + newFixedThreadPoolContext(2, "testCancellation").use { + val job = GlobalScope.launch(it) { + runBlocking(coroutineContext) { + while (true) { + yield() + } + } + } + + runBlocking { + job.cancelAndJoin() + } + } + } + + @Test + fun testCancelWithDelay() { + // see https://github.com/Kotlin/kotlinx.coroutines/issues/586 + try { + runBlocking { + expect(1) + coroutineContext.cancel() + expect(2) + try { + delay(1) + expectUnreached() + } finally { + expect(3) + } + } + expectUnreached() + } catch (e: CancellationException) { + finish(4) + } + } + + @Test + fun testDispatchOnShutdown(): Unit = assertFailsWith { + runBlocking { + expect(1) + val job = launch(NonCancellable) { + try { + expect(2) + delay(Long.MAX_VALUE) + } finally { + finish(4) + } + } + + yield() + expect(3) + coroutineContext.cancel() + job.cancel() + } + }.let { } + + @Test + fun testDispatchOnShutdown2(): Unit = assertFailsWith { + runBlocking { + coroutineContext.cancel() + expect(1) + val job = launch(NonCancellable, start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + delay(Long.MAX_VALUE) + } finally { + finish(4) + } + } + + expect(3) + job.cancel() + } + }.let { } + + @Test + fun testNestedRunBlocking() = runBlocking { + delay(100) + val value = runBlocking { + delay(100) + runBlocking { + delay(100) + 1 + } + } + + assertEquals(1, value) + } + + @Test + fun testIncompleteState() { + val handle = runBlocking { + // See #835 + coroutineContext[Job]!!.invokeOnCompletion { } + } + + handle.dispose() + } + + @Test + fun testCancelledParent() { + val job = Job() + job.cancel() + assertFailsWith { + runBlocking(job) { + expectUnreached() + } + } + } + + /** Tests that the delayed tasks scheduled on a closed `runBlocking` event loop get processed in reasonable time. */ + @Test + fun testReschedulingDelayedTasks() { + val job = runBlocking { + val dispatcher = coroutineContext[ContinuationInterceptor]!! + GlobalScope.launch(dispatcher) { + delay(1.milliseconds) + } + } + runBlocking { + withTimeout(10.seconds) { + job.join() + } + } + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/channels/BroadcastChannelSubStressTest.kt b/kotlinx-coroutines-core/concurrent/test/channels/BroadcastChannelSubStressTest.kt new file mode 100644 index 0000000000..8db6d8b9ec --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/channels/BroadcastChannelSubStressTest.kt @@ -0,0 +1,57 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlin.test.* + +/** + * Creates a broadcast channel and repeatedly opens new subscription, receives event, closes it, + * to stress test the logic of opening the subscription + * to broadcast channel while events are being concurrently sent to it. + */ +class BroadcastChannelSubStressTest: TestBase() { + + private val nSeconds = maxOf(5, stressTestMultiplier) + private val sentTotal = atomic(0L) + private val receivedTotal = atomic(0L) + + @Test + fun testStress() = runTest { + TestBroadcastChannelKind.entries.forEach { kind -> + println("--- BroadcastChannelSubStressTest $kind") + val broadcast = kind.create() + val sender = + launch(context = Dispatchers.Default + CoroutineName("Sender")) { + while (isActive) { + broadcast.send(sentTotal.incrementAndGet()) + } + } + val receiver = + launch(context = Dispatchers.Default + CoroutineName("Receiver")) { + var last = -1L + while (isActive) { + val channel = broadcast.openSubscription() + val i = channel.receive() + check(i >= last) { "Last was $last, got $i" } + if (!kind.isConflated) check(i != last) { "Last was $last, got it again" } + receivedTotal.incrementAndGet() + last = i + channel.cancel() + } + } + var prevSent = -1L + repeat(nSeconds) { sec -> + delay(1000) + val curSent = sentTotal.value + println("${sec + 1}: Sent $curSent, received ${receivedTotal.value}") + check(curSent > prevSent) { "Send stalled at $curSent events" } + prevSent = curSent + } + withTimeout(5000) { + sender.cancelAndJoin() + receiver.cancelAndJoin() + } + } + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/channels/ChannelCancelUndeliveredElementStressTest.kt b/kotlinx-coroutines-core/concurrent/test/channels/ChannelCancelUndeliveredElementStressTest.kt new file mode 100644 index 0000000000..292d5bda73 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/channels/ChannelCancelUndeliveredElementStressTest.kt @@ -0,0 +1,100 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import kotlin.random.* +import kotlin.test.* + +class ChannelCancelUndeliveredElementStressTest : TestBase() { + private val repeatTimes = (if (isNative) 1_000 else 10_000) * stressTestMultiplier + + // total counters + private var sendCnt = 0 + private var trySendFailedCnt = 0 + private var receivedCnt = 0 + private var undeliveredCnt = 0 + + // last operation + private var lastReceived = 0 + private var dSendCnt = 0 + private var dSendExceptionCnt = 0 + private var dTrySendFailedCnt = 0 + private var dReceivedCnt = 0 + private val dUndeliveredCnt = atomic(0) + + @Test + fun testStress() = runTest { + repeat(repeatTimes) { + val channel = Channel(1) { dUndeliveredCnt.incrementAndGet() } + val j1 = launch(Dispatchers.Default) { + sendOne(channel) // send first + sendOne(channel) // send second + } + val j2 = launch(Dispatchers.Default) { + receiveOne(channel) // receive one element from the channel + channel.cancel() // cancel the channel + } + + joinAll(j1, j2) + + // All elements must be either received or undelivered (IN every run) + if (dSendCnt - dTrySendFailedCnt != dReceivedCnt + dUndeliveredCnt.value) { + println(" Send: $dSendCnt") + println("Send exception: $dSendExceptionCnt") + println("trySend failed: $dTrySendFailedCnt") + println(" Received: $dReceivedCnt") + println(" Undelivered: ${dUndeliveredCnt.value}") + error("Failed") + } + (channel as? BufferedChannel<*>)?.checkSegmentStructureInvariants() + trySendFailedCnt += dTrySendFailedCnt + receivedCnt += dReceivedCnt + undeliveredCnt += dUndeliveredCnt.value + // clear for next run + dSendCnt = 0 + dSendExceptionCnt = 0 + dTrySendFailedCnt = 0 + dReceivedCnt = 0 + dUndeliveredCnt.value = 0 + } + // Stats + println(" Send: $sendCnt") + println("trySend failed: $trySendFailedCnt") + println(" Received: $receivedCnt") + println(" Undelivered: $undeliveredCnt") + assertEquals(sendCnt - trySendFailedCnt, receivedCnt + undeliveredCnt) + } + + private suspend fun sendOne(channel: Channel) { + dSendCnt++ + val i = ++sendCnt + try { + when (Random.nextInt(2)) { + 0 -> channel.send(i) + 1 -> if (!channel.trySend(i).isSuccess) { + dTrySendFailedCnt++ + } + } + } catch (e: Throwable) { + assertIs(e) // the only exception possible in this test + dSendExceptionCnt++ + throw e + } + } + + private suspend fun receiveOne(channel: Channel) { + val received = when (Random.nextInt(3)) { + 0 -> channel.receive() + 1 -> channel.receiveCatching().getOrElse { error("Cannot be closed yet") } + 2 -> select { + channel.onReceive { it } + } + else -> error("Cannot happen") + } + assertTrue(received > lastReceived) + dReceivedCnt++ + lastReceived = received + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/channels/ConflatedBroadcastChannelNotifyStressTest.kt b/kotlinx-coroutines-core/concurrent/test/channels/ConflatedBroadcastChannelNotifyStressTest.kt new file mode 100644 index 0000000000..99b3262560 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/channels/ConflatedBroadcastChannelNotifyStressTest.kt @@ -0,0 +1,86 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlin.test.* + +@Suppress("DEPRECATION_ERROR") +class ConflatedBroadcastChannelNotifyStressTest : TestBase() { + private val nSenders = 2 + private val nReceivers = 3 + private val nEvents = (if (isNative) 5_000 else 500_000) * stressTestMultiplier + private val timeLimit = 30_000L * stressTestMultiplier // 30 sec + + private val broadcast = ConflatedBroadcastChannel() + + private val sendersCompleted = atomic(0) + private val receiversCompleted = atomic(0) + private val sentTotal = atomic(0) + private val receivedTotal = atomic(0) + + @Test + fun testStressNotify()= runTest { + println("--- ConflatedBroadcastChannelNotifyStressTest") + val senders = List(nSenders) { senderId -> + launch(Dispatchers.Default + CoroutineName("Sender$senderId")) { + repeat(nEvents) { i -> + if (i % nSenders == senderId) { + broadcast.trySend(i) + sentTotal.incrementAndGet() + yield() + } + } + sendersCompleted.incrementAndGet() + } + } + val receivers = List(nReceivers) { receiverId -> + launch(Dispatchers.Default + CoroutineName("Receiver$receiverId")) { + var last = -1 + while (isActive) { + val i = waitForEvent() + if (i > last) { + receivedTotal.incrementAndGet() + last = i + } + if (i >= nEvents) break + yield() + } + receiversCompleted.incrementAndGet() + } + } + // print progress + val progressJob = launch { + var seconds = 0 + while (true) { + delay(1000) + println("${++seconds}: Sent ${sentTotal.value}, received ${receivedTotal.value}") + } + } + try { + withTimeout(timeLimit) { + senders.forEach { it.join() } + broadcast.trySend(nEvents) // last event to signal receivers termination + receivers.forEach { it.join() } + } + } catch (e: CancellationException) { + println("!!! Test timed out $e") + } + progressJob.cancel() + println("Tested with nSenders=$nSenders, nReceivers=$nReceivers") + println("Completed successfully ${sendersCompleted.value} sender coroutines") + println("Completed successfully ${receiversCompleted.value} receiver coroutines") + println(" Sent ${sentTotal.value} events") + println(" Received ${receivedTotal.value} events") + assertEquals(nSenders, sendersCompleted.value) + assertEquals(nReceivers, receiversCompleted.value) + assertEquals(nEvents, sentTotal.value) + } + + private suspend fun waitForEvent(): Int = + with(broadcast.openSubscription()) { + val value = receive() + cancel() + value + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/channels/TrySendBlockingTest.kt b/kotlinx-coroutines-core/concurrent/test/channels/TrySendBlockingTest.kt new file mode 100644 index 0000000000..4a485503a5 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/channels/TrySendBlockingTest.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class TrySendBlockingTest : TestBase() { + + @Test + fun testTrySendBlocking() = runBlocking { // For old MM + val ch = Channel() + val sum = GlobalScope.async { + var sum = 0 + ch.consumeEach { sum += it } + sum + } + repeat(10) { + assertTrue(ch.trySendBlocking(it).isSuccess) + } + ch.close() + assertEquals(45, runBlocking { sum.await() }) + } + + @Test + fun testTrySendBlockingClosedChannel() { + run { + val channel = Channel().also { it.close() } + channel.trySendBlocking(Unit) + .onSuccess { expectUnreached() } + .onFailure { assertIs(it) } + .also { assertTrue { it.isClosed } } + } + + run { + val channel = Channel().also { it.close(TestException()) } + channel.trySendBlocking(Unit) + .onSuccess { expectUnreached() } + .onFailure { assertIs(it) } + .also { assertTrue { it.isClosed } } + } + + run { + val channel = Channel().also { it.cancel(TestCancellationException()) } + channel.trySendBlocking(Unit) + .onSuccess { expectUnreached() } + .onFailure { assertIs(it) } + .also { assertTrue { it.isClosed } } + } + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/flow/CombineStressTest.kt b/kotlinx-coroutines-core/concurrent/test/flow/CombineStressTest.kt new file mode 100644 index 0000000000..102d7d2d55 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/flow/CombineStressTest.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class CombineStressTest : TestBase() { + + @Test + fun testCancellation() = runTest { + withContext(Dispatchers.Default + CoroutineExceptionHandler { _, _ -> expectUnreached() }) { + flow { + expect(1) + repeat(1_000 * stressTestMultiplier) { + emit(it) + } + }.flatMapLatest { + combine(flowOf(it), flowOf(it)) { arr -> arr[0] } + }.collect() + finish(2) + reset() + } + } + + @Test + fun testFailure() = runTest { + val innerIterations = 100 * stressTestMultiplierSqrt + val outerIterations = 10 * stressTestMultiplierSqrt + withContext(Dispatchers.Default + CoroutineExceptionHandler { _, _ -> expectUnreached() }) { + repeat(outerIterations) { + try { + flow { + expect(1) + repeat(innerIterations) { + emit(it) + } + }.flatMapLatest { + combine(flowOf(it), flowOf(it)) { arr -> arr[0] } + }.onEach { + if (it >= innerIterations / 2) throw TestException() + }.collect() + } catch (e: TestException) { + expect(2) + } + finish(3) + reset() + } + } + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/flow/FlowCancellationTest.kt b/kotlinx-coroutines-core/concurrent/test/flow/FlowCancellationTest.kt new file mode 100644 index 0000000000..4e36563738 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/flow/FlowCancellationTest.kt @@ -0,0 +1,60 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlin.test.* + +class FlowCancellationTest : TestBase() { + + @Test + fun testEmitIsCooperative() = runTest { + val latch = Channel(1) + val job = flow { + expect(1) + latch.send(Unit) + while (true) { + emit(42) + } + }.launchIn(this + Dispatchers.Default) + + latch.receive() + expect(2) + job.cancelAndJoin() + finish(3) + } + + @Test + fun testIsActiveOnCurrentContext() = runTest { + val latch = Channel(1) + val job = flow { + expect(1) + latch.send(Unit) + while (currentCoroutineContext().isActive) { + // Do nothing + } + }.launchIn(this + Dispatchers.Default) + + latch.receive() + expect(2) + job.cancelAndJoin() + finish(3) + } + + @Test + fun testFlowWithEmptyContext() = runTest { + expect(1) + withEmptyContext { + val flow = flow { + expect(2) + emit("OK") + } + flow.collect { + expect(3) + assertEquals("OK", it) + } + } + finish(4) + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/flow/StateFlowCommonStressTest.kt b/kotlinx-coroutines-core/concurrent/test/flow/StateFlowCommonStressTest.kt new file mode 100644 index 0000000000..e066052f52 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/flow/StateFlowCommonStressTest.kt @@ -0,0 +1,44 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.random.* +import kotlin.test.* + +// A simplified version of StateFlowStressTest +class StateFlowCommonStressTest : TestBase() { + private val state = MutableStateFlow(0) + + @Test + fun testSingleEmitterAndCollector() = runTest { + var collected = 0L + val collector = launch(Dispatchers.Default) { + // collect, but abort and collect again after every 1000 values to stress allocation/deallocation + do { + val batchSize = Random.nextInt(1..1000) + var index = 0 + val cnt = state.onEach { value -> + // the first value in batch is allowed to repeat, but cannot go back + val ok = if (index++ == 0) value >= collected else value > collected + check(ok) { + "Values must be monotonic, but $value is not, was $collected" + } + collected = value + }.take(batchSize).map { 1 }.sum() + } while (cnt == batchSize) + } + + var current = 1L + val emitter = launch { + while (true) { + state.value = current++ + if (current % 1000 == 0L) yield() // make it cancellable + } + } + + delay(3000) + emitter.cancelAndJoin() + collector.cancelAndJoin() + assertTrue { current >= collected / 2 } + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/flow/StateFlowUpdateCommonTest.kt b/kotlinx-coroutines-core/concurrent/test/flow/StateFlowUpdateCommonTest.kt new file mode 100644 index 0000000000..a4a5118ae9 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/flow/StateFlowUpdateCommonTest.kt @@ -0,0 +1,36 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.test.* + +// A simplified version of StateFlowUpdateStressTest +class StateFlowUpdateCommonTest : TestBase() { + private val iterations = 100_000 * stressTestMultiplier + + @Test + fun testUpdate() = doTest { update { it + 1 } } + + @Test + fun testUpdateAndGet() = doTest { updateAndGet { it + 1 } } + + @Test + fun testGetAndUpdate() = doTest { getAndUpdate { it + 1 } } + + private fun doTest(increment: MutableStateFlow.() -> Unit) = runTest { + val flow = MutableStateFlow(0) + val j1 = launch(Dispatchers.Default) { + repeat(iterations / 2) { + flow.increment() + } + } + + repeat(iterations / 2) { + flow.increment() + } + + joinAll(j1) + assertEquals(iterations, flow.value) + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/selects/SelectChannelStressTest.kt b/kotlinx-coroutines-core/concurrent/test/selects/SelectChannelStressTest.kt new file mode 100644 index 0000000000..32098b290e --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/selects/SelectChannelStressTest.kt @@ -0,0 +1,67 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class SelectChannelStressTest: TestBase() { + + // Running less iterations on native platforms because of some performance regression + private val iterations = (if (isNative) 1_000 else 1_000_000) * stressTestMultiplier + + @Test + fun testSelectSendResourceCleanupBufferedChannel() = runTest { + val channel = Channel(1) + expect(1) + channel.send(-1) // fill the buffer, so all subsequent sends cannot proceed + repeat(iterations) { i -> + select { + channel.onSend(i) { expectUnreached() } + default { expect(i + 2) } + } + } + finish(iterations + 2) + } + + @Test + fun testSelectReceiveResourceCleanupBufferedChannel() = runTest { + val channel = Channel(1) + expect(1) + repeat(iterations) { i -> + select { + channel.onReceive { expectUnreached() } + default { expect(i + 2) } + } + } + finish(iterations + 2) + } + + @Test + fun testSelectSendResourceCleanupRendezvousChannel() = runTest { + val channel = Channel(Channel.RENDEZVOUS) + expect(1) + repeat(iterations) { i -> + select { + channel.onSend(i) { expectUnreached() } + default { expect(i + 2) } + } + } + finish(iterations + 2) + } + + @Test + fun testSelectReceiveResourceRendezvousChannel() = runTest { + val channel = Channel(Channel.RENDEZVOUS) + expect(1) + repeat(iterations) { i -> + select { + channel.onReceive { expectUnreached() } + default { expect(i + 2) } + } + } + finish(iterations + 2) + } + + internal fun SelectBuilder.default(block: suspend () -> R) = onTimeout(0, block) +} diff --git a/kotlinx-coroutines-core/concurrent/test/selects/SelectMutexStressTest.kt b/kotlinx-coroutines-core/concurrent/test/selects/SelectMutexStressTest.kt new file mode 100644 index 0000000000..e8c2c822c8 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/selects/SelectMutexStressTest.kt @@ -0,0 +1,30 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import kotlin.test.* + +class SelectMutexStressTest : TestBase() { + @Test + fun testSelectCancelledResourceRelease() = runTest { + val n = 1_000 * stressTestMultiplier + val mutex = Mutex(true) as MutexImpl // locked + expect(1) + repeat(n) { i -> + val job = launch(kotlin.coroutines.coroutineContext) { + expect(i + 2) + select { + mutex.onLock { + expectUnreached() // never able to lock + } + } + } + yield() // to the launched job, so that it suspends + job.cancel() // cancel the job and select + yield() // so it can cleanup after itself + } + assertTrue(mutex.isLocked) + finish(n + 2) + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/sync/MutexStressTest.kt b/kotlinx-coroutines-core/concurrent/test/sync/MutexStressTest.kt new file mode 100644 index 0000000000..67ff03b91e --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/sync/MutexStressTest.kt @@ -0,0 +1,134 @@ +package kotlinx.coroutines.sync + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.exceptions.* +import kotlinx.coroutines.selects.* +import kotlin.test.* + +class MutexStressTest : TestBase() { + + private val n = 1000 * stressTestMultiplier // It mostly stresses K/N as JVM Mutex is tested by lincheck + + @Test + fun testDefaultDispatcher() = runTest { testBody(Dispatchers.Default) } + + @Test + fun testSingleThreadContext() = runTest { + newSingleThreadContext("testSingleThreadContext").use { + testBody(it) + } + } + + @Test + fun testMultiThreadedContextWithSingleWorker() = runTest { + newFixedThreadPoolContext(1, "testMultiThreadedContextWithSingleWorker").use { + testBody(it) + } + } + + @Test + fun testMultiThreadedContext() = runTest { + newFixedThreadPoolContext(8, "testMultiThreadedContext").use { + testBody(it) + } + } + + @Suppress("SuspendFunctionOnCoroutineScope") + private suspend fun CoroutineScope.testBody(dispatcher: CoroutineDispatcher) { + val k = 100 + var shared = 0 + val mutex = Mutex() + val jobs = List(n) { + launch(dispatcher) { + repeat(k) { + mutex.lock() + shared++ + mutex.unlock() + } + } + } + jobs.forEach { it.join() } + assertEquals(n * k, shared) + } + + @Test + fun stressUnlockCancelRace() = runTest { + val n = 10_000 * stressTestMultiplier + val mutex = Mutex(true) // create a locked mutex + newSingleThreadContext("SemaphoreStressTest").use { pool -> + repeat(n) { + // Initially, we hold the lock and no one else can `lock`, + // otherwise it's a bug. + assertTrue(mutex.isLocked) + var job1EnteredCriticalSection = false + val job1 = launch(start = CoroutineStart.UNDISPATCHED) { + mutex.lock() + job1EnteredCriticalSection = true + mutex.unlock() + } + // check that `job1` didn't finish the call to `acquire()` + assertEquals(false, job1EnteredCriticalSection) + val job2 = launch(pool) { + mutex.unlock() + } + // Because `job2` executes in a separate thread, this + // cancellation races with the call to `unlock()`. + job1.cancelAndJoin() + job2.join() + assertFalse(mutex.isLocked) + mutex.lock() + } + } + } + + @Test + fun stressUnlockCancelRaceWithSelect() = runTest { + val n = 10_000 * stressTestMultiplier + val mutex = Mutex(true) // create a locked mutex + newSingleThreadContext("SemaphoreStressTest").use { pool -> + repeat(n) { + // Initially, we hold the lock and no one else can `lock`, + // otherwise it's a bug. + assertTrue(mutex.isLocked) + var job1EnteredCriticalSection = false + val job1 = launch(start = CoroutineStart.UNDISPATCHED) { + select { + mutex.onLock { + job1EnteredCriticalSection = true + mutex.unlock() + } + } + } + // check that `job1` didn't finish the call to `acquire()` + assertEquals(false, job1EnteredCriticalSection) + val job2 = launch(pool) { + mutex.unlock() + } + // Because `job2` executes in a separate thread, this + // cancellation races with the call to `unlock()`. + job1.cancelAndJoin() + job2.join() + assertFalse(mutex.isLocked) + mutex.lock() + } + } + } + + @Test + fun testShouldBeUnlockedOnCancellation() = runTest { + val mutex = Mutex() + val n = 1000 * stressTestMultiplier + repeat(n) { + val job = launch(Dispatchers.Default) { + mutex.lock() + mutex.unlock() + } + mutex.withLock { + job.cancel() + } + job.join() + assertFalse { mutex.isLocked } + } + } +} diff --git a/kotlinx-coroutines-core/concurrent/test/sync/SemaphoreStressTest.kt b/kotlinx-coroutines-core/concurrent/test/sync/SemaphoreStressTest.kt new file mode 100644 index 0000000000..4be485d728 --- /dev/null +++ b/kotlinx-coroutines-core/concurrent/test/sync/SemaphoreStressTest.kt @@ -0,0 +1,135 @@ +package kotlinx.coroutines.sync + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.exceptions.* +import kotlin.test.* + +class SemaphoreStressTest : TestBase() { + + private val iterations = (if (isNative) 1_000 else 10_000) * stressTestMultiplier + + @Test + fun testStressTestAsMutex() = runTest { + val n = iterations + val k = 100 + var shared = 0 + val semaphore = Semaphore(1) + val jobs = List(n) { + launch(Dispatchers.Default) { + repeat(k) { + semaphore.acquire() + shared++ + semaphore.release() + } + } + } + jobs.forEach { it.join() } + assertEquals(n * k, shared) + } + + @Test + fun testStress() = runTest { + val n = iterations + val k = 100 + val semaphore = Semaphore(10) + val jobs = List(n) { + launch(Dispatchers.Default) { + repeat(k) { + semaphore.acquire() + semaphore.release() + } + } + } + jobs.forEach { it.join() } + } + + @Test + fun testStressAsMutex() = runTest { + runBlocking(Dispatchers.Default) { + val n = iterations + val k = 100 + var shared = 0 + val semaphore = Semaphore(1) + val jobs = List(n) { + launch { + repeat(k) { + semaphore.acquire() + shared++ + semaphore.release() + } + } + } + jobs.forEach { it.join() } + assertEquals(n * k, shared) + } + } + + @Test + fun testStressCancellation() = runTest { + val n = iterations + val semaphore = Semaphore(1) + semaphore.acquire() + repeat(n) { + val job = launch(Dispatchers.Default) { + semaphore.acquire() + } + yield() + job.cancelAndJoin() + } + assertEquals(0, semaphore.availablePermits) + semaphore.release() + assertEquals(1, semaphore.availablePermits) + } + + /** + * This checks if repeated releases that race with cancellations put + * the semaphore into an incorrect state where permits are leaked. + */ + @Test + fun testStressReleaseCancelRace() = runTest { + val n = iterations + val semaphore = Semaphore(1, 1) + newSingleThreadContext("SemaphoreStressTest").use { pool -> + repeat (n) { + // Initially, we hold the permit and no one else can `acquire`, + // otherwise it's a bug. + assertEquals(0, semaphore.availablePermits) + var job1EnteredCriticalSection = false + val job1 = launch(start = CoroutineStart.UNDISPATCHED) { + semaphore.acquire() + job1EnteredCriticalSection = true + semaphore.release() + } + // check that `job1` didn't finish the call to `acquire()` + assertEquals(false, job1EnteredCriticalSection) + val job2 = launch(pool) { + semaphore.release() + } + // Because `job2` executes in a separate thread, this + // cancellation races with the call to `release()`. + job1.cancelAndJoin() + job2.join() + assertEquals(1, semaphore.availablePermits) + semaphore.acquire() + } + } + } + + @Test + fun testShouldBeUnlockedOnCancellation() = runTest { + val semaphore = Semaphore(1) + val n = 1000 * stressTestMultiplier + repeat(n) { + val job = launch(Dispatchers.Default) { + semaphore.acquire() + semaphore.release() + } + semaphore.withPermit { + job.cancel() + } + job.join() + assertTrue { semaphore.availablePermits == 1 } + } + } +} diff --git a/kotlinx-coroutines-core/js/src/CoroutineContext.kt b/kotlinx-coroutines-core/js/src/CoroutineContext.kt new file mode 100644 index 0000000000..4384943c42 --- /dev/null +++ b/kotlinx-coroutines-core/js/src/CoroutineContext.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines + +import kotlinx.browser.* + +private external val navigator: dynamic +private const val UNDEFINED = "undefined" +internal external val process: dynamic + +internal actual fun createDefaultDispatcher(): CoroutineDispatcher = when { + // Check if we are running under jsdom. WindowDispatcher doesn't work under jsdom because it accesses MessageEvent#source. + // It is not implemented in jsdom, see https://github.com/jsdom/jsdom/blob/master/Changelog.md + // "It's missing a few semantics, especially around origins, as well as MessageEvent source." + isJsdom() -> NodeDispatcher + // Check if we are in the browser and must use window.postMessage to avoid setTimeout throttling + jsTypeOf(window) != UNDEFINED && window.asDynamic() != null && jsTypeOf(window.asDynamic().addEventListener) != UNDEFINED -> + window.asCoroutineDispatcher() + // If process is undefined (e.g. in NativeScript, #1404), use SetTimeout-based dispatcher + jsTypeOf(process) == UNDEFINED || jsTypeOf(process.nextTick) == UNDEFINED -> SetTimeoutDispatcher + // Fallback to NodeDispatcher when browser environment is not detected + else -> NodeDispatcher +} + +private fun isJsdom() = jsTypeOf(navigator) != UNDEFINED && + navigator != null && + navigator.userAgent != null && + jsTypeOf(navigator.userAgent) != UNDEFINED && + jsTypeOf(navigator.userAgent.match) != UNDEFINED && + navigator.userAgent.match("\\bjsdom\\b") diff --git a/kotlinx-coroutines-core/js/src/Debug.kt b/kotlinx-coroutines-core/js/src/Debug.kt new file mode 100644 index 0000000000..d36d673283 --- /dev/null +++ b/kotlinx-coroutines-core/js/src/Debug.kt @@ -0,0 +1,20 @@ +package kotlinx.coroutines + +private var counter = 0 + +internal actual val DEBUG: Boolean = false + +internal actual val Any.hexAddress: String + get() { + var result = this.asDynamic().__debug_counter + if (jsTypeOf(result) !== "number") { + result = ++counter + this.asDynamic().__debug_counter = result + + } + return (result as Int).toString() + } + +internal actual val Any.classSimpleName: String get() = this::class.simpleName ?: "Unknown" + +internal actual inline fun assert(value: () -> Boolean) {} diff --git a/kotlinx-coroutines-core/js/src/JSDispatcher.kt b/kotlinx-coroutines-core/js/src/JSDispatcher.kt new file mode 100644 index 0000000000..f0cd50a0b8 --- /dev/null +++ b/kotlinx-coroutines-core/js/src/JSDispatcher.kt @@ -0,0 +1,70 @@ +package kotlinx.coroutines + +import org.w3c.dom.* +import kotlin.js.Promise + +internal actual typealias W3CWindow = Window + +internal actual fun w3cSetTimeout(window: W3CWindow, handler: () -> Unit, timeout: Int): Int = + setTimeout(window, handler, timeout) + +internal actual fun w3cSetTimeout(handler: () -> Unit, timeout: Int): Int = + setTimeout(handler, timeout) + +internal actual fun w3cClearTimeout(window: W3CWindow, handle: Int) = + window.clearTimeout(handle) + +internal actual fun w3cClearTimeout(handle: Int) = + clearTimeout(handle) + +internal actual class ScheduledMessageQueue actual constructor(private val dispatcher: SetTimeoutBasedDispatcher) : MessageQueue() { + internal val processQueue: dynamic = { process() } + + actual override fun schedule() { + dispatcher.scheduleQueueProcessing() + } + + actual override fun reschedule() { + setTimeout(processQueue, 0) + } + + internal actual fun setTimeout(timeout: Int) { + setTimeout(processQueue, timeout) + } +} + +internal object NodeDispatcher : SetTimeoutBasedDispatcher() { + override fun scheduleQueueProcessing() { + process.nextTick(messageQueue.processQueue) + } +} + +internal actual class WindowMessageQueue actual constructor(private val window: W3CWindow) : MessageQueue() { + private val messageName = "dispatchCoroutine" + + init { + window.addEventListener("message", { event: dynamic -> + if (event.source == window && event.data == messageName) { + event.stopPropagation() + process() + } + }, true) + } + + actual override fun schedule() { + Promise.resolve(Unit).then({ process() }) + } + + actual override fun reschedule() { + window.postMessage(messageName, "*") + } +} + +// We need to reference global setTimeout and clearTimeout so that it works on Node.JS as opposed to +// using them via "window" (which only works in browser) +private external fun setTimeout(handler: dynamic, timeout: Int = definedExternally): Int + +private external fun clearTimeout(handle: Int = definedExternally) + +private fun setTimeout(window: Window, handler: () -> Unit, timeout: Int): Int = + window.setTimeout(handler, timeout) diff --git a/kotlinx-coroutines-core/js/src/Promise.kt b/kotlinx-coroutines-core/js/src/Promise.kt new file mode 100644 index 0000000000..5eb93d348e --- /dev/null +++ b/kotlinx-coroutines-core/js/src/Promise.kt @@ -0,0 +1,67 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* +import kotlin.js.* + +/** + * Starts new coroutine and returns its result as an implementation of [Promise]. + * + * Coroutine context is inherited from a [CoroutineScope], additional context elements can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * The parent job is inherited from a [CoroutineScope] as well, but it can also be overridden + * with corresponding [context] element. + * + * By default, the coroutine is immediately scheduled for execution. + * Other options can be specified via `start` parameter. See [CoroutineStart] for details. + * + * @param context additional to [CoroutineScope.coroutineContext] context of the coroutine. + * @param start coroutine start option. The default value is [CoroutineStart.DEFAULT]. + * @param block the coroutine code. + */ +public fun CoroutineScope.promise( + context: CoroutineContext = EmptyCoroutineContext, + start: CoroutineStart = CoroutineStart.DEFAULT, + block: suspend CoroutineScope.() -> T +): Promise = + async(context, start, block).asPromise() + +/** + * Converts this deferred value to the instance of [Promise]. + */ +public fun Deferred.asPromise(): Promise { + val promise = Promise { resolve, reject -> + invokeOnCompletion { + val e = getCompletionExceptionOrNull() + if (e != null) { + reject(e) + } else { + resolve(getCompleted()) + } + } + } + promise.asDynamic().deferred = this + return promise +} + +/** + * Converts this promise value to the instance of [Deferred]. + */ +public fun Promise.asDeferred(): Deferred { + val deferred = asDynamic().deferred + @Suppress("UnsafeCastFromDynamic") + return deferred ?: GlobalScope.async(start = CoroutineStart.UNDISPATCHED) { await() } +} + +/** + * Awaits for completion of the promise without blocking. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting on the promise, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + */ +public suspend fun Promise.await(): T = suspendCancellableCoroutine { cont: CancellableContinuation -> + this@await.then( + onFulfilled = { cont.resume(it) }, + onRejected = { cont.resumeWithException(it as? Throwable ?: Exception("Non-Kotlin exception $it")) }) +} diff --git a/kotlinx-coroutines-core/js/src/Window.kt b/kotlinx-coroutines-core/js/src/Window.kt new file mode 100644 index 0000000000..7fea6f3092 --- /dev/null +++ b/kotlinx-coroutines-core/js/src/Window.kt @@ -0,0 +1,58 @@ +package kotlinx.coroutines + +import org.w3c.dom.Window + +/** + * Converts an instance of [Window] to an implementation of [CoroutineDispatcher]. + */ +public fun Window.asCoroutineDispatcher(): CoroutineDispatcher = + @Suppress("UnsafeCastFromDynamic") + asDynamic().coroutineDispatcher ?: WindowDispatcher(this).also { + asDynamic().coroutineDispatcher = it + } + +/** + * Suspends coroutine until next JS animation frame and returns frame time on resumption. + * The time is consistent with [window.performance.now()][org.w3c.performance.Performance.now]. + * This function is cancellable. If the [Job] of the current coroutine is completed while this suspending + * function is waiting, this function immediately resumes with [CancellationException]. + */ +public suspend fun Window.awaitAnimationFrame(): Double = suspendCancellableCoroutine { cont -> + asWindowAnimationQueue().enqueue(cont) +} + +private fun Window.asWindowAnimationQueue(): WindowAnimationQueue = + @Suppress("UnsafeCastFromDynamic") + asDynamic().coroutineAnimationQueue ?: WindowAnimationQueue(this).also { + asDynamic().coroutineAnimationQueue = it + } + +private class WindowAnimationQueue(private val window: Window) { + private val dispatcher = window.asCoroutineDispatcher() + private var scheduled = false + private var current = ArrayDeque>() + private var next = ArrayDeque>() + private var timestamp = 0.0 + + fun enqueue(cont: CancellableContinuation) { + next.addLast(cont) + if (!scheduled) { + scheduled = true + window.requestAnimationFrame { ts -> + timestamp = ts + val prev = current + current = next + next = prev + scheduled = false + process() + } + } + } + + fun process() { + while(true) { + val element = current.removeFirstOrNull() ?: return + with(element) { dispatcher.resumeUndispatched(timestamp) } + } + } +} diff --git a/kotlinx-coroutines-core/js/src/internal/CopyOnWriteList.kt b/kotlinx-coroutines-core/js/src/internal/CopyOnWriteList.kt new file mode 100644 index 0000000000..26ad530a9e --- /dev/null +++ b/kotlinx-coroutines-core/js/src/internal/CopyOnWriteList.kt @@ -0,0 +1,94 @@ +package kotlinx.coroutines.internal + +/** + * Analogue of java.util.concurrent.CopyOnWriteArrayList for JS. + * Even though JS has no real concurrency, [CopyOnWriteList] is essential to manage any kinds + * of callbacks or continuations. + * + * Implementation note: most of the methods fallbacks to [AbstractMutableList] (thus inefficient for CoW pattern) + * and some methods are unsupported, because currently they are not required for this class consumers. + */ +internal class CopyOnWriteList(private var array: Array = emptyArray()) : AbstractMutableList() { + + override val size: Int get() = array.size + + override fun add(element: E): Boolean { + val copy = array.asDynamic().slice() + copy.push(element) + array = copy as Array + return true + } + + override fun add(index: Int, element: E) { + val copy = array.asDynamic().slice() + copy.splice(insertionRangeCheck(index), 0, element) + array = copy as Array + } + + override fun remove(element: E): Boolean { + for (index in array.indices) { + if (array[index] == element) { + val copy = array.asDynamic().slice() + copy.splice(index, 1) + array = copy as Array + return true + } + } + + return false + } + + override fun removeAt(index: Int): E { + rangeCheck(index) + val copy = array.asDynamic().slice() + val result = if (index == lastIndex) { + copy.pop() + } else { + copy.splice(index, 1)[0] + } + + array = copy as Array + return result as E + } + + override fun iterator(): MutableIterator = IteratorImpl(array) + + override fun listIterator(): MutableListIterator = throw UnsupportedOperationException("Operation is not supported") + + override fun listIterator(index: Int): MutableListIterator = throw UnsupportedOperationException("Operation is not supported") + + override fun isEmpty(): Boolean = size == 0 + + override fun set(index: Int, element: E): E = throw UnsupportedOperationException("Operation is not supported") + + override fun get(index: Int): E = array[rangeCheck(index)] + + private class IteratorImpl(private var array: Array) : MutableIterator { + + private var current = 0 + + override fun hasNext(): Boolean = current != array.size + + override fun next(): E { + if (!hasNext()) { + throw NoSuchElementException() + } + + return array[current++] + } + + override fun remove() = throw UnsupportedOperationException("Operation is not supported") + } + + private fun insertionRangeCheck(index: Int) { + if (index < 0 || index > size) { + throw IndexOutOfBoundsException("index: $index, size: $size") + } + } + + private fun rangeCheck(index: Int) = index.apply { + if (index < 0 || index >= size) { + throw IndexOutOfBoundsException("index: $index, size: $size") + } + } +} diff --git a/kotlinx-coroutines-core/js/src/internal/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/js/src/internal/CoroutineExceptionHandlerImpl.kt new file mode 100644 index 0000000000..b3c09e7c38 --- /dev/null +++ b/kotlinx-coroutines-core/js/src/internal/CoroutineExceptionHandlerImpl.kt @@ -0,0 +1,8 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* + +internal actual fun propagateExceptionFinalResort(exception: Throwable) { + // log exception + console.error(exception.toString()) +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/js/test/PromiseTest.kt b/kotlinx-coroutines-core/js/test/PromiseTest.kt new file mode 100644 index 0000000000..f9cb0ed56d --- /dev/null +++ b/kotlinx-coroutines-core/js/test/PromiseTest.kt @@ -0,0 +1,109 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.js.* +import kotlin.test.* + +class PromiseTest : TestBase() { + @Test + fun testPromiseResolvedAsDeferred() = GlobalScope.promise { + val promise = Promise { resolve, _ -> + resolve("OK") + } + val deferred = promise.asDeferred() + assertEquals("OK", deferred.await()) + } + + @Test + fun testPromiseRejectedAsDeferred() = GlobalScope.promise { + lateinit var promiseReject: (Throwable) -> Unit + val promise = Promise { _, reject -> + promiseReject = reject + } + val deferred = promise.asDeferred() + // reject after converting to deferred to avoid "Unhandled promise rejection" warnings + promiseReject(TestException("Rejected")) + try { + deferred.await() + expectUnreached() + } catch (e: Throwable) { + assertIs(e) + assertEquals("Rejected", e.message) + } + } + + @Test + fun testCompletedDeferredAsPromise() = GlobalScope.promise { + val deferred = async(start = CoroutineStart.UNDISPATCHED) { + // completed right away + "OK" + } + val promise = deferred.asPromise() + assertEquals("OK", promise.await()) + } + + @Test + fun testWaitForDeferredAsPromise() = GlobalScope.promise { + val deferred = async { + // will complete later + "OK" + } + val promise = deferred.asPromise() + assertEquals("OK", promise.await()) // await yields main thread to deferred coroutine + } + + @Test + fun testCancellableAwaitPromise() = GlobalScope.promise { + lateinit var r: (String) -> Unit + val toAwait = Promise { resolve, _ -> r = resolve } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + toAwait.await() // suspends + } + job.cancel() // cancel the job + r("fail") // too late, the waiting job was already cancelled + } + + @Test + fun testAsPromiseAsDeferred() = GlobalScope.promise { + val deferred = async { "OK" } + val promise = deferred.asPromise() + val d2 = promise.asDeferred() + assertSame(d2, deferred) + assertEquals("OK", d2.await()) + } + + @Test + fun testLeverageTestResult(): TestResult { + // Cannot use expect(..) here + var seq = 0 + val result = runTest { + ++seq + } + return result.then { + if (seq != 1) error("Unexpected result: $seq") + } + } + + @Test + fun testAwaitPromiseRejectedWithNonKotlinException() = GlobalScope.promise { + lateinit var r: (dynamic) -> Unit + val toAwait = Promise { _, reject -> r = reject } + val throwable = async(start = CoroutineStart.UNDISPATCHED) { + assertFails { toAwait.await() } + } + r("Rejected") + assertContains(throwable.await().message ?: "", "Rejected") + } + + @Test + fun testAwaitPromiseRejectedWithKotlinException() = GlobalScope.promise { + lateinit var r: (dynamic) -> Unit + val toAwait = Promise { _, reject -> r = reject } + val throwable = async(start = CoroutineStart.UNDISPATCHED) { + assertFails { toAwait.await() } + } + r(RuntimeException("Rejected")) + assertIs(throwable.await()) + assertEquals("Rejected", throwable.await().message) + } +} diff --git a/kotlinx-coroutines-core/jsAndWasmJsShared/src/EventLoop.kt b/kotlinx-coroutines-core/jsAndWasmJsShared/src/EventLoop.kt new file mode 100644 index 0000000000..90549eecf4 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmJsShared/src/EventLoop.kt @@ -0,0 +1,25 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* + +internal actual fun createEventLoop(): EventLoop = UnconfinedEventLoop() + +internal actual fun nanoTime(): Long = unsupported() + +internal class UnconfinedEventLoop : EventLoop() { + override fun dispatch(context: CoroutineContext, block: Runnable): Unit = unsupported() +} + +internal actual abstract class EventLoopImplPlatform : EventLoop() { + protected actual fun unpark(): Unit = unsupported() + protected actual fun reschedule(now: Long, delayedTask: EventLoopImplBase.DelayedTask): Unit = unsupported() +} + +internal actual object DefaultExecutor { + public actual fun enqueue(task: Runnable): Unit = unsupported() +} + +private fun unsupported(): Nothing = + throw UnsupportedOperationException("runBlocking event loop is not supported") + +internal actual inline fun platformAutoreleasePool(crossinline block: () -> Unit) = block() diff --git a/kotlinx-coroutines-core/jsAndWasmJsShared/src/internal/JSDispatcher.kt b/kotlinx-coroutines-core/jsAndWasmJsShared/src/internal/JSDispatcher.kt new file mode 100644 index 0000000000..c98a0672c9 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmJsShared/src/internal/JSDispatcher.kt @@ -0,0 +1,137 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +internal expect abstract class W3CWindow +internal expect fun w3cSetTimeout(window: W3CWindow, handler: () -> Unit, timeout: Int): Int +internal expect fun w3cSetTimeout(handler: () -> Unit, timeout: Int): Int +internal expect fun w3cClearTimeout(handle: Int) +internal expect fun w3cClearTimeout(window: W3CWindow, handle: Int) + +internal expect class ScheduledMessageQueue(dispatcher: SetTimeoutBasedDispatcher) : MessageQueue { + override fun schedule() + override fun reschedule() + internal fun setTimeout(timeout: Int) +} + +internal expect class WindowMessageQueue(window: W3CWindow) : MessageQueue { + override fun schedule() + override fun reschedule() +} + +private const val MAX_DELAY = Int.MAX_VALUE.toLong() + +private fun delayToInt(timeMillis: Long): Int = + timeMillis.coerceIn(0, MAX_DELAY).toInt() + +internal abstract class SetTimeoutBasedDispatcher: CoroutineDispatcher(), Delay { + internal val messageQueue = ScheduledMessageQueue(this) + + abstract fun scheduleQueueProcessing() + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + parallelism.checkParallelism() + return namedOrThis(name) + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + messageQueue.enqueue(block) + } + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { + val handle = w3cSetTimeout({ block.run() }, delayToInt(timeMillis)) + return ClearTimeout(handle) + } + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val handle = w3cSetTimeout({ with(continuation) { resumeUndispatched(Unit) } }, delayToInt(timeMillis)) + continuation.invokeOnCancellation(handler = ClearTimeout(handle)) + } +} + +internal class WindowDispatcher(private val window: W3CWindow) : CoroutineDispatcher(), Delay { + private val queue = WindowMessageQueue(window) + + override fun dispatch(context: CoroutineContext, block: Runnable) = queue.enqueue(block) + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val handle = w3cSetTimeout(window, { with(continuation) { resumeUndispatched(Unit) } }, delayToInt(timeMillis)) + continuation.invokeOnCancellation(handler = WindowClearTimeout(handle)) + } + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { + val handle = w3cSetTimeout(window, block::run, delayToInt(timeMillis)) + return WindowClearTimeout(handle) + } + + private inner class WindowClearTimeout(handle: Int) : ClearTimeout(handle) { + override fun dispose() { + w3cClearTimeout(window, handle) + } + } +} + +internal object SetTimeoutDispatcher : SetTimeoutBasedDispatcher() { + override fun scheduleQueueProcessing() { + messageQueue.setTimeout(0) + } +} + +private open class ClearTimeout(protected val handle: Int) : CancelHandler, DisposableHandle { + override fun dispose() { + w3cClearTimeout(handle) + } + + override fun invoke(cause: Throwable?) { + dispose() + } + + override fun toString(): String = "ClearTimeout[$handle]" +} + + +/** + * An abstraction over JS scheduling mechanism that leverages micro-batching of dispatched blocks without + * paying the cost of JS callbacks scheduling on every dispatch. + * + * Queue uses two scheduling mechanisms: + * 1) [schedule] is used to schedule the initial processing of the message queue. + * JS engine-specific microtask mechanism is used in order to boost performance on short runs and a dispatch batch + * 2) [reschedule] is used to schedule processing of the queue after yield to the JS event loop. + * JS engine-specific macrotask mechanism is used not to starve animations and non-coroutines macrotasks. + * + * Yet there could be a long tail of "slow" reschedules, but it should be amortized by the queue size. + */ +internal abstract class MessageQueue : MutableList by ArrayDeque() { + val yieldEvery = 16 // yield to JS macrotask event loop after this many processed messages + private var scheduled = false + + abstract fun schedule() + + abstract fun reschedule() + + fun enqueue(element: Runnable) { + add(element) + if (!scheduled) { + scheduled = true + schedule() + } + } + + fun process() { + try { + // limit number of processed messages + repeat(yieldEvery) { + val element = removeFirstOrNull() ?: return@process + element.run() + } + } finally { + if (isEmpty()) { + scheduled = false + } else { + reschedule() + } + } + } +} diff --git a/kotlinx-coroutines-core/jsAndWasmJsShared/test/MessageQueueTest.kt b/kotlinx-coroutines-core/jsAndWasmJsShared/test/MessageQueueTest.kt new file mode 100644 index 0000000000..5bf4d2db55 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmJsShared/test/MessageQueueTest.kt @@ -0,0 +1,83 @@ +package kotlinx.coroutines + +import kotlin.test.* + +class MessageQueueTest { + private var scheduled = false + private val processed = mutableListOf() + + private val queue = object : MessageQueue() { + override fun schedule() { + assertFalse(scheduled) + scheduled = true + } + + override fun reschedule() { + schedule() + } + } + + inner class Box(val i: Int): Runnable { + override fun run() { + processed += i + } + } + + inner class ReBox(val i: Int): Runnable { + override fun run() { + processed += i + queue.enqueue(Box(10 + i)) + } + } + + @Test + fun testBasic() { + assertTrue(queue.isEmpty()) + queue.enqueue(Box(1)) + assertFalse(queue.isEmpty()) + assertTrue(scheduled) + queue.enqueue(Box(2)) + assertFalse(queue.isEmpty()) + scheduled = false + queue.process() + assertEquals(listOf(1, 2), processed) + assertFalse(scheduled) + assertTrue(queue.isEmpty()) + } + + @Test fun testRescheduleFromProcess() { + assertTrue(queue.isEmpty()) + queue.enqueue(ReBox(1)) + assertFalse(queue.isEmpty()) + assertTrue(scheduled) + queue.enqueue(ReBox(2)) + assertFalse(queue.isEmpty()) + scheduled = false + queue.process() + assertEquals(listOf(1, 2, 11, 12), processed) + assertFalse(scheduled) + assertTrue(queue.isEmpty()) + } + + @Test + fun testResizeAndWrap() { + repeat(10) { phase -> + val n = 10 * (phase + 1) + assertTrue(queue.isEmpty()) + repeat(n) { + queue.enqueue(Box(it)) + assertFalse(queue.isEmpty()) + assertTrue(scheduled) + } + var countYields = 0 + while (scheduled) { + scheduled = false + queue.process() + countYields++ + } + assertEquals(List(n) { it }, processed) + assertEquals((n + queue.yieldEvery - 1) / queue.yieldEvery, countYields) + processed.clear() + } + } +} diff --git a/kotlinx-coroutines-core/jsAndWasmJsShared/test/SetTimeoutDispatcherTest.kt b/kotlinx-coroutines-core/jsAndWasmJsShared/test/SetTimeoutDispatcherTest.kt new file mode 100644 index 0000000000..3b4e422e0e --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmJsShared/test/SetTimeoutDispatcherTest.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class SetTimeoutDispatcherTest : TestBase() { + @Test + fun testDispatch() = runTest { + launch(SetTimeoutDispatcher) { + expect(1) + launch { + expect(3) + } + expect(2) + yield() + expect(4) + }.join() + finish(5) + } + + @Test + fun testDelay() = runTest { + withContext(SetTimeoutDispatcher) { + val job = launch(SetTimeoutDispatcher) { + expect(2) + delay(100) + expect(4) + } + expect(1) + yield() // Yield uses microtask, so should be in the same context + expect(3) + job.join() + finish(5) + } + } + + @Test + fun testWithTimeout() = runTest { + withContext(SetTimeoutDispatcher) { + val result = withTimeoutOrNull(10) { + expect(1) + delay(100) + expectUnreached() + 42 + } + assertNull(result) + finish(2) + } + } +} diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/CloseableCoroutineDispatcher.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/CloseableCoroutineDispatcher.kt new file mode 100644 index 0000000000..3ea73ad7a1 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/CloseableCoroutineDispatcher.kt @@ -0,0 +1,5 @@ +package kotlinx.coroutines + +public actual abstract class CloseableCoroutineDispatcher actual constructor() : CoroutineDispatcher(), AutoCloseable { + public actual abstract override fun close() +} diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/CoroutineContext.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/CoroutineContext.kt new file mode 100644 index 0000000000..82862ac8aa --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/CoroutineContext.kt @@ -0,0 +1,31 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.ScopeCoroutine +import kotlin.coroutines.* + +@PublishedApi // Used from kotlinx-coroutines-test via suppress, not part of ABI +internal actual val DefaultDelay: Delay + get() = Dispatchers.Default as Delay + +public actual fun CoroutineScope.newCoroutineContext(context: CoroutineContext): CoroutineContext { + val combined = coroutineContext + context + return if (combined !== Dispatchers.Default && combined[ContinuationInterceptor] == null) + combined + Dispatchers.Default else combined +} + +public actual fun CoroutineContext.newCoroutineContext(addedContext: CoroutineContext): CoroutineContext { + return this + addedContext +} + +// No debugging facilities on Wasm and JS +internal actual inline fun withCoroutineContext(context: CoroutineContext, countOrElement: Any?, block: () -> T): T = block() +internal actual inline fun withContinuationContext(continuation: Continuation<*>, countOrElement: Any?, block: () -> T): T = block() +internal actual fun Continuation<*>.toDebugString(): String = toString() +internal actual val CoroutineContext.coroutineName: String? get() = null // not supported on Wasm and JS + +internal actual class UndispatchedCoroutine actual constructor( + context: CoroutineContext, + uCont: Continuation +) : ScopeCoroutine(context, uCont) { + override fun afterResume(state: Any?) = uCont.resumeWith(recoverResult(state, uCont)) +} diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/Dispatchers.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/Dispatchers.kt new file mode 100644 index 0000000000..ca74fbae76 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/Dispatchers.kt @@ -0,0 +1,32 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* + +internal expect fun createDefaultDispatcher(): CoroutineDispatcher + +public actual object Dispatchers { + public actual val Default: CoroutineDispatcher = createDefaultDispatcher() + public actual val Main: MainCoroutineDispatcher + get() = injectedMainDispatcher ?: mainDispatcher + public actual val Unconfined: CoroutineDispatcher = kotlinx.coroutines.Unconfined + + private val mainDispatcher = JsMainDispatcher(Default, false) + private var injectedMainDispatcher: MainCoroutineDispatcher? = null + + @PublishedApi + internal fun injectMain(dispatcher: MainCoroutineDispatcher) { + injectedMainDispatcher = dispatcher + } +} + +private class JsMainDispatcher( + val delegate: CoroutineDispatcher, + private val invokeImmediately: Boolean +) : MainCoroutineDispatcher() { + override val immediate: MainCoroutineDispatcher = + if (invokeImmediately) this else JsMainDispatcher(delegate, true) + override fun isDispatchNeeded(context: CoroutineContext): Boolean = !invokeImmediately + override fun dispatch(context: CoroutineContext, block: Runnable) = delegate.dispatch(context, block) + override fun dispatchYield(context: CoroutineContext, block: Runnable) = delegate.dispatchYield(context, block) + override fun toString(): String = toStringInternalImpl() ?: delegate.toString() +} diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/Exceptions.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/Exceptions.kt new file mode 100644 index 0000000000..c6173afc26 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/Exceptions.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines + +/** + * Thrown by cancellable suspending functions if the [Job] of the coroutine is cancelled while it is suspending. + * It indicates _normal_ cancellation of a coroutine. + * **It is not printed to console/log by default uncaught exception handler**. + * (see [CoroutineExceptionHandler]). + */ +public actual typealias CancellationException = kotlin.coroutines.cancellation.CancellationException + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +@kotlin.internal.LowPriorityInOverloadResolution +public actual fun CancellationException(message: String?, cause: Throwable?): CancellationException = + CancellationException(message, cause) + +/** + * Thrown by cancellable suspending functions if the [Job] of the coroutine is cancelled or completed + * without cause, or with a cause or exception that is not [CancellationException] + * (see [Job.getCancellationException]). + */ +internal actual class JobCancellationException public actual constructor( + message: String, + cause: Throwable?, + internal actual val job: Job +) : CancellationException(message, cause) { + override fun toString(): String = "${super.toString()}; job=$job" + override fun equals(other: Any?): Boolean = + other === this || + other is JobCancellationException && other.message == message && other.job == job && other.cause == cause + override fun hashCode(): Int = + (message!!.hashCode() * 31 + job.hashCode()) * 31 + (cause?.hashCode() ?: 0) +} + +// For use in tests +internal actual val RECOVER_STACK_TRACES: Boolean = false diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/Runnable.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/Runnable.kt new file mode 100644 index 0000000000..d93e3f2073 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/Runnable.kt @@ -0,0 +1,24 @@ +package kotlinx.coroutines + +/** + * A runnable task for [CoroutineDispatcher.dispatch]. + * + * Equivalent to the type `() -> Unit`. + */ +public actual fun interface Runnable { + /** + * @suppress + */ + public actual fun run() +} + +@Deprecated( + "Preserved for binary compatibility, see https://github.com/Kotlin/kotlinx.coroutines/issues/4309", + level = DeprecationLevel.HIDDEN +) +public inline fun Runnable(crossinline block: () -> Unit): Runnable = + object : Runnable { + override fun run() { + block() + } + } diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/SchedulerTask.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/SchedulerTask.kt new file mode 100644 index 0000000000..24b2311268 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/SchedulerTask.kt @@ -0,0 +1,3 @@ +package kotlinx.coroutines + +internal actual abstract class SchedulerTask : Runnable diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/flow/internal/FlowExceptions.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/flow/internal/FlowExceptions.kt new file mode 100644 index 0000000000..0e780f53a0 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/flow/internal/FlowExceptions.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +internal actual class AbortFlowException actual constructor( + actual val owner: Any +) : CancellationException("Flow was aborted, no more elements needed") +internal actual class ChildCancelledException : CancellationException("Child of the scoped flow was cancelled") diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/flow/internal/SafeCollector.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/flow/internal/SafeCollector.kt new file mode 100644 index 0000000000..ded11e22f5 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/flow/internal/SafeCollector.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.coroutines.* + +internal actual class SafeCollector actual constructor( + internal actual val collector: FlowCollector, + internal actual val collectContext: CoroutineContext +) : FlowCollector { + + // Note, it is non-capturing lambda, so no extra allocation during init of SafeCollector + internal actual val collectContextSize = collectContext.fold(0) { count, _ -> count + 1 } + private var lastEmissionContext: CoroutineContext? = null + + actual override suspend fun emit(value: T) { + val currentContext = currentCoroutineContext() + currentContext.ensureActive() + if (lastEmissionContext !== currentContext) { + checkContext(currentContext) + lastEmissionContext = currentContext + } + collector.emit(value) + } + + public actual fun releaseIntercepted() { + } +} diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/Concurrent.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/Concurrent.kt new file mode 100644 index 0000000000..74e54bc477 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/Concurrent.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.internal + +internal actual typealias ReentrantLock = NoOpLock + +internal actual inline fun ReentrantLock.withLock(action: () -> T) = action() + +internal class NoOpLock { + fun tryLock() = true + fun unlock(): Unit {} +} + +internal actual fun identitySet(expectedSize: Int): MutableSet = HashSet(expectedSize) + +internal actual class WorkaroundAtomicReference actual constructor(private var value: V) { + + public actual fun get(): V = value + + public actual fun set(value: V) { + this.value = value + } + + public actual fun getAndSet(value: V): V { + val prev = this.value + this.value = value + return prev + } + + public actual fun compareAndSet(expected: V, value: V): Boolean { + if (this.value === expected) { + this.value = value + return true + } + return false + } +} diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/CoroutineExceptionHandlerImpl.kt new file mode 100644 index 0000000000..d664ce6406 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/CoroutineExceptionHandlerImpl.kt @@ -0,0 +1,17 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +private val platformExceptionHandlers_ = mutableSetOf() + +internal actual val platformExceptionHandlers: Collection + get() = platformExceptionHandlers_ + +internal actual fun ensurePlatformExceptionHandlerLoaded(callback: CoroutineExceptionHandler) { + platformExceptionHandlers_ += callback +} + +internal actual class DiagnosticCoroutineContextException actual constructor(context: CoroutineContext) : + RuntimeException(context.toString()) + diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/LinkedList.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/LinkedList.kt new file mode 100644 index 0000000000..6810d614d1 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/LinkedList.kt @@ -0,0 +1,73 @@ +@file:Suppress("unused", "NO_EXPLICIT_RETURN_TYPE_IN_API_MODE", "NO_EXPLICIT_VISIBILITY_IN_API_MODE") + +package kotlinx.coroutines.internal + +private typealias Node = LockFreeLinkedListNode + +/** @suppress **This is unstable API and it is subject to change.** */ +public actual open class LockFreeLinkedListNode { + @PublishedApi internal var _next = this + @PublishedApi internal var _prev = this + @PublishedApi internal var _removed: Boolean = false + + public actual inline val nextNode get() = _next + inline actual val prevNode get() = _prev + inline actual val isRemoved get() = _removed + + public actual fun addLast(node: Node, permissionsBitmask: Int): Boolean = when (val prev = this._prev) { + is ListClosed -> + prev.forbiddenElementsBitmask and permissionsBitmask == 0 && prev.addLast(node, permissionsBitmask) + else -> { + node._next = this + node._prev = prev + prev._next = node + this._prev = node + true + } + } + + public actual fun close(forbiddenElementsBit: Int) { + addLast(ListClosed(forbiddenElementsBit), forbiddenElementsBit) + } + + /* + * Remove that is invoked as a virtual function with a + * potentially augmented behaviour. + * I.g. `LockFreeLinkedListHead` throws, while `SendElementWithUndeliveredHandler` + * invokes handler on remove + */ + public actual open fun remove(): Boolean { + if (_removed) return false + val prev = this._prev + val next = this._next + prev._next = next + next._prev = prev + _removed = true + return true + } + + public actual fun addOneIfEmpty(node: Node): Boolean { + if (_next !== this) return false + addLast(node, Int.MIN_VALUE) + return true + } +} + +/** @suppress **This is unstable API and it is subject to change.** */ +public actual open class LockFreeLinkedListHead : Node() { + /** + * Iterates over all elements in this list of a specified type. + */ + public actual inline fun forEach(block: (Node) -> Unit) { + var cur: Node = _next + while (cur != this) { + block(cur) + cur = cur._next + } + } + + // just a defensive programming -- makes sure that list head sentinel is never removed + public actual final override fun remove(): Nothing = throw UnsupportedOperationException() +} + +private class ListClosed(val forbiddenElementsBitmask: Int): LockFreeLinkedListNode() diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/LocalAtomics.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/LocalAtomics.kt new file mode 100644 index 0000000000..e0e1bee56b --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/LocalAtomics.kt @@ -0,0 +1,11 @@ +package kotlinx.coroutines.internal + +internal actual class LocalAtomicInt actual constructor(private var value: Int) { + actual fun set(value: Int) { + this.value = value + } + + actual fun get(): Int = value + + actual fun decrementAndGet(): Int = --value +} diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/ProbesSupport.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/ProbesSupport.kt new file mode 100644 index 0000000000..7afce8f8a4 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/ProbesSupport.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +@Suppress("NOTHING_TO_INLINE") +internal actual inline fun probeCoroutineCreated(completion: Continuation): Continuation = completion + +@Suppress("NOTHING_TO_INLINE") +internal actual inline fun probeCoroutineResumed(completion: Continuation) { } diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/StackTraceRecovery.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/StackTraceRecovery.kt new file mode 100644 index 0000000000..7003a24fa2 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/StackTraceRecovery.kt @@ -0,0 +1,22 @@ +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +internal actual fun recoverStackTrace(exception: E, continuation: Continuation<*>): E = exception +internal actual fun recoverStackTrace(exception: E): E = exception +internal actual suspend inline fun recoverAndThrow(exception: Throwable): Nothing = throw exception + +@PublishedApi +internal actual fun unwrap(exception: E): E = exception + +@Suppress("UNUSED") +internal actual interface CoroutineStackFrame { + public actual val callerFrame: CoroutineStackFrame? + public actual fun getStackTraceElement(): StackTraceElement? +} + +@Suppress("ACTUAL_WITHOUT_EXPECT") +internal actual typealias StackTraceElement = Any + +internal actual fun Throwable.initCause(cause: Throwable) { +} diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/Synchronized.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/Synchronized.kt new file mode 100644 index 0000000000..e828f67588 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/Synchronized.kt @@ -0,0 +1,15 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public actual open class SynchronizedObject + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public actual inline fun synchronizedImpl(lock: SynchronizedObject, block: () -> T): T = block() diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/SystemProps.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/SystemProps.kt new file mode 100644 index 0000000000..accc247d37 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/SystemProps.kt @@ -0,0 +1,3 @@ +package kotlinx.coroutines.internal + +internal actual fun systemProp(propertyName: String): String? = null diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/ThreadContext.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/ThreadContext.kt new file mode 100644 index 0000000000..3f56f99d6c --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/ThreadContext.kt @@ -0,0 +1,5 @@ +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +internal actual fun threadContextElements(context: CoroutineContext): Any = 0 diff --git a/kotlinx-coroutines-core/jsAndWasmShared/src/internal/ThreadLocal.kt b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/ThreadLocal.kt new file mode 100644 index 0000000000..94eecfa0ee --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/src/internal/ThreadLocal.kt @@ -0,0 +1,10 @@ +package kotlinx.coroutines.internal + +internal actual class CommonThreadLocal { + private var value: T? = null + @Suppress("UNCHECKED_CAST") + actual fun get(): T = value as T + actual fun set(value: T) { this.value = value } +} + +internal actual fun commonThreadLocal(name: Symbol): CommonThreadLocal = CommonThreadLocal() \ No newline at end of file diff --git a/kotlinx-coroutines-core/jsAndWasmShared/test/ImmediateDispatcherTest.kt b/kotlinx-coroutines-core/jsAndWasmShared/test/ImmediateDispatcherTest.kt new file mode 100644 index 0000000000..936713b2b3 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/test/ImmediateDispatcherTest.kt @@ -0,0 +1,39 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +class ImmediateDispatcherTest : MainDispatcherTestBase.WithRealTimeDelay() { + + /** Tests that [MainCoroutineDispatcher.immediate] doesn't require dispatches from the test context. */ + @Test + fun testImmediate() = runTest { + expect(1) + val job = launch { expect(3) } + assertFalse(Dispatchers.Main.immediate.isDispatchNeeded(currentCoroutineContext())) + withContext(Dispatchers.Main.immediate) { + expect(2) + } + job.join() + finish(4) + } + + @Test + fun testMain() = runTest { + expect(1) + val job = launch { expect(2) } + withContext(Dispatchers.Main) { + expect(3) + } + job.join() + finish(4) + } + + override fun isMainThread(): Boolean? = null + + override fun scheduleOnMainQueue(block: () -> Unit) { + Dispatchers.Default.dispatch(EmptyCoroutineContext, Runnable { block() }) + } +} diff --git a/kotlinx-coroutines-core/jsAndWasmShared/test/internal/LinkedListTest.kt b/kotlinx-coroutines-core/jsAndWasmShared/test/internal/LinkedListTest.kt new file mode 100644 index 0000000000..305484f741 --- /dev/null +++ b/kotlinx-coroutines-core/jsAndWasmShared/test/internal/LinkedListTest.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines.internal + +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFalse +import kotlin.test.assertTrue + +class LinkedListTest { + data class IntNode(val i: Int) : LockFreeLinkedListNode() + + @Test + fun testSimpleAddLastRemove() { + val list = LockFreeLinkedListHead() + assertContents(list) + val n1 = IntNode(1).apply { list.addLast(this, Int.MAX_VALUE) } + assertContents(list, 1) + val n2 = IntNode(2).apply { list.addLast(this, Int.MAX_VALUE) } + assertContents(list, 1, 2) + val n3 = IntNode(3).apply { list.addLast(this, Int.MAX_VALUE) } + assertContents(list, 1, 2, 3) + val n4 = IntNode(4).apply { list.addLast(this, Int.MAX_VALUE) } + assertContents(list, 1, 2, 3, 4) + assertTrue(n1.remove()) + assertContents(list, 2, 3, 4) + assertTrue(n3.remove()) + assertContents(list, 2, 4) + assertTrue(n4.remove()) + assertContents(list, 2) + assertTrue(n2.remove()) + assertFalse(n2.remove()) + assertContents(list) + } + + private fun assertContents(list: LockFreeLinkedListHead, vararg expected: Int) { + val n = expected.size + val actual = IntArray(n) + var index = 0 + list.forEach { if (it is IntNode) actual[index++] = it.i } + assertEquals(n, index) + for (i in 0 until n) assertEquals(expected[i], actual[i], "item i") + } +} diff --git a/kotlinx-coroutines-core/jvm/resources/DebugProbesKt.bin b/kotlinx-coroutines-core/jvm/resources/DebugProbesKt.bin new file mode 100644 index 0000000000..cac12595e5 Binary files /dev/null and b/kotlinx-coroutines-core/jvm/resources/DebugProbesKt.bin differ diff --git a/kotlinx-coroutines-core/jvm/resources/META-INF/com.android.tools/proguard/coroutines.pro b/kotlinx-coroutines-core/jvm/resources/META-INF/com.android.tools/proguard/coroutines.pro new file mode 100644 index 0000000000..1380396073 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/resources/META-INF/com.android.tools/proguard/coroutines.pro @@ -0,0 +1,31 @@ +# When editing this file, update the following files as well: +# - META-INF/proguard/coroutines.pro +# - META-INF/com.android.tools/r8/coroutines.pro + +# ServiceLoader support +-keepnames class kotlinx.coroutines.internal.MainDispatcherFactory {} +-keepnames class kotlinx.coroutines.CoroutineExceptionHandler {} + +# Most of volatile fields are updated with AFU and should not be mangled +-keepclassmembers class kotlinx.coroutines.** { + volatile ; +} + +# Same story for the standard library's SafeContinuation that also uses AtomicReferenceFieldUpdater +-keepclassmembers class kotlin.coroutines.SafeContinuation { + volatile ; +} + +# These classes are only required by kotlinx.coroutines.debug.internal.AgentPremain, which is only loaded when +# kotlinx-coroutines-core is used as a Java agent, so these are not needed in contexts where ProGuard is used. +-dontwarn java.lang.instrument.ClassFileTransformer +-dontwarn sun.misc.SignalHandler +-dontwarn java.lang.instrument.Instrumentation +-dontwarn sun.misc.Signal + +# Only used in `kotlinx.coroutines.internal.ExceptionsConstructor`. +# The case when it is not available is hidden in a `try`-`catch`, as well as a check for Android. +-dontwarn java.lang.ClassValue + +# An annotation used for build tooling, won't be directly accessed. +-dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement diff --git a/kotlinx-coroutines-core/jvm/resources/META-INF/com.android.tools/r8/coroutines.pro b/kotlinx-coroutines-core/jvm/resources/META-INF/com.android.tools/r8/coroutines.pro new file mode 100644 index 0000000000..69a28956ac --- /dev/null +++ b/kotlinx-coroutines-core/jvm/resources/META-INF/com.android.tools/r8/coroutines.pro @@ -0,0 +1,27 @@ +# When editing this file, update the following files as well: +# - META-INF/proguard/coroutines.pro +# - META-INF/com.android.tools/proguard/coroutines.pro + +# Most of volatile fields are updated with AFU and should not be mangled +-keepclassmembers class kotlinx.coroutines.** { + volatile ; +} + +# Same story for the standard library's SafeContinuation that also uses AtomicReferenceFieldUpdater +-keepclassmembers class kotlin.coroutines.SafeContinuation { + volatile ; +} + +# These classes are only required by kotlinx.coroutines.debug.internal.AgentPremain, which is only loaded when +# kotlinx-coroutines-core is used as a Java agent, so these are not needed in contexts where ProGuard is used. +-dontwarn java.lang.instrument.ClassFileTransformer +-dontwarn sun.misc.SignalHandler +-dontwarn java.lang.instrument.Instrumentation +-dontwarn sun.misc.Signal + +# Only used in `kotlinx.coroutines.internal.ExceptionsConstructor`. +# The case when it is not available is hidden in a `try`-`catch`, as well as a check for Android. +-dontwarn java.lang.ClassValue + +# An annotation used for build tooling, won't be directly accessed. +-dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/resources/META-INF/proguard/coroutines.pro b/kotlinx-coroutines-core/jvm/resources/META-INF/proguard/coroutines.pro new file mode 100644 index 0000000000..874b097457 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/resources/META-INF/proguard/coroutines.pro @@ -0,0 +1,31 @@ +# When editing this file, update the following files as well: +# - META-INF/com.android.tools/proguard/coroutines.pro +# - META-INF/com.android.tools/r8/coroutines.pro + +# ServiceLoader support +-keepnames class kotlinx.coroutines.internal.MainDispatcherFactory {} +-keepnames class kotlinx.coroutines.CoroutineExceptionHandler {} + +# Most of volatile fields are updated with AFU and should not be mangled +-keepclassmembers class kotlinx.coroutines.** { + volatile ; +} + +# Same story for the standard library's SafeContinuation that also uses AtomicReferenceFieldUpdater +-keepclassmembers class kotlin.coroutines.SafeContinuation { + volatile ; +} + +# These classes are only required by kotlinx.coroutines.debug.internal.AgentPremain, which is only loaded when +# kotlinx-coroutines-core is used as a Java agent, so these are not needed in contexts where ProGuard is used. +-dontwarn java.lang.instrument.ClassFileTransformer +-dontwarn sun.misc.SignalHandler +-dontwarn java.lang.instrument.Instrumentation +-dontwarn sun.misc.Signal + +# Only used in `kotlinx.coroutines.internal.ExceptionsConstructor`. +# The case when it is not available is hidden in a `try`-`catch`, as well as a check for Android. +-dontwarn java.lang.ClassValue + +# An annotation used for build tooling, won't be directly accessed. +-dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement diff --git a/kotlinx-coroutines-core/jvm/src/AbstractTimeSource.kt b/kotlinx-coroutines-core/jvm/src/AbstractTimeSource.kt new file mode 100644 index 0000000000..f497dc803c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/AbstractTimeSource.kt @@ -0,0 +1,70 @@ +// Need InlineOnly for efficient bytecode on Android +@file:Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER", "NOTHING_TO_INLINE") + +package kotlinx.coroutines + +import java.util.concurrent.locks.* +import kotlin.internal.InlineOnly + +internal abstract class AbstractTimeSource { + abstract fun currentTimeMillis(): Long + abstract fun nanoTime(): Long + abstract fun wrapTask(block: Runnable): Runnable + abstract fun trackTask() + abstract fun unTrackTask() + abstract fun registerTimeLoopThread() + abstract fun unregisterTimeLoopThread() + abstract fun parkNanos(blocker: Any, nanos: Long) // should return immediately when nanos <= 0 + abstract fun unpark(thread: Thread) +} + +// For tests only +// @JvmField: Don't use JvmField here to enable R8 optimizations via "assumenosideeffects" +private var timeSource: AbstractTimeSource? = null + +// TODO: without this, there's a compilation error. Why? +internal inline fun mockTimeSource(source: AbstractTimeSource?) { + timeSource = source +} + +@InlineOnly +internal inline fun currentTimeMillis(): Long = + timeSource?.currentTimeMillis() ?: System.currentTimeMillis() + +@InlineOnly +internal actual inline fun nanoTime(): Long = + timeSource?.nanoTime() ?: System.nanoTime() + +@InlineOnly +internal inline fun wrapTask(block: Runnable): Runnable = + timeSource?.wrapTask(block) ?: block + +@InlineOnly +internal inline fun trackTask() { + timeSource?.trackTask() +} + +@InlineOnly +internal inline fun unTrackTask() { + timeSource?.unTrackTask() +} + +@InlineOnly +internal inline fun registerTimeLoopThread() { + timeSource?.registerTimeLoopThread() +} + +@InlineOnly +internal inline fun unregisterTimeLoopThread() { + timeSource?.unregisterTimeLoopThread() +} + +@InlineOnly +internal inline fun parkNanos(blocker: Any, nanos: Long) { + timeSource?.parkNanos(blocker, nanos) ?: LockSupport.parkNanos(blocker, nanos) +} + +@InlineOnly +internal inline fun unpark(thread: Thread) { + timeSource?.unpark(thread) ?: LockSupport.unpark(thread) +} diff --git a/kotlinx-coroutines-core/jvm/src/Builders.kt b/kotlinx-coroutines-core/jvm/src/Builders.kt new file mode 100644 index 0000000000..8f72e28606 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/Builders.kt @@ -0,0 +1,111 @@ +@file:JvmMultifileClass +@file:JvmName("BuildersKt") +@file:OptIn(ExperimentalContracts::class) +@file:Suppress("LEAKED_IN_PLACE_LAMBDA", "WRONG_INVOCATION_KIND") + +package kotlinx.coroutines + +import java.util.concurrent.locks.* +import kotlin.contracts.* +import kotlin.coroutines.* + +/** + * Runs a new coroutine and **blocks** the current thread _interruptibly_ until its completion. + * + * It is designed to bridge regular blocking code to libraries that are written in suspending style, to be used in + * `main` functions and in tests. + * + * Calling [runBlocking] from a suspend function is redundant. + * For example, the following code is incorrect: + * ``` + * suspend fun loadConfiguration() { + * // DO NOT DO THIS: + * val data = runBlocking { // <- redundant and blocks the thread, do not do that + * fetchConfigurationData() // suspending function + * } + * ``` + * + * Here, instead of releasing the thread on which `loadConfiguration` runs if `fetchConfigurationData` suspends, it will + * block, potentially leading to thread starvation issues. + * + * The default [CoroutineDispatcher] for this builder is an internal implementation of event loop that processes continuations + * in this blocked thread until the completion of this coroutine. + * See [CoroutineDispatcher] for the other implementations that are provided by `kotlinx.coroutines`. + * + * When [CoroutineDispatcher] is explicitly specified in the [context], then the new coroutine runs in the context of + * the specified dispatcher while the current thread is blocked. If the specified dispatcher is an event loop of another `runBlocking`, + * then this invocation uses the outer event loop. + * + * If this blocked thread is interrupted (see [Thread.interrupt]), then the coroutine job is cancelled and + * this `runBlocking` invocation throws [InterruptedException]. + * + * See [newCoroutineContext][CoroutineScope.newCoroutineContext] for a description of debugging facilities that are available + * for a newly created coroutine. + * + * @param context the context of the coroutine. The default value is an event loop on the current thread. + * @param block the coroutine code. + */ +@Throws(InterruptedException::class) +public actual fun runBlocking(context: CoroutineContext, block: suspend CoroutineScope.() -> T): T { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + val currentThread = Thread.currentThread() + val contextInterceptor = context[ContinuationInterceptor] + val eventLoop: EventLoop? + val newContext: CoroutineContext + if (contextInterceptor == null) { + // create or use private event loop if no dispatcher is specified + eventLoop = ThreadLocalEventLoop.eventLoop + newContext = GlobalScope.newCoroutineContext(context + eventLoop) + } else { + // See if context's interceptor is an event loop that we shall use (to support TestContext) + // or take an existing thread-local event loop if present to avoid blocking it (but don't create one) + eventLoop = (contextInterceptor as? EventLoop)?.takeIf { it.shouldBeProcessedFromContext() } + ?: ThreadLocalEventLoop.currentOrNull() + newContext = GlobalScope.newCoroutineContext(context) + } + val coroutine = BlockingCoroutine(newContext, currentThread, eventLoop) + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) + return coroutine.joinBlocking() +} + +private class BlockingCoroutine( + parentContext: CoroutineContext, + private val blockedThread: Thread, + private val eventLoop: EventLoop? +) : AbstractCoroutine(parentContext, true, true) { + + override val isScopedCoroutine: Boolean get() = true + + override fun afterCompletion(state: Any?) { + // wake up blocked thread + if (Thread.currentThread() != blockedThread) + unpark(blockedThread) + } + + @Suppress("UNCHECKED_CAST") + fun joinBlocking(): T { + registerTimeLoopThread() + try { + eventLoop?.incrementUseCount() + try { + while (true) { + val parkNanos = eventLoop?.processNextEvent() ?: Long.MAX_VALUE + // note: process next even may loose unpark flag, so check if completed before parking + if (isCompleted) break + parkNanos(this, parkNanos) + if (Thread.interrupted()) cancelCoroutine(InterruptedException()) + } + } finally { // paranoia + eventLoop?.decrementUseCount() + } + } finally { // paranoia + unregisterTimeLoopThread() + } + // now return result + val state = this.state.unboxState() + (state as? CompletedExceptionally)?.let { throw it.cause } + return state as T + } +} diff --git a/kotlinx-coroutines-core/jvm/src/CoroutineContext.kt b/kotlinx-coroutines-core/jvm/src/CoroutineContext.kt new file mode 100644 index 0000000000..7628d6ac85 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/CoroutineContext.kt @@ -0,0 +1,318 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.coroutines.jvm.internal.CoroutineStackFrame + +/** + * Creates a context for a new coroutine. It installs [Dispatchers.Default] when no other dispatcher or + * [ContinuationInterceptor] is specified and adds optional support for debugging facilities (when turned on) + * and copyable-thread-local facilities on JVM. + * See [DEBUG_PROPERTY_NAME] for description of debugging facilities on JVM. + */ +@ExperimentalCoroutinesApi +public actual fun CoroutineScope.newCoroutineContext(context: CoroutineContext): CoroutineContext { + val combined = foldCopies(coroutineContext, context, true) + val debug = if (DEBUG) combined + CoroutineId(COROUTINE_ID.incrementAndGet()) else combined + return if (combined !== Dispatchers.Default && combined[ContinuationInterceptor] == null) + debug + Dispatchers.Default else debug +} + +/** + * Creates a context for coroutine builder functions that do not launch a new coroutine, e.g. [withContext]. + * @suppress + */ +@InternalCoroutinesApi +public actual fun CoroutineContext.newCoroutineContext(addedContext: CoroutineContext): CoroutineContext { + /* + * Fast-path: we only have to copy/merge if 'addedContext' (which typically has one or two elements) + * contains copyable elements. + */ + if (!addedContext.hasCopyableElements()) return this + addedContext + return foldCopies(this, addedContext, false) +} + +private fun CoroutineContext.hasCopyableElements(): Boolean = + fold(false) { result, it -> result || it is CopyableThreadContextElement<*> } + +/** + * Folds two contexts properly applying [CopyableThreadContextElement] rules when necessary. + * The rules are the following: + * - If neither context has CTCE, the sum of two contexts is returned + * - Every CTCE from the left-hand side context that does not have a matching (by key) element from right-hand side context + * is [copied][CopyableThreadContextElement.copyForChild] if [isNewCoroutine] is `true`. + * - Every CTCE from the left-hand side context that has a matching element in the right-hand side context is [merged][CopyableThreadContextElement.mergeForChild] + * - Every CTCE from the right-hand side context that hasn't been merged is copied + * - Everything else is added to the resulting context as is. + */ +private fun foldCopies(originalContext: CoroutineContext, appendContext: CoroutineContext, isNewCoroutine: Boolean): CoroutineContext { + // Do we have something to copy left-hand side? + val hasElementsLeft = originalContext.hasCopyableElements() + val hasElementsRight = appendContext.hasCopyableElements() + + // Nothing to fold, so just return the sum of contexts + if (!hasElementsLeft && !hasElementsRight) { + return originalContext + appendContext + } + + var leftoverContext = appendContext + val folded = originalContext.fold(EmptyCoroutineContext) { result, element -> + if (element !is CopyableThreadContextElement<*>) return@fold result + element + // Will this element be overwritten? + val newElement = leftoverContext[element.key] + // No, just copy it + if (newElement == null) { + // For 'withContext'-like builders we do not copy as the element is not shared + return@fold result + if (isNewCoroutine) element.copyForChild() else element + } + // Yes, then first remove the element from append context + leftoverContext = leftoverContext.minusKey(element.key) + // Return the sum + @Suppress("UNCHECKED_CAST") + return@fold result + (element as CopyableThreadContextElement).mergeForChild(newElement) + } + + if (hasElementsRight) { + leftoverContext = leftoverContext.fold(EmptyCoroutineContext) { result, element -> + // We're appending new context element -- we have to copy it, otherwise it may be shared with others + if (element is CopyableThreadContextElement<*>) { + return@fold result + element.copyForChild() + } + return@fold result + element + } + } + return folded + leftoverContext +} + +/** + * Executes a block using a given coroutine context. + */ +internal actual inline fun withCoroutineContext(context: CoroutineContext, countOrElement: Any?, block: () -> T): T { + val oldValue = updateThreadContext(context, countOrElement) + try { + return block() + } finally { + restoreThreadContext(context, oldValue) + } +} + +/** + * Executes a block using a context of a given continuation. + */ +internal actual inline fun withContinuationContext(continuation: Continuation<*>, countOrElement: Any?, block: () -> T): T { + val context = continuation.context + val oldValue = updateThreadContext(context, countOrElement) + val undispatchedCompletion = if (oldValue !== NO_THREAD_ELEMENTS) { + // Only if some values were replaced we'll go to the slow path of figuring out where/how to restore them + continuation.updateUndispatchedCompletion(context, oldValue) + } else { + null // fast path -- don't even try to find undispatchedCompletion as there's nothing to restore in the context + } + try { + return block() + } finally { + if (undispatchedCompletion == null || undispatchedCompletion.clearThreadContext()) { + restoreThreadContext(context, oldValue) + } + } +} + +internal fun Continuation<*>.updateUndispatchedCompletion(context: CoroutineContext, oldValue: Any?): UndispatchedCoroutine<*>? { + if (this !is CoroutineStackFrame) return null + /* + * Fast-path to detect whether we have undispatched coroutine at all in our stack. + * + * Implementation note. + * If we ever find that stackwalking for thread-locals is way too slow, here is another idea: + * 1) Store undispatched coroutine right in the `UndispatchedMarker` instance + * 2) To avoid issues with cross-dispatch boundary, remove `UndispatchedMarker` + * from the context when creating dispatched coroutine in `withContext`. + * Another option is to "unmark it" instead of removing to save an allocation. + * Both options should work, but it requires more careful studying of the performance + * and, mostly, maintainability impact. + */ + val potentiallyHasUndispatchedCoroutine = context[UndispatchedMarker] !== null + if (!potentiallyHasUndispatchedCoroutine) return null + val completion = undispatchedCompletion() + completion?.saveThreadContext(context, oldValue) + return completion +} + +internal tailrec fun CoroutineStackFrame.undispatchedCompletion(): UndispatchedCoroutine<*>? { + // Find direct completion of this continuation + val completion: CoroutineStackFrame = when (this) { + is DispatchedCoroutine<*> -> return null + else -> callerFrame ?: return null // something else -- not supported + } + if (completion is UndispatchedCoroutine<*>) return completion // found UndispatchedCoroutine! + return completion.undispatchedCompletion() // walk up the call stack with tail call +} + +/** + * Marker indicating that [UndispatchedCoroutine] exists somewhere up in the stack. + * Used as a performance optimization to avoid stack walking where it is not necessary. + */ +private object UndispatchedMarker: CoroutineContext.Element, CoroutineContext.Key { + override val key: CoroutineContext.Key<*> + get() = this +} + +// Used by withContext when context changes, but dispatcher stays the same +internal actual class UndispatchedCoroutineactual constructor ( + context: CoroutineContext, + uCont: Continuation +) : ScopeCoroutine(if (context[UndispatchedMarker] == null) context + UndispatchedMarker else context, uCont) { + + /** + * The state of [ThreadContextElement]s associated with the current undispatched coroutine. + * It is stored in a thread local because this coroutine can be used concurrently in suspend-resume race scenario. + * See the followin, boiled down example with inlined `withContinuationContext` body: + * ``` + * val state = saveThreadContext(ctx) + * try { + * invokeSmthWithThisCoroutineAsCompletion() // Completion implies that 'afterResume' will be called + * // COROUTINE_SUSPENDED is returned + * } finally { + * thisCoroutine().clearThreadContext() // Concurrently the "smth" could've been already resumed on a different thread + * // and it also calls saveThreadContext and clearThreadContext + * } + * ``` + * + * Usage note: + * + * This part of the code is performance-sensitive. + * It is a well-established pattern to wrap various activities into system-specific undispatched + * `withContext` for the sake of logging, MDC, tracing etc., meaning that there exists thousands of + * undispatched coroutines. + * Each access to Java's [ThreadLocal] leaves a footprint in the corresponding Thread's `ThreadLocalMap` + * that is cleared automatically as soon as the associated thread-local (-> UndispatchedCoroutine) is garbage collected + * when either the corresponding thread is GC'ed or it cleans up its stale entries on other TL accesses. + * When such coroutines are promoted to old generation, `ThreadLocalMap`s become bloated and an arbitrary accesses to thread locals + * start to consume significant amount of CPU because these maps are open-addressed and cleaned up incrementally on each access. + * (You can read more about this effect as "GC nepotism"). + * + * To avoid that, we attempt to narrow down the lifetime of this thread local as much as possible: + * - It's never accessed when we are sure there are no thread context elements + * - It's cleaned up via [ThreadLocal.remove] as soon as the coroutine is suspended or finished. + */ + private val threadStateToRecover = ThreadLocal>() + + /* + * Indicates that a coroutine has at least one thread context element associated with it + * and that 'threadStateToRecover' is going to be set in case of dispatchhing in order to preserve them. + * Better than nullable thread-local for easier debugging. + * + * It is used as a performance optimization to avoid 'threadStateToRecover' initialization + * (note: tl.get() initializes thread local), + * and is prone to false-positives as it is never reset: otherwise + * it may lead to logical data races between suspensions point where + * coroutine is yet being suspended in one thread while already being resumed + * in another. + */ + @Volatile + private var threadLocalIsSet = false + + init { + /* + * This is a hack for a very specific case in #2930 unless #3253 is implemented. + * 'ThreadLocalStressTest' covers this change properly. + * + * The scenario this change covers is the following: + * 1) The coroutine is being started as plain non kotlinx.coroutines related suspend function, + * e.g. `suspend fun main` or, more importantly, Ktor `SuspendFunGun`, that is invoking + * `withContext(tlElement)` which creates `UndispatchedCoroutine`. + * 2) It (original continuation) is then not wrapped into `DispatchedContinuation` via `intercept()` + * and goes neither through `DC.run` nor through `resumeUndispatchedWith` that both + * do thread context element tracking. + * 3) So thread locals never got chance to get properly set up via `saveThreadContext`, + * but when `withContext` finishes, it attempts to recover thread locals in its `afterResume`. + * + * Here we detect precisely this situation and properly setup context to recover later. + * + */ + if (uCont.context[ContinuationInterceptor] !is CoroutineDispatcher) { + /* + * We cannot just "read" the elements as there is no such API, + * so we update-restore it immediately and use the intermediate value + * as the initial state, leveraging the fact that thread context element + * is idempotent and such situations are increasingly rare. + */ + val values = updateThreadContext(context, null) + restoreThreadContext(context, values) + saveThreadContext(context, values) + } + } + + fun saveThreadContext(context: CoroutineContext, oldValue: Any?) { + threadLocalIsSet = true // Specify that thread-local is touched at all + threadStateToRecover.set(context to oldValue) + } + + fun clearThreadContext(): Boolean { + return !(threadLocalIsSet && threadStateToRecover.get() == null).also { + threadStateToRecover.remove() + } + } + + override fun afterCompletionUndispatched() { + clearThreadLocal() + } + + override fun afterResume(state: Any?) { + clearThreadLocal() + // resume undispatched -- update context but stay on the same dispatcher + val result = recoverResult(state, uCont) + withContinuationContext(uCont, null) { + uCont.resumeWith(result) + } + } + + private fun clearThreadLocal() { + if (threadLocalIsSet) { + threadStateToRecover.get()?.let { (ctx, value) -> + restoreThreadContext(ctx, value) + } + threadStateToRecover.remove() + } + } +} + +internal actual val CoroutineContext.coroutineName: String? get() { + if (!DEBUG) return null + val coroutineId = this[CoroutineId] ?: return null + val coroutineName = this[CoroutineName]?.name ?: "coroutine" + return "$coroutineName#${coroutineId.id}" +} + +private const val DEBUG_THREAD_NAME_SEPARATOR = " @" + +@IgnoreJreRequirement // desugared hashcode implementation +@PublishedApi +internal data class CoroutineId( + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + val id: Long +) : ThreadContextElement, AbstractCoroutineContextElement(CoroutineId) { + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + companion object Key : CoroutineContext.Key + override fun toString(): String = "CoroutineId($id)" + + override fun updateThreadContext(context: CoroutineContext): String { + val coroutineName = context[CoroutineName]?.name ?: "coroutine" + val currentThread = Thread.currentThread() + val oldName = currentThread.name + var lastIndex = oldName.lastIndexOf(DEBUG_THREAD_NAME_SEPARATOR) + if (lastIndex < 0) lastIndex = oldName.length + currentThread.name = buildString(lastIndex + coroutineName.length + 10) { + append(oldName.substring(0, lastIndex)) + append(DEBUG_THREAD_NAME_SEPARATOR) + append(coroutineName) + append('#') + append(id) + } + return oldName + } + + override fun restoreThreadContext(context: CoroutineContext, oldState: String) { + Thread.currentThread().name = oldState + } +} diff --git a/kotlinx-coroutines-core/jvm/src/Debug.kt b/kotlinx-coroutines-core/jvm/src/Debug.kt new file mode 100644 index 0000000000..96166d14b1 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/Debug.kt @@ -0,0 +1,92 @@ +// Need InlineOnly for efficient bytecode on Android +@file:Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import java.util.concurrent.atomic.* +import kotlin.internal.InlineOnly + +/** + * Name of the property that controls coroutine debugging. + * + * ### Debugging facilities + * + * In debug mode every coroutine is assigned a unique consecutive identifier. + * Every thread that executes a coroutine has its name modified to include the name and identifier of + * the currently running coroutine. + * + * Enable debugging facilities with "`kotlinx.coroutines.debug`" ([DEBUG_PROPERTY_NAME]) system property, + * use the following values: + * + * - "`auto`" (default mode, [DEBUG_PROPERTY_VALUE_AUTO]) -- enabled when assertions are enabled with "`-ea`" JVM option. + * - "`on`" ([DEBUG_PROPERTY_VALUE_ON]) or empty string -- enabled. + * - "`off`" ([DEBUG_PROPERTY_VALUE_OFF]) -- disabled. + * + * Coroutine name can be explicitly assigned using [CoroutineName] context element. + * The string "coroutine" is used as a default name. + * + * Debugging facilities are implemented by [newCoroutineContext][CoroutineScope.newCoroutineContext] function that + * is used in all coroutine builders to create context of a new coroutine. + */ +public const val DEBUG_PROPERTY_NAME: String = "kotlinx.coroutines.debug" + +/** + * Name of the boolean property that controls stacktrace recovery (enabled by default) on JVM. + * Stacktrace recovery is enabled if both debug and stacktrace recovery modes are enabled. + * + * Stacktrace recovery mode wraps every exception into the exception of the same type with original exception + * as cause, but with stacktrace of the current coroutine. + * Exception is instantiated using reflection by using no-arg, cause or cause and message constructor. + * + * This mechanism is currently supported for channels, [async], [launch], [coroutineScope], [supervisorScope] + * and [withContext] builders. + */ +internal const val STACKTRACE_RECOVERY_PROPERTY_NAME = "kotlinx.coroutines.stacktrace.recovery" + +/** + * Automatic debug configuration value for [DEBUG_PROPERTY_NAME]. + */ +public const val DEBUG_PROPERTY_VALUE_AUTO: String = "auto" + +/** + * Debug turned on value for [DEBUG_PROPERTY_NAME]. + */ +public const val DEBUG_PROPERTY_VALUE_ON: String = "on" + +/** + * Debug turned off value for [DEBUG_PROPERTY_NAME]. + */ +public const val DEBUG_PROPERTY_VALUE_OFF: String = "off" + +// @JvmField: Don't use JvmField here to enable R8 optimizations via "assumenosideeffects" +internal val ASSERTIONS_ENABLED = CoroutineId::class.java.desiredAssertionStatus() + +// @JvmField: Don't use JvmField here to enable R8 optimizations via "assumenosideeffects" +internal actual val DEBUG = systemProp(DEBUG_PROPERTY_NAME).let { value -> + when (value) { + DEBUG_PROPERTY_VALUE_AUTO, null -> ASSERTIONS_ENABLED + DEBUG_PROPERTY_VALUE_ON, "" -> true + DEBUG_PROPERTY_VALUE_OFF -> false + else -> error("System property '$DEBUG_PROPERTY_NAME' has unrecognized value '$value'") + } +} + +// Note: stack-trace recovery is enabled only in debug mode +// @JvmField: Don't use JvmField here to enable R8 optimizations via "assumenosideeffects" +@PublishedApi +internal actual val RECOVER_STACK_TRACES: Boolean = + DEBUG && systemProp(STACKTRACE_RECOVERY_PROPERTY_NAME, true) + +// It is used only in debug mode +internal val COROUTINE_ID = AtomicLong(0) + +// for tests only +internal fun resetCoroutineId() { + COROUTINE_ID.set(0) +} + +@InlineOnly +internal actual inline fun assert(value: () -> Boolean) { + if (ASSERTIONS_ENABLED && !value()) throw AssertionError() +} diff --git a/kotlinx-coroutines-core/jvm/src/DebugStrings.kt b/kotlinx-coroutines-core/jvm/src/DebugStrings.kt new file mode 100644 index 0000000000..ce5adc5655 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/DebugStrings.kt @@ -0,0 +1,17 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +// internal debugging tools for string representation + +internal actual val Any.hexAddress: String + get() = Integer.toHexString(System.identityHashCode(this)) + +internal actual fun Continuation<*>.toDebugString(): String = when (this) { + is DispatchedContinuation -> toString() + // Workaround for #858 + else -> runCatching { "$this@$hexAddress" }.getOrElse { "${this::class.java.name}@$hexAddress" } +} + +internal actual val Any.classSimpleName: String get() = this::class.java.simpleName diff --git a/kotlinx-coroutines-core/jvm/src/DefaultExecutor.kt b/kotlinx-coroutines-core/jvm/src/DefaultExecutor.kt new file mode 100644 index 0000000000..3ce7e0d333 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/DefaultExecutor.kt @@ -0,0 +1,194 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import java.util.concurrent.* +import kotlin.coroutines.* + +private val defaultMainDelayOptIn = systemProp("kotlinx.coroutines.main.delay", false) + +@PublishedApi +internal actual val DefaultDelay: Delay = initializeDefaultDelay() + +private fun initializeDefaultDelay(): Delay { + // Opt-out flag + if (!defaultMainDelayOptIn) return DefaultExecutor + val main = Dispatchers.Main + /* + * When we already are working with UI and Main threads, it makes + * no sense to create a separate thread with timer that cannot be controller + * by the UI runtime. + */ + return if (main.isMissing() || main !is Delay) DefaultExecutor else main +} + +@Suppress("PLATFORM_CLASS_MAPPED_TO_KOTLIN") +internal actual object DefaultExecutor : EventLoopImplBase(), Runnable { + const val THREAD_NAME = "kotlinx.coroutines.DefaultExecutor" + + init { + incrementUseCount() // this event loop is never completed + } + + private const val DEFAULT_KEEP_ALIVE_MS = 1000L // in milliseconds + + private val KEEP_ALIVE_NANOS = TimeUnit.MILLISECONDS.toNanos( + try { + java.lang.Long.getLong("kotlinx.coroutines.DefaultExecutor.keepAlive", DEFAULT_KEEP_ALIVE_MS) + } catch (e: SecurityException) { + DEFAULT_KEEP_ALIVE_MS + }) + + @Suppress("ObjectPropertyName") + @Volatile + private var _thread: Thread? = null + + override val thread: Thread + get() = _thread ?: createThreadSync() + + private const val FRESH = 0 + private const val ACTIVE = 1 + private const val SHUTDOWN_REQ = 2 + private const val SHUTDOWN_ACK = 3 + private const val SHUTDOWN = 4 + + @Volatile + private var debugStatus: Int = FRESH + + private val isShutDown: Boolean get() = debugStatus == SHUTDOWN + + private val isShutdownRequested: Boolean get() { + val debugStatus = debugStatus + return debugStatus == SHUTDOWN_REQ || debugStatus == SHUTDOWN_ACK + } + + actual override fun enqueue(task: Runnable) { + if (isShutDown) shutdownError() + super.enqueue(task) + } + + override fun reschedule(now: Long, delayedTask: DelayedTask) { + // Reschedule on default executor can only be invoked after Dispatchers.shutdown + shutdownError() + } + + private fun shutdownError() { + throw RejectedExecutionException("DefaultExecutor was shut down. " + + "This error indicates that Dispatchers.shutdown() was invoked prior to completion of exiting coroutines, leaving coroutines in incomplete state. " + + "Please refer to Dispatchers.shutdown documentation for more details") + } + + override fun shutdown() { + debugStatus = SHUTDOWN + super.shutdown() + } + + /** + * All event loops are using DefaultExecutor#invokeOnTimeout to avoid livelock on + * ``` + * runBlocking(eventLoop) { withTimeout { while(isActive) { ... } } } + * ``` + * + * Livelock is possible only if `runBlocking` is called on internal default executed (which is used by default [delay]), + * but it's not exposed as public API. + */ + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + scheduleInvokeOnTimeout(timeMillis, block) + + override fun run() { + ThreadLocalEventLoop.setEventLoop(this) + registerTimeLoopThread() + try { + var shutdownNanos = Long.MAX_VALUE + if (!notifyStartup()) return + while (true) { + Thread.interrupted() // just reset interruption flag + var parkNanos = processNextEvent() + if (parkNanos == Long.MAX_VALUE) { + // nothing to do, initialize shutdown timeout + val now = nanoTime() + if (shutdownNanos == Long.MAX_VALUE) shutdownNanos = now + KEEP_ALIVE_NANOS + val tillShutdown = shutdownNanos - now + if (tillShutdown <= 0) return // shut thread down + parkNanos = parkNanos.coerceAtMost(tillShutdown) + } else + shutdownNanos = Long.MAX_VALUE + if (parkNanos > 0) { + // check if shutdown was requested and bail out in this case + if (isShutdownRequested) return + parkNanos(this, parkNanos) + } + } + } finally { + _thread = null // this thread is dead + acknowledgeShutdownIfNeeded() + unregisterTimeLoopThread() + // recheck if queues are empty after _thread reference was set to null (!!!) + if (!isEmpty) thread // recreate thread if it is needed + } + } + + @Synchronized + private fun createThreadSync(): Thread { + return _thread ?: Thread(this, THREAD_NAME).apply { + _thread = this + /* + * `DefaultExecutor` is a global singleton that creates its thread lazily. + * To isolate the classloaders properly, we are inherting the context classloader from + * the singleton itself instead of using parent' thread one + * in order not to accidentally capture temporary application classloader. + */ + contextClassLoader = this@DefaultExecutor.javaClass.classLoader + isDaemon = true + start() + } + } + + // used for tests + @Synchronized + internal fun ensureStarted() { + assert { _thread == null } // ensure we are at a clean state + assert { debugStatus == FRESH || debugStatus == SHUTDOWN_ACK } + debugStatus = FRESH + createThreadSync() // create fresh thread + while (debugStatus == FRESH) (this as Object).wait() + } + + @Synchronized + private fun notifyStartup(): Boolean { + if (isShutdownRequested) return false + debugStatus = ACTIVE + (this as Object).notifyAll() + return true + } + + @Synchronized // used _only_ for tests + fun shutdownForTests(timeout: Long) { + val deadline = System.currentTimeMillis() + timeout + if (!isShutdownRequested) debugStatus = SHUTDOWN_REQ + // loop while there is anything to do immediately or deadline passes + while (debugStatus != SHUTDOWN_ACK && _thread != null) { + _thread?.let { unpark(it) } // wake up thread if present + val remaining = deadline - System.currentTimeMillis() + if (remaining <= 0) break + (this as Object).wait(timeout) + } + // restore fresh status + debugStatus = FRESH + } + + @Synchronized + private fun acknowledgeShutdownIfNeeded() { + if (!isShutdownRequested) return + debugStatus = SHUTDOWN_ACK + resetAll() // clear queues + (this as Object).notifyAll() + } + + // User only for testing and nothing else + internal val isThreadPresent + get() = _thread != null + + override fun toString(): String { + return "DefaultExecutor" + } +} diff --git a/kotlinx-coroutines-core/jvm/src/Dispatchers.kt b/kotlinx-coroutines-core/jvm/src/Dispatchers.kt new file mode 100644 index 0000000000..a6acc129cc --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/Dispatchers.kt @@ -0,0 +1,102 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.scheduling.* + +/** + * Name of the property that defines the maximal number of threads that are used by [Dispatchers.IO] coroutines dispatcher. + */ +public const val IO_PARALLELISM_PROPERTY_NAME: String = "kotlinx.coroutines.io.parallelism" + +/** + * Groups various implementations of [CoroutineDispatcher]. + */ +public actual object Dispatchers { + @JvmStatic + public actual val Default: CoroutineDispatcher = DefaultScheduler + + @JvmStatic + public actual val Main: MainCoroutineDispatcher get() = MainDispatcherLoader.dispatcher + + @JvmStatic + public actual val Unconfined: CoroutineDispatcher = kotlinx.coroutines.Unconfined + + /** + * The [CoroutineDispatcher] that is designed for offloading blocking IO tasks to a shared pool of threads. + * + * Additional threads in this pool are created and are shutdown on demand. + * The number of threads used by tasks in this dispatcher is limited by the value of + * "`kotlinx.coroutines.io.parallelism`" ([IO_PARALLELISM_PROPERTY_NAME]) system property. + * It defaults to the limit of 64 threads or the number of cores (whichever is larger). + * + * ### Elasticity for limited parallelism + * + * `Dispatchers.IO` has a unique property of elasticity: its views + * obtained with [CoroutineDispatcher.limitedParallelism] are + * not restricted by the `Dispatchers.IO` parallelism. Conceptually, there is + * a dispatcher backed by an unlimited pool of threads, and both `Dispatchers.IO` + * and views of `Dispatchers.IO` are actually views of that dispatcher. In practice + * this means that, despite not abiding by `Dispatchers.IO`'s parallelism + * restrictions, its views share threads and resources with it. + * + * In the following example + * ``` + * // 100 threads for MySQL connection + * val myMysqlDbDispatcher = Dispatchers.IO.limitedParallelism(100) + * // 60 threads for MongoDB connection + * val myMongoDbDispatcher = Dispatchers.IO.limitedParallelism(60) + * ``` + * the system may have up to `64 + 100 + 60` threads dedicated to blocking tasks during peak loads, + * but during its steady state there is only a small number of threads shared + * among `Dispatchers.IO`, `myMysqlDbDispatcher` and `myMongoDbDispatcher`. + * + * ### Implementation note + * + * This dispatcher and its views share threads with the [Default][Dispatchers.Default] dispatcher, so using + * `withContext(Dispatchers.IO) { ... }` when already running on the [Default][Dispatchers.Default] + * dispatcher typically does not lead to an actual switching to another thread. In such scenarios, + * the underlying implementation attempts to keep the execution on the same thread on a best-effort basis. + * + * As a result of thread sharing, more than 64 (default parallelism) threads can be created (but not used) + * during operations over IO dispatcher. + */ + @JvmStatic + public val IO: CoroutineDispatcher get() = DefaultIoScheduler + + /** + * Shuts down built-in dispatchers, such as [Default] and [IO], + * stopping all the threads associated with them and making them reject all new tasks. + * Dispatcher used as a fallback for time-related operations (`delay`, `withTimeout`) + * and to handle rejected tasks from other dispatchers is also shut down. + * + * This is a **delicate** API. It is not supposed to be called from a general + * application-level code and its invocation is irreversible. + * The invocation of shutdown affects most of the coroutines machinery and + * leaves the coroutines framework in an inoperable state. + * The shutdown method should only be invoked when there are no pending tasks or active coroutines. + * Otherwise, the behavior is unspecified: the call to `shutdown` may throw an exception without completing + * the shutdown, or it may finish successfully, but the remaining jobs will be in a permanent dormant state, + * never completing nor executing. + * + * The main goal of the shutdown is to stop all background threads associated with the coroutines + * framework in order to make kotlinx.coroutines classes unloadable by Java Virtual Machine. + * It is only recommended to be used in containerized environments (OSGi, Gradle plugins system, + * IDEA plugins) at the end of the container lifecycle. + */ + @DelicateCoroutinesApi + public fun shutdown() { + DefaultExecutor.shutdown() + // Also shuts down Dispatchers.IO + DefaultScheduler.shutdown() + } +} + +/** + * `actual` counterpart of the corresponding `expect` declaration. + * Should never be used directly from JVM sources, all accesses + * to `Dispatchers.IO` should be resolved to the corresponding member of [Dispatchers] object. + * @suppress + */ +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +@Deprecated(message = "Should not be used directly", level = DeprecationLevel.HIDDEN) +public actual val Dispatchers.IO: CoroutineDispatcher get() = Dispatchers.IO diff --git a/kotlinx-coroutines-core/jvm/src/EventLoop.kt b/kotlinx-coroutines-core/jvm/src/EventLoop.kt new file mode 100644 index 0000000000..15d4ab5c85 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/EventLoop.kt @@ -0,0 +1,125 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.Runnable +import kotlinx.coroutines.scheduling.* +import kotlinx.coroutines.scheduling.CoroutineScheduler + +internal actual abstract class EventLoopImplPlatform: EventLoop() { + + protected abstract val thread: Thread + + protected actual fun unpark() { + val thread = thread // atomic read + if (Thread.currentThread() !== thread) + unpark(thread) + } + + protected actual open fun reschedule(now: Long, delayedTask: EventLoopImplBase.DelayedTask) { + DefaultExecutor.schedule(now, delayedTask) + } +} + +internal class BlockingEventLoop( + override val thread: Thread +) : EventLoopImplBase() + +internal actual fun createEventLoop(): EventLoop = BlockingEventLoop(Thread.currentThread()) + +/** + * Processes next event in the current thread's event loop. + * + * The result of this function is to be interpreted like this: + * - `<= 0` -- there are potentially more events for immediate processing; + * - `> 0` -- a number of nanoseconds to wait for the next scheduled event; + * - [Long.MAX_VALUE] -- no more events or no thread-local event loop. + * + * Sample usage of this function: + * + * ``` + * while (waitingCondition) { + * val time = processNextEventInCurrentThread() + * LockSupport.parkNanos(time) + * } + * ``` + * + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public fun processNextEventInCurrentThread(): Long = + // This API is used in Ktor for serverless integration where a single thread awaits a blocking call + // (and, to avoid actual blocking, does something via this call), see #850 + ThreadLocalEventLoop.currentOrNull()?.processNextEvent() ?: Long.MAX_VALUE + +internal actual inline fun platformAutoreleasePool(crossinline block: () -> Unit) = block() + +/** + * Retrieves and executes a single task from the current system dispatcher ([Dispatchers.Default] or [Dispatchers.IO]). + * Returns `0` if any task was executed, `>= 0` for number of nanoseconds to wait until invoking this method again + * (implying that there will be a task to steal in N nanoseconds), `-1` if there is no tasks in the corresponding dispatcher at all. + * + * ### Invariants + * + * - When invoked from [Dispatchers.Default] **thread** (even if the actual context is different dispatcher, + * [CoroutineDispatcher.limitedParallelism] or any in-place wrapper), it runs an arbitrary task that ended + * up being scheduled to [Dispatchers.Default] or its counterpart. Tasks scheduled to [Dispatchers.IO] + * **are not** executed[1]. + * - When invoked from [Dispatchers.IO] thread, the same rules apply, but for blocking tasks only. + * + * [1] -- this is purely technical limitation: the scheduler does not have "notify me when CPU token is available" API, + * and we cannot leave this method without leaving thread in its original state. + * + * ### Rationale + * + * This is an internal API that is intended to replace IDEA's core FJP decomposition. + * The following API is provided by IDEA core: + * ``` + * runDecomposedTaskAndJoinIt { // <- non-suspending call + * // spawn as many tasks as needed + * // these tasks can also invoke 'runDecomposedTaskAndJoinIt' + * } + * ``` + * The key observation here is that 'runDecomposedTaskAndJoinIt' can be invoked from `Dispatchers.Default` itself, + * thus blocking at least one thread. To avoid deadlocks and starvation during large hierarchical decompositions, + * 'runDecomposedTaskAndJoinIt' should not just block but also **help** execute the task or other tasks + * until an arbitrary condition is satisfied. + * + * See #3439 for additional details. + * + * ### Limitations and caveats + * + * - Executes tasks in-place, thus potentially leaking irrelevant thread-locals from the current thread + * - Is not 100% effective, because the caller should somehow "wait" (or do other work) for [Long] returned nanoseconds + * even when work arrives immediately after returning from this method. + * - When there is no more work, it's up to the caller to decide what to do. It's important to remember that + * work to current dispatcher may arrive **later** from external sources [1] + * + * [1] -- this is also a technicality that can be solved in kotlinx.coroutines itself, but unfortunately requires + * a tremendous effort. + * + * @throws IllegalStateException if the current thread is not system dispatcher thread + */ +@InternalCoroutinesApi +@DelicateCoroutinesApi +@PublishedApi +internal fun runSingleTaskFromCurrentSystemDispatcher(): Long { + val thread = Thread.currentThread() + if (thread !is CoroutineScheduler.Worker) throw IllegalStateException("Expected CoroutineScheduler.Worker, but got $thread") + return thread.runSingleTask() +} + +/** + * Checks whether the given thread belongs to Dispatchers.IO. + * Note that feature "is part of the Dispatchers.IO" is *dynamic*, meaning that the thread + * may change this status when switching between tasks. + * + * This function is inteded to be used on the result of `Thread.currentThread()` for diagnostic + * purposes, and is declared as an extension only to avoid top-level scope pollution. + */ +@InternalCoroutinesApi +@DelicateCoroutinesApi +@PublishedApi +internal fun Thread.isIoDispatcherThread(): Boolean { + if (this !is CoroutineScheduler.Worker) return false + return isIo() +} + diff --git a/kotlinx-coroutines-core/jvm/src/Exceptions.kt b/kotlinx-coroutines-core/jvm/src/Exceptions.kt new file mode 100644 index 0000000000..dafaaacbe3 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/Exceptions.kt @@ -0,0 +1,76 @@ +@file:Suppress("FunctionName") + +package kotlinx.coroutines + +/** + * Thrown by cancellable suspending functions if the [Job] of the coroutine is cancelled while it is suspending. + * It indicates _normal_ cancellation of a coroutine. + * **It is not printed to console/log by default uncaught exception handler**. + * See [CoroutineExceptionHandler] +*/ +public actual typealias CancellationException = java.util.concurrent.CancellationException + +/** + * Creates a cancellation exception with a specified message and [cause]. + */ +public actual fun CancellationException(message: String?, cause: Throwable?) : CancellationException = + CancellationException(message).apply { initCause(cause) } + +/** + * Thrown by cancellable suspending functions if the [Job] of the coroutine is cancelled or completed + * without cause, or with a cause or exception that is not [CancellationException] + * (see [Job.getCancellationException]). + */ +internal actual class JobCancellationException public actual constructor( + message: String, + cause: Throwable?, + job: Job +) : CancellationException(message), CopyableThrowable { + + @Transient + private val _job: Job? = job + + // The safest option for transient -- return something that meanigfully reject any attemp to interact with the job + internal actual val job get() = _job ?: NonCancellable + + init { + if (cause != null) initCause(cause) + } + + override fun fillInStackTrace(): Throwable { + if (DEBUG) { + return super.fillInStackTrace() + } + // Prevent Android <= 6.0 bug, #1866 + stackTrace = emptyArray() + /* + * In non-debug mode we don't want to have a stacktrace on every cancellation/close, + * parent job reference is enough. Stacktrace of JCE is not needed most of the time (e.g., it is not logged) + * and hurts performance. + */ + return this + } + + override fun createCopy(): JobCancellationException? { + if (DEBUG) { + return JobCancellationException(message!!, this, job) + } + + /* + * In non-debug mode we don't copy JCE for speed as it does not have the stack trace anyway. + */ + return null + } + + override fun toString(): String = "${super.toString()}; job=$job" + + override fun equals(other: Any?): Boolean = + other === this || + other is JobCancellationException && other.message == message && other.job == job && other.cause == cause + + override fun hashCode(): Int { + // since job is transient it is indeed nullable after deserialization + @Suppress("UNNECESSARY_SAFE_CALL") + return (message!!.hashCode() * 31 + (job?.hashCode() ?: 0)) * 31 + (cause?.hashCode() ?: 0) + } +} diff --git a/kotlinx-coroutines-core/jvm/src/Executors.kt b/kotlinx-coroutines-core/jvm/src/Executors.kt new file mode 100644 index 0000000000..9bd48b1537 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/Executors.kt @@ -0,0 +1,211 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import java.io.Closeable +import java.util.concurrent.* +import kotlin.coroutines.* +import kotlin.AutoCloseable + +/** + * [CoroutineDispatcher] that has underlying [Executor] for dispatching tasks. + * Instances of [ExecutorCoroutineDispatcher] should be closed by the owner of the dispatcher. + * + * This class is generally used as a bridge between coroutine-based API and + * asynchronous API that requires an instance of the [Executor]. + */ +public abstract class ExecutorCoroutineDispatcher : CoroutineDispatcher(), Closeable, AutoCloseable { + /** @suppress */ + @ExperimentalStdlibApi + public companion object Key : AbstractCoroutineContextKey( + CoroutineDispatcher, + { it as? ExecutorCoroutineDispatcher }) + + /** + * Underlying executor of current [CoroutineDispatcher]. + */ + public abstract val executor: Executor + + /** + * Closes this coroutine dispatcher and shuts down its executor. + * + * It may throw an exception if this dispatcher is global and cannot be closed. + */ + public abstract override fun close() +} + +@ExperimentalCoroutinesApi +public actual typealias CloseableCoroutineDispatcher = ExecutorCoroutineDispatcher + +/** + * Converts an instance of [ExecutorService] to an implementation of [ExecutorCoroutineDispatcher]. + * + * ## Interaction with [delay] and time-based coroutines. + * + * If the given [ExecutorService] is an instance of [ScheduledExecutorService], then all time-related + * coroutine operations such as [delay], [withTimeout] and time-based [Flow] operators will be scheduled + * on this executor using [schedule][ScheduledExecutorService.schedule] method. If the corresponding + * coroutine is cancelled, [ScheduledFuture.cancel] will be invoked on the corresponding future. + * + * If the given [ExecutorService] is an instance of [ScheduledThreadPoolExecutor], then prior to any scheduling, + * remove on cancel policy will be set via [ScheduledThreadPoolExecutor.setRemoveOnCancelPolicy] in order + * to reduce the memory pressure of cancelled coroutines. + * + * If the executor service is neither of this types, the separate internal thread will be used to + * _track_ the delay and time-related executions, but the coroutine itself will still be executed + * on top of the given executor. + * + * ## Rejected execution + * If the underlying executor throws [RejectedExecutionException] on + * attempt to submit a continuation task (it happens when [closing][ExecutorCoroutineDispatcher.close] the + * resulting dispatcher, on underlying executor [shutdown][ExecutorService.shutdown], or when it uses limited queues), + * then the [Job] of the affected task is [cancelled][Job.cancel] and the task is submitted to the + * [Dispatchers.IO], so that the affected coroutine can cleanup its resources and promptly complete. + */ +@JvmName("from") // this is for a nice Java API, see issue #255 +public fun ExecutorService.asCoroutineDispatcher(): ExecutorCoroutineDispatcher = + ExecutorCoroutineDispatcherImpl(this) + +/** + * Converts an instance of [Executor] to an implementation of [CoroutineDispatcher]. + * + * ## Interaction with [delay] and time-based coroutines. + * + * If the given [Executor] is an instance of [ScheduledExecutorService], then all time-related + * coroutine operations such as [delay], [withTimeout] and time-based [Flow] operators will be scheduled + * on this executor using [schedule][ScheduledExecutorService.schedule] method. If the corresponding + * coroutine is cancelled, [ScheduledFuture.cancel] will be invoked on the corresponding future. + * + * If the given [Executor] is an instance of [ScheduledThreadPoolExecutor], then prior to any scheduling, + * remove on cancel policy will be set via [ScheduledThreadPoolExecutor.setRemoveOnCancelPolicy] in order + * to reduce the memory pressure of cancelled coroutines. + * + * If the executor is neither of this types, the separate internal thread will be used to + * _track_ the delay and time-related executions, but the coroutine itself will still be executed + * on top of the given executor. + * + * ## Rejected execution + * + * If the underlying executor throws [RejectedExecutionException] on + * attempt to submit a continuation task (it happens when [closing][ExecutorCoroutineDispatcher.close] the + * resulting dispatcher, on underlying executor [shutdown][ExecutorService.shutdown], or when it uses limited queues), + * then the [Job] of the affected task is [cancelled][Job.cancel] and the task is submitted to the + * [Dispatchers.IO], so that the affected coroutine can cleanup its resources and promptly complete. + */ +@JvmName("from") // this is for a nice Java API, see issue #255 +public fun Executor.asCoroutineDispatcher(): CoroutineDispatcher = + (this as? DispatcherExecutor)?.dispatcher ?: ExecutorCoroutineDispatcherImpl(this) + +/** + * Converts an instance of [CoroutineDispatcher] to an implementation of [Executor]. + * + * It returns the original executor when used on the result of [Executor.asCoroutineDispatcher] extensions. + */ +public fun CoroutineDispatcher.asExecutor(): Executor = + (this as? ExecutorCoroutineDispatcher)?.executor ?: DispatcherExecutor(this) + +private class DispatcherExecutor(@JvmField val dispatcher: CoroutineDispatcher) : Executor { + override fun execute(block: Runnable) { + if (dispatcher.safeIsDispatchNeeded(EmptyCoroutineContext)) { + dispatcher.safeDispatch(EmptyCoroutineContext, block) + } else { + block.run() + } + } + + override fun toString(): String = dispatcher.toString() +} + +internal class ExecutorCoroutineDispatcherImpl(override val executor: Executor) : ExecutorCoroutineDispatcher(), Delay { + + /* + * Attempts to reflectively (to be Java 6 compatible) invoke + * ScheduledThreadPoolExecutor.setRemoveOnCancelPolicy in order to cleanup + * internal scheduler queue on cancellation. + */ + init { + removeFutureOnCancel(executor) + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + try { + executor.execute(wrapTask(block)) + } catch (e: RejectedExecutionException) { + unTrackTask() + cancelJobOnRejection(context, e) + Dispatchers.IO.dispatch(context, block) + } + } + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val future = (executor as? ScheduledExecutorService)?.scheduleBlock( + ResumeUndispatchedRunnable(this, continuation), + continuation.context, + timeMillis + ) + // If everything went fine and the scheduling attempt was not rejected -- use it + if (future != null) { + continuation.invokeOnCancellation(CancelFutureOnCancel(future)) + return + } + // Otherwise fallback to default executor + DefaultExecutor.scheduleResumeAfterDelay(timeMillis, continuation) + } + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { + val future = (executor as? ScheduledExecutorService)?.scheduleBlock(block, context, timeMillis) + return when { + future != null -> DisposableFutureHandle(future) + else -> DefaultExecutor.invokeOnTimeout(timeMillis, block, context) + } + } + + private fun ScheduledExecutorService.scheduleBlock(block: Runnable, context: CoroutineContext, timeMillis: Long): ScheduledFuture<*>? { + return try { + schedule(block, timeMillis, TimeUnit.MILLISECONDS) + } catch (e: RejectedExecutionException) { + cancelJobOnRejection(context, e) + null + } + } + + private fun cancelJobOnRejection(context: CoroutineContext, exception: RejectedExecutionException) { + context.cancel(CancellationException("The task was rejected", exception)) + } + + override fun close() { + (executor as? ExecutorService)?.shutdown() + } + + override fun toString(): String = executor.toString() + override fun equals(other: Any?): Boolean = other is ExecutorCoroutineDispatcherImpl && other.executor === executor + override fun hashCode(): Int = System.identityHashCode(executor) +} + +private class ResumeUndispatchedRunnable( + private val dispatcher: CoroutineDispatcher, + private val continuation: CancellableContinuation +) : Runnable { + override fun run() { + with(continuation) { dispatcher.resumeUndispatched(Unit) } + } +} + +/** + * An implementation of [DisposableHandle] that cancels the specified future on dispose. + * @suppress **This is unstable API and it is subject to change.** + */ +private class DisposableFutureHandle(private val future: Future<*>) : DisposableHandle { + override fun dispose() { + future.cancel(false) + } + override fun toString(): String = "DisposableFutureHandle[$future]" +} + +private class CancelFutureOnCancel(private val future: Future<*>) : CancelHandler { + override fun invoke(cause: Throwable?) { + // Don't interrupt when cancelling future on completion, because no one is going to reset this + // interruption flag and it will cause spurious failures elsewhere + future.cancel(false) + } + override fun toString() = "CancelFutureOnCancel[$future]" +} diff --git a/kotlinx-coroutines-core/jvm/src/Future.kt b/kotlinx-coroutines-core/jvm/src/Future.kt new file mode 100644 index 0000000000..be9466dea0 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/Future.kt @@ -0,0 +1,31 @@ +@file:JvmMultifileClass +@file:JvmName("JobKt") + +package kotlinx.coroutines + +import java.util.concurrent.* + +/** + * Cancels a specified [future] when this job is cancelled. + * This is a shortcut for the following code with slightly more efficient implementation (one fewer object created). + * ``` + * invokeOnCancellation { if (it != null) future.cancel(false) } + * ``` + */ +// Warning since 1.9.0 +@Deprecated( + "This function does not do what its name implies: it will not cancel the future if just cancel() was called.", + level = DeprecationLevel.WARNING, + replaceWith = ReplaceWith("this.invokeOnCancellation { future.cancel(false) }") +) +public fun CancellableContinuation<*>.cancelFutureOnCancellation(future: Future<*>): Unit = + invokeOnCancellation(handler = PublicCancelFutureOnCancel(future)) + +private class PublicCancelFutureOnCancel(private val future: Future<*>) : CancelHandler { + override fun invoke(cause: Throwable?) { + // Don't interrupt when cancelling future on completion, because no one is going to reset this + // interruption flag and it will cause spurious failures elsewhere + if (cause != null) future.cancel(false) + } + override fun toString() = "CancelFutureOnCancel[$future]" +} diff --git a/kotlinx-coroutines-core/jvm/src/Interruptible.kt b/kotlinx-coroutines-core/jvm/src/Interruptible.kt new file mode 100644 index 0000000000..6b52f499bf --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/Interruptible.kt @@ -0,0 +1,160 @@ +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlin.coroutines.* + +/** + * Calls the specified [block] with a given coroutine context in + * [an interruptible manner](https://docs.oracle.com/javase/tutorial/essential/concurrency/interrupt.html). + * The blocking code block will be interrupted and this function will throw [CancellationException] + * if the coroutine is cancelled. + * + * Example: + * + * ``` + * withTimeout(500L) { // Cancels coroutine on timeout + * runInterruptible { // Throws CancellationException if interrupted + * doSomethingBlocking() // Interrupted on coroutines cancellation + * } + * } + * ``` + * + * There is an optional [context] parameter to this function working just like [withContext]. + * It enables single-call conversion of interruptible Java methods into suspending functions. + * With one call here we are moving the call to [Dispatchers.IO] and supporting interruption: + * + * ``` + * suspend fun BlockingQueue.awaitTake(): T = + * runInterruptible(Dispatchers.IO) { queue.take() } + * ``` + * + * `runInterruptible` uses [withContext] as an underlying mechanism for switching context, + * meaning that the supplied [block] is invoked in an [undispatched][CoroutineStart.UNDISPATCHED] + * manner directly by the caller if [CoroutineDispatcher] from the current [coroutineContext][currentCoroutineContext] + * is the same as the one supplied in [context]. + */ +public suspend fun runInterruptible( + context: CoroutineContext = EmptyCoroutineContext, + block: () -> T +): T = withContext(context) { + runInterruptibleInExpectedContext(coroutineContext, block) +} + +private fun runInterruptibleInExpectedContext(coroutineContext: CoroutineContext, block: () -> T): T { + try { + val threadState = ThreadState() + threadState.setup(coroutineContext.job) + try { + return block() + } finally { + threadState.clearInterrupt() + } + } catch (e: InterruptedException) { + throw CancellationException("Blocking call was interrupted due to parent cancellation").initCause(e) + } +} + +private const val WORKING = 0 +private const val FINISHED = 1 +private const val INTERRUPTING = 2 +private const val INTERRUPTED = 3 + +private class ThreadState : JobNode() { + /* + === States === + + WORKING: running normally + FINISH: complete normally + INTERRUPTING: canceled, going to interrupt this thread + INTERRUPTED: this thread is interrupted + + === Possible Transitions === + + +----------------+ register job +-------------------------+ + | WORKING | cancellation listener | WORKING | + | (thread, null) | -------------------------> | (thread, cancel handle) | + +----------------+ +-------------------------+ + | | | + | cancel cancel | | complete + | | | + V | | + +---------------+ | | + | INTERRUPTING | <--------------------------------------+ | + +---------------+ | + | | + | interrupt | + | | + V V + +---------------+ +-------------------------+ + | INTERRUPTED | | FINISHED | + +---------------+ +-------------------------+ + */ + private val _state = atomic(WORKING) + private val targetThread = Thread.currentThread() + + // Registered cancellation handler + private var cancelHandle: DisposableHandle? = null + + override val onCancelling get() = true + + fun setup(job: Job) { + cancelHandle = job.invokeOnCompletion(handler = this) + // Either we successfully stored it or it was immediately cancelled + _state.loop { state -> + when (state) { + // Happy-path, move forward + WORKING -> if (_state.compareAndSet(state, WORKING)) return + // Immediately cancelled, just continue + INTERRUPTING, INTERRUPTED -> return + else -> invalidState(state) + } + } + } + + fun clearInterrupt() { + /* + * Do not allow to untriggered interrupt to leak + */ + _state.loop { state -> + when (state) { + WORKING -> if (_state.compareAndSet(state, FINISHED)) { + cancelHandle?.dispose() + return + } + INTERRUPTING -> { + /* + * Spin, cancellation mechanism is interrupting our thread right now + * and we have to wait it and then clear interrupt status + */ + } + INTERRUPTED -> { + // Clear it and bail out + Thread.interrupted() + return + } + else -> invalidState(state) + } + } + } + + // Cancellation handler + override fun invoke(cause: Throwable?) { + _state.loop { state -> + when (state) { + // Working -> try to transite state and interrupt the thread + WORKING -> { + if (_state.compareAndSet(state, INTERRUPTING)) { + targetThread.interrupt() + _state.value = INTERRUPTED + return + } + } + // Finished -- runInterruptible is already complete, INTERRUPTING - ignore + FINISHED, INTERRUPTING, INTERRUPTED -> return + else -> invalidState(state) + } + } + } + + private fun invalidState(state: Int): Nothing = error("Illegal state $state") +} diff --git a/kotlinx-coroutines-core/jvm/src/Runnable.kt b/kotlinx-coroutines-core/jvm/src/Runnable.kt new file mode 100644 index 0000000000..805bef7554 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/Runnable.kt @@ -0,0 +1,10 @@ +package kotlinx.coroutines + +/** + * A runnable task for [CoroutineDispatcher.dispatch]. + * + * It is a typealias for [java.lang.Runnable], which is widely used in Java APIs. + * This makes it possible to directly pass the argument of [CoroutineDispatcher.dispatch] + * to the underlying Java implementation without any additional wrapping. + */ +public actual typealias Runnable = java.lang.Runnable diff --git a/kotlinx-coroutines-core/jvm/src/SchedulerTask.kt b/kotlinx-coroutines-core/jvm/src/SchedulerTask.kt new file mode 100644 index 0000000000..ca1ab87f68 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/SchedulerTask.kt @@ -0,0 +1,5 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.scheduling.* + +internal actual typealias SchedulerTask = Task diff --git a/kotlinx-coroutines-core/jvm/src/ThreadContextElement.kt b/kotlinx-coroutines-core/jvm/src/ThreadContextElement.kt new file mode 100644 index 0000000000..5015a259f0 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/ThreadContextElement.kt @@ -0,0 +1,284 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +/** + * Defines elements in a [CoroutineContext] that are installed into the thread context + * every time the coroutine with this element in the context is resumed on a thread. + * + * Implementations of this interface define a type [S] of the thread-local state that they need to store + * upon resuming a coroutine and restore later upon suspension. + * The infrastructure provides the corresponding storage. + * + * Example usage looks like this: + * + * ``` + * // Appends "name" of a coroutine to a current thread name when coroutine is executed + * class CoroutineName(val name: String) : ThreadContextElement { + * // declare companion object for a key of this element in coroutine context + * companion object Key : CoroutineContext.Key + * + * // provide the key of the corresponding context element + * override val key: CoroutineContext.Key + * get() = Key + * + * // this is invoked before coroutine is resumed on current thread + * override fun updateThreadContext(context: CoroutineContext): String { + * val previousName = Thread.currentThread().name + * Thread.currentThread().name = "$previousName # $name" + * return previousName + * } + * + * // this is invoked after coroutine has suspended on current thread + * override fun restoreThreadContext(context: CoroutineContext, oldState: String) { + * Thread.currentThread().name = oldState + * } + * } + * + * // Usage + * launch(Dispatchers.Main + CoroutineName("Progress bar coroutine")) { ... } + * ``` + * + * Every time this coroutine is resumed on a thread, UI thread name is updated to + * "UI thread original name # Progress bar coroutine" and the thread name is restored to the original one when + * this coroutine suspends. + * + * To use [ThreadLocal] variable within the coroutine use [ThreadLocal.asContextElement][asContextElement] function. + * + * ### Reentrancy and thread-safety + * + * Correct implementations of this interface must expect that calls to [restoreThreadContext] + * may happen in parallel to the subsequent [updateThreadContext] and [restoreThreadContext] operations. + * See [CopyableThreadContextElement] for advanced interleaving details. + * + * All implementations of [ThreadContextElement] should be thread-safe and guard their internal mutable state + * within an element accordingly. + */ +public interface ThreadContextElement : CoroutineContext.Element { + /** + * Updates context of the current thread. + * This function is invoked before the coroutine in the specified [context] is resumed in the current thread + * when the context of the coroutine this element. + * The result of this function is the old value of the thread-local state that will be passed to [restoreThreadContext]. + * This method should handle its own exceptions and do not rethrow it. Thrown exceptions will leave coroutine which + * context is updated in an undefined state and may crash an application. + * + * @param context the coroutine context. + */ + public fun updateThreadContext(context: CoroutineContext): S + + /** + * Restores context of the current thread. + * This function is invoked after the coroutine in the specified [context] is suspended in the current thread + * if [updateThreadContext] was previously invoked on resume of this coroutine. + * The value of [oldState] is the result of the previous invocation of [updateThreadContext] and it should + * be restored in the thread-local state by this function. + * This method should handle its own exceptions and do not rethrow it. Thrown exceptions will leave coroutine which + * context is updated in an undefined state and may crash an application. + * + * @param context the coroutine context. + * @param oldState the value returned by the previous invocation of [updateThreadContext]. + */ + public fun restoreThreadContext(context: CoroutineContext, oldState: S) +} + +/** + * A [ThreadContextElement] copied whenever a child coroutine inherits a context containing it. + * + * When an API uses a _mutable_ [ThreadLocal] for consistency, a [CopyableThreadContextElement] + * can give coroutines "coroutine-safe" write access to that `ThreadLocal`. + * + * A write made to a `ThreadLocal` with a matching [CopyableThreadContextElement] by a coroutine + * will be visible to _itself_ and any child coroutine launched _after_ that write. + * + * Writes will not be visible to the parent coroutine, peer coroutines, or coroutines that happen + * to use the same thread. Writes made to the `ThreadLocal` by the parent coroutine _after_ + * launching a child coroutine will not be visible to that child coroutine. + * + * This can be used to allow a coroutine to use a mutable ThreadLocal API transparently and + * correctly, regardless of the coroutine's structured concurrency. + * + * This example adapts a `ThreadLocal` method trace to be "coroutine local" while the method trace + * is in a coroutine: + * + * ``` + * class TraceContextElement(private val traceData: TraceData?) : CopyableThreadContextElement { + * companion object Key : CoroutineContext.Key + * + * override val key: CoroutineContext.Key = Key + * + * override fun updateThreadContext(context: CoroutineContext): TraceData? { + * val oldState = traceThreadLocal.get() + * traceThreadLocal.set(traceData) + * return oldState + * } + * + * override fun restoreThreadContext(context: CoroutineContext, oldState: TraceData?) { + * traceThreadLocal.set(oldState) + * } + * + * override fun copyForChild(): TraceContextElement { + * // Copy from the ThreadLocal source of truth at child coroutine launch time. This makes + * // ThreadLocal writes between resumption of the parent coroutine and the launch of the + * // child coroutine visible to the child. + * return TraceContextElement(traceThreadLocal.get()?.copy()) + * } + * + * override fun mergeForChild(overwritingElement: CoroutineContext.Element): CoroutineContext { + * // Merge operation defines how to handle situations when both + * // the parent coroutine has an element in the context and + * // an element with the same key was also + * // explicitly passed to the child coroutine. + * // If merging does not require special behavior, + * // the copy of the element can be returned. + * return TraceContextElement(traceThreadLocal.get()?.copy()) + * } + * } + * ``` + * + * A coroutine using this mechanism can safely call Java code that assumes the corresponding thread local element's + * value is installed into the target thread local. + * + * ### Reentrancy and thread-safety + * + * Correct implementations of this interface must expect that calls to [restoreThreadContext] + * may happen in parallel to the subsequent [updateThreadContext] and [restoreThreadContext] operations. + * + * Even though an element is copied for each child coroutine, an implementation should be able to handle the following + * interleaving when a coroutine with the corresponding element is launched on a multithreaded dispatcher: + * + * ``` + * coroutine.updateThreadContext() // Thread #1 + * ... coroutine body ... + * // suspension + immediate dispatch happen here + * coroutine.updateThreadContext() // Thread #2, coroutine is already resumed + * // ... coroutine body after suspension point on Thread #2 ... + * coroutine.restoreThreadContext() // Thread #1, is invoked late because Thread #1 is slow + * coroutine.restoreThreadContext() // Thread #2, may happen in parallel with the previous restore + * ``` + * + * All implementations of [CopyableThreadContextElement] should be thread-safe and guard their internal mutable state + * within an element accordingly. + */ +@DelicateCoroutinesApi +@ExperimentalCoroutinesApi +public interface CopyableThreadContextElement : ThreadContextElement { + + /** + * Returns a [CopyableThreadContextElement] to replace `this` `CopyableThreadContextElement` in the child + * coroutine's context that is under construction if the added context does not contain an element with the same [key]. + * + * This function is called on the element each time a new coroutine inherits a context containing it, + * and the returned value is folded into the context given to the child. + * + * Since this method is called whenever a new coroutine is launched in a context containing this + * [CopyableThreadContextElement], implementations are performance-sensitive. + */ + public fun copyForChild(): CopyableThreadContextElement + + /** + * Returns a [CopyableThreadContextElement] to replace `this` `CopyableThreadContextElement` in the child + * coroutine's context that is under construction if the added context does contain an element with the same [key]. + * + * This method is invoked on the original element, accepting as the parameter + * the element that is supposed to overwrite it. + */ + public fun mergeForChild(overwritingElement: CoroutineContext.Element): CoroutineContext +} + +/** + * Wraps [ThreadLocal] into [ThreadContextElement]. The resulting [ThreadContextElement] + * maintains the given [value] of the given [ThreadLocal] for a coroutine regardless of the actual thread it is resumed on. + * By default [ThreadLocal.get] is used as a value for the thread-local variable, but it can be overridden with the [value] parameter. + * Beware that the context element **does not track** modifications of the thread-local and accessing thread-local from a coroutine + * without the corresponding context element returns an **undefined** value. See the examples for a detailed description. + * + * + * Example usage: + * ``` + * val myThreadLocal = ThreadLocal() + * ... + * println(myThreadLocal.get()) // Prints "null" + * launch(Dispatchers.Default + myThreadLocal.asContextElement(value = "foo")) { + * println(myThreadLocal.get()) // Prints "foo" + * withContext(Dispatchers.Main) { + * println(myThreadLocal.get()) // Prints "foo", but it's on UI thread + * } + * } + * println(myThreadLocal.get()) // Prints "null" + * ``` + * + * The context element does not track modifications of the thread-local variable, for example: + * + * ``` + * myThreadLocal.set("main") + * withContext(Dispatchers.Main) { + * println(myThreadLocal.get()) // Prints "main" + * myThreadLocal.set("UI") + * } + * println(myThreadLocal.get()) // Prints "main", not "UI" + * ``` + * + * Use `withContext` to update the corresponding thread-local variable to a different value, for example: + * ``` + * withContext(myThreadLocal.asContextElement("foo")) { + * println(myThreadLocal.get()) // Prints "foo" + * } + * ``` + * + * Accessing the thread-local without corresponding context element leads to undefined value: + * ``` + * val tl = ThreadLocal.withInitial { "initial" } + * + * runBlocking { + * println(tl.get()) // Will print "initial" + * // Change context + * withContext(tl.asContextElement("modified")) { + * println(tl.get()) // Will print "modified" + * } + * // Context is changed again + * println(tl.get()) // <- WARN: can print either "modified" or "initial" + * } + * ``` + * to fix this behaviour use `runBlocking(tl.asContextElement())` + */ +public fun ThreadLocal.asContextElement(value: T = get()): ThreadContextElement = + ThreadLocalElement(value, this) + +/** + * Return `true` when current thread local is present in the coroutine context, `false` otherwise. + * Thread local can be present in the context only if it was added via [asContextElement] to the context. + * + * Example of usage: + * ``` + * suspend fun processRequest() { + * if (traceCurrentRequestThreadLocal.isPresent()) { // Probabilistic tracing + * // Do some heavy-weight tracing + * } + * // Process request regularly + * } + * ``` + */ +public suspend inline fun ThreadLocal<*>.isPresent(): Boolean = coroutineContext[ThreadLocalKey(this)] !== null + +/** + * Checks whether current thread local is present in the coroutine context and throws [IllegalStateException] if it is not. + * It is a good practice to validate that thread local is present in the context, especially in large code-bases, + * to avoid stale thread-local values and to have a strict invariants. + * + * E.g. one may use the following method to enforce proper use of the thread locals with coroutines: + * ``` + * public suspend inline fun ThreadLocal.getSafely(): T { + * ensurePresent() + * return get() + * } + * + * // Usage + * withContext(...) { + * val value = threadLocal.getSafely() // Fail-fast in case of improper context + * } + * ``` + */ +public suspend inline fun ThreadLocal<*>.ensurePresent(): Unit = + check(isPresent()) { "ThreadLocal $this is missing from context $coroutineContext" } diff --git a/kotlinx-coroutines-core/jvm/src/ThreadPoolDispatcher.kt b/kotlinx-coroutines-core/jvm/src/ThreadPoolDispatcher.kt new file mode 100644 index 0000000000..151f598168 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/ThreadPoolDispatcher.kt @@ -0,0 +1,17 @@ +@file:JvmMultifileClass +@file:JvmName("ThreadPoolDispatcherKt") +package kotlinx.coroutines + +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicInteger + +@DelicateCoroutinesApi +public actual fun newFixedThreadPoolContext(nThreads: Int, name: String): ExecutorCoroutineDispatcher { + require(nThreads >= 1) { "Expected at least one thread, but $nThreads specified" } + val threadNo = AtomicInteger() + val executor = Executors.newScheduledThreadPool(nThreads) { runnable -> + Thread(runnable, if (nThreads == 1) name else name + "-" + threadNo.incrementAndGet()) + .apply { isDaemon = true } + } + return Executors.unconfigurableExecutorService(executor).asCoroutineDispatcher() +} diff --git a/kotlinx-coroutines-core/jvm/src/channels/Actor.kt b/kotlinx-coroutines-core/jvm/src/channels/Actor.kt new file mode 100644 index 0000000000..68ba76e47e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/channels/Actor.kt @@ -0,0 +1,198 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.intrinsics.* +import kotlinx.coroutines.selects.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +/** + * Scope for [actor][GlobalScope.actor] coroutine builder. + * + * **Note: This API will become obsolete in future updates with introduction of complex actors.** + * See [issue #87](https://github.com/Kotlin/kotlinx.coroutines/issues/87). + */ +@ObsoleteCoroutinesApi +public interface ActorScope : CoroutineScope, ReceiveChannel { + /** + * A reference to the mailbox channel that this coroutine [receives][receive] messages from. + * It is provided for convenience, so that the code in the coroutine can refer + * to the channel as `channel` as apposed to `this`. + * All the [ReceiveChannel] functions on this interface delegate to + * the channel instance returned by this function. + */ + public val channel: Channel +} + +/** + * Launches new coroutine that is receiving messages from its mailbox channel + * and returns a reference to its mailbox channel as a [SendChannel]. The resulting + * object can be used to [send][SendChannel.send] messages to this coroutine. + * + * The scope of the coroutine contains [ActorScope] interface, which implements + * both [CoroutineScope] and [ReceiveChannel], so that coroutine can invoke + * [receive][ReceiveChannel.receive] directly. The channel is [closed][SendChannel.close] + * when the coroutine completes. + * + * Coroutine context is inherited from a [CoroutineScope], additional context elements can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * The parent job is inherited from a [CoroutineScope] as well, but it can also be overridden + * with corresponding [context] element. + * + * By default, the coroutine is immediately scheduled for execution. + * Other options can be specified via `start` parameter. See [CoroutineStart] for details. + * An optional [start] parameter can be set to [CoroutineStart.LAZY] to start coroutine _lazily_. In this case, + * it will be started implicitly on the first message + * [sent][SendChannel.send] to this actors's mailbox channel. + * + * Uncaught exceptions in this coroutine close the channel with this exception as a cause, + * so that any attempt to send to such a channel throws exception. + * + * The kind of the resulting channel depends on the specified [capacity] parameter. + * See [Channel] interface documentation for details. + * + * See [newCoroutineContext][CoroutineScope.newCoroutineContext] for a description of debugging facilities that are available for newly created coroutine. + * + * ### Using actors + * + * A typical usage of the actor builder looks like this: + * + * ``` + * val c = actor { + * // initialize actor's state + * for (msg in channel) { + * // process message here + * } + * } + * // send messages to the actor + * c.send(...) + * ... + * // stop the actor when it is no longer needed + * c.close() + * ``` + * + * ### Stopping and cancelling actors + * + * When the inbox channel of the actor is [closed][SendChannel.close] it sends a special "close token" to the actor. + * The actor still processes all the messages that were already sent and then "`for (msg in channel)`" loop terminates + * and the actor completes. + * + * If the actor needs to be aborted without processing all the messages that were already sent to it, then + * it shall be created with a parent job: + * + * ``` + * val job = Job() + * val c = actor(context = job) { ... } + * ... + * // abort the actor + * job.cancel() + * ``` + * + * When actor's parent job is [cancelled][Job.cancel], then actor's job becomes cancelled. It means that + * "`for (msg in channel)`" and other cancellable suspending functions throw [CancellationException] and actor + * completes without processing remaining messages. + * + * **Note: This API will become obsolete in future updates with introduction of complex actors.** + * See [issue #87](https://github.com/Kotlin/kotlinx.coroutines/issues/87). + * + * @param context additional to [CoroutineScope.coroutineContext] context of the coroutine. + * @param capacity capacity of the channel's buffer (no buffer by default). + * @param start coroutine start option. The default value is [CoroutineStart.DEFAULT]. + * @param onCompletion optional completion handler for the actor coroutine (see [Job.invokeOnCompletion]) + * @param block the coroutine code. + */ +@ObsoleteCoroutinesApi +public fun CoroutineScope.actor( + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = 0, // todo: Maybe Channel.DEFAULT here? + start: CoroutineStart = CoroutineStart.DEFAULT, + onCompletion: CompletionHandler? = null, + block: suspend ActorScope.() -> Unit +): SendChannel { + val newContext = newCoroutineContext(context) + val channel = Channel(capacity) + val coroutine = if (start.isLazy) + LazyActorCoroutine(newContext, channel, block) else + ActorCoroutine(newContext, channel, active = true) + if (onCompletion != null) coroutine.invokeOnCompletion(handler = onCompletion) + coroutine.start(start, coroutine, block) + return coroutine +} + +@Suppress("MULTIPLE_DEFAULTS_INHERITED_FROM_SUPERTYPES_WHEN_NO_EXPLICIT_OVERRIDE_DEPRECATION_WARNING") +private open class ActorCoroutine( + parentContext: CoroutineContext, + channel: Channel, + active: Boolean +) : ChannelCoroutine(parentContext, channel, initParentJob = false, active = active), ActorScope { + + init { + initParentJob(parentContext[Job]) + } + + override fun onCancelling(cause: Throwable?) { + _channel.cancel(cause?.let { + it as? CancellationException ?: CancellationException("$classSimpleName was cancelled", it) + }) + } + + override fun handleJobException(exception: Throwable): Boolean { + handleCoroutineException(context, exception) + return true + } +} + +private class LazyActorCoroutine( + parentContext: CoroutineContext, + channel: Channel, + block: suspend ActorScope.() -> Unit +) : ActorCoroutine(parentContext, channel, active = false) { + + private var continuation = block.createCoroutineUnintercepted(this, this) + + override fun onStart() { + continuation.startCoroutineCancellable(this) + } + + override suspend fun send(element: E) { + start() + return super.send(element) + } + + @Suppress("DEPRECATION_ERROR") + @Deprecated( + level = DeprecationLevel.ERROR, + message = "Deprecated in the favour of 'trySend' method", + replaceWith = ReplaceWith("trySend(element).isSuccess") + ) // See super() + override fun offer(element: E): Boolean { + start() + return super.offer(element) + } + + override fun trySend(element: E): ChannelResult { + start() + return super.trySend(element) + } + + @Suppress("MULTIPLE_DEFAULTS_INHERITED_FROM_SUPERTYPES_DEPRECATION_WARNING") // do not remove the MULTIPLE_DEFAULTS suppression: required in K2 + override fun close(cause: Throwable?): Boolean { + // close the channel _first_ + val closed = super.close(cause) + // then start the coroutine (it will promptly fail if it was not started yet) + start() + return closed + } + + @Suppress("UNCHECKED_CAST") + override val onSend: SelectClause2> get() = SelectClause2Impl( + clauseObject = this, + regFunc = LazyActorCoroutine<*>::onSendRegFunction as RegistrationFunction, + processResFunc = super.onSend.processResFunc + ) + + private fun onSendRegFunction(select: SelectInstance<*>, element: Any?) { + onStart() + super.onSend.regFunc(this, select, element) + } +} diff --git a/kotlinx-coroutines-core/jvm/src/channels/TickerChannels.kt b/kotlinx-coroutines-core/jvm/src/channels/TickerChannels.kt new file mode 100644 index 0000000000..c07e68dc4e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/channels/TickerChannels.kt @@ -0,0 +1,107 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * Mode for [ticker] function. + * + * **Note: Ticker channels are not currently integrated with structured concurrency and their api will change in the future.** + */ +@ObsoleteCoroutinesApi +public enum class TickerMode { + /** + * Adjust delay to maintain fixed period if consumer cannot keep up or is otherwise slow. + * **This is a default mode.** + * + * ``` + * val channel = ticker(delay = 100) + * delay(350) // 250 ms late + * println(channel.tryReceive().getOrNull()) // prints Unit + * println(channel.tryReceive().getOrNull()) // prints null + * + * delay(50) + * println(channel.tryReceive().getOrNull()) // prints Unit, delay was adjusted + * delay(50) + * println(channel.tryReceive().getOrNull()) // prints null, we're not late relatively to previous element + * ``` + */ + FIXED_PERIOD, + + /** + * Maintains fixed delay between produced elements if consumer cannot keep up or it otherwise slow. + */ + FIXED_DELAY +} + +/** + * Creates a channel that produces the first item after the given initial delay and subsequent items with the + * given delay between them. + * + * The resulting channel is a _rendezvous channel_. When receiver from this channel does not keep + * up with receiving the elements from this channel, they are not being sent due to backpressure. The actual + * timing behavior of ticker in this case is controlled by [mode] parameter which + * is set to [TickerMode.FIXED_PERIOD] by default. See [TickerMode] for other details. + * + * This channel stops producing elements immediately after [ReceiveChannel.cancel] invocation. + * + * **Note** producer to this channel is dispatched via [Dispatchers.Unconfined] by default and started eagerly. + * + * **Note: Ticker channels are not currently integrated with structured concurrency and their api will change in the future.** + * + * @param delayMillis delay between each element in milliseconds. + * @param initialDelayMillis delay after which the first element will be produced (it is equal to [delayMillis] by default) in milliseconds. + * @param context context of the producing coroutine. + * @param mode specifies behavior when elements are not received ([FIXED_PERIOD][TickerMode.FIXED_PERIOD] by default). + */ +@ObsoleteCoroutinesApi +public fun ticker( + delayMillis: Long, + initialDelayMillis: Long = delayMillis, + context: CoroutineContext = EmptyCoroutineContext, + mode: TickerMode = TickerMode.FIXED_PERIOD +): ReceiveChannel { + require(delayMillis >= 0) { "Expected non-negative delay, but has $delayMillis ms" } + require(initialDelayMillis >= 0) { "Expected non-negative initial delay, but has $initialDelayMillis ms" } + return GlobalScope.produce(Dispatchers.Unconfined + context, capacity = 0) { + when (mode) { + TickerMode.FIXED_PERIOD -> fixedPeriodTicker(delayMillis, initialDelayMillis, channel) + TickerMode.FIXED_DELAY -> fixedDelayTicker(delayMillis, initialDelayMillis, channel) + } + } +} + +private suspend fun fixedPeriodTicker( + delayMillis: Long, + initialDelayMillis: Long, + channel: SendChannel +) { + var deadline = nanoTime() + delayToNanos(initialDelayMillis) + delay(initialDelayMillis) + val delayNs = delayToNanos(delayMillis) + while (true) { + deadline += delayNs + channel.send(Unit) + val now = nanoTime() + val nextDelay = (deadline - now).coerceAtLeast(0) + if (nextDelay == 0L && delayNs != 0L) { + val adjustedDelay = delayNs - (now - deadline) % delayNs + deadline = now + adjustedDelay + delay(delayNanosToMillis(adjustedDelay)) + } else { + delay(delayNanosToMillis(nextDelay)) + } + } +} + +private suspend fun fixedDelayTicker( + delayMillis: Long, + initialDelayMillis: Long, + channel: SendChannel +) { + delay(initialDelayMillis) + while (true) { + channel.send(Unit) + delay(delayMillis) + } +} diff --git a/kotlinx-coroutines-core/jvm/src/debug/CoroutineDebugging.kt b/kotlinx-coroutines-core/jvm/src/debug/CoroutineDebugging.kt new file mode 100644 index 0000000000..7029994276 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/CoroutineDebugging.kt @@ -0,0 +1,61 @@ +/* This package name is like this so that +1) the artificial stack frames look pretty, and +2) the IDE reliably navigates to this file. */ +package _COROUTINE + +/** + * A collection of artificial stack trace elements to be included in stack traces by the coroutines machinery. + * + * There are typically two ways in which one can encounter an artificial stack frame: + * 1. By using the debug mode, via the stacktrace recovery mechanism; see + * [stacktrace recovery](https://github.com/Kotlin/kotlinx.coroutines/blob/master/docs/topics/debugging.md#stacktrace-recovery) + * documentation. The usual way to enable the debug mode is with the [kotlinx.coroutines.DEBUG_PROPERTY_NAME] system + * property. + * 2. By looking at the output of DebugProbes; see the + * [kotlinx-coroutines-debug](https://github.com/Kotlin/kotlinx.coroutines/tree/master/kotlinx-coroutines-debug) module. + */ +internal class ArtificialStackFrames { + /** + * Returns an artificial stack trace element denoting the boundary between coroutine creation and its execution. + * + * Appearance of this function in stack traces does not mean that it was called. Instead, it is used as a marker + * that separates the part of the stack trace with the code executed in a coroutine from the stack trace of the code + * that launched the coroutine. + * + * In earlier versions of kotlinx-coroutines, this was displayed as "(Coroutine creation stacktrace)", which caused + * problems for tooling that processes stack traces: https://github.com/Kotlin/kotlinx.coroutines/issues/2291 + * + * Note that presence of this marker in a stack trace implies that coroutine creation stack traces were enabled. + */ + fun coroutineCreation(): StackTraceElement = Exception().artificialFrame(_CREATION::class.java.simpleName) + + /** + * Returns an artificial stack trace element denoting a coroutine boundary. + * + * Appearance of this function in stack traces does not mean that it was called. Instead, when one coroutine invokes + * another, this is used as a marker in the stack trace to denote where the execution of one coroutine ends and that + * of another begins. + * + * In earlier versions of kotlinx-coroutines, this was displayed as "(Coroutine boundary)", which caused + * problems for tooling that processes stack traces: https://github.com/Kotlin/kotlinx.coroutines/issues/2291 + */ + fun coroutineBoundary(): StackTraceElement = Exception().artificialFrame(_BOUNDARY::class.java.simpleName) +} + +// These are needed for the IDE navigation to detect that this file does contain the definition. +private class _CREATION +private class _BOUNDARY + +internal val ARTIFICIAL_FRAME_PACKAGE_NAME = "_COROUTINE" + +/** + * Forms an artificial stack frame with the given class name. + * + * It consists of the following parts: + * 1. The package name, it seems, is needed for the IDE to detect stack trace elements reliably. It is `_COROUTINE` since + * this is a valid identifier. + * 2. Class names represents what type of artificial frame this is. + * 3. The method name is `_`. The methods not being present in class definitions does not seem to affect navigation. + */ +private fun Throwable.artificialFrame(name: String): StackTraceElement = + with(stackTrace[0]) { StackTraceElement(ARTIFICIAL_FRAME_PACKAGE_NAME + "." + name, "_", fileName, lineNumber) } diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/AgentInstallationType.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/AgentInstallationType.kt new file mode 100644 index 0000000000..be37ff36e7 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/AgentInstallationType.kt @@ -0,0 +1,18 @@ +package kotlinx.coroutines.debug.internal + +/** + * Object used to differentiate between agent installed statically or dynamically. + * This is done in a separate object so [DebugProbesImpl] can check for static installation + * without having to depend on [AgentPremain], which is not compatible with Android. + * Otherwise, access to `AgentPremain.isInstalledStatically` triggers the load of its internal `ClassFileTransformer` + * that is not available on Android. + * + * The entity (despite being internal) has usages in the following products + * - Fleet (Reflection): FleetDebugProbes + * - Android (Hard Coded, ignored for Leak Detection) + * - IntelliJ (Suppress KotlinInternalInJava): CoroutineDumpState + */ +@PublishedApi +internal object AgentInstallationType { + internal var isInstalledStatically = false +} diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/AgentPremain.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/AgentPremain.kt new file mode 100644 index 0000000000..8d0c557ed2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/AgentPremain.kt @@ -0,0 +1,71 @@ +package kotlinx.coroutines.debug.internal + +import android.annotation.* +import org.codehaus.mojo.animal_sniffer.* +import sun.misc.* +import java.lang.instrument.* +import java.lang.instrument.ClassFileTransformer +import java.security.* + +/* + * This class is loaded if and only if kotlinx-coroutines-core was used as -javaagent argument, + * but Android complains anyway (java.lang.instrument.*), so we suppress all lint checks here + */ +@Suppress("unused") +@SuppressLint("all") +@IgnoreJRERequirement // Never touched on Android +internal object AgentPremain { + + private val enableCreationStackTraces = runCatching { + System.getProperty("kotlinx.coroutines.debug.enable.creation.stack.trace")?.toBoolean() + }.getOrNull() ?: DebugProbesImpl.enableCreationStackTraces + + @JvmStatic + @Suppress("UNUSED_PARAMETER") + fun premain(args: String?, instrumentation: Instrumentation) { + AgentInstallationType.isInstalledStatically = true + instrumentation.addTransformer(DebugProbesTransformer) + DebugProbesImpl.enableCreationStackTraces = enableCreationStackTraces + DebugProbesImpl.install() + installSignalHandler() + } + + internal object DebugProbesTransformer : ClassFileTransformer { + override fun transform( + loader: ClassLoader?, + className: String, + classBeingRedefined: Class<*>?, + protectionDomain: ProtectionDomain, + classfileBuffer: ByteArray? + ): ByteArray? { + if (loader == null || className != "kotlin/coroutines/jvm/internal/DebugProbesKt") { + return null + } + /* + * DebugProbesKt.bin contains `kotlin.coroutines.jvm.internal.DebugProbesKt` class + * with method bodies that delegate all calls directly to their counterparts in + * kotlinx.coroutines.debug.DebugProbesImpl. This is done to avoid classfile patching + * on the fly (-> get rid of ASM dependency). + * You can verify its content either by using javap on it or looking at out integration test module. + */ + AgentInstallationType.isInstalledStatically = true + return loader.getResourceAsStream("DebugProbesKt.bin").readBytes() + } + } + + private fun installSignalHandler() { + try { + Signal.handle(Signal("TRAP")) { // kill -5 + if (DebugProbesImpl.isInstalled) { + // Case with 'isInstalled' changed between this check-and-act is not considered + // a real debug probes use-case, thus is not guarded against. + DebugProbesImpl.dumpCoroutines(System.out) + } else { + println("Cannot perform coroutines dump, debug probes are disabled") + } + } + } catch (t: Throwable) { + // Do nothing, signal cannot be installed, e.g. because we are on Windows + } + } +} diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/ConcurrentWeakMap.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/ConcurrentWeakMap.kt new file mode 100644 index 0000000000..f28d9a6d42 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/ConcurrentWeakMap.kt @@ -0,0 +1,279 @@ +package kotlinx.coroutines.debug.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.internal.* +import java.lang.ref.* + +// This is very limited implementation, not suitable as a generic map replacement. +// It has lock-free get and put with synchronized rehash for simplicity (and better CPU usage on contention) +@Suppress("UNCHECKED_CAST") +internal class ConcurrentWeakMap( + /** + * Weak reference queue is needed when a small key is mapped to a large value, and we need to promptly release a + * reference to the value when the key was already disposed. + */ + weakRefQueue: Boolean = false +) : AbstractMutableMap() { + private val _size = atomic(0) + private val core = atomic(Core(MIN_CAPACITY)) + private val weakRefQueue: ReferenceQueue? = if (weakRefQueue) ReferenceQueue() else null + + override val size: Int + get() = _size.value + + private fun decrementSize() { _size.decrementAndGet() } + + override fun get(key: K): V? = core.value.getImpl(key) + + override fun put(key: K, value: V): V? { + var oldValue = core.value.putImpl(key, value) + if (oldValue === REHASH) oldValue = putSynchronized(key, value) + if (oldValue == null) _size.incrementAndGet() + return oldValue as V? + } + + override fun remove(key: K): V? { + var oldValue = core.value.putImpl(key, null) + if (oldValue === REHASH) oldValue = putSynchronized(key, null) + if (oldValue != null) _size.decrementAndGet() + return oldValue as V? + } + + @Synchronized + private fun putSynchronized(key: K, value: V?): V? { + // Note: concurrent put leaves chance that we fail to put even after rehash, we retry until successful + var curCore = core.value + while (true) { + val oldValue = curCore.putImpl(key, value) + if (oldValue !== REHASH) return oldValue as V? + curCore = curCore.rehash() + core.value = curCore + } + } + + override val keys: MutableSet + get() = KeyValueSet { k, _ -> k } + + override val entries: MutableSet> + get() = KeyValueSet { k, v -> Entry(k, v) } + + // We don't care much about clear's efficiency + override fun clear() { + for (k in keys) remove(k) + } + + fun runWeakRefQueueCleaningLoopUntilInterrupted() { + check(weakRefQueue != null) { "Must be created with weakRefQueue = true" } + try { + while (true) { + cleanWeakRef(weakRefQueue.remove() as HashedWeakRef<*>) + } + } catch (e: InterruptedException) { + Thread.currentThread().interrupt() + } + } + + private fun cleanWeakRef(w: HashedWeakRef<*>) { + core.value.cleanWeakRef(w) + } + + @Suppress("UNCHECKED_CAST") + private inner class Core(private val allocated: Int) { + private val shift = allocated.countLeadingZeroBits() + 1 + private val threshold = 2 * allocated / 3 // max fill factor at 66% to ensure speedy lookups + private val load = atomic(0) // counts how many slots are occupied in this core + private val keys = atomicArrayOfNulls?>(allocated) + private val values = atomicArrayOfNulls(allocated) + + private fun index(hash: Int) = (hash * MAGIC) ushr shift + + // get is always lock-free, unwraps the value that was marked by concurrent rehash + fun getImpl(key: K): V? { + var index = index(key.hashCode()) + while (true) { + val w = keys[index].value ?: return null // not found + val k = w.get() + if (key == k) { + val value = values[index].value + return (if (value is Marked) value.ref else value) as V? + } + if (k == null) removeCleanedAt(index) // weak ref was here, but collected + if (index == 0) index = allocated + index-- + } + } + + private fun removeCleanedAt(index: Int) { + while (true) { + val oldValue = values[index].value ?: return // return when already removed + if (oldValue is Marked) return // cannot remove marked (rehash is working on it, will not copy) + if (values[index].compareAndSet(oldValue, null)) { // removed + decrementSize() + return + } + } + } + + // returns REHASH when rehash is needed (the value was not put) + fun putImpl(key: K, value: V?, weakKey0: HashedWeakRef? = null): Any? { + var index = index(key.hashCode()) + var loadIncremented = false + var weakKey: HashedWeakRef? = weakKey0 + while (true) { + val w = keys[index].value + if (w == null) { // slot empty => not found => try reserving slot + if (value == null) return null // removing missing value, nothing to do here + if (!loadIncremented) { + // We must increment load before we even try to occupy a slot to avoid overfill during concurrent put + load.update { n -> + if (n >= threshold) return REHASH // the load is already too big -- rehash + n + 1 // otherwise increment + } + loadIncremented = true + } + if (weakKey == null) weakKey = HashedWeakRef(key, weakRefQueue) + if (keys[index].compareAndSet(null, weakKey)) break // slot reserved !!! + continue // retry at this slot on CAS failure (somebody already reserved this slot) + } + val k = w.get() + if (key == k) { // found already reserved slot at index + if (loadIncremented) load.decrementAndGet() // undo increment, because found a slot + break + } + if (k == null) removeCleanedAt(index) // weak ref was here, but collected + if (index == 0) index = allocated + index-- + } + // update value + var oldValue: Any? + while (true) { + oldValue = values[index].value + if (oldValue is Marked) return REHASH // rehash started, cannot work here + if (values[index].compareAndSet(oldValue, value)) break + } + return oldValue as V? + } + + // only one thread can rehash, but may have concurrent puts/gets + fun rehash(): Core { + // use size to approximate new required capacity to have at least 25-50% fill factor, + // may fail due to concurrent modification, will retry + retry@while (true) { + val newCapacity = size.coerceAtLeast(MIN_CAPACITY / 4).takeHighestOneBit() * 4 + val newCore = Core(newCapacity) + for (index in 0 until allocated) { + // load the key + val w = keys[index].value + val k = w?.get() + if (w != null && k == null) removeCleanedAt(index) // weak ref was here, but collected + // mark value so that it cannot be changed while we rehash to new core + var value: Any? + while (true) { + value = values[index].value + if (value is Marked) { // already marked -- good + value = value.ref + break + } + // try mark + if (values[index].compareAndSet(value, value.mark())) break + } + if (k != null && value != null) { + val oldValue = newCore.putImpl(k, value as V, w) + if (oldValue === REHASH) continue@retry // retry if we underestimated capacity + assert(oldValue == null) + } + } + return newCore // rehashed everything successfully + } + } + + fun cleanWeakRef(weakRef: HashedWeakRef<*>) { + var index = index(weakRef.hash) + while (true) { + val w = keys[index].value ?: return // return when slots are over + if (w === weakRef) { // found + removeCleanedAt(index) + return + } + if (index == 0) index = allocated + index-- + } + } + + fun keyValueIterator(factory: (K, V) -> E): MutableIterator = KeyValueIterator(factory) + + private inner class KeyValueIterator(private val factory: (K, V) -> E) : MutableIterator { + private var index = -1 + private lateinit var key: K + private lateinit var value: V + + init { findNext() } + + private fun findNext() { + while (++index < allocated) { + key = keys[index].value?.get() ?: continue + var value = values[index].value + if (value is Marked) value = value.ref + if (value != null) { + this.value = value as V + return + } + } + } + + override fun hasNext(): Boolean = index < allocated + + override fun next(): E { + if (index >= allocated) throw NoSuchElementException() + return factory(key, value).also { findNext() } + } + + override fun remove() = noImpl() + } + } + + private class Entry(override val key: K, override val value: V) : MutableMap.MutableEntry { + override fun setValue(newValue: V): V = noImpl() + } + + private inner class KeyValueSet( + private val factory: (K, V) -> E + ) : AbstractMutableSet() { + override val size: Int get() = this@ConcurrentWeakMap.size + override fun add(element: E): Boolean = noImpl() + override fun iterator(): MutableIterator = core.value.keyValueIterator(factory) + } +} + +private const val MAGIC = 2654435769L.toInt() // golden ratio +private const val MIN_CAPACITY = 16 +private val REHASH = Symbol("REHASH") +private val MARKED_NULL = Marked(null) +private val MARKED_TRUE = Marked(true) // When using map as set "true" used as value, optimize its mark allocation + +/** + * Weak reference that stores the original hash code so that we can use reference queue to promptly clean them up + * from the hashtable even in the absence of ongoing modifications. + */ +internal class HashedWeakRef( + ref: T, queue: ReferenceQueue? +) : WeakReference(ref, queue) { + @JvmField + val hash = ref.hashCode() +} + +/** + * Marked values cannot be modified. The marking is performed when rehash has started to ensure that concurrent + * modifications (that are lock-free) cannot perform any changes and are forced to synchronize with ongoing rehash. + */ +private class Marked(@JvmField val ref: Any?) + +private fun Any?.mark(): Marked = when(this) { + null -> MARKED_NULL + true -> MARKED_TRUE + else -> Marked(this) +} + +private fun noImpl(): Nothing { + throw UnsupportedOperationException("not implemented") +} diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfo.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfo.kt new file mode 100644 index 0000000000..91a4cadc59 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfo.kt @@ -0,0 +1,26 @@ +package kotlinx.coroutines.debug.internal + +import kotlin.coroutines.* +import kotlin.coroutines.jvm.internal.* + +/** + * This class represents the data required by IDEA debugger. + * IDEA debugger either directly reads data from the corresponding JVM fields of this class or calls the getters, + * so we keep both for maximal flexibility for now. + * **DO NOT MAKE BINARY-INCOMPATIBLE CHANGES TO THIS CLASS**. + */ +@Suppress("unused") +@PublishedApi +internal class DebugCoroutineInfo internal constructor( + source: DebugCoroutineInfoImpl, + public val context: CoroutineContext // field is used as of 1.4-M3 +) { + internal val creationStackBottom: CoroutineStackFrame? = source.creationStackBottom // field is used as of 1.4-M3 + public val sequenceNumber: Long = source.sequenceNumber // field is used as of 1.4-M3 + public val creationStackTrace = source.creationStackTrace // getter is used as of 1.4-M3 + public val state: String = source.state // getter is used as of 1.4-M3 + public val lastObservedThread: Thread? = source.lastObservedThread // field is used as of 1.4-M3 + public val lastObservedFrame: CoroutineStackFrame? = source.lastObservedFrame // field is used as of 1.4-M3 + @get:JvmName("lastObservedStackTrace") // method with this name is used as of 1.4-M3 + public val lastObservedStackTrace: List = source.lastObservedStackTrace() +} diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfoImpl.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfoImpl.kt new file mode 100644 index 0000000000..47d69363c8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugCoroutineInfoImpl.kt @@ -0,0 +1,177 @@ +package kotlinx.coroutines.debug.internal + +import java.lang.ref.* +import kotlin.coroutines.* +import kotlin.coroutines.jvm.internal.* + +internal const val CREATED = "CREATED" +internal const val RUNNING = "RUNNING" +internal const val SUSPENDED = "SUSPENDED" + +/** + * Internal implementation class where debugger tracks details it knows about each coroutine. + * Its mutable fields can be updated concurrently, thus marked with `@Volatile` + */ +@PublishedApi +internal class DebugCoroutineInfoImpl internal constructor( + context: CoroutineContext?, + /** + * A reference to a stack-trace that is converted to a [StackTraceFrame] which implements [CoroutineStackFrame]. + * The actual reference to the coroutine is not stored here, so we keep a strong reference. + */ + internal val creationStackBottom: StackTraceFrame?, + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @JvmField public val sequenceNumber: Long +) { + /** + * We cannot keep a strong reference to the context, because with the [Job] in the context it will indirectly + * keep a reference to the last frame of an abandoned coroutine which the debugger should not be preventing + * garbage-collection of. The reference to context will not disappear as long as the coroutine itself is not lost. + */ + private val _context = WeakReference(context) + public val context: CoroutineContext? // can be null when the coroutine was already garbage-collected + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + get() = _context.get() + + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + public val creationStackTrace: List get() = creationStackTrace() + + /** + * Last observed state of the coroutine. + * Can be CREATED, RUNNING, SUSPENDED. + */ + internal val state: String get() = _state + + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @Volatile + @JvmField + public var _state: String = CREATED + + /* + * How many consecutive unmatched 'updateState(RESUMED)' this object has received. + * It can be `> 1` in two cases: + * + * - The coroutine is finishing and its state is being unrolled in BaseContinuationImpl, see comment to DebugProbesImpl#callerInfoCache + * Such resumes are not expected to be matched and are ignored. + * - We encountered suspend-resume race explained above, and we do wait for a match. + */ + private var unmatchedResume = 0 + + /** + * Here we orchestrate overlapping state updates that are coming asynchronously. + * In a nutshell, `probeCoroutineSuspended` can arrive **later** than its matching `probeCoroutineResumed`, + * e.g. for the following code: + * ``` + * suspend fun foo() = yield() + * ``` + * + * we have this sequence: + * ``` + * fun foo(...) { + * uCont.intercepted().dispatchUsingDispatcher() // 1 + * // Notify the debugger the coroutine is suspended + * probeCoroutineSuspended() // 2 + * return COROUTINE_SUSPENDED // Unroll the stack + * } + * ``` + * Nothing prevents coroutine to be dispatched and invoke `probeCoroutineResumed` right between '1' and '2'. + * See also: https://github.com/Kotlin/kotlinx.coroutines/issues/3193 + * + * [shouldBeMatched] -- `false` if it is an expected consecutive `probeCoroutineResumed` from BaseContinuationImpl, + * `true` otherwise. + */ + @Synchronized + internal fun updateState(state: String, frame: Continuation<*>, shouldBeMatched: Boolean) { + /** + * We observe consecutive resume that had to be matched, but it wasn't, + * increment + */ + if (_state == RUNNING && state == RUNNING && shouldBeMatched) { + ++unmatchedResume + } else if (unmatchedResume > 0 && state == SUSPENDED) { + /* + * We received late 'suspend' probe for unmatched resume, skip it. + * Here we deliberately allow the very unlikely race; + * Consider the following scenario ('[r:a]' means "probeCoroutineResumed at a()"): + * ``` + * [r:a] a() -> b() [s:b] [r:b] -> (back to a) a() -> c() [s:c] + * ``` + * We can, in theory, observe the following probes interleaving: + * ``` + * r:a + * r:b // Unmatched resume + * s:c // Matched suspend, discard + * s:b + * ``` + * Thus mis-attributing 'lastObservedFrame' to a previously-observed. + * It is possible in theory (though I've failed to reproduce it), yet + * is more preferred than indefinitely mismatched state (-> mismatched real/enhanced stacktrace) + */ + --unmatchedResume + return + } + + // Propagate only non-duplicating transitions to running, see KT-29997 + if (_state == state && state == SUSPENDED && lastObservedFrame != null) return + + _state = state + lastObservedFrame = frame as? CoroutineStackFrame + lastObservedThread = if (state == RUNNING) { + Thread.currentThread() + } else { + null + } + } + + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @JvmField + @Volatile + public var lastObservedThread: Thread? = null + + /** + * We cannot keep a strong reference to the last observed frame of the coroutine, because this will + * prevent garbage-collection of a coroutine that was lost. + * + * Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + */ + @Volatile + @JvmField + public var _lastObservedFrame: WeakReference? = null + internal var lastObservedFrame: CoroutineStackFrame? + get() = _lastObservedFrame?.get() + set(value) { + _lastObservedFrame = value?.let { WeakReference(it) } + } + + /** + * Last observed stacktrace of the coroutine captured on its suspension or resumption point. + * It means that for [running][State.RUNNING] coroutines resulting stacktrace is inaccurate and + * reflects stacktrace of the resumption point, not the actual current stacktrace. + */ + internal fun lastObservedStackTrace(): List { + var frame: CoroutineStackFrame? = lastObservedFrame ?: return emptyList() + val result = ArrayList() + while (frame != null) { + frame.getStackTraceElement()?.let { result.add(it) } + frame = frame.callerFrame + } + return result + } + + private fun creationStackTrace(): List { + val bottom = creationStackBottom ?: return emptyList() + // Skip "Coroutine creation stacktrace" frame + return sequence { yieldFrames(bottom.callerFrame) }.toList() + } + + private tailrec suspend fun SequenceScope.yieldFrames(frame: CoroutineStackFrame?) { + if (frame == null) return + frame.getStackTraceElement()?.let { yield(it) } + val caller = frame.callerFrame + if (caller != null) { + yieldFrames(caller) + } + } + + override fun toString(): String = "DebugCoroutineInfo(state=$state,context=$context)" +} diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbes.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbes.kt new file mode 100644 index 0000000000..672990e16e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbes.kt @@ -0,0 +1,18 @@ +@file:Suppress("unused") + +package kotlinx.coroutines.debug.internal + +import kotlin.coroutines.* + +/* + * This class is used by ByteBuddy from kotlinx-coroutines-debug as kotlin.coroutines.jvm.internal.DebugProbesKt replacement. + * In theory, it should belong to kotlinx-coroutines-debug, but placing it here significantly simplifies the + * Android AS debugger that does on-load DEX transformation + */ + +// Stubs which are injected as coroutine probes. Require direct match of signatures +internal fun probeCoroutineResumed(frame: Continuation<*>) = DebugProbesImpl.probeCoroutineResumed(frame) + +internal fun probeCoroutineSuspended(frame: Continuation<*>) = DebugProbesImpl.probeCoroutineSuspended(frame) +internal fun probeCoroutineCreated(completion: Continuation): Continuation = + DebugProbesImpl.probeCoroutineCreated(completion) diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbesImpl.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbesImpl.kt new file mode 100644 index 0000000000..25594ad01a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/DebugProbesImpl.kt @@ -0,0 +1,615 @@ +package kotlinx.coroutines.debug.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.ScopeCoroutine +import java.io.* +import java.lang.StackTraceElement +import java.text.* +import java.util.concurrent.locks.* +import kotlin.collections.ArrayList +import kotlin.concurrent.* +import kotlin.coroutines.* +import kotlin.coroutines.jvm.internal.CoroutineStackFrame +import kotlin.synchronized +import _COROUTINE.ArtificialStackFrames + +@PublishedApi +internal object DebugProbesImpl { + private val ARTIFICIAL_FRAME = ArtificialStackFrames().coroutineCreation() + private val dateFormat = SimpleDateFormat("yyyy/MM/dd HH:mm:ss") + + private var weakRefCleanerThread: Thread? = null + + // Values are boolean, so this map does not need to use a weak reference queue + private val capturedCoroutinesMap = ConcurrentWeakMap, Boolean>() + private val capturedCoroutines: Set> get() = capturedCoroutinesMap.keys + + private val installations = atomic(0) + + /** + * This internal method is used by the IDEA debugger under the JVM name + * "isInstalled$kotlinx_coroutines_debug" and must be kept binary-compatible, see KTIJ-24102 + */ + val isInstalled: Boolean + // IDEA depended on "internal val isInstalled", thus the mangling. Public + JvmName in order to make this getter part of the ABI + @JvmName("isInstalled\$kotlinx_coroutines_debug") + get() = installations.value > 0 + + // To sort coroutines by creation order, used as a unique id + private val sequenceNumber = atomic(0L) + + internal var sanitizeStackTraces: Boolean = true + internal var enableCreationStackTraces: Boolean = false + public var ignoreCoroutinesWithEmptyContext: Boolean = true + + /* + * Substitute for service loader, DI between core and debug modules. + * If the agent was installed via command line -javaagent parameter, do not use byte-buddy to avoid dynamic attach. + */ + private val dynamicAttach = getDynamicAttach() + + @Suppress("UNCHECKED_CAST") + private fun getDynamicAttach(): Function1? = runCatching { + val clz = Class.forName("kotlinx.coroutines.debug.ByteBuddyDynamicAttach") + val ctor = clz.constructors[0] + ctor.newInstance() as Function1 + }.getOrNull() + + /** + * Because `probeCoroutinesResumed` is called for every resumed continuation (see KT-29997 and the related code), + * we perform a performance optimization: + * Imagine a suspending call stack a()->b()->c(), where c() completes its execution and every call is + * "almost" in tail position. + * + * Then at least three RUNNING -> RUNNING transitions will occur consecutively, the complexity of each O(depth). + * To avoid this quadratic complexity, we are caching lookup result for such chains in this map and update it incrementally. + * + * [DebugCoroutineInfoImpl] keeps a lot of auxiliary information about a coroutine, so we use a weak reference queue + * to promptly release the corresponding memory when the reference to the coroutine itself was already collected. + */ + private val callerInfoCache = ConcurrentWeakMap(weakRefQueue = true) + + internal fun install() { + if (installations.incrementAndGet() > 1) return + startWeakRefCleanerThread() + if (AgentInstallationType.isInstalledStatically) return + dynamicAttach?.invoke(true) // attach + } + + internal fun uninstall() { + check(isInstalled) { "Agent was not installed" } + if (installations.decrementAndGet() != 0) return + stopWeakRefCleanerThread() + capturedCoroutinesMap.clear() + callerInfoCache.clear() + if (AgentInstallationType.isInstalledStatically) return + dynamicAttach?.invoke(false) // detach + } + + private fun startWeakRefCleanerThread() { + weakRefCleanerThread = thread(isDaemon = true, name = "Coroutines Debugger Cleaner") { + callerInfoCache.runWeakRefQueueCleaningLoopUntilInterrupted() + } + } + + private fun stopWeakRefCleanerThread() { + val thread = weakRefCleanerThread ?: return + weakRefCleanerThread = null + thread.interrupt() + thread.join() + } + + internal fun hierarchyToString(job: Job): String { + check(isInstalled) { "Debug probes are not installed" } + val jobToStack = capturedCoroutines + .filter { it.delegate.context[Job] != null } + .associateBy({ it.delegate.context.job }, { it.info }) + return buildString { + job.build(jobToStack, this, "") + } + } + + private fun Job.build(map: Map, builder: StringBuilder, indent: String) { + val info = map[this] + val newIndent: String + if (info == null) { // Append coroutine without stacktrace + // Do not print scoped coroutines and do not increase indentation level + @Suppress("INVISIBLE_REFERENCE") + if (this !is ScopeCoroutine<*>) { + builder.append("$indent$debugString\n") + newIndent = indent + "\t" + } else { + newIndent = indent + } + } else { + // Append coroutine with its last stacktrace element + val element = info.lastObservedStackTrace().firstOrNull() + val state = info.state + builder.append("$indent$debugString, continuation is $state at line $element\n") + newIndent = indent + "\t" + } + // Append children with new indent + for (child in children) { + child.build(map, builder, newIndent) + } + } + + @Suppress("DEPRECATION_ERROR") // JobSupport + private val Job.debugString: String get() = if (this is JobSupport) toDebugString() else toString() + + /** + * Private method that dumps coroutines so that different public-facing method can use + * to produce different result types. + */ + private inline fun dumpCoroutinesInfoImpl(crossinline create: (CoroutineOwner<*>, CoroutineContext) -> R): List { + check(isInstalled) { "Debug probes are not installed" } + return capturedCoroutines + .asSequence() + // Stable ordering of coroutines by their sequence number + .sortedBy { it.info.sequenceNumber } + // Leave in the dump only the coroutines that were not collected while we were dumping them + .mapNotNull { owner -> + // Fuse map and filter into one operation to save an inline + if (owner.isFinished()) null + else owner.info.context?.let { context -> create(owner, context) } + }.toList() + } + + /* + * This method optimises the number of packages sent by the IDEA debugger + * to a client VM to speed up fetching of coroutine information. + * + * The return value is an array of objects, which consists of four elements: + * 1) A string in a JSON format that stores information that is needed to display + * every coroutine in the coroutine panel in the IDEA debugger. + * 2) An array of last observed threads. + * 3) An array of last observed frames. + * 4) An array of DebugCoroutineInfo. + * + * ### Implementation note + * For methods like `dumpCoroutinesInfo` JDWP provides `com.sun.jdi.ObjectReference` + * that does a roundtrip to client VM for *each* field or property read. + * To avoid that, we serialize most of the critical for UI data into a primitives + * to save an exponential number of roundtrips. + * + * Internal (JVM-public) method used by IDEA debugger as of 1.6.0-RC. + * See KTIJ-24102. + */ + @OptIn(ExperimentalStdlibApi::class) + fun dumpCoroutinesInfoAsJsonAndReferences(): Array { + val coroutinesInfo = dumpCoroutinesInfo() + val size = coroutinesInfo.size + val lastObservedThreads = ArrayList(size) + val lastObservedFrames = ArrayList(size) + val coroutinesInfoAsJson = ArrayList(size) + for (info in coroutinesInfo) { + val context = info.context + val name = context[CoroutineName.Key]?.name?.toStringRepr() + val dispatcher = context[CoroutineDispatcher.Key]?.toStringRepr() + coroutinesInfoAsJson.add( + """ + { + "name": $name, + "id": ${context[CoroutineId.Key]?.id}, + "dispatcher": $dispatcher, + "sequenceNumber": ${info.sequenceNumber}, + "state": "${info.state}" + } + """.trimIndent() + ) + lastObservedFrames.add(info.lastObservedFrame) + lastObservedThreads.add(info.lastObservedThread) + } + + return arrayOf( + "[${coroutinesInfoAsJson.joinToString()}]", + lastObservedThreads.toTypedArray(), + lastObservedFrames.toTypedArray(), + coroutinesInfo.toTypedArray() + ) + } + + /* + * Internal (JVM-public) method used by IDEA debugger as of 1.6.0-RC, must be kept binary-compatible, see KTIJ-24102 + */ + fun enhanceStackTraceWithThreadDumpAsJson(info: DebugCoroutineInfo): String { + val stackTraceElements = enhanceStackTraceWithThreadDump(info, info.lastObservedStackTrace) + val stackTraceElementsInfoAsJson = mutableListOf() + for (element in stackTraceElements) { + stackTraceElementsInfoAsJson.add( + """ + { + "declaringClass": "${element.className}", + "methodName": "${element.methodName}", + "fileName": ${element.fileName?.toStringRepr()}, + "lineNumber": ${element.lineNumber} + } + """.trimIndent() + ) + } + + return "[${stackTraceElementsInfoAsJson.joinToString()}]" + } + + private fun Any.toStringRepr() = toString().repr() + + /* + * Internal (JVM-public) method used by IDEA debugger as of 1.4-M3. See KTIJ-24102 + */ + fun dumpCoroutinesInfo(): List = + dumpCoroutinesInfoImpl { owner, context -> DebugCoroutineInfo(owner.info, context) } + + /* + * Internal (JVM-public) method to be used by IDEA debugger in the future (not used as of 1.4-M3). + * It is equivalent to [dumpCoroutinesInfo], but returns serializable (and thus less typed) objects. + */ + fun dumpDebuggerInfo(): List = + dumpCoroutinesInfoImpl { owner, context -> DebuggerInfo(owner.info, context) } + + @JvmName("dumpCoroutines") + internal fun dumpCoroutines(out: PrintStream): Unit = synchronized(out) { + /* + * This method synchronizes both on `out` and `this` for a reason: + * 1) Taking a write lock is required to have a consistent snapshot of coroutines. + * 2) Synchronization on `out` is not required, but prohibits interleaving with any other + * (asynchronous) attempt to write to this `out` (System.out by default). + * Yet this prevents the progress of coroutines until they are fully dumped to the out which we find acceptable compromise. + */ + dumpCoroutinesSynchronized(out) + } + + /* + * Filters out coroutines that do not call probeCoroutineCompleted, + * are completed, but not yet garbage collected. + * + * Typically, we intercept completion of the coroutine so it invokes "probeCoroutineCompleted", + * but it's not the case for lazy coroutines that get cancelled before start. + */ + private fun CoroutineOwner<*>.isFinished(): Boolean { + // Guarded by lock + val job = info.context?.get(Job) ?: return false + if (!job.isCompleted) return false + capturedCoroutinesMap.remove(this) // Clean it up by the way + return true + } + + private fun dumpCoroutinesSynchronized(out: PrintStream) { + check(isInstalled) { "Debug probes are not installed" } + out.print("Coroutines dump ${dateFormat.format(System.currentTimeMillis())}") + capturedCoroutines + .asSequence() + .filter { !it.isFinished() } + .sortedBy { it.info.sequenceNumber } + .forEach { owner -> + val info = owner.info + val observedStackTrace = info.lastObservedStackTrace() + val enhancedStackTrace = enhanceStackTraceWithThreadDumpImpl(info.state, info.lastObservedThread, observedStackTrace) + val state = if (info.state == RUNNING && enhancedStackTrace === observedStackTrace) + "${info.state} (Last suspension stacktrace, not an actual stacktrace)" + else + info.state + out.print("\n\nCoroutine ${owner.delegate}, state: $state") + if (observedStackTrace.isEmpty()) { + out.print("\n\tat $ARTIFICIAL_FRAME") + printStackTrace(out, info.creationStackTrace) + } else { + printStackTrace(out, enhancedStackTrace) + } + } + } + + private fun printStackTrace(out: PrintStream, frames: List) { + frames.forEach { frame -> + out.print("\n\tat $frame") + } + } + + /* + * Internal (JVM-public) method used by IDEA debugger as of 1.4-M3, must be kept binary-compatible. See KTIJ-24102. + * It is similar to [enhanceStackTraceWithThreadDumpImpl], but uses debugger-facing [DebugCoroutineInfo] type. + */ + @Suppress("unused") + fun enhanceStackTraceWithThreadDump( + info: DebugCoroutineInfo, + coroutineTrace: List + ): List = + enhanceStackTraceWithThreadDumpImpl(info.state, info.lastObservedThread, coroutineTrace) + + /** + * Tries to enhance [coroutineTrace] (obtained by call to [DebugCoroutineInfoImpl.lastObservedStackTrace]) with + * thread dump of [DebugCoroutineInfoImpl.lastObservedThread]. + * + * Returns [coroutineTrace] if enhancement was unsuccessful or the enhancement result. + */ + private fun enhanceStackTraceWithThreadDumpImpl( + state: String, + thread: Thread?, + coroutineTrace: List + ): List { + if (state != RUNNING || thread == null) return coroutineTrace + // Avoid security manager issues + val actualTrace = runCatching { thread.stackTrace }.getOrNull() + ?: return coroutineTrace + + /* + * Here goes heuristic that tries to merge two stacktraces: real one + * (that has at least one but usually not so many suspend function frames) + * and coroutine one that has only suspend function frames. + * + * Heuristic: + * 1) Dump lastObservedThread + * 2) Find the next frame after BaseContinuationImpl.resumeWith (continuation machinery). + * Invariant: this method is called under the lock, so such method **should** be present + * in continuation stacktrace. + * 3) Find target method in continuation stacktrace (metadata-based) + * 4) Prepend dumped stacktrace (trimmed by target frame) to continuation stacktrace + * + * Heuristic may fail on recursion and overloads, but it will be automatically improved + * with KT-29997. + */ + val indexOfResumeWith = actualTrace.indexOfFirst { + it.className == "kotlin.coroutines.jvm.internal.BaseContinuationImpl" && + it.methodName == "resumeWith" && + it.fileName == "ContinuationImpl.kt" + } + + val (continuationStartFrame, delta) = findContinuationStartIndex( + indexOfResumeWith, + actualTrace, + coroutineTrace + ) + + if (continuationStartFrame == -1) return coroutineTrace + + val expectedSize = indexOfResumeWith + coroutineTrace.size - continuationStartFrame - 1 - delta + val result = ArrayList(expectedSize) + for (index in 0 until indexOfResumeWith - delta) { + result += actualTrace[index] + } + + for (index in continuationStartFrame + 1 until coroutineTrace.size) { + result += coroutineTrace[index] + } + + return result + } + + /** + * Tries to find the lowest meaningful frame above `resumeWith` in the real stacktrace and + * its match in a coroutines stacktrace (steps 2-3 in heuristic). + * + * This method does more than just matching `realTrace.indexOf(resumeWith) - 1`: + * If method above `resumeWith` has no line number (thus it is `stateMachine.invokeSuspend`), + * it's skipped and attempt to match next one is made because state machine could have been missing in the original coroutine stacktrace. + * + * Returns index of such frame (or -1) and number of skipped frames (up to 2, for state machine and for access$). + */ + private fun findContinuationStartIndex( + indexOfResumeWith: Int, + actualTrace: Array, + coroutineTrace: List + ): Pair { + /* + * Since Kotlin 1.5.0 we have these access$ methods that we have to skip. + * So we have to test next frame for invokeSuspend, for $access and for actual suspending call. + */ + repeat(3) { + val result = findIndexOfFrame(indexOfResumeWith - 1 - it, actualTrace, coroutineTrace) + if (result != -1) return result to it + } + return -1 to 0 + } + + private fun findIndexOfFrame( + frameIndex: Int, + actualTrace: Array, + coroutineTrace: List + ): Int { + val continuationFrame = actualTrace.getOrNull(frameIndex) + ?: return -1 + + return coroutineTrace.indexOfFirst { + it.fileName == continuationFrame.fileName && + it.className == continuationFrame.className && + it.methodName == continuationFrame.methodName + } + } + + internal fun probeCoroutineResumed(frame: Continuation<*>) = updateState(frame, RUNNING) + + internal fun probeCoroutineSuspended(frame: Continuation<*>) = updateState(frame, SUSPENDED) + + private fun updateState(frame: Continuation<*>, state: String) { + if (!isInstalled) return + if (ignoreCoroutinesWithEmptyContext && frame.context === EmptyCoroutineContext) return // See ignoreCoroutinesWithEmptyContext + if (state == RUNNING) { + val stackFrame = frame as? CoroutineStackFrame ?: return + updateRunningState(stackFrame, state) + return + } + + // Find ArtificialStackFrame of the coroutine + val owner = frame.owner() ?: return + updateState(owner, frame, state) + } + + // See comment to callerInfoCache + private fun updateRunningState(frame: CoroutineStackFrame, state: String) { + if (!isInstalled) return + // Lookup coroutine info in cache or by traversing stack frame + val info: DebugCoroutineInfoImpl + val cached = callerInfoCache.remove(frame) + val shouldBeMatchedWithProbeSuspended: Boolean + if (cached != null) { + info = cached + shouldBeMatchedWithProbeSuspended = false + } else { + info = frame.owner()?.info ?: return + shouldBeMatchedWithProbeSuspended = true + // Guard against improper implementations of CoroutineStackFrame and bugs in the compiler + val realCaller = info.lastObservedFrame?.realCaller() + if (realCaller != null) callerInfoCache.remove(realCaller) + } + info.updateState(state, frame as Continuation<*>, shouldBeMatchedWithProbeSuspended) + // Do not cache it for proxy-classes such as ScopeCoroutines + val caller = frame.realCaller() ?: return + callerInfoCache[caller] = info + } + + private tailrec fun CoroutineStackFrame.realCaller(): CoroutineStackFrame? { + val caller = callerFrame ?: return null + return if (caller.getStackTraceElement() != null) caller else caller.realCaller() + } + + private fun updateState(owner: CoroutineOwner<*>, frame: Continuation<*>, state: String) { + if (!isInstalled) return + owner.info.updateState(state, frame, true) + } + + private fun Continuation<*>.owner(): CoroutineOwner<*>? = (this as? CoroutineStackFrame)?.owner() + + private tailrec fun CoroutineStackFrame.owner(): CoroutineOwner<*>? = + if (this is CoroutineOwner<*>) this else callerFrame?.owner() + + // Not guarded by the lock at all, does not really affect consistency + internal fun probeCoroutineCreated(completion: Continuation): Continuation { + if (!isInstalled) return completion + // See DebugProbes.ignoreCoroutinesWithEmptyContext for the additional details. + if (ignoreCoroutinesWithEmptyContext && completion.context === EmptyCoroutineContext) return completion + /* + * If completion already has an owner, it means that we are in scoped coroutine (coroutineScope, withContext etc.), + * then piggyback on its already existing owner and do not replace completion + */ + val owner = completion.owner() + if (owner != null) return completion + /* + * Here we replace completion with a sequence of StackTraceFrame objects + * which represents creation stacktrace, thus making stacktrace recovery mechanism + * even more verbose (it will attach coroutine creation stacktrace to all exceptions), + * and then using CoroutineOwner completion as unique identifier of coroutineSuspended/resumed calls. + */ + val frame = if (enableCreationStackTraces) { + sanitizeStackTrace(Exception()).toStackTraceFrame() + } else { + null + } + return createOwner(completion, frame) + } + + private fun List.toStackTraceFrame(): StackTraceFrame = + StackTraceFrame( + foldRight(null) { frame, acc -> + StackTraceFrame(acc, frame) + }, ARTIFICIAL_FRAME + ) + + private fun createOwner(completion: Continuation, frame: StackTraceFrame?): Continuation { + if (!isInstalled) return completion + val info = DebugCoroutineInfoImpl(completion.context, frame, sequenceNumber.incrementAndGet()) + val owner = CoroutineOwner(completion, info) + capturedCoroutinesMap[owner] = true + if (!isInstalled) capturedCoroutinesMap.clear() + return owner + } + + // Not guarded by the lock at all, does not really affect consistency + private fun probeCoroutineCompleted(owner: CoroutineOwner<*>) { + capturedCoroutinesMap.remove(owner) + /* + * This removal is a guard against improperly implemented CoroutineStackFrame + * and bugs in the compiler. + */ + val caller = owner.info.lastObservedFrame?.realCaller() ?: return + callerInfoCache.remove(caller) + } + + /** + * This class is injected as completion of all continuations in [probeCoroutineCompleted]. + * It is owning the coroutine info and responsible for managing all its external info related to debug agent. + */ + public class CoroutineOwner internal constructor( + @JvmField internal val delegate: Continuation, + // Used by the IDEA debugger via reflection and must be kept binary-compatible, see KTIJ-24102 + @JvmField public val info: DebugCoroutineInfoImpl + ) : Continuation by delegate, CoroutineStackFrame { + private val frame get() = info.creationStackBottom + + override val callerFrame: CoroutineStackFrame? + get() = frame?.callerFrame + + override fun getStackTraceElement(): StackTraceElement? = frame?.getStackTraceElement() + + override fun resumeWith(result: Result) { + probeCoroutineCompleted(this) + delegate.resumeWith(result) + } + + override fun toString(): String = delegate.toString() + } + + private fun sanitizeStackTrace(throwable: T): List { + val stackTrace = throwable.stackTrace + val size = stackTrace.size + val traceStart = 1 + stackTrace.indexOfLast { it.className == "kotlin.coroutines.jvm.internal.DebugProbesKt" } + + if (!sanitizeStackTraces) { + return List(size - traceStart) { stackTrace[it + traceStart] } + } + + /* + * Trim intervals of internal methods from the stacktrace (bounds are excluded from trimming) + * E.g. for sequence [e, i1, i2, i3, e, i4, e, i5, i6, i7] + * output will be [e, i1, i3, e, i4, e, i5, i7] + * + * If an interval of internal methods ends in a synthetic method, the outermost non-synthetic method in that + * interval will also be included. + */ + val result = ArrayList(size - traceStart + 1) + var i = traceStart + while (i < size) { + if (stackTrace[i].isInternalMethod) { + result += stackTrace[i] // we include the boundary of the span in any case + // first index past the end of the span of internal methods that starts from `i` + var j = i + 1 + while (j < size && stackTrace[j].isInternalMethod) { + ++j + } + // index of the last non-synthetic internal methods in this span, or `i` if there are no such methods + var k = j - 1 + while (k > i && stackTrace[k].fileName == null) { + k -= 1 + } + if (k > i && k < j - 1) { + /* there are synthetic internal methods at the end of this span, but there is a non-synthetic method + after `i`, so we include it. */ + result += stackTrace[k] + } + result += stackTrace[j - 1] // we include the other boundary of this span in any case, too + i = j + } else { + result += stackTrace[i] + ++i + } + } + return result + } + + private val StackTraceElement.isInternalMethod: Boolean get() = className.startsWith("kotlinx.coroutines") +} + +private fun String.repr(): String = buildString { + append('"') + for (c in this@repr) { + when (c) { + '"' -> append("\\\"") + '\\' -> append("\\\\") + '\b' -> append("\\b") + '\n' -> append("\\n") + '\r' -> append("\\r") + '\t' -> append("\\t") + else -> append(c) + } + } + append('"') +} diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/DebuggerInfo.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/DebuggerInfo.kt new file mode 100644 index 0000000000..342c12381e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/DebuggerInfo.kt @@ -0,0 +1,24 @@ +@file:Suppress("UNUSED") + +package kotlinx.coroutines.debug.internal + +import java.io.Serializable +import kotlin.coroutines.* +import kotlinx.coroutines.* + +/* + * This class represents all the data required by IDEA debugger. + * It is serializable in order to speedup JDWP interactions. + * **DO NOT MAKE BINARY-INCOMPATIBLE CHANGES TO THIS CLASS**. + */ +@PublishedApi +internal class DebuggerInfo(source: DebugCoroutineInfoImpl, context: CoroutineContext) : Serializable { + public val coroutineId: Long? = context[CoroutineId]?.id + public val dispatcher: String? = context[ContinuationInterceptor]?.toString() + public val name: String? = context[CoroutineName]?.name + public val state: String = source.state + public val lastObservedThreadState: String? = source.lastObservedThread?.state?.toString() + public val lastObservedThreadName = source.lastObservedThread?.name + public val lastObservedStackTrace: List = source.lastObservedStackTrace() + public val sequenceNumber: Long = source.sequenceNumber +} diff --git a/kotlinx-coroutines-core/jvm/src/debug/internal/StackTraceFrame.kt b/kotlinx-coroutines-core/jvm/src/debug/internal/StackTraceFrame.kt new file mode 100644 index 0000000000..5ab67dcaee --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/debug/internal/StackTraceFrame.kt @@ -0,0 +1,13 @@ +package kotlinx.coroutines.debug.internal + +import kotlin.coroutines.jvm.internal.* + +/** + * A stack-trace represented as [CoroutineStackFrame]. + */ +internal class StackTraceFrame( + override val callerFrame: CoroutineStackFrame?, + private val stackTraceElement: StackTraceElement +) : CoroutineStackFrame { + override fun getStackTraceElement(): StackTraceElement = stackTraceElement +} diff --git a/kotlinx-coroutines-core/jvm/src/flow/internal/FlowExceptions.kt b/kotlinx-coroutines-core/jvm/src/flow/internal/FlowExceptions.kt new file mode 100644 index 0000000000..02817396e8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/flow/internal/FlowExceptions.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* + +/** + * Implementation note: `owner` is an internal marked that is used ONLY for identity checks by coroutines machinery, + * and it's never exposed, thus it's safe to have it both `@Transient` and non-nullable. + */ +internal actual class AbortFlowException actual constructor( + @JvmField @Transient actual val owner: Any +) : CancellationException("Flow was aborted, no more elements needed") { + + override fun fillInStackTrace(): Throwable { + if (DEBUG) return super.fillInStackTrace() + // Prevent Android <= 6.0 bug, #1866 + stackTrace = emptyArray() + return this + } +} + +internal actual class ChildCancelledException : CancellationException("Child of the scoped flow was cancelled") { + override fun fillInStackTrace(): Throwable { + if (DEBUG) return super.fillInStackTrace() + // Prevent Android <= 6.0 bug, #1866 + stackTrace = emptyArray() + return this + } +} diff --git a/kotlinx-coroutines-core/jvm/src/flow/internal/SafeCollector.kt b/kotlinx-coroutines-core/jvm/src/flow/internal/SafeCollector.kt new file mode 100644 index 0000000000..a02feee1e2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/flow/internal/SafeCollector.kt @@ -0,0 +1,181 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlin.coroutines.jvm.internal.* + +@Suppress("UNCHECKED_CAST") +private val emitFun = + FlowCollector::emit as Function3, Any?, Continuation, Any?> + +/** + * A safe collector is an instance of [FlowCollector] that ensures that neither context preservation + * nor exception transparency invariants are broken. Instances of [SafeCollector] are used in flow + * operators that provide raw access to the [FlowCollector] e.g. [Flow.transform]. + * Mechanically, each [emit] call captures [currentCoroutineContext], ensures it is not different from the + * previously caught one and proceeds further. If an exception is thrown from the downstream, + * it is caught, and any further attempts to [emit] lead to the [IllegalStateException]. + * + * ### Performance hacks + * + * Implementor of [ContinuationImpl] (that will be preserved as ABI nearly forever) + * in order to properly control `intercepted()` lifecycle. + * The safe collector implements [ContinuationImpl] to pretend it *is* a state-machine of its own `emit` method. + * It is [ContinuationImpl] and not any other [Continuation] subclass because only [ContinuationImpl] supports `intercepted()` caching. + * This is the most performance-sensitive place in the overall flow pipeline, because otherwise safe collector is forced to allocate + * a state machine on each element being emitted for each intermediate stage where the safe collector is present. + * + * See a comment to [emit] for the explanation of what and how is being optimized. + */ +@Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER", "INVISIBLE_MEMBER", "INVISIBLE_REFERENCE", "UNCHECKED_CAST") +internal actual class SafeCollector actual constructor( + @JvmField internal actual val collector: FlowCollector, + @JvmField internal actual val collectContext: CoroutineContext +) : FlowCollector, ContinuationImpl(NoOpContinuation, EmptyCoroutineContext), CoroutineStackFrame { + + override val callerFrame: CoroutineStackFrame? get() = completion_ as? CoroutineStackFrame + + override fun getStackTraceElement(): StackTraceElement? = null + + @JvmField // Note, it is non-capturing lambda, so no extra allocation during init of SafeCollector + internal actual val collectContextSize = collectContext.fold(0) { count, _ -> count + 1 } + + // Either context of the last emission or wrapper 'DownstreamExceptionContext' + private var lastEmissionContext: CoroutineContext? = null + // Completion if we are currently suspended or within completion_ body or null otherwise + private var completion_: Continuation? = null + + /* + * This property is accessed in two places: + * - ContinuationImpl invokes this in its `releaseIntercepted` as `context[ContinuationInterceptor]!!` + * - When we are within a callee, it is used to create its continuation object with this collector as completion_ + */ + override val context: CoroutineContext + get() = lastEmissionContext ?: EmptyCoroutineContext + + override fun invokeSuspend(result: Result): Any { + result.onFailure { lastEmissionContext = DownstreamExceptionContext(it, context) } + completion_?.resumeWith(result as Result) + return COROUTINE_SUSPENDED + } + + // Escalate visibility to manually release intercepted continuation + public actual override fun releaseIntercepted() { + super.releaseIntercepted() + } + + /** + * This is a crafty implementation of state-machine reusing. + * + * First it checks that it is not used concurrently (which we explicitly prohibit), and + * then just caches an instance of the completion_ in order to avoid extra allocation on each emit, + * making it effectively garbage-free on its hot-path. + * + * See `emit` overload. + */ + actual override suspend fun emit(value: T) { + // NB: it is a tail-call, so we are sure `uCont` is the completion of the emit's **caller**. + return suspendCoroutineUninterceptedOrReturn sc@{ uCont -> + try { + emit(uCont, value) + } catch (e: Throwable) { + // Save the fact that exception from emit (or even check context) has been thrown + // Note, that this can the first emit and lastEmissionContext may not be saved yet, + // hence we use `uCont.context` here. + lastEmissionContext = DownstreamExceptionContext(e, uCont.context) + throw e + } + } + } + + /** + * Here we use the following trick: + * - Perform all the required checks + * - Having a non-intercepted, non-cancellable caller's `uCont`, we leverage our implementation knowledge + * and invoke `collector.emit(T)` as `collector.emit(value: T, completion: Continuation), passing `this` + * as the completion. We also setup `this` state, so if the `completion.resume` is invoked, we are + * invoking `uCont.resume` properly in accordance with `ContinuationImpl`/`BaseContinuationImpl` internal invariants. + * + * Note that in such scenarios, `collector.emit` completion is the current instance of SafeCollector and thus is reused. + */ + private fun emit(uCont: Continuation, value: T): Any? { + val currentContext = uCont.context + currentContext.ensureActive() + // This check is triggered once per flow on a happy path. + val previousContext = lastEmissionContext + if (previousContext !== currentContext) { + checkContext(currentContext, previousContext, value) + lastEmissionContext = currentContext + } + completion_ = uCont + val result = emitFun(collector as FlowCollector, value, this as Continuation) + /* + * If the callee hasn't suspended, that means that it won't (it's forbidden) call 'resumeWith` (-> `invokeSuspend`) + * and we don't have to retain a strong reference to it to avoid memory leaks. + */ + if (result != COROUTINE_SUSPENDED) { + completion_ = null + } + return result + } + + private fun checkContext( + currentContext: CoroutineContext, + previousContext: CoroutineContext?, + value: T + ) { + if (previousContext is DownstreamExceptionContext) { + exceptionTransparencyViolated(previousContext, value) + } + checkContext(currentContext) + } + + private fun exceptionTransparencyViolated(exception: DownstreamExceptionContext, value: Any?) { + /* + * Exception transparency ensures that if a `collect` block or any intermediate operator + * throws an exception, then no more values will be received by it. + * For example, the following code: + * ``` + * val flow = flow { + * emit(1) + * try { + * emit(2) + * } catch (e: Exception) { + * emit(3) + * } + * } + * // Collector + * flow.collect { value -> + * if (value == 2) { + * throw CancellationException("No more elements required, received enough") + * } else { + * println("Collected $value") + * } + * } + * ``` + * is expected to print "Collected 1" and then "No more elements required, received enough" exception, + * but if exception transparency wasn't enforced, "Collected 1" and "Collected 3" would be printed instead. + */ + error(""" + Flow exception transparency is violated: + Previous 'emit' call has thrown exception ${exception.e}, but then emission attempt of value '$value' has been detected. + Emissions from 'catch' blocks are prohibited in order to avoid unspecified behaviour, 'Flow.catch' operator can be used instead. + For a more detailed explanation, please refer to Flow documentation. + """.trimIndent()) + } +} + +internal class DownstreamExceptionContext( + @JvmField val e: Throwable, + originalContext: CoroutineContext +) : CoroutineContext by originalContext + +private object NoOpContinuation : Continuation { + override val context: CoroutineContext = EmptyCoroutineContext + + override fun resumeWith(result: Result) { + // Nothing + } +} diff --git a/kotlinx-coroutines-core/jvm/src/future/Future.kt b/kotlinx-coroutines-core/jvm/src/future/Future.kt new file mode 100644 index 0000000000..37620bbbdb --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/future/Future.kt @@ -0,0 +1,207 @@ +package kotlinx.coroutines.future + +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import java.util.concurrent.* +import java.util.function.* +import kotlin.coroutines.* + +/** + * Starts a new coroutine and returns its result as an implementation of [CompletableFuture]. + * The running coroutine is cancelled when the resulting future is cancelled or otherwise completed. + * + * The coroutine context is inherited from a [CoroutineScope], additional context elements can be specified with the [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * The parent job is inherited from a [CoroutineScope] as well, but it can also be overridden + * with corresponding [context] element. + * + * By default, the coroutine is immediately scheduled for execution. + * Other options can be specified via `start` parameter. See [CoroutineStart] for details. + * A value of [CoroutineStart.LAZY] is not supported + * (since `CompletableFuture` framework does not provide the corresponding capability) and + * produces [IllegalArgumentException]. + * + * See [newCoroutineContext][CoroutineScope.newCoroutineContext] for a description of debugging facilities that are available for newly created coroutine. + * + * @param context additional to [CoroutineScope.coroutineContext] context of the coroutine. + * @param start coroutine start option. The default value is [CoroutineStart.DEFAULT]. + * @param block the coroutine code. + */ +public fun CoroutineScope.future( + context: CoroutineContext = EmptyCoroutineContext, + start: CoroutineStart = CoroutineStart.DEFAULT, + block: suspend CoroutineScope.() -> T +) : CompletableFuture { + require(!start.isLazy) { "$start start is not supported" } + val newContext = this.newCoroutineContext(context) + val future = CompletableFuture() + val coroutine = CompletableFutureCoroutine(newContext, future) + future.handle(coroutine) // Cancel coroutine if future was completed externally + coroutine.start(start, coroutine, block) + return future +} + +private class CompletableFutureCoroutine( + context: CoroutineContext, + private val future: CompletableFuture +) : AbstractCoroutine(context, initParentJob = true, active = true), BiFunction { + override fun apply(value: T?, exception: Throwable?) { + cancel() + } + + override fun onCompleted(value: T) { + future.complete(value) + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + /* + * Here we can potentially lose the cause if the failure is racing with future's + * external cancellation. We are consistent with other future implementations + * (LF, FT, CF) and give up on such exception. + */ + future.completeExceptionally(cause) + } +} + +/** + * Converts this deferred value to the instance of [CompletableFuture]. + * The deferred value is cancelled when the resulting future is cancelled or otherwise completed. + */ +public fun Deferred.asCompletableFuture(): CompletableFuture { + val future = CompletableFuture() + setupCancellation(future) + invokeOnCompletion { + try { + future.complete(getCompleted()) + } catch (t: Throwable) { + future.completeExceptionally(t) + } + } + return future +} + +/** + * Converts this job to the instance of [CompletableFuture]. + * The job is cancelled when the resulting future is cancelled or otherwise completed. + */ +public fun Job.asCompletableFuture(): CompletableFuture { + val future = CompletableFuture() + setupCancellation(future) + invokeOnCompletion { cause -> + if (cause === null) future.complete(Unit) + else future.completeExceptionally(cause) + } + return future +} + +private fun Job.setupCancellation(future: CompletableFuture<*>) { + future.handle { _, exception -> + cancel(exception?.let { + it as? CancellationException ?: CancellationException("CompletableFuture was completed exceptionally", it) + }) + } +} + +/** + * Converts this [CompletionStage] to an instance of [Deferred]. + * + * The [CompletableFuture] that corresponds to this [CompletionStage] (see [CompletionStage.toCompletableFuture]) + * is cancelled when the resulting deferred is cancelled. + */ +@Suppress("DeferredIsResult") +public fun CompletionStage.asDeferred(): Deferred { + val future = toCompletableFuture() // retrieve the future + // Fast path if already completed + if (future.isDone) { + return try { + @Suppress("UNCHECKED_CAST") + CompletableDeferred(future.get() as T) + } catch (e: Throwable) { + // unwrap original cause from ExecutionException + val original = (e as? ExecutionException)?.cause ?: e + CompletableDeferred().also { it.completeExceptionally(original) } + } + } + val result = CompletableDeferred() + handle { value, exception -> + try { + if (exception == null) { + // the future has completed normally + result.complete(value) + } else { + // the future has completed with an exception, unwrap it consistently with fast path + // Note: In the fast-path the implementation of CompletableFuture.get() does unwrapping + result.completeExceptionally((exception as? CompletionException)?.cause ?: exception) + } + } catch (e: Throwable) { + // We come here iff the internals of Deferred threw an exception during its completion + handleCoroutineException(EmptyCoroutineContext, e) + } + } + result.invokeOnCompletion(handler = CancelFutureOnCompletion(future)) + return result +} + +/** + * Awaits for completion of [CompletionStage] without blocking a thread. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this function + * stops waiting for the completion stage and immediately resumes with [CancellationException][kotlinx.coroutines.CancellationException]. + * + * This method is intended to be used with one-shot futures, so on coroutine cancellation the [CompletableFuture] that + * corresponds to this [CompletionStage] (see [CompletionStage.toCompletableFuture]) + * is cancelled. If cancelling the given stage is undesired, `stage.asDeferred().await()` should be used instead. + */ +public suspend fun CompletionStage.await(): T { + val future = toCompletableFuture() // retrieve the future + // fast path when CompletableFuture is already done (does not suspend) + if (future.isDone) { + try { + @Suppress("UNCHECKED_CAST", "BlockingMethodInNonBlockingContext") + return future.get() as T + } catch (e: ExecutionException) { + throw e.cause ?: e // unwrap original cause from ExecutionException + } + } + // slow path -- suspend + return suspendCancellableCoroutine { cont: CancellableContinuation -> + val consumer = ContinuationHandler(cont) + handle(consumer) + cont.invokeOnCancellation { + future.cancel(false) + consumer.cont = null // shall clear reference to continuation to aid GC + } + } +} + +private class ContinuationHandler( + @Volatile @JvmField var cont: Continuation? +) : BiFunction { + @Suppress("UNCHECKED_CAST") + override fun apply(result: T?, exception: Throwable?) { + val cont = this.cont ?: return // atomically read current value unless null + if (exception == null) { + // the future has completed normally + cont.resume(result as T) + } else { + // the future has completed with an exception, unwrap it to provide consistent view of .await() result and to propagate only original exception + cont.resumeWithException((exception as? CompletionException)?.cause ?: exception) + } + } +} + +private class CancelFutureOnCompletion( + private val future: Future<*> +) : JobNode() { + override val onCancelling get() = false + + override fun invoke(cause: Throwable?) { + // Don't interrupt when cancelling future on completion, because no one is going to reset this + // interruption flag and it will cause spurious failures elsewhere. + // We do not cancel the future if it's already completed in some way, + // because `cancel` on a completed future won't change the state but is not guaranteed to behave well + // on reentrancy. See https://github.com/Kotlin/kotlinx.coroutines/issues/4156 + if (cause != null && !future.isDone) future.cancel(false) + } +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/Concurrent.kt b/kotlinx-coroutines-core/jvm/src/internal/Concurrent.kt new file mode 100644 index 0000000000..db2cb0da39 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/Concurrent.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines.internal + +import java.lang.reflect.Method +import java.util.* +import java.util.concurrent.Executor +import java.util.concurrent.ScheduledThreadPoolExecutor +import kotlin.concurrent.withLock as withLockJvm + +internal actual typealias ReentrantLock = java.util.concurrent.locks.ReentrantLock + +internal actual inline fun ReentrantLock.withLock(action: () -> T) = this.withLockJvm(action) + +internal actual typealias WorkaroundAtomicReference = java.util.concurrent.atomic.AtomicReference + +// BenignDataRace is OptionalExpectation and doesn't have to be here +// but then IC breaks. See KT-66317 +@Retention(AnnotationRetention.SOURCE) +@Target(AnnotationTarget.FIELD) +internal actual annotation class BenignDataRace() + +@Suppress("NOTHING_TO_INLINE") // So that R8 can completely remove ConcurrentKt class +internal actual inline fun identitySet(expectedSize: Int): MutableSet = + Collections.newSetFromMap(IdentityHashMap(expectedSize)) + +private val REMOVE_FUTURE_ON_CANCEL: Method? = try { + ScheduledThreadPoolExecutor::class.java.getMethod("setRemoveOnCancelPolicy", Boolean::class.java) +} catch (_: Throwable) { + null +} + +/* We can not simply call `setRemoveOnCancelPolicy`, even though the code would compile and tests would pass, + * because older Android versions don't support it. */ +@Suppress("NAME_SHADOWING") +internal fun removeFutureOnCancel(executor: Executor): Boolean { + try { + val executor = executor as? ScheduledThreadPoolExecutor ?: return false + (REMOVE_FUTURE_ON_CANCEL ?: return false).invoke(executor, true) + return true + } catch (_: Throwable) { + return false // failed to setRemoveOnCancelPolicy, assume it does not remove the future on cancel + } +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/jvm/src/internal/CoroutineExceptionHandlerImpl.kt new file mode 100644 index 0000000000..2d048ac71a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/CoroutineExceptionHandlerImpl.kt @@ -0,0 +1,49 @@ +package kotlinx.coroutines.internal + +import java.util.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * A list of globally installed [CoroutineExceptionHandler] instances. + * + * Note that Android may have dummy [Thread.contextClassLoader] which is used by one-argument [ServiceLoader.load] function, + * see (https://stackoverflow.com/questions/13407006/android-class-loader-may-fail-for-processes-that-host-multiple-applications). + * So here we explicitly use two-argument `load` with a class-loader of [CoroutineExceptionHandler] class. + * + * We are explicitly using the `ServiceLoader.load(MyClass::class.java, MyClass::class.java.classLoader).iterator()` + * form of the ServiceLoader call to enable R8 optimization when compiled on Android. + */ +internal actual val platformExceptionHandlers: Collection = ServiceLoader.load( + CoroutineExceptionHandler::class.java, + CoroutineExceptionHandler::class.java.classLoader +).iterator().asSequence().toList() + +internal actual fun ensurePlatformExceptionHandlerLoaded(callback: CoroutineExceptionHandler) { + // we use JVM's mechanism of ServiceLoader, so this should be a no-op on JVM. + // The only thing we do is make sure that the ServiceLoader did work correctly. + check(callback in platformExceptionHandlers) { "Exception handler was not found via a ServiceLoader" } +} + +internal actual fun propagateExceptionFinalResort(exception: Throwable) { + // use the thread's handler + val currentThread = Thread.currentThread() + currentThread.uncaughtExceptionHandler.uncaughtException(currentThread, exception) +} + +// This implementation doesn't store a stacktrace, which is good because a stacktrace doesn't make sense for this. +internal actual class DiagnosticCoroutineContextException actual constructor(context: CoroutineContext) : RuntimeException() { + + @Transient + private val context: CoroutineContext? = context + + override fun getLocalizedMessage(): String { + return context.toString() + } + + override fun fillInStackTrace(): Throwable { + // Prevent Android <= 6.0 bug, #1866 + stackTrace = emptyArray() + return this + } +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/ExceptionsConstructor.kt b/kotlinx-coroutines-core/jvm/src/internal/ExceptionsConstructor.kt new file mode 100644 index 0000000000..de1225ada5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/ExceptionsConstructor.kt @@ -0,0 +1,111 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import java.lang.reflect.* +import java.util.* +import java.util.concurrent.locks.* +import kotlin.concurrent.* + +private val throwableFields = Throwable::class.java.fieldsCountOrDefault(-1) +private typealias Ctor = (Throwable) -> Throwable? + +private val ctorCache = try { + if (ANDROID_DETECTED) WeakMapCtorCache + else ClassValueCtorCache +} catch (e: Throwable) { + // Fallback on Java 6 or exotic setups + WeakMapCtorCache +} + +@Suppress("UNCHECKED_CAST") +internal fun tryCopyException(exception: E): E? { + // Fast path for CopyableThrowable + if (exception is CopyableThrowable<*>) { + return runCatching { exception.createCopy() as E? }.getOrNull() + } + return ctorCache.get(exception.javaClass).invoke(exception) as E? +} + +private fun createConstructor(clz: Class): Ctor { + val nullResult: Ctor = { null } // Pre-cache class + // Skip reflective copy if an exception has additional fields (that are typically populated in user-defined constructors) + if (throwableFields != clz.fieldsCountOrDefault(0)) return nullResult + /* + * Try to reflectively find constructor(message, cause), constructor(message), constructor(cause), or constructor(), + * in that order of priority. + * Exceptions are shared among coroutines, so we should copy exception before recovering current stacktrace. + * + * By default, Java's reflection iterates over ctors in the source-code order and the sorting is stable, so we can + * not rely on the order of iteration. Instead, we assign a unique priority to each ctor type. + */ + return clz.constructors.map { constructor -> + val p = constructor.parameterTypes + when (p.size) { + 2 -> when { + p[0] == String::class.java && p[1] == Throwable::class.java -> + safeCtor { e -> constructor.newInstance(e.message, e) as Throwable } to 3 + else -> null to -1 + } + 1 -> when (p[0]) { + String::class.java -> + safeCtor { e -> (constructor.newInstance(e.message) as Throwable).also { it.initCause(e) } } to 2 + Throwable::class.java -> + safeCtor { e -> constructor.newInstance(e) as Throwable } to 1 + else -> null to -1 + } + 0 -> safeCtor { e -> (constructor.newInstance() as Throwable).also { it.initCause(e) } } to 0 + else -> null to -1 + } + }.maxByOrNull(Pair<*, Int>::second)?.first ?: nullResult +} + +private fun safeCtor(block: (Throwable) -> Throwable): Ctor = { e -> + runCatching { + val result = block(e) + /* + * Verify that the new exception has the same message as the original one (bail out if not, see #1631) + * or if the new message complies the contract from `Throwable(cause).message` contract. + */ + if (e.message != result.message && result.message != e.toString()) null + else result + }.getOrNull() +} + +private fun Class<*>.fieldsCountOrDefault(defaultValue: Int) = + kotlin.runCatching { fieldsCount() }.getOrDefault(defaultValue) + +private tailrec fun Class<*>.fieldsCount(accumulator: Int = 0): Int { + val fieldsCount = declaredFields.count { !Modifier.isStatic(it.modifiers) } + val totalFields = accumulator + fieldsCount + val superClass = superclass ?: return totalFields + return superClass.fieldsCount(totalFields) +} + +internal abstract class CtorCache { + abstract fun get(key: Class): Ctor +} + +private object WeakMapCtorCache : CtorCache() { + private val cacheLock = ReentrantReadWriteLock() + private val exceptionCtors: WeakHashMap, Ctor> = WeakHashMap() + + override fun get(key: Class): Ctor { + cacheLock.read { exceptionCtors[key]?.let { return it } } + cacheLock.write { + exceptionCtors[key]?.let { return it } + return createConstructor(key).also { exceptionCtors[key] = it } + } + } +} + +@IgnoreJreRequirement +private object ClassValueCtorCache : CtorCache() { + private val cache = object : ClassValue() { + override fun computeValue(type: Class<*>?): Ctor { + @Suppress("UNCHECKED_CAST") + return createConstructor(type as Class) + } + } + + override fun get(key: Class): Ctor = cache.get(key) +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/FastServiceLoader.kt b/kotlinx-coroutines-core/jvm/src/internal/FastServiceLoader.kt new file mode 100644 index 0000000000..eb2c4869ae --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/FastServiceLoader.kt @@ -0,0 +1,168 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.CoroutineExceptionHandler +import java.io.* +import java.net.* +import java.util.* +import java.util.jar.* +import java.util.zip.* + +/** + * Don't use JvmField here to enable R8 optimizations via "assumenosideeffects" + */ +internal val ANDROID_DETECTED = runCatching { Class.forName("android.os.Build") }.isSuccess + +/** + * A simplified version of [ServiceLoader]. + * FastServiceLoader locates and instantiates all service providers named in configuration + * files placed in the resource directory META-INF/services. + * + * The main difference between this class and classic service loader is in skipping + * verification JARs. A verification requires reading the whole JAR (and it causes problems and ANRs on Android devices) + * and prevents only trivial checksum issues. See #878. + * + * If any error occurs during loading, it fallbacks to [ServiceLoader], mostly to prevent R8 issues. + */ +internal object FastServiceLoader { + private const val PREFIX: String = "META-INF/services/" + + /** + * This method attempts to load [MainDispatcherFactory] in Android-friendly way. + * + * If we are not on Android, this method fallbacks to a regular service loading, + * else we attempt to do `Class.forName` lookup for + * `AndroidDispatcherFactory` and `TestMainDispatcherFactory`. + * If lookups are successful, we return resultinAg instances because we know that + * `MainDispatcherFactory` API is internal and this is the only possible classes of `MainDispatcherFactory` Service on Android. + * + * Such an intricate dance is required to avoid calls to `ServiceLoader.load` for multiple reasons: + * 1) It eliminates disk lookup on potentially slow devices on the Main thread. + * 2) Various Android toolchain versions by various vendors don't tend to handle ServiceLoader calls properly. + * Sometimes META-INF is removed from the resulting APK, sometimes class names are mangled, etc. + * While it is not the problem of `kotlinx.coroutines`, it significantly worsens user experience, thus we are workarounding it. + * Examples of such issues are #932, #1072, #1557, #1567 + * + * We also use SL for [CoroutineExceptionHandler], but we do not experience the same problems and CEH is a public API + * that may already be injected vis SL, so we are not using the same technique for it. + */ + internal fun loadMainDispatcherFactory(): List { + val clz = MainDispatcherFactory::class.java + if (!ANDROID_DETECTED) { + return load(clz, clz.classLoader) + } + + /* + * If `ANDROID_DETECTED` is true, it is still possible to have `AndroidDispatcherFactory` missing. + * The most notable case of it is firebase-sdk that repackages some Android classes but can be used from an arbitrary + * K/JVM application. + * See also #3914. + */ + return try { + val result = ArrayList(2) + val mainFactory = createInstanceOf(clz, "kotlinx.coroutines.android.AndroidDispatcherFactory") + if (mainFactory == null) { + // Fallback to regular service loading + return load(clz, clz.classLoader) + } + result.add(mainFactory) + // Also search for test-module factory + createInstanceOf(clz, "kotlinx.coroutines.test.internal.TestMainDispatcherFactory")?.apply { result.add(this) } + result + } catch (_: Throwable) { + // Fallback to the regular SL in case of any unexpected exception + load(clz, clz.classLoader) + } + } + + /* + * This method is inline to have a direct Class.forName("string literal") in the byte code to avoid weird interactions with ProGuard/R8. + */ + @Suppress("NOTHING_TO_INLINE") + private inline fun createInstanceOf( + baseClass: Class, + serviceClass: String + ): MainDispatcherFactory? { + return try { + val clz = Class.forName(serviceClass, true, baseClass.classLoader) + baseClass.cast(clz.getDeclaredConstructor().newInstance()) + } catch (_: ClassNotFoundException) { // Do not fail if TestMainDispatcherFactory is not found + null + } + } + + private fun load(service: Class, loader: ClassLoader): List { + return try { + loadProviders(service, loader) + } catch (_: Throwable) { + // Fallback to default service loader + ServiceLoader.load(service, loader).toList() + } + } + + // Visible for tests + internal fun loadProviders(service: Class, loader: ClassLoader): List { + val fullServiceName = PREFIX + service.name + // Filter out situations when both JAR and regular files are in the classpath (e.g. IDEA) + val urls = loader.getResources(fullServiceName) + val providers = urls.toList().flatMap { parse(it) }.toSet() + require(providers.isNotEmpty()) { "No providers were loaded with FastServiceLoader" } + return providers.map { getProviderInstance(it, loader, service) } + } + + private fun getProviderInstance(name: String, loader: ClassLoader, service: Class): S { + val clazz = Class.forName(name, false, loader) + require(service.isAssignableFrom(clazz)) { "Expected service of class $service, but found $clazz" } + return service.cast(clazz.getDeclaredConstructor().newInstance()) + } + + private fun parse(url: URL): List { + val path = url.toString() + // Fast-path for JARs + if (path.startsWith("jar")) { + val pathToJar = path.substringAfter("jar:file:").substringBefore('!') + val entry = path.substringAfter("!/") + // mind the verify = false flag! + (JarFile(pathToJar, false)).use { file -> + BufferedReader(InputStreamReader(file.getInputStream(ZipEntry(entry)), "UTF-8")).use { r -> + return parseFile(r) + } + } + } + // Regular path for everything else + return BufferedReader(InputStreamReader(url.openStream())).use { reader -> + parseFile(reader) + } + } + + // JarFile does no implement Closesable on Java 1.6 + private inline fun JarFile.use(block: (JarFile) -> R): R { + var cause: Throwable? = null + try { + return block(this) + } catch (e: Throwable) { + cause = e + throw e + } finally { + try { + close() + } catch (closeException: Throwable) { + if (cause === null) throw closeException + cause.addSuppressed(closeException) + throw cause + } + } + } + + private fun parseFile(r: BufferedReader): List { + val names = mutableSetOf() + while (true) { + val line = r.readLine() ?: break + val serviceName = line.substringBefore("#").trim() + require(serviceName.all { it == '.' || Character.isJavaIdentifierPart(it) }) { "Illegal service provider class name: $serviceName" } + if (serviceName.isNotEmpty()) { + names.add(serviceName) + } + } + return names.toList() + } +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/InternalAnnotations.kt b/kotlinx-coroutines-core/jvm/src/internal/InternalAnnotations.kt new file mode 100644 index 0000000000..01279f23ba --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/InternalAnnotations.kt @@ -0,0 +1,3 @@ +package kotlinx.coroutines.internal + +internal actual typealias IgnoreJreRequirement = org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement diff --git a/kotlinx-coroutines-core/jvm/src/internal/LocalAtomics.kt b/kotlinx-coroutines-core/jvm/src/internal/LocalAtomics.kt new file mode 100644 index 0000000000..19398ed543 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/LocalAtomics.kt @@ -0,0 +1,3 @@ +package kotlinx.coroutines.internal + +internal actual typealias LocalAtomicInt = java.util.concurrent.atomic.AtomicInteger diff --git a/kotlinx-coroutines-core/jvm/src/internal/MainDispatchers.kt b/kotlinx-coroutines-core/jvm/src/internal/MainDispatchers.kt new file mode 100644 index 0000000000..0cace58cf9 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/MainDispatchers.kt @@ -0,0 +1,129 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import java.util.* +import kotlin.coroutines.* + +/** + * Name of the boolean property that enables using of [FastServiceLoader]. + */ +private const val FAST_SERVICE_LOADER_PROPERTY_NAME = "kotlinx.coroutines.fast.service.loader" + +// Lazy loader for the main dispatcher +internal object MainDispatcherLoader { + + private val FAST_SERVICE_LOADER_ENABLED = systemProp(FAST_SERVICE_LOADER_PROPERTY_NAME, true) + + @JvmField + val dispatcher: MainCoroutineDispatcher = loadMainDispatcher() + + private fun loadMainDispatcher(): MainCoroutineDispatcher { + return try { + val factories = if (FAST_SERVICE_LOADER_ENABLED) { + FastServiceLoader.loadMainDispatcherFactory() + } else { + // We are explicitly using the + // `ServiceLoader.load(MyClass::class.java, MyClass::class.java.classLoader).iterator()` + // form of the ServiceLoader call to enable R8 optimization when compiled on Android. + ServiceLoader.load( + MainDispatcherFactory::class.java, + MainDispatcherFactory::class.java.classLoader + ).iterator().asSequence().toList() + } + @Suppress("ConstantConditionIf") + factories.maxByOrNull { it.loadPriority }?.tryCreateDispatcher(factories) + ?: createMissingDispatcher() + } catch (e: Throwable) { + // Service loader can throw an exception as well + createMissingDispatcher(e) + } + } +} + +/** + * If anything goes wrong while trying to create main dispatcher (class not found, + * initialization failed, etc), then replace the main dispatcher with a special + * stub that throws an error message on any attempt to actually use it. + * + * @suppress internal API + */ +@InternalCoroutinesApi +public fun MainDispatcherFactory.tryCreateDispatcher(factories: List): MainCoroutineDispatcher = + try { + createDispatcher(factories) + } catch (cause: Throwable) { + createMissingDispatcher(cause, hintOnError()) + } + +/** @suppress */ +@InternalCoroutinesApi +public fun MainCoroutineDispatcher.isMissing(): Boolean = + // not checking `this`, as it may be wrapped in a `TestMainDispatcher`, whereas `immediate` never is. + this.immediate is MissingMainCoroutineDispatcher + +// R8 optimization hook, not const on purpose to enable R8 optimizations via "assumenosideeffects" +@Suppress("MayBeConstant") +private val SUPPORT_MISSING = true + +@Suppress( + "ConstantConditionIf", + "IMPLICIT_NOTHING_TYPE_ARGUMENT_AGAINST_NOT_NOTHING_EXPECTED_TYPE" // KT-47626 +) +private fun createMissingDispatcher(cause: Throwable? = null, errorHint: String? = null) = + if (SUPPORT_MISSING) MissingMainCoroutineDispatcher(cause, errorHint) else + cause?.let { throw it } ?: throwMissingMainDispatcherException() + +internal fun throwMissingMainDispatcherException(): Nothing { + throw IllegalStateException( + "Module with the Main dispatcher is missing. " + + "Add dependency providing the Main dispatcher, e.g. 'kotlinx-coroutines-android' " + + "and ensure it has the same version as 'kotlinx-coroutines-core'" + ) +} + +private class MissingMainCoroutineDispatcher( + private val cause: Throwable?, + private val errorHint: String? = null +) : MainCoroutineDispatcher(), Delay { + + override val immediate: MainCoroutineDispatcher get() = this + + override fun isDispatchNeeded(context: CoroutineContext): Boolean = + missing() + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher = + missing() + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + missing() + + override fun dispatch(context: CoroutineContext, block: Runnable) = + missing() + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) = + missing() + + private fun missing(): Nothing { + if (cause == null) { + throwMissingMainDispatcherException() + } else { + val message = "Module with the Main dispatcher had failed to initialize" + (errorHint?.let { ". $it" } ?: "") + throw IllegalStateException(message, cause) + } + } + + override fun toString(): String = "Dispatchers.Main[missing${if (cause != null) ", cause=$cause" else ""}]" +} + +/** + * @suppress + */ +@InternalCoroutinesApi +public object MissingMainCoroutineDispatcherFactory : MainDispatcherFactory { + override val loadPriority: Int + get() = -1 + + override fun createDispatcher(allFactories: List): MainCoroutineDispatcher { + return MissingMainCoroutineDispatcher(null) + } +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/ProbesSupport.kt b/kotlinx-coroutines-core/jvm/src/internal/ProbesSupport.kt new file mode 100644 index 0000000000..47c8189778 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/ProbesSupport.kt @@ -0,0 +1,12 @@ +@file:Suppress("NOTHING_TO_INLINE", "INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +internal actual inline fun probeCoroutineCreated(completion: Continuation): Continuation = + kotlin.coroutines.jvm.internal.probeCoroutineCreated(completion) + +internal actual inline fun probeCoroutineResumed(completion: Continuation) { + kotlin.coroutines.jvm.internal.probeCoroutineResumed(completion) +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/ResizableAtomicArray.kt b/kotlinx-coroutines-core/jvm/src/internal/ResizableAtomicArray.kt new file mode 100644 index 0000000000..9cc2983ffe --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/ResizableAtomicArray.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.internal + +import java.util.concurrent.atomic.* + +/** + * Atomic array with lock-free reads and synchronized modifications. It logically has an unbounded size, + * is implicitly filled with nulls, and is resized on updates as needed to grow. + */ +internal class ResizableAtomicArray(initialLength: Int) { + @Volatile + private var array = AtomicReferenceArray(initialLength) + + // for debug output + public fun currentLength(): Int = array.length() + + public operator fun get(index: Int): T? { + val array = this.array // volatile read + return if (index < array.length()) array[index] else null + } + + // Must not be called concurrently, e.g. always use synchronized(this) to call this function + fun setSynchronized(index: Int, value: T?) { + val curArray = this.array + val curLen = curArray.length() + if (index < curLen) { + curArray[index] = value + return + } + // It would be nice to copy array in batch instead of 1 by 1, but it seems like Java has no API for that + val newArray = AtomicReferenceArray((index + 1).coerceAtLeast(2 * curLen)) + for (i in 0 until curLen) newArray[i] = curArray[i] + newArray[index] = value + array = newArray // copy done + } +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/StackTraceRecovery.kt b/kotlinx-coroutines-core/jvm/src/internal/StackTraceRecovery.kt new file mode 100644 index 0000000000..acf07a553b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/StackTraceRecovery.kt @@ -0,0 +1,209 @@ +@file:Suppress("UNCHECKED_CAST") + +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import _COROUTINE.ARTIFICIAL_FRAME_PACKAGE_NAME +import _COROUTINE.ArtificialStackFrames +import java.util.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* + +/* + * `Class.forName(name).canonicalName` instead of plain `name` is required to properly handle + * Android's minifier that renames these classes and breaks our recovery heuristic without such lookup. + */ +private const val baseContinuationImplClass = "kotlin.coroutines.jvm.internal.BaseContinuationImpl" +private const val stackTraceRecoveryClass = "kotlinx.coroutines.internal.StackTraceRecoveryKt" + +private val ARTIFICIAL_FRAME = ArtificialStackFrames().coroutineBoundary() + +private val baseContinuationImplClassName = runCatching { + Class.forName(baseContinuationImplClass).canonicalName +}.getOrElse { baseContinuationImplClass } + +private val stackTraceRecoveryClassName = runCatching { + Class.forName(stackTraceRecoveryClass).canonicalName +}.getOrElse { stackTraceRecoveryClass } + +internal actual fun recoverStackTrace(exception: E): E { + if (!RECOVER_STACK_TRACES) return exception + // No unwrapping on continuation-less path: exception is not reported multiple times via slow paths + val copy = tryCopyException(exception) ?: return exception + return copy.sanitizeStackTrace() +} + +private fun E.sanitizeStackTrace(): E { + val stackTrace = stackTrace + val size = stackTrace.size + val lastIntrinsic = stackTrace.indexOfLast { stackTraceRecoveryClassName == it.className } + val startIndex = lastIntrinsic + 1 + val endIndex = stackTrace.firstFrameIndex(baseContinuationImplClassName) + val adjustment = if (endIndex == -1) 0 else size - endIndex + val trace = Array(size - lastIntrinsic - adjustment) { + if (it == 0) { + ARTIFICIAL_FRAME + } else { + stackTrace[startIndex + it - 1] + } + } + + setStackTrace(trace) + return this +} + +@Suppress("NOTHING_TO_INLINE") // Inline for better R8 optimization +internal actual inline fun recoverStackTrace(exception: E, continuation: Continuation<*>): E { + if (!RECOVER_STACK_TRACES || continuation !is CoroutineStackFrame) return exception + return recoverFromStackFrame(exception, continuation) +} + +private fun recoverFromStackFrame(exception: E, continuation: CoroutineStackFrame): E { + /* + * Here we are checking whether exception has already recovered stacktrace. + * If so, we extract initial and merge recovered stacktrace and current one + */ + val (cause, recoveredStacktrace) = exception.causeAndStacktrace() + + // Try to create an exception of the same type and get stacktrace from continuation + val newException = tryCopyException(cause) ?: return exception + // Update stacktrace + val stacktrace = createStackTrace(continuation) + if (stacktrace.isEmpty()) return exception + // Merge if necessary + if (cause !== exception) { + mergeRecoveredTraces(recoveredStacktrace, stacktrace) + } + // Take recovered stacktrace, merge it with existing one if necessary and return + return createFinalException(cause, newException, stacktrace) +} + +/* + * Here we partially copy original exception stackTrace to make current one much prettier. + * E.g. for + * ``` + * fun foo() = async { error(...) } + * suspend fun bar() = foo().await() + * ``` + * we would like to produce following exception: + * IllegalStateException + * at foo + * at kotlin.coroutines.resumeWith + * at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + * at bar + * ...real stackTrace... + * caused by "IllegalStateException" (original one) + */ +private fun createFinalException(cause: E, result: E, resultStackTrace: ArrayDeque): E { + resultStackTrace.addFirst(ARTIFICIAL_FRAME) + val causeTrace = cause.stackTrace + val size = causeTrace.firstFrameIndex(baseContinuationImplClassName) + if (size == -1) { + result.stackTrace = resultStackTrace.toTypedArray() + return result + } + + val mergedStackTrace = arrayOfNulls(resultStackTrace.size + size) + for (i in 0 until size) { + mergedStackTrace[i] = causeTrace[i] + } + + for ((index, element) in resultStackTrace.withIndex()) { + mergedStackTrace[size + index] = element + } + + result.stackTrace = mergedStackTrace + return result +} + +/** + * Find initial cause of the exception without restored stacktrace. + * Returns intermediate stacktrace as well in order to avoid excess cloning of array as an optimization. + */ +private fun E.causeAndStacktrace(): Pair> { + val cause = cause + return if (cause != null && cause.javaClass == javaClass) { + val currentTrace = stackTrace + if (currentTrace.any { it.isArtificial() }) + cause as E to currentTrace + else this to emptyArray() + } else { + this to emptyArray() + } +} + +private fun mergeRecoveredTraces(recoveredStacktrace: Array, result: ArrayDeque) { + // Merge two stacktraces and trim common prefix + val startIndex = recoveredStacktrace.indexOfFirst { it.isArtificial() } + 1 + val lastFrameIndex = recoveredStacktrace.size - 1 + for (i in lastFrameIndex downTo startIndex) { + val element = recoveredStacktrace[i] + if (element.elementWiseEquals(result.last)) { + result.removeLast() + } + result.addFirst(recoveredStacktrace[i]) + } +} + +internal actual suspend inline fun recoverAndThrow(exception: Throwable): Nothing { + if (!RECOVER_STACK_TRACES) throw exception + suspendCoroutineUninterceptedOrReturn { + if (it !is CoroutineStackFrame) throw exception + throw recoverFromStackFrame(exception, it) + } +} + +@PublishedApi +@Suppress("NOTHING_TO_INLINE") // Inline for better R8 optimizations +internal actual inline fun unwrap(exception: E): E = + if (!RECOVER_STACK_TRACES) exception else unwrapImpl(exception) + +@PublishedApi +internal fun unwrapImpl(exception: E): E { + val cause = exception.cause + // Fast-path to avoid array cloning + if (cause == null || cause.javaClass != exception.javaClass) { + return exception + } + // Slow path looks for artificial frames in a stack-trace + if (exception.stackTrace.any { it.isArtificial() }) { + @Suppress("UNCHECKED_CAST") + return cause as E + } else { + return exception + } +} + +private fun createStackTrace(continuation: CoroutineStackFrame): ArrayDeque { + val stack = ArrayDeque() + continuation.getStackTraceElement()?.let { stack.add(it) } + + var last = continuation + while (true) { + last = (last as? CoroutineStackFrame)?.callerFrame ?: break + last.getStackTraceElement()?.let { stack.add(it) } + } + return stack +} + +internal fun StackTraceElement.isArtificial() = className.startsWith(ARTIFICIAL_FRAME_PACKAGE_NAME) +private fun Array.firstFrameIndex(methodName: String) = indexOfFirst { methodName == it.className } + +private fun StackTraceElement.elementWiseEquals(e: StackTraceElement): Boolean { + /* + * In order to work on Java 9 where modules and classloaders of enclosing class + * are part of the comparison + */ + return lineNumber == e.lineNumber && methodName == e.methodName + && fileName == e.fileName && className == e.className +} + +internal actual typealias CoroutineStackFrame = kotlin.coroutines.jvm.internal.CoroutineStackFrame + +internal actual typealias StackTraceElement = java.lang.StackTraceElement + +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +internal actual fun Throwable.initCause(cause: Throwable) { + // Resolved to member, verified by test + initCause(cause) +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/Synchronized.kt b/kotlinx-coroutines-core/jvm/src/internal/Synchronized.kt new file mode 100644 index 0000000000..47273d6ceb --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/Synchronized.kt @@ -0,0 +1,16 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public actual typealias SynchronizedObject = Any + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public actual inline fun synchronizedImpl(lock: SynchronizedObject, block: () -> T): T = + kotlin.synchronized(lock, block) diff --git a/kotlinx-coroutines-core/jvm/src/internal/SystemProps.kt b/kotlinx-coroutines-core/jvm/src/internal/SystemProps.kt new file mode 100644 index 0000000000..837c2ff9ea --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/SystemProps.kt @@ -0,0 +1,16 @@ +@file:JvmName("SystemPropsKt") +@file:JvmMultifileClass + +package kotlinx.coroutines.internal + +// number of processors at startup for consistent prop initialization +internal val AVAILABLE_PROCESSORS = Runtime.getRuntime().availableProcessors() + +internal actual fun systemProp( + propertyName: String +): String? = + try { + System.getProperty(propertyName) + } catch (e: SecurityException) { + null + } diff --git a/kotlinx-coroutines-core/jvm/src/internal/ThreadContext.kt b/kotlinx-coroutines-core/jvm/src/internal/ThreadContext.kt new file mode 100644 index 0000000000..8f21b13c25 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/ThreadContext.kt @@ -0,0 +1,126 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +@JvmField +internal val NO_THREAD_ELEMENTS = Symbol("NO_THREAD_ELEMENTS") + +// Used when there are >= 2 active elements in the context +@Suppress("UNCHECKED_CAST") +private class ThreadState(@JvmField val context: CoroutineContext, n: Int) { + private val values = arrayOfNulls(n) + private val elements = arrayOfNulls>(n) + private var i = 0 + + fun append(element: ThreadContextElement<*>, value: Any?) { + values[i] = value + elements[i++] = element as ThreadContextElement + } + + fun restore(context: CoroutineContext) { + for (i in elements.indices.reversed()) { + elements[i]!!.restoreThreadContext(context, values[i]) + } + } +} + +// Counts ThreadContextElements in the context +// Any? here is Int | ThreadContextElement (when count is one) +private val countAll = + fun (countOrElement: Any?, element: CoroutineContext.Element): Any? { + if (element is ThreadContextElement<*>) { + val inCount = countOrElement as? Int ?: 1 + return if (inCount == 0) element else inCount + 1 + } + return countOrElement + } + +// Find one (first) ThreadContextElement in the context, it is used when we know there is exactly one +private val findOne = + fun (found: ThreadContextElement<*>?, element: CoroutineContext.Element): ThreadContextElement<*>? { + if (found != null) return found + return element as? ThreadContextElement<*> + } + +// Updates state for ThreadContextElements in the context using the given ThreadState +private val updateState = + fun (state: ThreadState, element: CoroutineContext.Element): ThreadState { + if (element is ThreadContextElement<*>) { + state.append(element, element.updateThreadContext(state.context)) + } + return state + } + +internal actual fun threadContextElements(context: CoroutineContext): Any = context.fold(0, countAll)!! + +// countOrElement is pre-cached in dispatched continuation +// returns NO_THREAD_ELEMENTS if the contest does not have any ThreadContextElements +internal fun updateThreadContext(context: CoroutineContext, countOrElement: Any?): Any? { + @Suppress("NAME_SHADOWING") + val countOrElement = countOrElement ?: threadContextElements(context) + @Suppress("IMPLICIT_BOXING_IN_IDENTITY_EQUALS") + return when { + countOrElement === 0 -> NO_THREAD_ELEMENTS // very fast path when there are no active ThreadContextElements + // ^^^ identity comparison for speed, we know zero always has the same identity + countOrElement is Int -> { + // slow path for multiple active ThreadContextElements, allocates ThreadState for multiple old values + context.fold(ThreadState(context, countOrElement), updateState) + } + else -> { + // fast path for one ThreadContextElement (no allocations, no additional context scan) + @Suppress("UNCHECKED_CAST") + val element = countOrElement as ThreadContextElement + element.updateThreadContext(context) + } + } +} + +internal fun restoreThreadContext(context: CoroutineContext, oldState: Any?) { + when { + oldState === NO_THREAD_ELEMENTS -> return // very fast path when there are no ThreadContextElements + oldState is ThreadState -> { + // slow path with multiple stored ThreadContextElements + oldState.restore(context) + } + else -> { + // fast path for one ThreadContextElement, but need to find it + @Suppress("UNCHECKED_CAST") + val element = context.fold(null, findOne) as ThreadContextElement + element.restoreThreadContext(context, oldState) + } + } +} + +// top-level data class for a nicer out-of-the-box toString representation and class name +@PublishedApi +internal data class ThreadLocalKey(private val threadLocal: ThreadLocal<*>) : CoroutineContext.Key> + +internal class ThreadLocalElement( + private val value: T, + private val threadLocal: ThreadLocal +) : ThreadContextElement { + override val key: CoroutineContext.Key<*> = ThreadLocalKey(threadLocal) + + override fun updateThreadContext(context: CoroutineContext): T { + val oldState = threadLocal.get() + threadLocal.set(value) + return oldState + } + + override fun restoreThreadContext(context: CoroutineContext, oldState: T) { + threadLocal.set(oldState) + } + + // this method is overridden to perform value comparison (==) on key + override fun minusKey(key: CoroutineContext.Key<*>): CoroutineContext { + return if (this.key == key) EmptyCoroutineContext else this + } + + // this method is overridden to perform value comparison (==) on key + public override operator fun get(key: CoroutineContext.Key): E? = + @Suppress("UNCHECKED_CAST") + if (this.key == key) this as E else null + + override fun toString(): String = "ThreadLocal(value=$value, threadLocal = $threadLocal)" +} diff --git a/kotlinx-coroutines-core/jvm/src/internal/ThreadLocal.kt b/kotlinx-coroutines-core/jvm/src/internal/ThreadLocal.kt new file mode 100644 index 0000000000..209c3d5f32 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/internal/ThreadLocal.kt @@ -0,0 +1,7 @@ +package kotlinx.coroutines.internal + +import java.lang.ThreadLocal + +internal actual typealias CommonThreadLocal = ThreadLocal + +internal actual fun commonThreadLocal(name: Symbol): CommonThreadLocal = ThreadLocal() diff --git a/kotlinx-coroutines-core/jvm/src/module-info.java b/kotlinx-coroutines-core/jvm/src/module-info.java new file mode 100644 index 0000000000..ad4c2ee066 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/module-info.java @@ -0,0 +1,28 @@ +import kotlinx.coroutines.CoroutineExceptionHandler; +import kotlinx.coroutines.internal.MainDispatcherFactory; + +module kotlinx.coroutines.core { + requires transitive kotlin.stdlib; + requires kotlinx.atomicfu; + + // these are used by kotlinx.coroutines.debug.internal.AgentPremain + requires static java.instrument; // contains java.lang.instrument.* + requires static jdk.unsupported; // contains sun.misc.Signal + + exports kotlinx.coroutines; + exports kotlinx.coroutines.channels; + exports kotlinx.coroutines.debug.internal; + exports kotlinx.coroutines.flow; + exports kotlinx.coroutines.flow.internal; + exports kotlinx.coroutines.future; + exports kotlinx.coroutines.internal; + exports kotlinx.coroutines.intrinsics; + exports kotlinx.coroutines.scheduling; + exports kotlinx.coroutines.selects; + exports kotlinx.coroutines.stream; + exports kotlinx.coroutines.sync; + exports kotlinx.coroutines.time; + + uses CoroutineExceptionHandler; + uses MainDispatcherFactory; +} diff --git a/kotlinx-coroutines-core/jvm/src/scheduling/CoroutineScheduler.kt b/kotlinx-coroutines-core/jvm/src/scheduling/CoroutineScheduler.kt new file mode 100644 index 0000000000..3430ebadec --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/scheduling/CoroutineScheduler.kt @@ -0,0 +1,1040 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import java.io.* +import java.util.concurrent.* +import java.util.concurrent.locks.* +import kotlin.jvm.internal.Ref.ObjectRef +import kotlin.math.* + +/** + * Coroutine scheduler (pool of shared threads) with a primary target to distribute dispatched coroutines + * over worker threads, including both CPU-intensive and potentially blocking tasks, in the most efficient manner. + * + * The current scheduler implementation has two optimization targets: + * - Efficiency in the face of communication patterns (e.g. actors communicating via channel). + * - Dynamic thread state and resizing to schedule blocking calls without re-dispatching coroutine to a separate "blocking" thread pool. + * + * ### Structural overview + * + * The scheduler consists of [corePoolSize] worker threads to execute CPU-bound tasks and up to + * [maxPoolSize] lazily created threads to execute blocking tasks. + * The scheduler has two global queues -- one for CPU tasks and one for blocking tasks. + * These queues are used for tasks that a submited externally (from threads not belonging to the scheduler) + * and as overflow buffers for thread-local queues. + * + * Every worker has a local queue in addition to global scheduler queues. + * The queue to pick the task from is selected randomly to avoid starvation of both local queue and + * global queue submitted tasks. + * Work-stealing is implemented on top of that queues to provide even load distribution and an illusion of centralized run queue. + * + * ### Scheduling policy + * + * When a coroutine is dispatched from within a scheduler worker, it's placed into the head of worker run queue. + * If the head is not empty, the task from the head is moved to the tail. Though it is an unfair scheduling policy, + * it effectively couples communicating coroutines into one and eliminates scheduling latency + * that arises from placing tasks to the end of the queue. + * Placing former head to the tail is necessary to provide semi-FIFO order, otherwise, queue degenerates to a stack. + * When a coroutine is dispatched from an external thread, it's put into the global queue. + * The original idea with a single-slot LIFO buffer comes from Golang runtime scheduler by D. Vyukov. + * It was proven to be "fair enough", performant and generally well accepted and initially was a significant inspiration + * source for the coroutine scheduler. + * + * ### Work stealing and affinity + * + * To provide even tasks distribution worker tries to steal tasks from other workers queues + * before parking when his local queue is empty. + * A non-standard solution is implemented to provide tasks affinity: a task from FIFO buffer may be stolen + * only if it is stale enough based on the value of [WORK_STEALING_TIME_RESOLUTION_NS]. + * For this purpose, monotonic global clock is used, and every task has a submission time associated with task. + * This approach shows outstanding results when coroutines are cooperative, + * but as a downside, the scheduler now depends on a high-resolution global clock, + * which may limit scalability on NUMA machines. + * + * ### Thread management + * + * One of the hardest parts of the scheduler is decentralized management of the threads with progress guarantees + * similar to the regular centralized executors. + * The state of the threads consists of [controlState] and [parkedWorkersStack] fields. + * The former field incorporates the number of created threads, CPU-tokens and blocking tasks + * that require thread compensation, + * while the latter represents an intrusive versioned Treiber stack of idle workers. + * When a worker cannot find any work, it first adds itself to the stack, + * then re-scans the queue to avoid missing signals and then attempts to park + * with an additional rendezvous against unnecessary parking. + * If a worker finds a task that it cannot yet steal due to time constraints, it stores this fact in its state + * (to be uncounted when additional work is signalled) and parks for such duration. + * + * When a new task arrives to the scheduler (whether it is a local or a global queue), + * either an idle worker is being signalled, or a new worker is attempted to be created. + * (Only [corePoolSize] workers can be created for regular CPU tasks) + * + * ### Support for blocking tasks + * + * The scheduler also supports the notion of [blocking][Task.isBlocking] tasks. + * When executing or enqueuing blocking tasks, the scheduler notifies or creates an additional worker in + * addition to the core pool size, so at any given moment, it has [corePoolSize] threads (potentially not yet created) + * available to serve CPU-bound tasks. To properly guarantee liveness, the scheduler maintains + * "CPU permits" -- #[corePoolSize] special tokens that allow an arbitrary worker to execute and steal CPU-bound tasks. + * When a worker encounters a blocking tasks, it releases its permit to the scheduler to + * keep an invariant "scheduler always has at least min(pending CPU tasks, core pool size) + * and at most core pool size threads to execute CPU tasks". + * To avoid overprovision, workers without CPU permit are allowed to scan [globalBlockingQueue] + * and steal **only** blocking tasks from other workers which imposes a non-trivial complexity to the queue management. + * + * The scheduler does not limit the count of pending blocking tasks, potentially creating up to [maxPoolSize] threads. + * End users do not have access to the scheduler directly and can dispatch blocking tasks only with + * [LimitingDispatcher] that does control concurrency level by its own mechanism. + */ +@Suppress("NOTHING_TO_INLINE") +internal class CoroutineScheduler( + @JvmField val corePoolSize: Int, + @JvmField val maxPoolSize: Int, + @JvmField val idleWorkerKeepAliveNs: Long = IDLE_WORKER_KEEP_ALIVE_NS, + @JvmField val schedulerName: String = DEFAULT_SCHEDULER_NAME +) : Executor, Closeable { + init { + require(corePoolSize >= MIN_SUPPORTED_POOL_SIZE) { + "Core pool size $corePoolSize should be at least $MIN_SUPPORTED_POOL_SIZE" + } + require(maxPoolSize >= corePoolSize) { + "Max pool size $maxPoolSize should be greater than or equals to core pool size $corePoolSize" + } + require(maxPoolSize <= MAX_SUPPORTED_POOL_SIZE) { + "Max pool size $maxPoolSize should not exceed maximal supported number of threads $MAX_SUPPORTED_POOL_SIZE" + } + require(idleWorkerKeepAliveNs > 0) { + "Idle worker keep alive time $idleWorkerKeepAliveNs must be positive" + } + } + + @JvmField + val globalCpuQueue = GlobalQueue() + + @JvmField + val globalBlockingQueue = GlobalQueue() + + private fun addToGlobalQueue(task: Task): Boolean { + return if (task.isBlocking) { + globalBlockingQueue.addLast(task) + } else { + globalCpuQueue.addLast(task) + } + } + + /** + * The stack of parker workers. + * Every worker registers itself in a stack before parking (if it was not previously registered), + * so it can be signalled when new tasks arrive. + * This is a form of intrusive garbage-free Treiber stack where [Worker] also is a stack node. + * + * The stack is better than a queue (even with the contention on top) because it unparks threads + * in most-recently used order, improving both performance and locality. + * Moreover, it decreases threads thrashing, if the pool has n threads when only n / 2 is required, + * the latter half will never be unparked and will terminate itself after [IDLE_WORKER_KEEP_ALIVE_NS]. + * + * This long version consist of version bits with [PARKED_VERSION_MASK] + * and top worker thread index bits with [PARKED_INDEX_MASK]. + */ + private val parkedWorkersStack = atomic(0L) + + /** + * Updates index of the worker at the top of [parkedWorkersStack]. + * It always updates version to ensure interference with [parkedWorkersStackPop] operation + * that might have already decided to put this index to the top. + * + * Note, [newIndex] can be zero for the worker that is being terminated (removed from [workers]). + */ + fun parkedWorkersStackTopUpdate(worker: Worker, oldIndex: Int, newIndex: Int) { + parkedWorkersStack.loop { top -> + val index = (top and PARKED_INDEX_MASK).toInt() + val updVersion = (top + PARKED_VERSION_INC) and PARKED_VERSION_MASK + val updIndex = if (index == oldIndex) { + if (newIndex == 0) { + parkedWorkersStackNextIndex(worker) + } else { + newIndex + } + } else { + index // no change to index, but update version + } + if (updIndex < 0) return@loop // retry + if (parkedWorkersStack.compareAndSet(top, updVersion or updIndex.toLong())) return + } + } + + /** + * Pushes worker into [parkedWorkersStack]. + * It does nothing is this worker is already physically linked to the stack. + * This method is invoked only from the worker thread itself. + * This invocation always precedes [LockSupport.parkNanos]. + * See [Worker.tryPark]. + * + * Returns `true` if worker was added to the stack by this invocation, `false` if it was already + * registered in the stack. + */ + fun parkedWorkersStackPush(worker: Worker): Boolean { + if (worker.nextParkedWorker !== NOT_IN_STACK) return false // already in stack, bail out + /* + * The below loop can be entered only if this worker was not in the stack and, since no other thread + * can add it to the stack (only the worker itself), this invariant holds while this loop executes. + */ + parkedWorkersStack.loop { top -> + val index = (top and PARKED_INDEX_MASK).toInt() + val updVersion = (top + PARKED_VERSION_INC) and PARKED_VERSION_MASK + val updIndex = worker.indexInArray + assert { updIndex != 0 } // only this worker can push itself, cannot be terminated + worker.nextParkedWorker = workers[index] + /* + * Other thread can be changing this worker's index at this point, but it + * also invokes parkedWorkersStackTopUpdate which updates version to make next CAS fail. + * Successful CAS of the stack top completes successful push. + */ + if (parkedWorkersStack.compareAndSet(top, updVersion or updIndex.toLong())) return true + } + } + + /** + * Pops worker from [parkedWorkersStack]. + * It can be invoked concurrently from any thread that is looking for help and needs to unpark some worker. + * This invocation is always followed by an attempt to [LockSupport.unpark] resulting worker. + * See [tryUnpark]. + */ + private fun parkedWorkersStackPop(): Worker? { + parkedWorkersStack.loop { top -> + val index = (top and PARKED_INDEX_MASK).toInt() + val worker = workers[index] ?: return null // stack is empty + val updVersion = (top + PARKED_VERSION_INC) and PARKED_VERSION_MASK + val updIndex = parkedWorkersStackNextIndex(worker) + if (updIndex < 0) return@loop // retry + /* + * Other thread can be changing this worker's index at this point, but it + * also invokes parkedWorkersStackTopUpdate which updates version to make next CAS fail. + * Successful CAS of the stack top completes successful pop. + */ + if (parkedWorkersStack.compareAndSet(top, updVersion or updIndex.toLong())) { + /* + * We've just took worker out of the stack, but nextParkerWorker is not reset yet, so if a worker is + * currently invoking parkedWorkersStackPush it would think it is in the stack and bail out without + * adding itself again. It does not matter, since we are going it invoke unpark on the thread + * that was popped out of parkedWorkersStack anyway. + */ + worker.nextParkedWorker = NOT_IN_STACK + return worker + } + } + } + + /** + * Finds next usable index for [parkedWorkersStack]. The problem is that workers can + * be terminated at their [Worker.indexInArray] becomes zero, so they cannot be + * put at the top of the stack. In which case we are looking for next. + * + * Returns `index >= 0` or `-1` for retry. + */ + private fun parkedWorkersStackNextIndex(worker: Worker): Int { + var next = worker.nextParkedWorker + findNext@ while (true) { + when { + next === NOT_IN_STACK -> return -1 // we are too late -- other thread popped this element, retry + next === null -> return 0 // stack becomes empty + else -> { + val nextWorker = next as Worker + val updIndex = nextWorker.indexInArray + if (updIndex != 0) return updIndex // found good index for next worker + // Otherwise, this worker was terminated and we cannot put it to top anymore, check next + next = nextWorker.nextParkedWorker + } + } + } + } + + /** + * State of worker threads. + * [workers] is a dynamic array of lazily created workers up to [maxPoolSize] workers. + * [createdWorkers] is count of already created workers (worker with index lesser than [createdWorkers] exists). + * [blockingTasks] is count of pending (either in the queue or being executed) blocking tasks. + * + * Workers array is also used as a lock for workers' creation and termination sequence. + * + * **NOTE**: `workers[0]` is always `null` (never used, works as sentinel value), so + * workers are 1-indexed, code path in [Worker.trySteal] is a bit faster and index swap during termination + * works properly. + * + * Initial size is `Dispatchers.Default` size * 2 to prevent unnecessary resizes for slightly or steadily loaded + * applications. + */ + @JvmField + val workers = ResizableAtomicArray((corePoolSize + 1) * 2) + + /** + * The `Long` value describing the state of workers in this pool. + * Currently, includes created, CPU-acquired, and blocking workers, each occupying [BLOCKING_SHIFT] bits. + * + * State layout (highest to lowest bits): + * | --- number of cpu permits, 22 bits --- | --- number of blocking tasks, 21 bits --- | --- number of created threads, 21 bits --- | + */ + private val controlState = atomic(corePoolSize.toLong() shl CPU_PERMITS_SHIFT) + + private val createdWorkers: Int inline get() = (controlState.value and CREATED_MASK).toInt() + private val availableCpuPermits: Int inline get() = availableCpuPermits(controlState.value) + + private inline fun createdWorkers(state: Long): Int = (state and CREATED_MASK).toInt() + private inline fun blockingTasks(state: Long): Int = (state and BLOCKING_MASK shr BLOCKING_SHIFT).toInt() + inline fun availableCpuPermits(state: Long): Int = (state and CPU_PERMITS_MASK shr CPU_PERMITS_SHIFT).toInt() + + // Guarded by synchronization + private inline fun incrementCreatedWorkers(): Int = createdWorkers(controlState.incrementAndGet()) + private inline fun decrementCreatedWorkers(): Int = createdWorkers(controlState.getAndDecrement()) + + private inline fun incrementBlockingTasks() = controlState.addAndGet(1L shl BLOCKING_SHIFT) + + private inline fun decrementBlockingTasks() { + controlState.addAndGet(-(1L shl BLOCKING_SHIFT)) + } + + private inline fun tryAcquireCpuPermit(): Boolean = controlState.loop { state -> + val available = availableCpuPermits(state) + if (available == 0) return false + val update = state - (1L shl CPU_PERMITS_SHIFT) + if (controlState.compareAndSet(state, update)) return true + } + + private inline fun releaseCpuPermit() = controlState.addAndGet(1L shl CPU_PERMITS_SHIFT) + + // This is used a "stop signal" for close and shutdown functions + private val _isTerminated = atomic(false) + val isTerminated: Boolean get() = _isTerminated.value + + companion object { + // A symbol to mark workers that are not in parkedWorkersStack + @JvmField + val NOT_IN_STACK = Symbol("NOT_IN_STACK") + + // Worker ctl states + private const val PARKED = -1 + private const val CLAIMED = 0 + private const val TERMINATED = 1 + + // Masks of control state + private const val BLOCKING_SHIFT = 21 // 2M threads max + private const val CREATED_MASK: Long = (1L shl BLOCKING_SHIFT) - 1 + private const val BLOCKING_MASK: Long = CREATED_MASK shl BLOCKING_SHIFT + private const val CPU_PERMITS_SHIFT = BLOCKING_SHIFT * 2 + private const val CPU_PERMITS_MASK = CREATED_MASK shl CPU_PERMITS_SHIFT + + internal const val MIN_SUPPORTED_POOL_SIZE = 1 // we support 1 for test purposes, but it is not usually used + internal const val MAX_SUPPORTED_POOL_SIZE = (1 shl BLOCKING_SHIFT) - 2 + + // Masks of parkedWorkersStack + private const val PARKED_INDEX_MASK = CREATED_MASK + private const val PARKED_VERSION_MASK = CREATED_MASK.inv() + private const val PARKED_VERSION_INC = 1L shl BLOCKING_SHIFT + } + + override fun execute(command: Runnable) = dispatch(command) + + override fun close() = shutdown(10_000L) + + // Shuts down current scheduler and waits until all work is done and all threads are stopped. + fun shutdown(timeout: Long) { + // atomically set termination flag which is checked when workers are added or removed + if (!_isTerminated.compareAndSet(false, true)) return + // make sure we are not waiting for the current thread + val currentWorker = currentWorker() + // Capture # of created workers that cannot change anymore (mind the synchronized block!) + val created = synchronized(workers) { createdWorkers } + // Shutdown all workers with the only exception of the current thread + for (i in 1..created) { + val worker = workers[i]!! + if (worker !== currentWorker) { + // Note: this is java.lang.Thread.getState() of type java.lang.Thread.State + while (worker.getState() != Thread.State.TERMINATED) { + LockSupport.unpark(worker) + worker.join(timeout) + } + // Note: this is CoroutineScheduler.Worker.state of type CoroutineScheduler.WorkerState + assert { worker.state === WorkerState.TERMINATED } // Expected TERMINATED state + worker.localQueue.offloadAllWorkTo(globalBlockingQueue) // Doesn't actually matter which queue to use + } + } + // Make sure no more work is added to GlobalQueue from anywhere + globalBlockingQueue.close() + globalCpuQueue.close() + // Finish processing tasks from globalQueue and/or from this worker's local queue + while (true) { + val task = currentWorker?.findTask(true) + ?: globalCpuQueue.removeFirstOrNull() + ?: globalBlockingQueue.removeFirstOrNull() + ?: break + runSafely(task) + } + // Shutdown current thread + currentWorker?.tryReleaseCpu(WorkerState.TERMINATED) + // check & cleanup state + assert { availableCpuPermits == corePoolSize } + parkedWorkersStack.value = 0L + controlState.value = 0L + } + + /** + * Dispatches execution of a runnable [block] with a hint to a scheduler whether + * this [block] may execute blocking operations (IO, system calls, locking primitives etc.) + * + * [taskContext] -- concurrency context of given [block]. + * [fair] -- whether this [dispatch] call is fair. + * If `true` then the task will be dispatched in a FIFO manner. + * Note that caller cannot be ensured that it is being executed on worker thread for the following reasons: + * - [CoroutineStart.UNDISPATCHED] + * - Concurrent [close] that effectively shutdowns the worker thread. + * Used for [yield]. + */ + fun dispatch(block: Runnable, taskContext: TaskContext = NonBlockingContext, fair: Boolean = false) { + trackTask() // this is needed for virtual time support + val task = createTask(block, taskContext) + val isBlockingTask = task.isBlocking + // Invariant: we increment counter **before** publishing the task + // so executing thread can safely decrement the number of blocking tasks + val stateSnapshot = if (isBlockingTask) incrementBlockingTasks() else 0 + // try to submit the task to the local queue and act depending on the result + val currentWorker = currentWorker() + val notAdded = currentWorker.submitToLocalQueue(task, fair) + if (notAdded != null) { + if (!addToGlobalQueue(notAdded)) { + // Global queue is closed in the last step of close/shutdown -- no more tasks should be accepted + throw RejectedExecutionException("$schedulerName was terminated") + } + } + // Checking 'task' instead of 'notAdded' is completely okay + if (isBlockingTask) { + // Use state snapshot to better estimate the number of running threads + signalBlockingWork(stateSnapshot) + } else { + signalCpuWork() + } + } + + fun createTask(block: Runnable, taskContext: TaskContext): Task { + val nanoTime = schedulerTimeSource.nanoTime() + if (block is Task) { + block.submissionTime = nanoTime + block.taskContext = taskContext + return block + } + return block.asTask(nanoTime, taskContext) + } + + // NB: should only be called from 'dispatch' method due to blocking tasks increment + private fun signalBlockingWork(stateSnapshot: Long) { + if (tryUnpark()) return + // Use state snapshot to avoid accidental thread overprovision + if (tryCreateWorker(stateSnapshot)) return + tryUnpark() // Try unpark again in case there was race between permit release and parking + } + + fun signalCpuWork() { + if (tryUnpark()) return + if (tryCreateWorker()) return + tryUnpark() + } + + private fun tryCreateWorker(state: Long = controlState.value): Boolean { + val created = createdWorkers(state) + val blocking = blockingTasks(state) + val cpuWorkers = (created - blocking).coerceAtLeast(0) + /* + * We check how many threads are there to handle non-blocking work, + * and create one more if we have not enough of them. + */ + if (cpuWorkers < corePoolSize) { + val newCpuWorkers = createNewWorker() + // If we've created the first cpu worker and corePoolSize > 1 then create + // one more (second) cpu worker, so that stealing between them is operational + if (newCpuWorkers == 1 && corePoolSize > 1) createNewWorker() + if (newCpuWorkers > 0) return true + } + return false + } + + private fun tryUnpark(): Boolean { + while (true) { + val worker = parkedWorkersStackPop() ?: return false + if (worker.workerCtl.compareAndSet(PARKED, CLAIMED)) { + LockSupport.unpark(worker) + return true + } + } + } + + /** + * Returns the number of CPU workers after this function (including new worker) or + * 0 if no worker was created. + */ + private fun createNewWorker(): Int { + val worker: Worker + return synchronized(workers) { + // Make sure we're not trying to resurrect terminated scheduler + if (isTerminated) return -1 + val state = controlState.value + val created = createdWorkers(state) + val blocking = blockingTasks(state) + val cpuWorkers = (created - blocking).coerceAtLeast(0) + // Double check for overprovision + if (cpuWorkers >= corePoolSize) return 0 + if (created >= maxPoolSize) return 0 + // start & register new worker, commit index only after successful creation + val newIndex = createdWorkers + 1 + require(newIndex > 0 && workers[newIndex] == null) + /* + * 1) Claim the slot (under a lock) by the newly created worker + * 2) Make it observable by increment created workers count + * 3) Only then start the worker, otherwise it may miss its own creation + */ + worker = Worker(newIndex) + workers.setSynchronized(newIndex, worker) + require(newIndex == incrementCreatedWorkers()) + cpuWorkers + 1 + }.also { worker.start() } // Start worker when the lock is released to reduce contention, see #3652 + } + + /** + * Returns `null` if task was successfully added or an instance of the + * task that was not added or replaced (thus should be added to global queue). + */ + private fun Worker?.submitToLocalQueue(task: Task, fair: Boolean): Task? { + if (this == null) return task + /* + * This worker could have been already terminated from this thread by close/shutdown and it should not + * accept any more tasks into its local queue. + */ + if (state === WorkerState.TERMINATED) return task + // Do not add CPU tasks in local queue if we are not able to execute it + if (!task.isBlocking && state === WorkerState.BLOCKING) { + return task + } + mayHaveLocalTasks = true + return localQueue.add(task, fair = fair) + } + + private fun currentWorker(): Worker? = (Thread.currentThread() as? Worker)?.takeIf { it.scheduler == this } + + /** + * Returns a string identifying the state of this scheduler for nicer debugging. + * Note that this method is not atomic and represents rough state of pool. + * + * State of the queues: + * b for blocking, c for CPU, r for retiring. + * E.g. for [1b, 1b, 2c, 1d] means that pool has + * two blocking workers with queue size 1, one worker with CPU permit and queue size 1 + * and one dormant (executing his local queue before parking) worker with queue size 1. + */ + override fun toString(): String { + var parkedWorkers = 0 + var blockingWorkers = 0 + var cpuWorkers = 0 + var dormant = 0 + var terminated = 0 + val queueSizes = arrayListOf() + for (index in 1 until workers.currentLength()) { + val worker = workers[index] ?: continue + val queueSize = worker.localQueue.size + when (worker.state) { + WorkerState.PARKING -> ++parkedWorkers + WorkerState.BLOCKING -> { + ++blockingWorkers + queueSizes += queueSize.toString() + "b" // Blocking + } + + WorkerState.CPU_ACQUIRED -> { + ++cpuWorkers + queueSizes += queueSize.toString() + "c" // CPU + } + + WorkerState.DORMANT -> { + ++dormant + if (queueSize > 0) queueSizes += queueSize.toString() + "d" // Retiring + } + + WorkerState.TERMINATED -> ++terminated + } + } + val state = controlState.value + return "$schedulerName@$hexAddress[" + + "Pool Size {" + + "core = $corePoolSize, " + + "max = $maxPoolSize}, " + + "Worker States {" + + "CPU = $cpuWorkers, " + + "blocking = $blockingWorkers, " + + "parked = $parkedWorkers, " + + "dormant = $dormant, " + + "terminated = $terminated}, " + + "running workers queues = $queueSizes, " + + "global CPU queue size = ${globalCpuQueue.size}, " + + "global blocking queue size = ${globalBlockingQueue.size}, " + + "Control State {" + + "created workers= ${createdWorkers(state)}, " + + "blocking tasks = ${blockingTasks(state)}, " + + "CPUs acquired = ${corePoolSize - availableCpuPermits(state)}" + + "}]" + } + + fun runSafely(task: Task) { + try { + task.run() + } catch (e: Throwable) { + val thread = Thread.currentThread() + thread.uncaughtExceptionHandler.uncaughtException(thread, e) + } finally { + unTrackTask() + } + } + + internal inner class Worker private constructor() : Thread() { + init { + isDaemon = true + /* + * `Dispatchers.Default` is used as *the* dispatcher in the containerized environments, + * isolated by their own classloaders. Workers are populated lazily, thus we are inheriting + * `Dispatchers.Default` context class loader here instead of using parent' thread one + * in order not to accidentally capture temporary application classloader. + */ + contextClassLoader = this@CoroutineScheduler.javaClass.classLoader + } + + // guarded by scheduler lock, index in workers array, 0 when not in array (terminated) + @Volatile // volatile for push/pop operation into parkedWorkersStack + var indexInArray = 0 + set(index) { + name = "$schedulerName-worker-${if (index == 0) "TERMINATED" else index.toString()}" + field = index + } + + constructor(index: Int) : this() { + indexInArray = index + } + + inline val scheduler get() = this@CoroutineScheduler + + @JvmField + val localQueue: WorkQueue = WorkQueue() + + /** + * Slot that is used to steal tasks into to avoid re-adding them + * to the local queue. See [trySteal] + */ + private val stolenTask: ObjectRef = ObjectRef() + + /** + * Worker state. **Updated only by this worker thread**. + * By default, worker is in DORMANT state in the case when it was created, but all CPU tokens or tasks were taken. + * Is used locally by the worker to maintain its own invariants. + */ + @JvmField + var state = WorkerState.DORMANT + + /** + * Worker control state responsible for worker claiming, parking and termination. + * List of states: + * [PARKED] -- worker is parked and can self-terminate after a termination deadline. + * [CLAIMED] -- worker is claimed by an external submitter. + * [TERMINATED] -- worker is terminated and no longer usable. + */ + val workerCtl = atomic(CLAIMED) + + /** + * It is set to the termination deadline when started doing [park] and it reset + * when there is a task. It serves as protection against spurious wakeups of parkNanos. + */ + private var terminationDeadline = 0L + + /** + * Reference to the next worker in the [parkedWorkersStack]. + * It may be `null` if there is no next parked worker. + * This reference is set to [NOT_IN_STACK] when worker is physically not in stack. + */ + @Volatile + var nextParkedWorker: Any? = NOT_IN_STACK + + /* + * The delay until at least one task in other worker queues will become stealable. + */ + private var minDelayUntilStealableTaskNs = 0L + + /** + * The state of embedded Marsaglia xorshift random number generator, used for work-stealing purposes. + * It is initialized with a seed. + */ + private var rngState: Int = run { + // This could've been Random.nextInt(), but we are shaving an extra initialization cost, see #4051 + val seed = System.nanoTime().toInt() + // rngState shouldn't be zero, as required for the xorshift algorithm + if (seed != 0) return@run seed + 42 + } + + /** + * Tries to acquire CPU token if worker doesn't have one + * @return whether worker acquired (or already had) CPU token + */ + private fun tryAcquireCpuPermit(): Boolean = when { + state == WorkerState.CPU_ACQUIRED -> true + this@CoroutineScheduler.tryAcquireCpuPermit() -> { + state = WorkerState.CPU_ACQUIRED + true + } + + else -> false + } + + /** + * Releases CPU token if worker has any and changes state to [newState]. + * Returns `true` if CPU permit was returned to the pool + */ + fun tryReleaseCpu(newState: WorkerState): Boolean { + val previousState = state + val hadCpu = previousState == WorkerState.CPU_ACQUIRED + if (hadCpu) releaseCpuPermit() + if (previousState != newState) state = newState + return hadCpu + } + + override fun run() = runWorker() + + @JvmField + var mayHaveLocalTasks = false + + private fun runWorker() { + var rescanned = false + while (!isTerminated && state != WorkerState.TERMINATED) { + val task = findTask(mayHaveLocalTasks) + // Task found. Execute and repeat + if (task != null) { + rescanned = false + minDelayUntilStealableTaskNs = 0L + executeTask(task) + continue + } else { + mayHaveLocalTasks = false + } + /* + * No tasks were found: + * 1) Either at least one of the workers has stealable task in its FIFO-buffer with a stealing deadline. + * Then its deadline is stored in [minDelayUntilStealableTask] + * // '2)' can be found below + * + * Then just park for that duration (ditto re-scanning). + * While it could potentially lead to short (up to WORK_STEALING_TIME_RESOLUTION_NS ns) starvations, + * excess unparks and managing "one unpark per signalling" invariant become unfeasible, instead we are going to resolve + * it with "spinning via scans" mechanism. + * NB: this short potential parking does not interfere with `tryUnpark` + */ + if (minDelayUntilStealableTaskNs != 0L) { + if (!rescanned) { + rescanned = true + } else { + rescanned = false + tryReleaseCpu(WorkerState.PARKING) + interrupted() + LockSupport.parkNanos(minDelayUntilStealableTaskNs) + minDelayUntilStealableTaskNs = 0L + } + continue + } + /* + * 2) Or no tasks available, time to park and, potentially, shut down the thread. + * Add itself to the stack of parked workers, re-scans all the queues + * to avoid missing wake-up (requestCpuWorker) and either starts executing discovered tasks or parks itself awaiting for new tasks. + */ + tryPark() + } + tryReleaseCpu(WorkerState.TERMINATED) + } + + /** + * See [runSingleTaskFromCurrentSystemDispatcher] for rationale and details. + * This is a fine-tailored method for a specific use-case not expected to be used widely. + */ + fun runSingleTask(): Long { + val stateSnapshot = state + val isCpuThread = state == WorkerState.CPU_ACQUIRED + val task = if (isCpuThread) { + findCpuTask() + } else { + findBlockingTask() + } + if (task == null) { + if (minDelayUntilStealableTaskNs == 0L) return -1L + return minDelayUntilStealableTaskNs + } + runSafely(task) + if (!isCpuThread) decrementBlockingTasks() + assert { state == stateSnapshot } + return 0L + } + + fun isIo() = state == WorkerState.BLOCKING + + // Counterpart to "tryUnpark" + private fun tryPark() { + if (!inStack()) { + parkedWorkersStackPush(this) + return + } + workerCtl.value = PARKED // Update value once + /* + * inStack() prevents spurious wakeups, while workerCtl.value == PARKED + * prevents the following race: + * + * - T2 scans the queue, adds itself to the stack, goes to rescan + * - T2 suspends in 'workerCtl.value = PARKED' line + * - T1 pops T2 from the stack, claims workerCtl, suspends + * - T2 fails 'while (inStack())' check, goes to full rescan + * - T2 adds itself to the stack, parks + * - T1 unparks T2, bails out with success + * - T2 unparks and loops in 'while (inStack())' + */ + while (inStack() && workerCtl.value == PARKED) { // Prevent spurious wakeups + if (isTerminated || state == WorkerState.TERMINATED) break + tryReleaseCpu(WorkerState.PARKING) + interrupted() // Cleanup interruptions + park() + } + } + + private fun inStack(): Boolean = nextParkedWorker !== NOT_IN_STACK + + private fun executeTask(task: Task) { + terminationDeadline = 0L // reset deadline for termination + if (state == WorkerState.PARKING) { + assert { task.isBlocking } + state = WorkerState.BLOCKING + } + if (task.isBlocking) { + // Always notify about new work when releasing CPU-permit to execute some blocking task + if (tryReleaseCpu(WorkerState.BLOCKING)) { + signalCpuWork() + } + runSafely(task) + decrementBlockingTasks() + val currentState = state + // Shutdown sequence of blocking dispatcher + if (currentState !== WorkerState.TERMINATED) { + assert { currentState == WorkerState.BLOCKING } // "Expected BLOCKING state, but has $currentState" + state = WorkerState.DORMANT + } + } else { + runSafely(task) + } + } + + /* + * Marsaglia xorshift RNG with period 2^32-1 for work stealing purposes. + * ThreadLocalRandom cannot be used to support Android and ThreadLocal is up to 15% slower on Ktor benchmarks + */ + fun nextInt(upperBound: Int): Int { + var r = rngState + r = r xor (r shl 13) + r = r xor (r shr 17) + r = r xor (r shl 5) + rngState = r + val mask = upperBound - 1 + // Fast path for power of two bound + if (mask and upperBound == 0) { + return r and mask + } + return (r and Int.MAX_VALUE) % upperBound + } + + private fun park() { + // set termination deadline the first time we are here (it is reset in idleReset) + if (terminationDeadline == 0L) terminationDeadline = System.nanoTime() + idleWorkerKeepAliveNs + // actually park + LockSupport.parkNanos(idleWorkerKeepAliveNs) + // try terminate when we are idle past termination deadline + // note that comparison is written like this to protect against potential nanoTime wraparound + if (System.nanoTime() - terminationDeadline >= 0) { + terminationDeadline = 0L // if attempt to terminate worker fails we'd extend deadline again + tryTerminateWorker() + } + } + + /** + * Stops execution of current thread and removes it from [createdWorkers]. + */ + private fun tryTerminateWorker() { + synchronized(workers) { + // Make sure we're not trying race with termination of scheduler + if (isTerminated) return + // Someone else terminated, bail out + if (createdWorkers <= corePoolSize) return + /* + * See tryUnpark for state reasoning. + * If this CAS fails, then we were successfully unparked by other worker and cannot terminate. + */ + if (!workerCtl.compareAndSet(PARKED, TERMINATED)) return + /* + * At this point this thread is no longer considered as usable for scheduling. + * We need multi-step choreography to reindex workers. + * + * 1) Read current worker's index and reset it to zero. + */ + val oldIndex = indexInArray + indexInArray = 0 + /* + * Now this worker cannot become the top of parkedWorkersStack, but it can + * still be at the stack top via oldIndex. + * + * 2) Update top of stack if it was pointing to oldIndex and make sure no + * pending push/pop operation that might have already retrieved oldIndex could complete. + */ + parkedWorkersStackTopUpdate(this, oldIndex, 0) + /* + * 3) Move last worker into an index in array that was previously occupied by this worker, + * if last worker was a different one (sic!). + */ + val lastIndex = decrementCreatedWorkers() + if (lastIndex != oldIndex) { + val lastWorker = workers[lastIndex]!! + workers.setSynchronized(oldIndex, lastWorker) + lastWorker.indexInArray = oldIndex + /* + * Now lastWorker is available at both indices in the array, but it can + * still be at the stack top on via its lastIndex + * + * 4) Update top of stack lastIndex -> oldIndex and make sure no + * pending push/pop operation that might have already retrieved lastIndex could complete. + */ + parkedWorkersStackTopUpdate(lastWorker, lastIndex, oldIndex) + } + /* + * 5) It is safe to clear reference from workers array now. + */ + workers.setSynchronized(lastIndex, null) + } + state = WorkerState.TERMINATED + } + + fun findTask(mayHaveLocalTasks: Boolean): Task? { + if (tryAcquireCpuPermit()) return findAnyTask(mayHaveLocalTasks) + /* + * If we can't acquire a CPU permit, attempt to find blocking task: + * - Check if our queue has one (maybe mixed in with CPU tasks) + * - Poll global and try steal + */ + return findBlockingTask() + } + + // NB: ONLY for runSingleTask method + private fun findBlockingTask(): Task? { + return localQueue.pollBlocking() + ?: globalBlockingQueue.removeFirstOrNull() + ?: trySteal(STEAL_BLOCKING_ONLY) + } + + // NB: ONLY for runSingleTask method + private fun findCpuTask(): Task? { + return localQueue.pollCpu() + ?: globalBlockingQueue.removeFirstOrNull() + ?: trySteal(STEAL_CPU_ONLY) + } + + private fun findAnyTask(scanLocalQueue: Boolean): Task? { + /* + * Anti-starvation mechanism: probabilistically poll either local + * or global queue to ensure progress for both external and internal tasks. + */ + if (scanLocalQueue) { + val globalFirst = nextInt(2 * corePoolSize) == 0 + if (globalFirst) pollGlobalQueues()?.let { return it } + localQueue.poll()?.let { return it } + if (!globalFirst) pollGlobalQueues()?.let { return it } + } else { + pollGlobalQueues()?.let { return it } + } + return trySteal(STEAL_ANY) + } + + private fun pollGlobalQueues(): Task? { + if (nextInt(2) == 0) { + globalCpuQueue.removeFirstOrNull()?.let { return it } + return globalBlockingQueue.removeFirstOrNull() + } else { + globalBlockingQueue.removeFirstOrNull()?.let { return it } + return globalCpuQueue.removeFirstOrNull() + } + } + + private fun trySteal(stealingMode: StealingMode): Task? { + val created = createdWorkers + // 0 to await an initialization and 1 to avoid excess stealing on single-core machines + if (created < 2) { + return null + } + + var currentIndex = nextInt(created) + var minDelay = Long.MAX_VALUE + repeat(created) { + ++currentIndex + if (currentIndex > created) currentIndex = 1 + val worker = workers[currentIndex] + if (worker !== null && worker !== this) { + val stealResult = worker.localQueue.trySteal(stealingMode, stolenTask) + if (stealResult == TASK_STOLEN) { + val result = stolenTask.element + stolenTask.element = null + return result + } else if (stealResult > 0) { + minDelay = min(minDelay, stealResult) + } + } + } + minDelayUntilStealableTaskNs = if (minDelay != Long.MAX_VALUE) minDelay else 0 + return null + } + } + + enum class WorkerState { + /** + * Has CPU token and either executes a [Task.isBlocking]` == false` task or tries to find one. + */ + CPU_ACQUIRED, + + /** + * Executing task with [Task.isBlocking]. + */ + BLOCKING, + + /** + * Currently parked. + */ + PARKING, + + /** + * Tries to execute its local work and then goes to infinite sleep as no longer needed worker. + */ + DORMANT, + + /** + * Terminal state, will no longer be used + */ + TERMINATED + } +} + +/** + * Checks if the thread is part of a thread pool that supports coroutines. + * This function is needed for integration with BlockHound. + */ +@JvmName("isSchedulerWorker") +internal fun isSchedulerWorker(thread: Thread) = thread is CoroutineScheduler.Worker + +/** + * Checks if the thread is running a CPU-bound task. + * This function is needed for integration with BlockHound. + */ +@JvmName("mayNotBlock") +internal fun mayNotBlock(thread: Thread) = thread is CoroutineScheduler.Worker && + thread.state == CoroutineScheduler.WorkerState.CPU_ACQUIRED diff --git a/kotlinx-coroutines-core/jvm/src/scheduling/Dispatcher.kt b/kotlinx-coroutines-core/jvm/src/scheduling/Dispatcher.kt new file mode 100644 index 0000000000..28d5537108 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/scheduling/Dispatcher.kt @@ -0,0 +1,152 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import java.util.concurrent.* +import kotlin.coroutines.* + +// Instance of Dispatchers.Default +internal object DefaultScheduler : SchedulerCoroutineDispatcher( + CORE_POOL_SIZE, MAX_POOL_SIZE, + IDLE_WORKER_KEEP_ALIVE_NS, DEFAULT_SCHEDULER_NAME +) { + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + parallelism.checkParallelism() + if (parallelism >= CORE_POOL_SIZE) { + return namedOrThis(name) + } + return super.limitedParallelism(parallelism, name) + } + + // Shuts down the dispatcher, used only by Dispatchers.shutdown() + internal fun shutdown() { + super.close() + } + + // Overridden in case anyone writes (Dispatchers.Default as ExecutorCoroutineDispatcher).close() + override fun close() { + throw UnsupportedOperationException("Dispatchers.Default cannot be closed") + } + + override fun toString(): String = "Dispatchers.Default" +} + +// The unlimited instance of Dispatchers.IO that utilizes all the threads CoroutineScheduler provides +private object UnlimitedIoScheduler : CoroutineDispatcher() { + + @InternalCoroutinesApi + override fun dispatchYield(context: CoroutineContext, block: Runnable) { + DefaultScheduler.dispatchWithContext(block, BlockingContext, true) + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + DefaultScheduler.dispatchWithContext(block, BlockingContext, false) + } + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + parallelism.checkParallelism() + if (parallelism >= MAX_POOL_SIZE) { + return namedOrThis(name) + } + return super.limitedParallelism(parallelism, name) + } + + // This name only leaks to user code as part of .limitedParallelism machinery + override fun toString(): String { + return "Dispatchers.IO" + } +} + +// Dispatchers.IO +internal object DefaultIoScheduler : ExecutorCoroutineDispatcher(), Executor { + + private val default = UnlimitedIoScheduler.limitedParallelism( + systemProp( + IO_PARALLELISM_PROPERTY_NAME, + 64.coerceAtLeast(AVAILABLE_PROCESSORS) + ) + ) + + override val executor: Executor + get() = this + + override fun execute(command: java.lang.Runnable) = dispatch(EmptyCoroutineContext, command) + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + // See documentation to Dispatchers.IO for the rationale + return UnlimitedIoScheduler.limitedParallelism(parallelism, name) + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + default.dispatch(context, block) + } + + @InternalCoroutinesApi + override fun dispatchYield(context: CoroutineContext, block: Runnable) { + default.dispatchYield(context, block) + } + + override fun close() { + error("Cannot be invoked on Dispatchers.IO") + } + + override fun toString(): String = "Dispatchers.IO" +} + +// Instantiated in tests so we can test it in isolation +internal open class SchedulerCoroutineDispatcher( + private val corePoolSize: Int = CORE_POOL_SIZE, + private val maxPoolSize: Int = MAX_POOL_SIZE, + private val idleWorkerKeepAliveNs: Long = IDLE_WORKER_KEEP_ALIVE_NS, + private val schedulerName: String = "CoroutineScheduler", +) : ExecutorCoroutineDispatcher() { + + override val executor: Executor + get() = coroutineScheduler + + // This is variable for test purposes, so that we can reinitialize from clean state + private var coroutineScheduler = createScheduler() + + private fun createScheduler() = + CoroutineScheduler(corePoolSize, maxPoolSize, idleWorkerKeepAliveNs, schedulerName) + + override fun dispatch(context: CoroutineContext, block: Runnable): Unit = coroutineScheduler.dispatch(block) + + override fun dispatchYield(context: CoroutineContext, block: Runnable): Unit { + /* + * 'dispatchYield' implementation is needed to address the scheduler's scheduling policy. + * By default, the scheduler dispatches tasks in a semi-LIFO order, meaning that for the + * task sequence [#1, #2, #3], the scheduling of task #4 will produce + * [#4, #1, #2, #3], allocates new worker and makes #4 stealable after some time. + * On a fast enough system, it means that `while (true) { yield() }` might obstruct the progress + * of the system and potentially starve it. + * To mitigate that, `dispatchYield` is a dedicated entry point that produces [#1, #2, #3, #4] + */ + coroutineScheduler.dispatch(block, fair = true) + } + + internal fun dispatchWithContext(block: Runnable, context: TaskContext, fair: Boolean) { + coroutineScheduler.dispatch(block, context, fair) + } + + override fun close() { + coroutineScheduler.close() + } + + // fot tests only + @Synchronized + internal fun usePrivateScheduler() { + coroutineScheduler.shutdown(1_000L) + coroutineScheduler = createScheduler() + } + + // for tests only + @Synchronized + internal fun shutdown(timeout: Long) { + coroutineScheduler.shutdown(timeout) + } + + // for tests only + internal fun restore() = usePrivateScheduler() // recreate scheduler +} diff --git a/kotlinx-coroutines-core/jvm/src/scheduling/Tasks.kt b/kotlinx-coroutines-core/jvm/src/scheduling/Tasks.kt new file mode 100644 index 0000000000..bdf6335d98 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/scheduling/Tasks.kt @@ -0,0 +1,106 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import java.util.concurrent.* + + +/** + * The name of the default scheduler. The names of the worker threads of [Dispatchers.Default] have it as their prefix. + */ +@JvmField +internal val DEFAULT_SCHEDULER_NAME = systemProp( + "kotlinx.coroutines.scheduler.default.name", "DefaultDispatcher" +) + +// 100us as default +@JvmField +internal val WORK_STEALING_TIME_RESOLUTION_NS = systemProp( + "kotlinx.coroutines.scheduler.resolution.ns", 100000L +) + +/** + * The maximum number of threads allocated for CPU-bound tasks at the default set of dispatchers. + * + * NOTE: we coerce default to at least two threads to give us chances that multi-threading problems + * get reproduced even on a single-core machine, but support explicit setting of 1 thread scheduler if needed + */ +@JvmField +internal val CORE_POOL_SIZE = systemProp( + "kotlinx.coroutines.scheduler.core.pool.size", + AVAILABLE_PROCESSORS.coerceAtLeast(2), + minValue = CoroutineScheduler.MIN_SUPPORTED_POOL_SIZE +) + +/** The maximum number of threads allocated for blocking tasks at the default set of dispatchers. */ +@JvmField +internal val MAX_POOL_SIZE = systemProp( + "kotlinx.coroutines.scheduler.max.pool.size", + CoroutineScheduler.MAX_SUPPORTED_POOL_SIZE, + maxValue = CoroutineScheduler.MAX_SUPPORTED_POOL_SIZE +) + +@JvmField +internal val IDLE_WORKER_KEEP_ALIVE_NS = TimeUnit.SECONDS.toNanos( + systemProp("kotlinx.coroutines.scheduler.keep.alive.sec", 60L) +) + +@JvmField +internal var schedulerTimeSource: SchedulerTimeSource = NanoTimeSource + +/** + * Concurrency context of a task. + * + * Currently, it only signifies whether the task is blocking or non-blocking. + */ +internal typealias TaskContext = Boolean + +/** + * This would be [TaskContext.toString] if [TaskContext] was a proper class. + */ +private fun taskContextString(taskContext: TaskContext): String = if (taskContext) "Blocking" else "Non-blocking" + +internal const val NonBlockingContext: TaskContext = false + +internal const val BlockingContext: TaskContext = true + +/** + * A scheduler task. + */ +internal abstract class Task( + @JvmField var submissionTime: Long, + @JvmField var taskContext: TaskContext +) : Runnable { + internal constructor() : this(0, NonBlockingContext) +} + +internal inline val Task.isBlocking get() = taskContext + +internal fun Runnable.asTask(submissionTime: Long, taskContext: TaskContext): Task = + TaskImpl(this, submissionTime, taskContext) + +// Non-reusable Task implementation to wrap Runnable instances that do not otherwise implement task +private class TaskImpl( + @JvmField val block: Runnable, + submissionTime: Long, + taskContext: TaskContext +) : Task(submissionTime, taskContext) { + override fun run() { + block.run() + } + + override fun toString(): String = + "Task[${block.classSimpleName}@${block.hexAddress}, $submissionTime, ${taskContextString(taskContext)}]" +} + +// Open for tests +internal class GlobalQueue : LockFreeTaskQueue(singleConsumer = false) + +// Was previously TimeSource, renamed due to KT-42625 and KT-23727 +internal abstract class SchedulerTimeSource { + abstract fun nanoTime(): Long +} + +internal object NanoTimeSource : SchedulerTimeSource() { + override fun nanoTime() = System.nanoTime() +} diff --git a/kotlinx-coroutines-core/jvm/src/scheduling/WorkQueue.kt b/kotlinx-coroutines-core/jvm/src/scheduling/WorkQueue.kt new file mode 100644 index 0000000000..a048d75d55 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/scheduling/WorkQueue.kt @@ -0,0 +1,250 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import java.util.concurrent.atomic.* +import kotlin.jvm.internal.Ref.ObjectRef + +internal const val BUFFER_CAPACITY_BASE = 7 +internal const val BUFFER_CAPACITY = 1 shl BUFFER_CAPACITY_BASE +internal const val MASK = BUFFER_CAPACITY - 1 // 128 by default + +internal const val TASK_STOLEN = -1L +internal const val NOTHING_TO_STEAL = -2L + +internal typealias StealingMode = Int +internal const val STEAL_ANY: StealingMode = 3 +internal const val STEAL_CPU_ONLY: StealingMode = 2 +internal const val STEAL_BLOCKING_ONLY: StealingMode = 1 + +internal inline val Task.maskForStealingMode: Int + get() = if (isBlocking) STEAL_BLOCKING_ONLY else STEAL_CPU_ONLY + +/** + * Tightly coupled with [CoroutineScheduler] queue of pending tasks, but extracted to separate file for simplicity. + * At any moment queue is used only by [CoroutineScheduler.Worker] threads, has only one producer (worker owning this queue) + * and any amount of consumers, other pool workers which are trying to steal work. + * + * ### Fairness + * + * [WorkQueue] provides semi-FIFO order, but with priority for most recently submitted task assuming + * that these two (current one and submitted) are communicating and sharing state thus making such communication extremely fast. + * E.g. submitted jobs [1, 2, 3, 4] will be executed in [4, 1, 2, 3] order. + * + * ### Algorithm and implementation details + * This is a regular SPMC bounded queue with the additional property that tasks can be removed from the middle of the queue + * (scheduler workers without a CPU permit steal blocking tasks via this mechanism). Such property enforces us to use CAS in + * order to properly claim value from the buffer. + * Moreover, [Task] objects are reusable, so it may seem that this queue is prone to ABA problem. + * Indeed, it formally has ABA-problem, but the whole processing logic is written in the way that such ABA is harmless. + * I have discovered a truly marvelous proof of this, which this KDoc is too narrow to contain. + */ +internal class WorkQueue { + + /* + * We read two independent counter here. + * Producer index is incremented only by owner + * Consumer index is incremented both by owner and external threads + * + * The only harmful race is: + * [T1] readProducerIndex (1) preemption(2) readConsumerIndex(5) + * [T2] changeProducerIndex (3) + * [T3] changeConsumerIndex (4) + * + * Which can lead to resulting size being negative or bigger than actual size at any moment of time. + * This is in general harmless because steal will be blocked by timer. + * Negative sizes can be observed only when non-owner reads the size, which happens only + * for diagnostic toString(). + */ + private val bufferSize: Int get() = producerIndex.value - consumerIndex.value + internal val size: Int get() = if (lastScheduledTask.value != null) bufferSize + 1 else bufferSize + private val buffer: AtomicReferenceArray = AtomicReferenceArray(BUFFER_CAPACITY) + private val lastScheduledTask = atomic(null) + + private val producerIndex = atomic(0) + private val consumerIndex = atomic(0) + // Shortcut to avoid scanning queue without blocking tasks + private val blockingTasksInBuffer = atomic(0) + + /** + * Retrieves and removes task from the head of the queue + * Invariant: this method is called only by the owner of the queue. + */ + fun poll(): Task? = lastScheduledTask.getAndSet(null) ?: pollBuffer() + + /** + * Invariant: Called only by the owner of the queue, returns + * `null` if task was added, task that wasn't added otherwise. + */ + fun add(task: Task, fair: Boolean = false): Task? { + if (fair) return addLast(task) + val previous = lastScheduledTask.getAndSet(task) ?: return null + return addLast(previous) + } + + /** + * Invariant: Called only by the owner of the queue, returns + * `null` if task was added, task that wasn't added otherwise. + */ + private fun addLast(task: Task): Task? { + if (bufferSize == BUFFER_CAPACITY - 1) return task + if (task.isBlocking) blockingTasksInBuffer.incrementAndGet() + val nextIndex = producerIndex.value and MASK + /* + * If current element is not null then we're racing with a really slow consumer that committed the consumer index, + * but hasn't yet nulled out the slot, effectively preventing us from using it. + * Such situations are very rare in practise (although possible) and we decided to give up a progress guarantee + * to have a stronger invariant "add to queue with bufferSize == 0 is always successful". + * This algorithm can still be wait-free for add, but if and only if tasks are not reusable, otherwise + * nulling out the buffer wouldn't be possible. + */ + while (buffer[nextIndex] != null) { + Thread.yield() + } + buffer.lazySet(nextIndex, task) + producerIndex.incrementAndGet() + return null + } + + /** + * Tries stealing from this queue into the [stolenTaskRef] argument. + * + * Returns [NOTHING_TO_STEAL] if queue has nothing to steal, [TASK_STOLEN] if at least task was stolen + * or positive value of how many nanoseconds should pass until the head of this queue will be available to steal. + * + * [StealingMode] controls what tasks to steal: + * - [STEAL_ANY] is default mode for scheduler, task from the head (in FIFO order) is stolen + * - [STEAL_BLOCKING_ONLY] is mode for stealing *an arbitrary* blocking task, which is used by the scheduler when helping in Dispatchers.IO mode + * - [STEAL_CPU_ONLY] is a kludge for `runSingleTaskFromCurrentSystemDispatcher` + */ + fun trySteal(stealingMode: StealingMode, stolenTaskRef: ObjectRef): Long { + val task = when (stealingMode) { + STEAL_ANY -> pollBuffer() + else -> stealWithExclusiveMode(stealingMode) + } + + if (task != null) { + stolenTaskRef.element = task + return TASK_STOLEN + } + return tryStealLastScheduled(stealingMode, stolenTaskRef) + } + + // Steal only tasks of a particular kind, potentially invoking full queue scan + private fun stealWithExclusiveMode(stealingMode: StealingMode): Task? { + var start = consumerIndex.value + val end = producerIndex.value + val onlyBlocking = stealingMode == STEAL_BLOCKING_ONLY + // Bail out if there is no blocking work for us + while (start != end) { + if (onlyBlocking && blockingTasksInBuffer.value == 0) return null + return tryExtractFromTheMiddle(start++, onlyBlocking) ?: continue + } + + return null + } + + // Polls for blocking task, invoked only by the owner + // NB: ONLY for runSingleTask method + fun pollBlocking(): Task? = pollWithExclusiveMode(onlyBlocking = true /* only blocking */) + + // Polls for CPU task, invoked only by the owner + // NB: ONLY for runSingleTask method + fun pollCpu(): Task? = pollWithExclusiveMode(onlyBlocking = false /* only cpu */) + + private fun pollWithExclusiveMode(/* Only blocking OR only CPU */ onlyBlocking: Boolean): Task? { + while (true) { // Poll the slot + val lastScheduled = lastScheduledTask.value ?: break + if (lastScheduled.isBlocking != onlyBlocking) break + if (lastScheduledTask.compareAndSet(lastScheduled, null)) { + return lastScheduled + } // Failed -> someone else stole it + } + + // Failed to poll the slot, scan the queue + val start = consumerIndex.value + var end = producerIndex.value + // Bail out if there is no blocking work for us + while (start != end) { + if (onlyBlocking && blockingTasksInBuffer.value == 0) return null + val task = tryExtractFromTheMiddle(--end, onlyBlocking) + if (task != null) { + return task + } + } + return null + } + + private fun tryExtractFromTheMiddle(index: Int, onlyBlocking: Boolean): Task? { + val arrayIndex = index and MASK + val value = buffer[arrayIndex] + if (value != null && value.isBlocking == onlyBlocking && buffer.compareAndSet(arrayIndex, value, null)) { + if (onlyBlocking) blockingTasksInBuffer.decrementAndGet() + return value + } + return null + } + + fun offloadAllWorkTo(globalQueue: GlobalQueue) { + lastScheduledTask.getAndSet(null)?.let { globalQueue.addLast(it) } + while (pollTo(globalQueue)) { + // Steal everything + } + } + + /** + * Contract on return value is the same as for [trySteal] + */ + private fun tryStealLastScheduled(stealingMode: StealingMode, stolenTaskRef: ObjectRef): Long { + while (true) { + val lastScheduled = lastScheduledTask.value ?: return NOTHING_TO_STEAL + if ((lastScheduled.maskForStealingMode and stealingMode) == 0) { + return NOTHING_TO_STEAL + } + + // TODO time wraparound ? + val time = schedulerTimeSource.nanoTime() + val staleness = time - lastScheduled.submissionTime + if (staleness < WORK_STEALING_TIME_RESOLUTION_NS) { + return WORK_STEALING_TIME_RESOLUTION_NS - staleness + } + + /* + * If CAS has failed, either someone else had stolen this task or the owner executed this task + * and dispatched another one. In the latter case we should retry to avoid missing task. + */ + if (lastScheduledTask.compareAndSet(lastScheduled, null)) { + stolenTaskRef.element = lastScheduled + return TASK_STOLEN + } + continue + } + } + + private fun pollTo(queue: GlobalQueue): Boolean { + val task = pollBuffer() ?: return false + queue.addLast(task) + return true + } + + private fun pollBuffer(): Task? { + while (true) { + val tailLocal = consumerIndex.value + if (tailLocal - producerIndex.value == 0) return null + val index = tailLocal and MASK + if (consumerIndex.compareAndSet(tailLocal, tailLocal + 1)) { + // Nulls are allowed when blocking tasks are stolen from the middle of the queue. + val value = buffer.getAndSet(index, null) ?: continue + value.decrementIfBlocking() + return value + } + } + } + + private fun Task?.decrementIfBlocking() { + if (this != null && isBlocking) { + val value = blockingTasksInBuffer.decrementAndGet() + assert { value >= 0 } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/src/stream/Stream.kt b/kotlinx-coroutines-core/jvm/src/stream/Stream.kt new file mode 100644 index 0000000000..bbbac1f237 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/stream/Stream.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines.stream + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import java.util.stream.* + +/** + * Represents the given stream as a flow and [closes][Stream.close] the stream afterwards. + * The resulting flow can be [collected][Flow.collect] only once + * and throws [IllegalStateException] when trying to collect it more than once. + */ +public fun Stream.consumeAsFlow(): Flow = StreamFlow(this) + +private class StreamFlow(private val stream: Stream) : Flow { + private val consumed = atomic(false) + + override suspend fun collect(collector: FlowCollector) { + if (!consumed.compareAndSet(false, true)) error("Stream.consumeAsFlow can be collected only once") + try { + for (value in stream.iterator()) { + collector.emit(value) + } + } finally { + stream.close() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/src/time/Time.kt b/kotlinx-coroutines-core/jvm/src/time/Time.kt new file mode 100644 index 0000000000..3bb1d054e6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/src/time/Time.kt @@ -0,0 +1,73 @@ +@file:OptIn(ExperimentalContracts::class) + +package kotlinx.coroutines.time + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.selects.* +import java.time.* +import java.time.temporal.* +import kotlin.contracts.* + +/** + * "java.time" adapter method for [kotlinx.coroutines.delay]. + */ +public suspend fun delay(duration: Duration): Unit = delay(duration.coerceToMillis()) + +/** + * "java.time" adapter method for [kotlinx.coroutines.flow.debounce]. + */ +@FlowPreview +public fun Flow.debounce(timeout: Duration): Flow = debounce(timeout.coerceToMillis()) + +/** + * "java.time" adapter method for [kotlinx.coroutines.flow.sample]. + */ +@FlowPreview +public fun Flow.sample(period: Duration): Flow = sample(period.coerceToMillis()) + +/** + * "java.time" adapter method for [SelectBuilder.onTimeout]. + */ +public fun SelectBuilder.onTimeout(duration: Duration, block: suspend () -> R): Unit = + onTimeout(duration.coerceToMillis(), block) + +/** + * "java.time" adapter method for [kotlinx.coroutines.withTimeout]. + */ +public suspend fun withTimeout(duration: Duration, block: suspend CoroutineScope.() -> T): T { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + return kotlinx.coroutines.withTimeout(duration.coerceToMillis(), block) +} + +/** + * "java.time" adapter method for [kotlinx.coroutines.withTimeoutOrNull]. + */ +public suspend fun withTimeoutOrNull(duration: Duration, block: suspend CoroutineScope.() -> T): T? = + kotlinx.coroutines.withTimeoutOrNull(duration.coerceToMillis(), block) + +/** + * Coerces the given [Duration] to a millisecond delay. + * Negative values are coerced to zero, values that cannot + * be represented in milliseconds as long ("infinite" duration) are coerced to [Long.MAX_VALUE] + * and durations lesser than a millisecond are coerced to 1 millisecond. + * + * The rationale of coercion: + * 1) Too large durations typically indicate infinity and Long.MAX_VALUE is the + * best approximation of infinity we can provide. + * 2) Coercing too small durations to 1 instead of 0 is crucial for two patterns: + * - Programming with deadlines and delays + * - Non-suspending fast-paths (e.g. `withTimeout(1 nanosecond) { 42 }` should not throw) + */ +private fun Duration.coerceToMillis(): Long { + if (this <= Duration.ZERO) return 0 + if (this <= ChronoUnit.MILLIS.duration) return 1 + + // Maximum scalar values of Duration.ofMillis(Long.MAX_VALUE) + val maxSeconds = 9223372036854775 + val maxNanos = 807000000 + return if (seconds < maxSeconds || seconds == maxSeconds && nano < maxNanos) toMillis() + else Long.MAX_VALUE +} diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferFromScope.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferFromScope.txt new file mode 100644 index 0000000000..de14e9da00 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferFromScope.txt @@ -0,0 +1,10 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:109) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.sendInChannel(StackTraceRecoveryChannelsTest.kt:167) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendWithContext$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:162) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendFromScope$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:172) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:112) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:109) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithContextWrapped.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithContextWrapped.txt new file mode 100644 index 0000000000..1a153d3e06 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithContextWrapped.txt @@ -0,0 +1,9 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithContextWrapped$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:98) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.sendInChannel(StackTraceRecoveryChannelsTest.kt:199) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendWithContext$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:194) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithContextWrapped$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:100) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithContextWrapped$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:98) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithCurrentContext.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithCurrentContext.txt new file mode 100644 index 0000000000..5bc7124c6e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testOfferWithCurrentContext.txt @@ -0,0 +1,9 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithCurrentContext$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:86) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.sendInChannel(StackTraceRecoveryChannelsTest.kt:210) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendWithContext$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:205) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithCurrentContext$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:89) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testOfferWithCurrentContext$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:86) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromChannel.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromChannel.txt new file mode 100644 index 0000000000..e02f709a64 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromChannel.txt @@ -0,0 +1,9 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.channels.BufferedChannel.receive$suspendImpl(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.receive(BufferedChannel.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelReceive(StackTraceRecoveryChannelsTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.access$channelReceive(StackTraceRecoveryChannelsTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$channelReceive$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromClosedChannel.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromClosedChannel.txt new file mode 100644 index 0000000000..d48630054a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromClosedChannel.txt @@ -0,0 +1,8 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:110) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelReceive(StackTraceRecoveryChannelsTest.kt:116) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:111) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:110) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendFromScope.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendFromScope.txt new file mode 100644 index 0000000000..bc0f9cd7ef --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendFromScope.txt @@ -0,0 +1,10 @@ +kotlinx.coroutines.testing.RecoverableTestCancellationException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:136) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.sendInChannel(StackTraceRecoveryChannelsTest.kt:167) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendWithContext$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:162) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$sendFromScope$2.invokeSuspend(StackTraceRecoveryChannelsTest.kt:172) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendFromScope$1$deferred$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:126) +Caused by: kotlinx.coroutines.testing.RecoverableTestCancellationException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendFromScope$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:136) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToChannel.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToChannel.txt new file mode 100644 index 0000000000..af6e564210 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToChannel.txt @@ -0,0 +1,20 @@ +java.util.concurrent.CancellationException: Channel was cancelled + at kotlinx.coroutines.channels.BufferedChannel.cancelImpl$(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.cancel(BufferedChannel.kt) + at kotlinx.coroutines.channels.ReceiveChannel$DefaultImpls.cancel$default(Channel.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelSend(StackTraceRecoveryChannelsTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) +Caused by: java.util.concurrent.CancellationException: Channel was cancelled + at kotlinx.coroutines.channels.BufferedChannel.cancelImpl$(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.cancel(BufferedChannel.kt) + at kotlinx.coroutines.channels.ReceiveChannel$DefaultImpls.cancel$default(Channel.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt) + at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt) + at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt) + at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt) + at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt) + at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source) + at kotlinx.coroutines.testing.TestBase.runTest(TestBase.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToClosedChannel.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToClosedChannel.txt new file mode 100644 index 0000000000..cd200924e8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testSendToClosedChannel.txt @@ -0,0 +1,8 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:43) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelSend(StackTraceRecoveryChannelsTest.kt:74) + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:44) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testSendToClosedChannel$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt:43) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcher.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcher.txt new file mode 100644 index 0000000000..3c7ee94b4c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcher.txt @@ -0,0 +1,12 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testEventLoopDispatcher$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:40) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testEventLoopDispatcher$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:40) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testEventLoopDispatcher$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:40) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcherSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcherSuspending.txt new file mode 100644 index 0000000000..100c8f04a4 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testEventLoopDispatcherSuspending.txt @@ -0,0 +1,10 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:99) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:116) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:110) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:101) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testEventLoopDispatcherSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:89) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:99) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContext.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContext.txt new file mode 100644 index 0000000000..18d2293eea --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContext.txt @@ -0,0 +1,13 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopChangedContext$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:54) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopChangedContext$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:54) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopChangedContext$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:53) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopChangedContext$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:54) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContextSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContextSuspending.txt new file mode 100644 index 0000000000..fb44df47fc --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopChangedContextSuspending.txt @@ -0,0 +1,11 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:113) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:130) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:124) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:115) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopChangedContextSuspending$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:103) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopChangedContextSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:102) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:113) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcher.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcher.txt new file mode 100644 index 0000000000..13b12c18c3 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcher.txt @@ -0,0 +1,13 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopDispatcher$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:47) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopDispatcher$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:47) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopDispatcher$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:46) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopDispatcher$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:47) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcherSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcherSuspending.txt new file mode 100644 index 0000000000..2b302644ee --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedEventLoopDispatcherSuspending.txt @@ -0,0 +1,11 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:113) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:130) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:124) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:115) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopDispatcherSuspending$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:96) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedEventLoopDispatcherSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:95) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:113) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfined.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfined.txt new file mode 100644 index 0000000000..5740f12332 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfined.txt @@ -0,0 +1,13 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfined$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:27) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfined$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:27) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfined$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:26) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfined$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:27) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContext.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContext.txt new file mode 100644 index 0000000000..a3bc4edbb2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContext.txt @@ -0,0 +1,13 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedChangedContext$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:34) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:76) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doFastPath(StackTraceRecoveryResumeModeTest.kt:71) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:62) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedChangedContext$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:34) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedChangedContext$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:33) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt:61) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedChangedContext$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:34) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContextSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContextSuspending.txt new file mode 100644 index 0000000000..ffbcd58e41 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedChangedContextSuspending.txt @@ -0,0 +1,11 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:148) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:140) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:130) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedChangedContextSuspending$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:95) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedChangedContextSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:94) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedSuspending.txt new file mode 100644 index 0000000000..e115a69850 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testNestedUnconfinedSuspending.txt @@ -0,0 +1,11 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$4.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:148) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:140) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:130) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedSuspending$1$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:88) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testNestedUnconfinedSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:87) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfined.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfined.txt new file mode 100644 index 0000000000..21252458f8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfined.txt @@ -0,0 +1,9 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.channels.BufferedChannel.receive$suspendImpl(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.receive(BufferedChannel.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$withContext$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.access$testResumeModeFastPath(StackTraceRecoveryResumeModeTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testUnconfined$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfinedSuspending.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfinedSuspending.txt new file mode 100644 index 0000000000..1199fe6461 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/resume-mode/testUnconfinedSuspending.txt @@ -0,0 +1,9 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.doSuspendingPath(StackTraceRecoveryResumeModeTest.kt:140) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest.testResumeModeSuspending(StackTraceRecoveryResumeModeTest.kt:130) + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testUnconfinedSuspending$1.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:82) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoveryResumeModeTest$testResumeModeSuspending$2.invokeSuspend(StackTraceRecoveryResumeModeTest.kt:128) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectCompletedAwait.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectCompletedAwait.txt new file mode 100644 index 0000000000..900cb6c2f8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectCompletedAwait.txt @@ -0,0 +1,7 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectCompletedAwait$1.invokeSuspend(StackTraceRecoverySelectTest.kt:40) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectCompletedAwait$1.invokeSuspend(StackTraceRecoverySelectTest.kt:41) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectCompletedAwait$1.invokeSuspend(StackTraceRecoverySelectTest.kt:40) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectJoin.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectJoin.txt new file mode 100644 index 0000000000..15a689b724 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectJoin.txt @@ -0,0 +1,8 @@ +kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$doSelect$2$1.invokeSuspend(StackTraceRecoverySelectTest.kt) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.selects.SelectImplementation.processResultAndInvokeBlockRecoveringException(Select.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectJoin$1.invokeSuspend(StackTraceRecoverySelectTest.kt) +Caused by: kotlinx.coroutines.testing.RecoverableTestException + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$doSelect$2$1.invokeSuspend(StackTraceRecoverySelectTest.kt) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectOnReceive.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectOnReceive.txt new file mode 100644 index 0000000000..8b958d2058 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/select/testSelectOnReceive.txt @@ -0,0 +1,32 @@ +kotlinx.coroutines.channels.ClosedReceiveChannelException: Channel was closed + at kotlinx.coroutines.channels.BufferedChannel.getReceiveException(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.processResultSelectReceive(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.access$processResultSelectReceive(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel$onReceive$2.invoke(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel$onReceive$2.invoke(BufferedChannel.kt) + at kotlinx.coroutines.selects.SelectImplementation$ClauseData.processResult(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.processResultAndInvokeBlockRecoveringException(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.complete(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.doSelect$suspendImpl(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.doSelect(Select.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest.doSelectOnReceive(StackTraceRecoverySelectTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest.access$doSelectOnReceive(StackTraceRecoverySelectTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectOnReceive$1.invokeSuspend(StackTraceRecoverySelectTest.kt) + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.selects.SelectImplementation.processResultAndInvokeBlockRecoveringException(Select.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectOnReceive$1.invokeSuspend(StackTraceRecoverySelectTest.kt) +Caused by: kotlinx.coroutines.channels.ClosedReceiveChannelException: Channel was closed + at kotlinx.coroutines.channels.BufferedChannel.getReceiveException(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.processResultSelectReceive(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel.access$processResultSelectReceive(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel$onReceive$2.invoke(BufferedChannel.kt) + at kotlinx.coroutines.channels.BufferedChannel$onReceive$2.invoke(BufferedChannel.kt) + at kotlinx.coroutines.selects.SelectImplementation$ClauseData.processResult(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.processResultAndInvokeBlockRecoveringException(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.complete(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.doSelect$suspendImpl(Select.kt) + at kotlinx.coroutines.selects.SelectImplementation.doSelect(Select.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest.doSelectOnReceive(StackTraceRecoverySelectTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest.access$doSelectOnReceive(StackTraceRecoverySelectTest.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoverySelectTest$testSelectOnReceive$1.invokeSuspend(StackTraceRecoverySelectTest.kt) + at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild.txt new file mode 100644 index 0000000000..ac40dc152b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild.txt @@ -0,0 +1,7 @@ +kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.outerChildWithTimeout(StackTraceRecoveryWithTimeoutTest.kt:48) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest$testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild$1.invokeSuspend(StackTraceRecoveryWithTimeoutTest.kt:40) +Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms + at kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:116) + at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:86) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPoint.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPoint.txt new file mode 100644 index 0000000000..9d5ddb6621 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPoint.txt @@ -0,0 +1,10 @@ +kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.suspendForever(StackTraceRecoveryWithTimeoutTest.kt:42) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest$outerWithTimeout$2.invokeSuspend(StackTraceRecoveryWithTimeoutTest.kt:32) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.outerWithTimeout(StackTraceRecoveryWithTimeoutTest.kt:31) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest$testStacktraceIsRecoveredFromSuspensionPoint$1.invokeSuspend(StackTraceRecoveryWithTimeoutTest.kt:19) +Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms + at kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:116) + at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:86) + at kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:492) diff --git a/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPointWithChild.txt b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPointWithChild.txt new file mode 100644 index 0000000000..6f21cc6b30 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test-resources/stacktraces/timeout/testStacktraceIsRecoveredFromSuspensionPointWithChild.txt @@ -0,0 +1,9 @@ +kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms + at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.suspendForever(StackTraceRecoveryWithTimeoutTest.kt:92) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest$outerChild$2.invokeSuspend(StackTraceRecoveryWithTimeoutTest.kt:78) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest.outerChild(StackTraceRecoveryWithTimeoutTest.kt:74) + at kotlinx.coroutines.exceptions.StackTraceRecoveryWithTimeoutTest$testStacktraceIsRecoveredFromSuspensionPointWithChild$1.invokeSuspend(StackTraceRecoveryWithTimeoutTest.kt:66) +Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 200 ms + at kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:116) + at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:86) diff --git a/kotlinx-coroutines-core/jvm/test/AbstractLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/AbstractLincheckTest.kt new file mode 100644 index 0000000000..6287be0081 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/AbstractLincheckTest.kt @@ -0,0 +1,41 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.jetbrains.kotlinx.lincheck.* +import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* +import org.jetbrains.kotlinx.lincheck.strategy.stress.* +import org.junit.* + +abstract class AbstractLincheckTest { + open fun > O.customize(isStressTest: Boolean): O = this + open fun ModelCheckingOptions.customize(isStressTest: Boolean): ModelCheckingOptions = this + open fun StressOptions.customize(isStressTest: Boolean): StressOptions = this + + @Test + fun modelCheckingTest() = ModelCheckingOptions() + .iterations(20 * stressTestMultiplierSqrt) + .invocationsPerIteration(1_000 * stressTestMultiplierSqrt) + .commonConfiguration() + .customize(isStressTest) + .check(this::class) + + @Test + fun stressTest() = StressOptions() + .iterations(20 * stressTestMultiplierSqrt) + .invocationsPerIteration(1_000 * stressTestMultiplierSqrt) + .commonConfiguration() + .customize(isStressTest) + .check(this::class) + + private fun > O.commonConfiguration(): O = this + .actorsBefore(if (isStressTest) 3 else 1) + // All the bugs we have discovered so far + // were reproducible on at most 3 threads + .threads(3) + // 3 operations per thread is sufficient, + // while increasing this number declines + // the model checking coverage. + .actorsPerThread(if (isStressTest) 3 else 2) + .actorsAfter(if (isStressTest) 3 else 0) + .customize(isStressTest) +} diff --git a/kotlinx-coroutines-core/jvm/test/AsyncJvmTest.kt b/kotlinx-coroutines-core/jvm/test/AsyncJvmTest.kt new file mode 100644 index 0000000000..05bc505f0a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/AsyncJvmTest.kt @@ -0,0 +1,46 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + + +class AsyncJvmTest : TestBase() { + // We have the same test in common module, but the maintainer uses this particular file + // and semi-automatically types cmd+N + AsyncJvm in order to duck-tape any JVM samples/repros, + // please do not remove this test + + @Test + fun testAsyncWithFinally() = runTest { + expect(1) + + @Suppress("UNREACHABLE_CODE") + val d = async { + expect(3) + try { + yield() // to main, will cancel + } finally { + expect(6) // will go there on await + return@async "Fail" // result will not override cancellation + } + expectUnreached() + "Fail2" + } + expect(2) + yield() // to async + expect(4) + check(d.isActive && !d.isCompleted && !d.isCancelled) + d.cancel() + check(!d.isActive && !d.isCompleted && d.isCancelled) + check(!d.isActive && !d.isCompleted && d.isCancelled) + expect(5) + try { + d.await() // awaits + expectUnreached() // does not complete normally + } catch (e: Throwable) { + expect(7) + check(e is CancellationException) + } + check(!d.isActive && d.isCompleted && d.isCancelled) + finish(8) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/AwaitJvmTest.kt b/kotlinx-coroutines-core/jvm/test/AwaitJvmTest.kt new file mode 100644 index 0000000000..e43611e808 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/AwaitJvmTest.kt @@ -0,0 +1,24 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* + +class AwaitJvmTest : TestBase() { + @Test + public fun testSecondLeak() = runTest { + // This test is to make sure that handlers installed on the second deferred do not leak + val d1 = CompletableDeferred() + val d2 = CompletableDeferred() + d1.completeExceptionally(TestException()) // first is crashed + val iterations = 3_000_000 * stressTestMultiplier + for (iter in 1..iterations) { + try { + awaitAll(d1, d2) + expectUnreached() + } catch (e: TestException) { + expect(iter) + } + } + finish(iterations + 1) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/AwaitStressTest.kt b/kotlinx-coroutines-core/jvm/test/AwaitStressTest.kt new file mode 100644 index 0000000000..8c84e7da3b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/AwaitStressTest.kt @@ -0,0 +1,122 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* + +class AwaitStressTest : TestBase() { + + private val iterations = 50_000 * stressTestMultiplier + @get:Rule + public val pool = ExecutorRule(4) + + @Test + fun testMultipleExceptions() = runTest { + val ctx = pool + NonCancellable + repeat(iterations) { + val barrier = CyclicBarrier(4) + val d1 = async(ctx) { + barrier.await() + throw TestException() + } + val d2 = async(ctx) { + barrier.await() + throw TestException() + } + val d3 = async(ctx) { + barrier.await() + 1L + } + try { + barrier.await() + awaitAll(d1, d2, d3) + expectUnreached() + } catch (e: TestException) { + // Expected behaviour + } + + barrier.reset() + } + } + + @Test + fun testAwaitAll() = runTest { + val barrier = CyclicBarrier(3) + repeat(iterations) { + val d1 = async(pool) { + barrier.await() + 1L + } + val d2 = async(pool) { + barrier.await() + 2L + } + barrier.await() + awaitAll(d1, d2) + require(d1.isCompleted && d2.isCompleted) + barrier.reset() + } + } + + @Test + fun testConcurrentCancellation() = runTest { + var cancelledOnce = false + repeat(iterations) { + val barrier = CyclicBarrier(3) + + val d1 = async(pool) { + barrier.await() + delay(10_000) + yield() + } + + val d2 = async(pool) { + barrier.await() + d1.cancel() + } + + barrier.await() + try { + awaitAll(d1, d2) + } catch (e: CancellationException) { + cancelledOnce = true + } + } + + require(cancelledOnce) { "Cancellation exception wasn't properly caught" } + } + + @Test + fun testMutatingCollection() = runTest { + val barrier = CyclicBarrier(4) + + repeat(iterations) { + // thread-safe collection that we are going to modify + val deferreds = CopyOnWriteArrayList>() + + deferreds += async(pool) { + barrier.await() + 1L + } + + deferreds += async(pool) { + barrier.await() + 2L + } + + deferreds += async(pool) { + barrier.await() + deferreds.removeAt(2) + 3L + } + + val allJobs = ArrayList(deferreds) + barrier.await() + val results = deferreds.awaitAll() // shouldn't hang + check(results == listOf(1L, 2L, 3L) || results == listOf(1L, 2L)) + allJobs.awaitAll() + barrier.reset() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/CancellableContinuationJvmTest.kt b/kotlinx-coroutines-core/jvm/test/CancellableContinuationJvmTest.kt new file mode 100644 index 0000000000..e53cd0cc30 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/CancellableContinuationJvmTest.kt @@ -0,0 +1,79 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +class CancellableContinuationJvmTest : TestBase() { + @Test + fun testToString() = runTest { + checkToString() + } + + private suspend fun checkToString() { + suspendCancellableCoroutine { + it.resume(Unit) + assertTrue(it.toString().contains("kotlinx.coroutines.CancellableContinuationJvmTest.checkToString(CancellableContinuationJvmTest.kt")) + } + suspend {}() // Eliminate tail-call optimization + } + + @Test + fun testExceptionIsNotReported() = runTest({ it is CancellationException }) { + val ctx = coroutineContext + suspendCancellableCoroutine { + ctx.cancel() + it.resumeWith(Result.failure(TestException())) + } + } + + @Test + fun testBlockingIntegration() = runTest { + val source = BlockingSource() + val job = launch(Dispatchers.Default) { + source.await() + } + source.cancelAndJoin(job) + } + + @Test + fun testBlockingIntegrationAlreadyCancelled() = runTest { + val source = BlockingSource() + val job = launch(Dispatchers.Default) { + cancel() + source.await() + } + source.cancelAndJoin(job) + } + + private suspend fun BlockingSource.cancelAndJoin(job: Job) { + while (!hasSubscriber) { + Thread.sleep(10) + } + job.cancelAndJoin() + } + + private suspend fun BlockingSource.await() = suspendCancellableCoroutine { + it.invokeOnCancellation { this.cancel() } + subscribe() + } + + private class BlockingSource { + @Volatile + private var isCancelled = false + + @Volatile + public var hasSubscriber = false + + public fun subscribe() { + hasSubscriber = true + while (!isCancelled) { + Thread.sleep(10) + } + } + + public fun cancel() { + isCancelled = true + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/CancellableContinuationResumeCloseStressTest.kt b/kotlinx-coroutines-core/jvm/test/CancellableContinuationResumeCloseStressTest.kt new file mode 100644 index 0000000000..fa6030d2c9 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/CancellableContinuationResumeCloseStressTest.kt @@ -0,0 +1,61 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import org.junit.* +import java.util.concurrent.* +import kotlin.test.* +import kotlin.test.Test + +class CancellableContinuationResumeCloseStressTest : TestBase() { + @get:Rule + public val dispatcher = ExecutorRule(2) + + private val startBarrier = CyclicBarrier(3) + private val doneBarrier = CyclicBarrier(2) + private val nRepeats = 1_000 * stressTestMultiplier + + private val closed = atomic(false) + private var returnedOk = false + + @Test + @Suppress("BlockingMethodInNonBlockingContext") + fun testStress() = runTest { + repeat(nRepeats) { + closed.value = false + returnedOk = false + val job = testJob() + startBarrier.await() + job.cancel() // (1) cancel job + job.join() + // check consistency + doneBarrier.await() + if (returnedOk) { + assertFalse(closed.value, "should not have closed resource -- returned Ok") + } else { + assertTrue(closed.value, "should have closed resource -- was cancelled") + } + } + } + + private fun CoroutineScope.testJob(): Job = launch(dispatcher, start = CoroutineStart.ATOMIC) { + val ok = resumeClose() // might be cancelled + assertEquals("OK", ok) + returnedOk = true + } + + private suspend fun resumeClose() = suspendCancellableCoroutine { cont -> + dispatcher.executor.execute { + startBarrier.await() // (2) resume at the same time + cont.resume("OK") { + close() + } + doneBarrier.await() + } + startBarrier.await() // (3) return at the same time + } + + fun close() { + assertFalse(closed.getAndSet(true)) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/CancelledAwaitStressTest.kt b/kotlinx-coroutines-core/jvm/test/CancelledAwaitStressTest.kt new file mode 100644 index 0000000000..5a85aebb1a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/CancelledAwaitStressTest.kt @@ -0,0 +1,52 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* + +class CancelledAwaitStressTest : TestBase() { + private val n = 1000 * stressTestMultiplier + + /** + * Tests that memory does not leak from cancelled [Deferred.await] + */ + @Test + fun testCancelledAwait() = runTest { + val d = async { + delay(Long.MAX_VALUE) + } + repeat(n) { + val waiter = launch(start = CoroutineStart.UNDISPATCHED) { + val a = ByteArray(10000000) // allocate 10M of memory here + d.await() + keepMe(a) // make sure it is kept in state machine + } + waiter.cancel() // cancel await + yield() // complete the waiter job, release its memory + } + d.cancel() // done test + } + + /** + * Tests that memory does not leak from cancelled [Job.join] + */ + @Test + fun testCancelledJoin() = runTest { + val j = launch { + delay(Long.MAX_VALUE) + } + repeat(n) { + val joiner = launch(start = CoroutineStart.UNDISPATCHED) { + val a = ByteArray(10000000) // allocate 10M of memory here + j.join() + keepMe(a) // make sure it is kept in state machine + } + joiner.cancel() // cancel join + yield() // complete the joiner job, release its memory + } + j.cancel() // done test + } + + private fun keepMe(a: ByteArray) { + // does nothing, makes sure the variable is kept in state-machine + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ConcurrentTestUtilities.kt b/kotlinx-coroutines-core/jvm/test/ConcurrentTestUtilities.kt new file mode 100644 index 0000000000..6d6bc2127f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ConcurrentTestUtilities.kt @@ -0,0 +1,5 @@ +package kotlinx.coroutines.exceptions + +actual inline fun yieldThread() { Thread.yield() } + +actual fun currentThreadName(): String = Thread.currentThread().name diff --git a/kotlinx-coroutines-core/jvm/test/CoroutinesJvmTest.kt b/kotlinx-coroutines-core/jvm/test/CoroutinesJvmTest.kt new file mode 100644 index 0000000000..416226cb9f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/CoroutinesJvmTest.kt @@ -0,0 +1,36 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class CoroutinesJvmTest : TestBase() { + @Test + fun testNotCancellableCodeWithExceptionCancelled() = runTest(expected = {e -> e is TestException}) { + expect(1) + // CoroutineStart.ATOMIC makes sure it will not get cancelled for it starts executing + val job = launch(start = CoroutineStart.ATOMIC) { + Thread.sleep(100) // cannot be cancelled + throwTestException() // will throw + expectUnreached() + } + expect(2) + job.cancel() + finish(3) + } + + @Test + fun testCancelManyCompletedAttachedChildren() = runTest { + val parent = launch { /* do nothing */ } + val n = 10_000 * stressTestMultiplier + repeat(n) { + // create a child that already completed + val child = launch(start = CoroutineStart.UNDISPATCHED) { /* do nothing */ } + // attach it manually via internal API + @Suppress("DEPRECATION_ERROR") + parent.attachChild(child as ChildJob) + } + parent.cancelAndJoin() // cancel parent, make sure no stack overflow + } + + private fun throwTestException(): Unit = throw TestException() +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/DebugThreadNameTest.kt b/kotlinx-coroutines-core/jvm/test/DebugThreadNameTest.kt new file mode 100644 index 0000000000..4505f39dca --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/DebugThreadNameTest.kt @@ -0,0 +1,70 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class DebugThreadNameTest : TestBase() { + @BeforeTest + fun resetName() { + resetCoroutineId() + } + + @Test + fun testLaunchId() = runTest { + assertName("coroutine#1") + launch { + assertName("coroutine#2") + yield() + assertName("coroutine#2") + } + assertName("coroutine#1") + } + + @Test + fun testLaunchIdUndispatched() = runTest { + assertName("coroutine#1") + launch(start = CoroutineStart.UNDISPATCHED) { + assertName("coroutine#2") + yield() + assertName("coroutine#2") + } + assertName("coroutine#1") + } + + @Test + fun testLaunchName() = runTest { + assertName("coroutine#1") + launch(CoroutineName("TEST")) { + assertName("TEST#2") + yield() + assertName("TEST#2") + } + assertName("coroutine#1") + } + + @Test + fun testWithContext() = runTest { + assertName("coroutine#1") + withContext(Dispatchers.Default) { + assertName("coroutine#1") + yield() + assertName("coroutine#1") + withContext(CoroutineName("TEST")) { + assertName("TEST#1") + yield() + assertName("TEST#1") + } + assertName("coroutine#1") + yield() + assertName("coroutine#1") + } + assertName("coroutine#1") + } + + private fun assertName(expected: String) { + val name = Thread.currentThread().name + val split = name.split(Regex(" @")) + assertEquals(2, split.size, "Thread name '$name' is expected to contain one coroutine name") + assertEquals(expected, split[1], "Thread name '$name' is expected to end with coroutine name '$expected'") + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/DefaultExecutorStressTest.kt b/kotlinx-coroutines-core/jvm/test/DefaultExecutorStressTest.kt new file mode 100644 index 0000000000..58b8024547 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/DefaultExecutorStressTest.kt @@ -0,0 +1,62 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import kotlin.test.* + +class DefaultExecutorStressTest : TestBase() { + @Test + fun testDelay() = runTest { + val iterations = 100_000 * stressTestMultiplier + withContext(DefaultExecutor) { + expect(1) + var expected = 1 + repeat(iterations) { + expect(++expected) + val deferred = async { + expect(++expected) + val largeArray = IntArray(10_000) { it } + delay(Long.MAX_VALUE) + println(largeArray) // consume to avoid DCE, actually unreachable + } + + expect(++expected) + yield() + deferred.cancel() + try { + deferred.await() + } catch (e: CancellationException) { + expect(++expected) + } + } + + } + finish(2 + iterations * 4) + } + + @Test + fun testWorkerShutdown() = withVirtualTimeSource { + val iterations = 1_000 * stressTestMultiplier + // wait for the worker to shut down + suspend fun awaitWorkerShutdown() { + val executorTimeoutMs = 1000L + delay(executorTimeoutMs) + while (DefaultExecutor.isThreadPresent) { delay(10) } // hangs if the thread refuses to stop + assertFalse(DefaultExecutor.isThreadPresent) // just to make sure + } + runTest { + awaitWorkerShutdown() // so that the worker shuts down after the initial launch + repeat (iterations) { + val job = launch(Dispatchers.Unconfined) { + // this line runs in the main thread + delay(1) + // this line runs in the DefaultExecutor worker + } + delay(100) // yield the execution, allow the worker to spawn + assertTrue(DefaultExecutor.isThreadPresent) // the worker spawned + job.join() + awaitWorkerShutdown() + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/DelayJvmTest.kt b/kotlinx-coroutines-core/jvm/test/DelayJvmTest.kt new file mode 100644 index 0000000000..710f773f44 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/DelayJvmTest.kt @@ -0,0 +1,68 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import java.util.concurrent.* +import kotlin.coroutines.* +import kotlin.test.assertEquals + +class DelayJvmTest : TestBase() { + /** + * Test that delay works properly in contexts with custom [ContinuationInterceptor] + */ + @Test + fun testDelayInArbitraryContext() = runBlocking { + var thread: Thread? = null + val pool = Executors.newFixedThreadPool(1) { runnable -> + Thread(runnable).also { thread = it } + } + val context = CustomInterceptor(pool) + val c = async(context) { + assertEquals(thread, Thread.currentThread()) + delay(100) + assertEquals(thread, Thread.currentThread()) + 42 + } + assertEquals(42, c.await()) + pool.shutdown() + } + + @Test + fun testDelayWithoutDispatcher() = runBlocking(CoroutineName("testNoDispatcher.main")) { + // launch w/o a specified dispatcher + val c = async(CoroutineName("testNoDispatcher.inner")) { + delay(100) + 42 + } + assertEquals(42, c.await()) + } + + @Test + fun testNegativeDelay() = runBlocking { + expect(1) + val job = async { + expect(3) + delay(0) + expect(4) + } + + delay(-1) + expect(2) + job.await() + finish(5) + } + + class CustomInterceptor(val pool: Executor) : AbstractCoroutineContextElement(ContinuationInterceptor), ContinuationInterceptor { + override fun interceptContinuation(continuation: Continuation): Continuation = + Wrapper(pool, continuation) + } + + class Wrapper(val pool: Executor, private val cont: Continuation) : Continuation { + override val context: CoroutineContext + get() = cont.context + + override fun resumeWith(result: Result) { + pool.execute { cont.resumeWith(result) } + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/DispatcherKeyTest.kt b/kotlinx-coroutines-core/jvm/test/DispatcherKeyTest.kt new file mode 100644 index 0000000000..8b6c219c8e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/DispatcherKeyTest.kt @@ -0,0 +1,49 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import kotlin.coroutines.* +import kotlin.test.* + +@OptIn(ExperimentalStdlibApi::class) +class DispatcherKeyTest : TestBase() { + + companion object CustomInterceptor : AbstractCoroutineContextElement(ContinuationInterceptor), + ContinuationInterceptor { + override fun interceptContinuation(continuation: Continuation): Continuation { + return continuation + } + } + + private val name = CoroutineName("test") + + @Test + fun testDispatcher() { + val context = name + CustomInterceptor + assertNull(context[CoroutineDispatcher]) + assertSame(CustomInterceptor, context[ContinuationInterceptor]) + + val updated = context + Dispatchers.Main + val result: CoroutineDispatcher? = updated[CoroutineDispatcher] + assertSame(Dispatchers.Main, result) + assertSame(Dispatchers.Main, updated[ContinuationInterceptor]) + assertEquals(name, updated.minusKey(CoroutineDispatcher)) + assertEquals(name, updated.minusKey(ContinuationInterceptor)) + } + + @Test + fun testExecutorCoroutineDispatcher() { + val context = name + CustomInterceptor + assertNull(context[ExecutorCoroutineDispatcher]) + val updated = context + Dispatchers.Main + assertNull(updated[ExecutorCoroutineDispatcher]) + val executor = Dispatchers.Default + val updated2 = updated + executor + assertSame(Dispatchers.Default, updated2[ContinuationInterceptor]) + assertSame(Dispatchers.Default, updated2[CoroutineDispatcher]) + assertSame(Dispatchers.Default as ExecutorCoroutineDispatcher, updated2[ExecutorCoroutineDispatcher]) + assertEquals(name, updated2.minusKey(ContinuationInterceptor)) + assertEquals(name, updated2.minusKey(CoroutineDispatcher)) + assertEquals(name, updated2.minusKey(ExecutorCoroutineDispatcher)) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/DispatchersToStringTest.kt b/kotlinx-coroutines-core/jvm/test/DispatchersToStringTest.kt new file mode 100644 index 0000000000..32573ca1f6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/DispatchersToStringTest.kt @@ -0,0 +1,56 @@ +@file:OptIn(ExperimentalStdlibApi::class) + +package kotlinx.coroutines + +import kotlinx.coroutines.scheduling.CORE_POOL_SIZE +import kotlinx.coroutines.scheduling.MAX_POOL_SIZE +import kotlin.test.* + +class DispatchersToStringTest { + @Test + fun testStrings() { + assertEquals("Dispatchers.Unconfined", Dispatchers.Unconfined.toString()) + assertEquals("Dispatchers.Default", Dispatchers.Default.toString()) + assertEquals("Dispatchers.IO", Dispatchers.IO.toString()) + assertEquals("Dispatchers.Main[missing]", Dispatchers.Main.toString()) + assertEquals("Dispatchers.Main[missing]", Dispatchers.Main.immediate.toString()) + } + + @Test + fun testLimitedParallelism() { + for (parallelism in 1..100) { + assertEquals( + "Dispatchers.IO" + if (parallelism < MAX_POOL_SIZE) ".limitedParallelism($parallelism)" else "", + Dispatchers.IO.limitedParallelism(parallelism).toString() + ) + assertEquals( + "Dispatchers.Default" + if (parallelism < CORE_POOL_SIZE) ".limitedParallelism($parallelism)" else "", + Dispatchers.Default.limitedParallelism(parallelism).toString() + ) + } + // Not overridden at all, limited parallelism returns `this` + assertEquals("DefaultExecutor", (DefaultDelay as CoroutineDispatcher).limitedParallelism(42).toString()) + + assertEquals("filesDispatcher", Dispatchers.IO.limitedParallelism(1, "filesDispatcher").toString()) + assertEquals("json", Dispatchers.Default.limitedParallelism(2, "json").toString()) + assertEquals("\uD80C\uDE11", (DefaultDelay as CoroutineDispatcher).limitedParallelism(42, "\uD80C\uDE11").toString()) + assertEquals("DefaultExecutor", (DefaultDelay as CoroutineDispatcher).limitedParallelism(42).toString()) + + val limitedNamed = Dispatchers.IO.limitedParallelism(10, "limited") + assertEquals("limited.limitedParallelism(2)", limitedNamed.limitedParallelism(2).toString()) + assertEquals("2", limitedNamed.limitedParallelism(2, "2").toString()) + // We asked for too many threads with no name, this was returned + assertEquals("limited", limitedNamed.limitedParallelism(12).toString()) + assertEquals("12", limitedNamed.limitedParallelism(12, "12").toString()) + + runBlocking { + val d = coroutineContext[CoroutineDispatcher]!! + assertContains(d.toString(), "BlockingEventLoop") + val limited = d.limitedParallelism(2) + assertContains(limited.toString(), "BlockingEventLoop") + assertFalse(limited.toString().contains("limitedParallelism")) + val named = d.limitedParallelism(2, "Named") + assertEquals("Named", named.toString()) + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/EventLoopsTest.kt b/kotlinx-coroutines-core/jvm/test/EventLoopsTest.kt new file mode 100644 index 0000000000..551d1977c0 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/EventLoopsTest.kt @@ -0,0 +1,162 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.channels.* +import org.junit.Test +import java.util.concurrent.locks.* +import kotlin.concurrent.* +import kotlin.test.* + +/** + * Tests event loops integration. + * See [https://github.com/Kotlin/kotlinx.coroutines/issues/860]. + */ +class EventLoopsTest : TestBase() { + @Test + fun testNestedRunBlocking() { + runBlocking { // outer event loop + // Produce string "OK" + val ch = produce { send("OK") } + // try receive this string in a blocking way: + assertEquals("OK", runBlocking { ch.receive() }) // it should not hang here + } + } + + @Test + fun testUnconfinedInRunBlocking() { + var completed = false + runBlocking { + launch(Dispatchers.Unconfined) { + completed = true + } + // should not go into runBlocking loop, but complete here + assertTrue(completed) + } + } + + @Test + fun testNestedUnconfined() { + expect(1) + GlobalScope.launch(Dispatchers.Unconfined) { + expect(2) + GlobalScope.launch(Dispatchers.Unconfined) { + // this gets scheduled into outer unconfined loop + expect(4) + } + expect(3) // ^^ executed before the above unconfined + } + finish(5) + } + + @Test + fun testEventLoopInDefaultExecutor() = runTest { + expect(1) + withContext(Dispatchers.Unconfined) { + delay(1) + assertTrue(Thread.currentThread().name.startsWith(DefaultExecutor.THREAD_NAME)) + expect(2) + // now runBlocking inside default executor thread --> should use outer event loop + DefaultExecutor.enqueue(Runnable { + expect(4) // will execute when runBlocking runs loop + }) + expect(3) + runBlocking { + expect(5) + } + } + finish(6) + } + + /** + * Simple test for [processNextEventInCurrentThread] API use-case. + */ + @Test + fun testProcessNextEventInCurrentThreadSimple() = runTest { + expect(1) + val event = EventSync() + // this coroutine fires event + launch { + expect(3) + event.fireEvent() + } + // main coroutine waits for event (same thread!) + expect(2) + event.blockingAwait() + finish(4) + } + + @Test + fun testSecondThreadRunBlocking() = runTest { + val testThread = Thread.currentThread() + val testContext = coroutineContext + val event = EventSync() // will signal completion + var thread = thread { + runBlocking { // outer event loop + // Produce string "OK" + val ch = produce { send("OK") } + // try receive this string in a blocking way using test context (another thread) + assertEquals("OK", runBlocking(testContext) { + assertEquals(testThread, Thread.currentThread()) + ch.receive() // it should not hang here + }) + } + event.fireEvent() // done thread + } + event.blockingAwait() // wait for thread to complete + thread.join() // it is safe to join thread now + } + + /** + * Test for [processNextEventInCurrentThread] API use-case with delay. + */ + @Test + fun testProcessNextEventInCurrentThreadDelay() = runTest { + expect(1) + val event = EventSync() + // this coroutine fires event + launch { + expect(3) + delay(100) + event.fireEvent() + } + // main coroutine waits for event (same thread!) + expect(2) + event.blockingAwait() + finish(4) + } + + /** + * Tests that, when delayed tasks are due on an event loop, they will execute earlier than the newly-scheduled + * non-delayed tasks. + */ + @Test + fun testPendingDelayedBeingDueEarlier() = runTest { + launch(start = CoroutineStart.UNDISPATCHED) { + delay(1) + expect(1) + } + Thread.sleep(100) + yield() + finish(2) + } + + class EventSync { + private val waitingThread = atomic(null) + private val fired = atomic(false) + + fun fireEvent() { + fired.value = true + waitingThread.value?.let { LockSupport.unpark(it) } + } + + fun blockingAwait() { + check(waitingThread.getAndSet(Thread.currentThread()) == null) + while (!fired.getAndSet(false)) { + val time = processNextEventInCurrentThread() + LockSupport.parkNanos(time) + } + waitingThread.value = null + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/ExecutorAsCoroutineDispatcherDelayTest.kt b/kotlinx-coroutines-core/jvm/test/ExecutorAsCoroutineDispatcherDelayTest.kt new file mode 100644 index 0000000000..819b05e9ed --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ExecutorAsCoroutineDispatcherDelayTest.kt @@ -0,0 +1,58 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import java.lang.Runnable +import java.util.concurrent.* +import kotlin.test.* + +class ExecutorAsCoroutineDispatcherDelayTest : TestBase() { + + private var callsToSchedule = 0 + + private inner class STPE : ScheduledThreadPoolExecutor(1) { + override fun schedule(command: Runnable, delay: Long, unit: TimeUnit): ScheduledFuture<*> { + if (delay != 0L) ++callsToSchedule + return super.schedule(command, delay, unit) + } + } + + private inner class SES : ScheduledExecutorService by STPE() + + @Test + fun testScheduledThreadPool() = runTest { + val executor = STPE() + withContext(executor.asCoroutineDispatcher()) { + delay(100) + } + executor.shutdown() + assertEquals(1, callsToSchedule) + } + + @Test + fun testScheduledExecutorService() = runTest { + val executor = SES() + withContext(executor.asCoroutineDispatcher()) { + delay(100) + } + executor.shutdown() + assertEquals(1, callsToSchedule) + } + + @Test + fun testCancelling() = runTest { + val executor = STPE() + launch(start = CoroutineStart.UNDISPATCHED) { + suspendCancellableCoroutine { cont -> + expect(1) + (executor.asCoroutineDispatcher() as Delay).scheduleResumeAfterDelay(1_000_000, cont) + cont.cancel() + expect(2) + } + } + expect(3) + assertTrue(executor.getQueue().isEmpty()) + executor.shutdown() + finish(4) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ExecutorsTest.kt b/kotlinx-coroutines-core/jvm/test/ExecutorsTest.kt new file mode 100644 index 0000000000..3c60407bb9 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ExecutorsTest.kt @@ -0,0 +1,149 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.coroutines.* +import kotlin.test.* + +class ExecutorsTest : TestBase() { + private fun checkThreadName(prefix: String) { + val name = Thread.currentThread().name + check(name.startsWith(prefix)) { "Expected thread name to start with '$prefix', found: '$name'" } + } + + @Test + fun testSingleThread() { + val context = newSingleThreadContext("TestThread") + runBlocking(context) { + checkThreadName("TestThread") + } + context.close() + } + + @Test + fun testFixedThreadPool() { + val context = newFixedThreadPoolContext(2, "TestPool") + runBlocking(context) { + checkThreadName("TestPool") + delay(10) + checkThreadName("TestPool") // should dispatch on the right thread + } + context.close() + } + + @Test + fun testExecutorToDispatcher() { + val executor = Executors.newSingleThreadExecutor { r -> Thread(r, "TestExecutor") } + runBlocking(executor.asCoroutineDispatcher()) { + checkThreadName("TestExecutor") + delay(10) + checkThreadName("TestExecutor") // should dispatch on the right thread + } + executor.shutdown() + } + + @Test + fun testConvertedDispatcherToExecutor() { + val executor: ExecutorService = Executors.newSingleThreadExecutor { r -> Thread(r, "TestExecutor") } + val dispatcher: CoroutineDispatcher = executor.asCoroutineDispatcher() + assertSame(executor, dispatcher.asExecutor()) + executor.shutdown() + } + + @Test + fun testDefaultDispatcherToExecutor() { + val latch = CountDownLatch(1) + Dispatchers.Default.asExecutor().execute { + checkThreadName("DefaultDispatcher") + latch.countDown() + } + latch.await() + } + + @Test + fun testCustomDispatcherToExecutor() { + expect(1) + val dispatcher = object : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + expect(2) + block.run() + } + } + val executor = dispatcher.asExecutor() + assertSame(dispatcher, executor.asCoroutineDispatcher()) + executor.execute { + expect(3) + } + finish(4) + } + + @Test + fun testCustomDispatcherToExecutorDispatchNotNeeded() { + expect(1) + val dispatcher = object : CoroutineDispatcher() { + override fun isDispatchNeeded(context: CoroutineContext) = false + + override fun dispatch(context: CoroutineContext, block: Runnable) { + fail("should not dispatch") + } + } + dispatcher.asExecutor().execute { + expect(2) + } + finish(3) + } + + @Test + fun testTwoThreads() { + val ctx1 = newSingleThreadContext("Ctx1") + val ctx2 = newSingleThreadContext("Ctx2") + runBlocking(ctx1) { + checkThreadName("Ctx1") + withContext(ctx2) { + checkThreadName("Ctx2") + } + checkThreadName("Ctx1") + } + ctx1.close() + ctx2.close() + } + + @Test + fun testShutdownExecutorService() { + val executorService = Executors.newSingleThreadExecutor { r -> Thread(r, "TestExecutor") } + val dispatcher = executorService.asCoroutineDispatcher() + runBlocking (dispatcher) { + checkThreadName("TestExecutor") + } + dispatcher.close() + check(executorService.isShutdown) + } + + @Test + fun testExceptionInIsDispatchNeeded() { + val dispatcher = object : CoroutineDispatcher() { + override fun isDispatchNeeded(context: CoroutineContext): Boolean { + expect(2) + throw TestException() + } + override fun dispatch(context: CoroutineContext, block: Runnable) = expectUnreached() + } + try { + runBlocking { + expect(1) + try { + launch(dispatcher) { + expectUnreached() + } + expectUnreached() + } catch (_: TestException) { + expect(3) + } + + } + } catch (_: TestException) { + finish(4) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/FailFastOnStartTest.kt b/kotlinx-coroutines-core/jvm/test/FailFastOnStartTest.kt new file mode 100644 index 0000000000..022fb6b630 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/FailFastOnStartTest.kt @@ -0,0 +1,110 @@ +@file:Suppress("DeferredResultUnused") + +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.emptyFlow +import kotlinx.coroutines.flow.flow +import kotlinx.coroutines.flow.flowOn +import org.junit.* +import org.junit.Test +import org.junit.rules.* +import kotlin.coroutines.Continuation +import kotlin.coroutines.EmptyCoroutineContext +import kotlin.coroutines.startCoroutine +import kotlin.test.* + +class FailFastOnStartTest : TestBase() { + + @Rule + @JvmField + public val timeout: Timeout = Timeout.seconds(5) + + @Test + fun testLaunch() = runTest(expected = ::mainException) { + launch(Dispatchers.Main) {} + } + + @Test + fun testLaunchLazy() = runTest(expected = ::mainException) { + val job = launch(Dispatchers.Main, start = CoroutineStart.LAZY) { fail() } + job.join() + } + + @Test + fun testLaunchUndispatched() = runTest(expected = ::mainException) { + launch(Dispatchers.Main, start = CoroutineStart.UNDISPATCHED) { + yield() + fail() + } + } + + @Test + fun testAsync() = runTest(expected = ::mainException) { + async(Dispatchers.Main) {} + } + + @Test + fun testAsyncLazy() = runTest(expected = ::mainException) { + val job = async(Dispatchers.Main, start = CoroutineStart.LAZY) { fail() } + job.await() + } + + @Test + fun testWithContext() = runTest(expected = ::mainException) { + withContext(Dispatchers.Main) { + fail() + } + } + + @Test + fun testProduce() = runTest(expected = ::mainException) { + produce(Dispatchers.Main) { fail() } + } + + @Test + fun testActor() = runTest(expected = ::mainException) { + actor(Dispatchers.Main) { fail() } + } + + @Test + fun testActorLazy() = runTest(expected = ::mainException) { + val actor = actor(Dispatchers.Main, start = CoroutineStart.LAZY) { fail() } + actor.send(1) + } + + private fun mainException(e: Throwable): Boolean { + return e is IllegalStateException && e.message?.contains("Module with the Main dispatcher is missing") ?: false + } + + @Test + fun testProduceNonChild() = runTest(expected = ::mainException) { + produce(Job() + Dispatchers.Main) { fail() } + } + + @Test + fun testAsyncNonChild() = runTest(expected = ::mainException) { + async(Job() + Dispatchers.Main) { fail() } + } + + @Test + fun testFlowOn() { + // See #4142, this test ensures that `coroutineScope { produce(failingDispatcher, ATOMIC) }` + // rethrows an exception. It does not help with the completion of such a coroutine though. + // `suspend {}` + start coroutine with custom `completion` to avoid waiting for test completion + expect(1) + val caller = suspend { + try { + emptyFlow().flowOn(Dispatchers.Main).collect { fail() } + } catch (e: Throwable) { + assertTrue(mainException(e)) + expect(2) + } + } + + caller.startCoroutine(Continuation(EmptyCoroutineContext) { + finish(3) + }) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/FailingCoroutinesMachineryTest.kt b/kotlinx-coroutines-core/jvm/test/FailingCoroutinesMachineryTest.kt new file mode 100644 index 0000000000..144e4e9dc4 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/FailingCoroutinesMachineryTest.kt @@ -0,0 +1,151 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import java.util.concurrent.* +import kotlin.coroutines.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class FailingCoroutinesMachineryTest( + private val element: CoroutineContext.Element, + private val dispatcher: TestDispatcher +) : TestBase() { + class TestDispatcher(val name: String, val block: () -> CoroutineDispatcher) { + private var _value: CoroutineDispatcher? = null + + val value: CoroutineDispatcher + get() = _value ?: block().also { _value = it } + + override fun toString(): String = name + + fun reset() { + runCatching { (_value as? ExecutorCoroutineDispatcher)?.close() } + _value = null + } + } + + private var caught: Throwable? = null + private val latch = CountDownLatch(1) + private var exceptionHandler = CoroutineExceptionHandler { _, t -> caught = t; latch.countDown() } + private val lazyOuterDispatcher = lazy { newFixedThreadPoolContext(1, "") } + + private object FailingUpdate : ThreadContextElement { + private object Key : CoroutineContext.Key + + override val key: CoroutineContext.Key<*> get() = Key + + override fun restoreThreadContext(context: CoroutineContext, oldState: Unit) { + } + + override fun updateThreadContext(context: CoroutineContext) { + throw TestException("Prevent a coroutine from starting right here for some reason") + } + + override fun toString() = "FailingUpdate" + } + + private object FailingRestore : ThreadContextElement { + private object Key : CoroutineContext.Key + + override val key: CoroutineContext.Key<*> get() = Key + + override fun restoreThreadContext(context: CoroutineContext, oldState: Unit) { + throw TestException("Prevent a coroutine from starting right here for some reason") + } + + override fun updateThreadContext(context: CoroutineContext) { + } + + override fun toString() = "FailingRestore" + } + + private object ThrowingDispatcher : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + throw TestException() + } + + override fun toString() = "ThrowingDispatcher" + } + + private object ThrowingDispatcher2 : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + block.run() + } + + override fun isDispatchNeeded(context: CoroutineContext): Boolean { + throw TestException() + } + + override fun toString() = "ThrowingDispatcher2" + } + + @After + fun tearDown() { + dispatcher.reset() + if (lazyOuterDispatcher.isInitialized()) lazyOuterDispatcher.value.close() + } + + companion object { + @JvmStatic + @Parameterized.Parameters(name = "Element: {0}, dispatcher: {1}") + fun dispatchers(): List> { + val elements = listOf(FailingRestore, FailingUpdate) + val dispatchers = listOf( + TestDispatcher("Dispatchers.Unconfined") { Dispatchers.Unconfined }, + TestDispatcher("Dispatchers.Default") { Dispatchers.Default }, + TestDispatcher("Executors.newFixedThreadPool(1)") { Executors.newFixedThreadPool(1).asCoroutineDispatcher() }, + TestDispatcher("Executors.newScheduledThreadPool(1)") { Executors.newScheduledThreadPool(1).asCoroutineDispatcher() }, + TestDispatcher("ThrowingDispatcher") { ThrowingDispatcher }, + TestDispatcher("ThrowingDispatcher2") { ThrowingDispatcher2 } + ) + return elements.flatMap { element -> + dispatchers.map { dispatcher -> + arrayOf(element, dispatcher) + } + } + } + } + + @Test + fun testElement() = runTest { + // Top-level throwing dispatcher may rethrow an exception right here + runCatching { + launch(NonCancellable + dispatcher.value + exceptionHandler + element) {} + } + checkException() + } + + @Test + fun testNestedElement() = runTest { + // Top-level throwing dispatcher may rethrow an exception right here + runCatching { + launch(NonCancellable + dispatcher.value + exceptionHandler) { + launch(element) { } + } + } + checkException() + } + + @Test + fun testNestedDispatcherAndElement() = runTest { + launch(lazyOuterDispatcher.value + NonCancellable + exceptionHandler) { + launch(element + dispatcher.value) { } + } + checkException() + } + + private fun checkException() { + latch.await(2, TimeUnit.SECONDS) + val e = caught + assertNotNull(e) + // First condition -- failure in context element + val firstCondition = e is CoroutinesInternalError && e.cause is TestException + // Second condition -- failure from isDispatchNeeded (#880) + val secondCondition = e is TestException + assertTrue(firstCondition xor secondCondition) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/IODispatcherTest.kt b/kotlinx-coroutines-core/jvm/test/IODispatcherTest.kt new file mode 100644 index 0000000000..db81f12c79 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/IODispatcherTest.kt @@ -0,0 +1,22 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import kotlin.test.* + +class IODispatcherTest : TestBase() { + @Test + fun testWithIOContext() = runTest { + // just a very basic test that is dispatcher works and indeed uses background thread + val mainThread = Thread.currentThread() + expect(1) + withContext(Dispatchers.IO) { + expect(2) + assertNotSame(mainThread, Thread.currentThread()) + } + + expect(3) + assertSame(mainThread, Thread.currentThread()) + finish(4) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/IntellijIdeaDebuggerEvaluatorCompatibilityTest.kt b/kotlinx-coroutines-core/jvm/test/IntellijIdeaDebuggerEvaluatorCompatibilityTest.kt new file mode 100644 index 0000000000..56f2f3b809 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/IntellijIdeaDebuggerEvaluatorCompatibilityTest.kt @@ -0,0 +1,52 @@ +package kotlinx.coroutines + +import org.junit.Test +import kotlin.coroutines.* +import kotlin.test.* + +class IntellijIdeaDebuggerEvaluatorCompatibilityTest { + + /* + * This test verifies that our CoroutineScope is accessible to IDEA debugger. + * + * Consider the following scenario: + * ``` + * runBlocking { // this: CoroutineScope + * println("runBlocking") + * } + * ``` + * user puts breakpoint to `println` line, opens "Evaluate" window + * and executes `launch { println("launch") }`. They (obviously) expect it to work, but + * it won't: `{}` in `runBlocking` is `SuspendLambda` and `this` is an unused implicit receiver + * that is removed by the compiler (because it's unused). + * + * But we still want to provide consistent user experience for functions with `CoroutineScope` receiver, + * for that IDEA debugger tries to retrieve the scope via `kotlin.coroutines.coroutineContext[Job] as? CoroutineScope` + * and with this test we're fixing this behaviour. + * + * Note that this behaviour is not carved in stone: IDEA fallbacks to `kotlin.coroutines.coroutineContext` for the context if necessary. + */ + + @Test + fun testScopeIsAccessible() = runBlocking { + verify() + + withContext(Job()) { + verify() + } + + coroutineScope { + verify() + } + + supervisorScope { + verify() + } + + } + + private suspend fun verify() { + val ctx = coroutineContext + assertTrue { ctx.job is CoroutineScope } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/JobActivationStressTest.kt b/kotlinx-coroutines-core/jvm/test/JobActivationStressTest.kt new file mode 100644 index 0000000000..c8bfbf01fe --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/JobActivationStressTest.kt @@ -0,0 +1,68 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class JobActivationStressTest : TestBase() { + private val N_ITERATIONS = 10_000 * stressTestMultiplier + private val pool = newFixedThreadPoolContext(3, "JobActivationStressTest") + + @After + fun tearDown() { + pool.close() + } + + /** + * Perform concurrent start & cancel of a job with prior installed completion handlers + */ + @Test + @Suppress("PLATFORM_CLASS_MAPPED_TO_KOTLIN") + fun testActivation() = runTest { + val barrier = CyclicBarrier(3) + val scope = CoroutineScope(pool) + repeat(N_ITERATIONS) { + var wasStarted = false + val d = scope.async(NonCancellable, start = CoroutineStart.LAZY) { + wasStarted = true + throw TestException() + } + // need to add on completion handler + val causeHolder = object { + var cause: Throwable? = null + } + // we use synchronization on causeHolder to work around the fact that completion listeners + // are invoked after the job is in the final state, so when "d.join()" completes there is + // no guarantee that this listener was already invoked + d.invokeOnCompletion { + synchronized(causeHolder) { + causeHolder.cause = it ?: Error("Empty cause") + (causeHolder as Object).notifyAll() + } + } + // concurrent cancel + val canceller = scope.launch { + barrier.await() + d.cancel() + } + // concurrent cancel + val starter = scope.launch { + barrier.await() + d.start() + } + barrier.await() + joinAll(d, canceller, starter) + if (wasStarted) { + val exception = d.getCompletionExceptionOrNull() + assertIs(exception, "exception=$exception") + val cause = synchronized(causeHolder) { + while (causeHolder.cause == null) (causeHolder as Object).wait() + causeHolder.cause + } + assertIs(cause, "cause=$cause") + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/JobCancellationExceptionSerializerTest.kt b/kotlinx-coroutines-core/jvm/test/JobCancellationExceptionSerializerTest.kt new file mode 100644 index 0000000000..18c3d29db5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/JobCancellationExceptionSerializerTest.kt @@ -0,0 +1,65 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import java.io.* + + +@Suppress("BlockingMethodInNonBlockingContext") +class JobCancellationExceptionSerializerTest : TestBase() { + + @Test + fun testSerialization() = runTest { + try { + coroutineScope { + expect(1) + + launch { + expect(2) + try { + hang {} + } catch (e: CancellationException) { + throw RuntimeException("RE2", e) + } + } + + launch { + expect(3) + throw RuntimeException("RE1") + } + } + } catch (e: Throwable) { + // Should not fail + ObjectOutputStream(ByteArrayOutputStream()).use { + it.writeObject(e) + } + finish(4) + } + } + + @Test + fun testHashCodeAfterDeserialization() = runTest { + try { + coroutineScope { + expect(1) + throw JobCancellationException( + message = "Job Cancelled", + job = Job(), + cause = null, + ) + } + } catch (e: Throwable) { + finish(2) + val outputStream = ByteArrayOutputStream() + ObjectOutputStream(outputStream).use { + it.writeObject(e) + } + val deserializedException = + ObjectInputStream(outputStream.toByteArray().inputStream()).use { + it.readObject() as JobCancellationException + } + // verify hashCode does not fail even though Job is transient + assert(deserializedException.hashCode() != 0) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/JobChildStressTest.kt b/kotlinx-coroutines-core/jvm/test/JobChildStressTest.kt new file mode 100644 index 0000000000..16fc64e83e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/JobChildStressTest.kt @@ -0,0 +1,115 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import kotlin.test.* + +/** + * Testing the procedure of attaching a child to the parent job. + */ +class JobChildStressTest : TestBase() { + private val N_ITERATIONS = 10_000 * stressTestMultiplier + private val pool = newFixedThreadPoolContext(3, "JobChildStressTest") + + @AfterTest + fun tearDown() { + pool.close() + } + + /** + * Tests attaching a child while the parent is trying to finalize its state. + * + * Checks the following interleavings: + * - A child attaches before the parent is cancelled. + * - A child attaches after the parent is cancelled, but before the parent notifies anyone about it. + * - A child attaches after the parent notifies the children about being cancelled, + * but before it starts waiting for its children. + * - A child attempts to attach after the parent stops waiting for its children, + * which immediately cancels the child. + */ + @Test + @Suppress("PLATFORM_CLASS_MAPPED_TO_KOTLIN") + fun testChildAttachmentRacingWithCancellation() = runTest { + val barrier = CyclicBarrier(3) + repeat(N_ITERATIONS) { + var wasLaunched = false + var unhandledException: Throwable? = null + val handler = CoroutineExceptionHandler { _, ex -> + unhandledException = ex + } + val scope = CoroutineScope(pool + handler) + val parent = createCompletableDeferredForTesting(it) + // concurrent child launcher + val launcher = scope.launch { + barrier.await() + // A: launch child for a parent job + launch(parent) { + wasLaunched = true + throw TestException() + } + } + // concurrent cancel + val canceller = scope.launch { + barrier.await() + // B: cancel parent job of a child + parent.cancel() + } + barrier.await() + joinAll(launcher, canceller, parent) + assertNull(unhandledException) + if (wasLaunched) { + val exception = parent.getCompletionExceptionOrNull() + assertIs(exception, "exception=$exception") + } + } + } + + /** + * Tests attaching a child while the parent is waiting for the last child job to complete. + * + * Checks the following interleavings: + * - A child attaches while the parent is already completing, but is waiting for its children. + * - A child attempts to attach after the parent stops waiting for its children, + * which immediately cancels the child. + */ + @Test + fun testChildAttachmentRacingWithLastChildCompletion() { + // All exceptions should get aggregated here + repeat(N_ITERATIONS) { + val canCloseThePool = CountDownLatch(1) + runBlocking { + val rogueJob = AtomicReference() + /** not using [createCompletableDeferredForTesting] because we don't need extra children. */ + val deferred = CompletableDeferred() + // optionally, add a completion handler to the parent job, so that the child tries to enter a list with + // multiple elements, not just one. + if (it.mod(2) == 0) { + deferred.invokeOnCompletion { } + } + launch(pool + deferred) { + deferred.complete(Unit) // Transition deferred into "completing" state waiting for current child + // **Asynchronously** submit task that launches a child so it races with completion + pool.executor.execute { + rogueJob.set(launch(pool + deferred) { + throw TestException("isCancelled: ${coroutineContext.job.isCancelled}") + }) + canCloseThePool.countDown() + } + } + + deferred.join() + val rogue = rogueJob.get() + if (rogue?.isActive == true) { + throw TestException("Rogue job $rogue with parent " + rogue.parent + " and children list: " + rogue.parent?.children?.toList()) + } else { + canCloseThePool.await() + rogueJob.get().let { + assertNotNull(it) + assertTrue(it.isCancelled) + } + } + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/JobDisposeStressTest.kt b/kotlinx-coroutines-core/jvm/test/JobDisposeStressTest.kt new file mode 100644 index 0000000000..22b5d5992c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/JobDisposeStressTest.kt @@ -0,0 +1,77 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import kotlin.concurrent.thread + +/** + * Tests concurrent cancel & dispose of the jobs. + */ +class JobDisposeStressTest: TestBase() { + private val TEST_DURATION = 3 * stressTestMultiplier // seconds + + @Volatile + private var done = false + @Volatile + private var job: TestJob? = null + @Volatile + private var handle: DisposableHandle? = null + + @Volatile + private var exception: Throwable? = null + + private fun testThread(name: String, block: () -> Unit): Thread = + thread(start = false, name = name, block = block).apply { + setUncaughtExceptionHandler { t, e -> + exception = e + println("Exception in ${t.name}: $e") + e.printStackTrace() + } + } + + @Test + fun testConcurrentDispose() { + // create threads + val threads = mutableListOf() + threads += testThread("creator") { + while (!done) { + val job = TestJob() + val handle = job.invokeOnCompletion(onCancelling = true) { /* nothing */ } + this.job = job // post job to cancelling thread + this.handle = handle // post handle to concurrent disposer thread + handle.dispose() // dispose of handle from this thread (concurrently with other disposer) + } + } + + threads += testThread("canceller") { + while (!done) { + val job = this.job ?: continue + job.cancel() + // Always returns true, TestJob never completes + } + } + + threads += testThread("disposer") { + while (!done) { + handle?.dispose() + } + } + + // start threads + threads.forEach { it.start() } + // wait + for (i in 1..TEST_DURATION) { + println("$i: Running") + Thread.sleep(1000) + if (exception != null) break + } + // done + done = true + // join threads + threads.forEach { it.join() } + // rethrow exception if any + } + + @Suppress("DEPRECATION_ERROR") + private class TestJob : JobSupport(active = true) +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/JobHandlersUpgradeStressTest.kt b/kotlinx-coroutines-core/jvm/test/JobHandlersUpgradeStressTest.kt new file mode 100644 index 0000000000..3f085b6f20 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/JobHandlersUpgradeStressTest.kt @@ -0,0 +1,97 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import java.util.* +import java.util.concurrent.* +import kotlin.concurrent.* +import kotlin.test.* + +class JobHandlersUpgradeStressTest : TestBase() { + private val nSeconds = 3 * stressTestMultiplier + private val nThreads = 4 + + private val cyclicBarrier = CyclicBarrier(1 + nThreads) + private val threads = mutableListOf() + + private val inters = atomic(0) + private val removed = atomic(0) + private val fired = atomic(0) + + private val sink = atomic(0) + + @Volatile + private var done = false + + @Volatile + private var job: Job? = null + + internal class State { + val state = atomic(0) + } + + /** + * Tests handlers not being invoked more than once. + */ + @Test + fun testStress() { + println("--- JobHandlersUpgradeStressTest") + threads += thread(name = "creator", start = false) { + val rnd = Random() + while (true) { + job = if (done) null else Job() + cyclicBarrier.await() + val job = job ?: break + // burn some time + repeat(rnd.nextInt(3000)) { sink.incrementAndGet() } + // cancel job + job.cancel() + cyclicBarrier.await() + inters.incrementAndGet() + } + } + threads += List(nThreads) { threadId -> + thread(name = "handler-$threadId", start = false) { + val rnd = Random() + while (true) { + val onCancelling = rnd.nextBoolean() + val invokeImmediately: Boolean = rnd.nextBoolean() + cyclicBarrier.await() + val job = job ?: break + val state = State() + // burn some time + repeat(rnd.nextInt(1000)) { sink.incrementAndGet() } + val handle = + job.invokeOnCompletion(onCancelling = onCancelling, invokeImmediately = invokeImmediately) { + if (!state.state.compareAndSet(0, 1)) + error("Fired more than once or too late: state=${state.state.value}") + } + // burn some time + repeat(rnd.nextInt(1000)) { sink.incrementAndGet() } + // dispose + handle.dispose() + cyclicBarrier.await() + val resultingState = state.state.value + when (resultingState) { + 0 -> removed.incrementAndGet() + 1 -> fired.incrementAndGet() + else -> error("Cannot happen") + } + if (!state.state.compareAndSet(resultingState, 2)) + error("Cannot fire late: resultingState=$resultingState") + } + } + } + threads.forEach { it.start() } + repeat(nSeconds) { second -> + Thread.sleep(1000) + println("${second + 1}: ${inters.value} iterations") + } + done = true + threads.forEach { it.join() } + println(" Completed ${inters.value} iterations") + println(" Removed handler ${removed.value} times") + println(" Fired handler ${fired.value} times") + + } +} diff --git a/kotlinx-coroutines-core/jvm/test/JobOnCompletionStressTest.kt b/kotlinx-coroutines-core/jvm/test/JobOnCompletionStressTest.kt new file mode 100644 index 0000000000..3df62b666e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/JobOnCompletionStressTest.kt @@ -0,0 +1,192 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.testing.* +import java.util.concurrent.CyclicBarrier +import java.util.concurrent.atomic.* +import kotlin.test.* +import kotlin.time.Duration.Companion.seconds + +class JobOnCompletionStressTest: TestBase() { + private val N_ITERATIONS = 10_000 * stressTestMultiplier + private val pool = newFixedThreadPoolContext(2, "JobOnCompletionStressTest") + + private val completionHandlerSeesCompletedParent = AtomicBoolean(false) + private val completionHandlerSeesCancelledParent = AtomicBoolean(false) + private val encounteredException = AtomicReference(null) + + @AfterTest + fun tearDown() { + pool.close() + } + + @Test + fun testOnCompletionRacingWithCompletion() = runTest { + testHandlerRacingWithCancellation( + onCancelling = false, + invokeImmediately = true, + parentCompletion = { complete(Unit) } + ) { + assertNull(encounteredException.get()) + assertTrue(completionHandlerSeesCompletedParent.get()) + assertFalse(completionHandlerSeesCancelledParent.get()) + } + } + + @Test + fun testOnCompletionRacingWithCancellation() = runTest { + testHandlerRacingWithCancellation( + onCancelling = false, + invokeImmediately = true, + parentCompletion = { completeExceptionally(TestException()) } + ) { + assertIs(encounteredException.get()) + assertTrue(completionHandlerSeesCompletedParent.get()) + assertTrue(completionHandlerSeesCancelledParent.get()) + } + } + + @Test + fun testOnCancellingRacingWithCompletion() = runTest { + testHandlerRacingWithCancellation( + onCancelling = true, + invokeImmediately = true, + parentCompletion = { complete(Unit) } + ) { + assertNull(encounteredException.get()) + assertTrue(completionHandlerSeesCompletedParent.get()) + assertFalse(completionHandlerSeesCancelledParent.get()) + } + } + + @Test + fun testOnCancellingRacingWithCancellation() = runTest { + testHandlerRacingWithCancellation( + onCancelling = true, + invokeImmediately = true, + parentCompletion = { completeExceptionally(TestException()) } + ) { + assertIs(encounteredException.get()) + assertTrue(completionHandlerSeesCancelledParent.get()) + } + } + + @Test + fun testNonImmediateOnCompletionRacingWithCompletion() = runTest { + testHandlerRacingWithCancellation( + onCancelling = false, + invokeImmediately = false, + parentCompletion = { complete(Unit) } + ) { + assertNull(encounteredException.get()) + assertTrue(completionHandlerSeesCompletedParent.get()) + assertFalse(completionHandlerSeesCancelledParent.get()) + } + } + + @Test + fun testNonImmediateOnCompletionRacingWithCancellation() = runTest { + testHandlerRacingWithCancellation( + onCancelling = false, + invokeImmediately = false, + parentCompletion = { completeExceptionally(TestException()) } + ) { + assertIs(encounteredException.get()) + assertTrue(completionHandlerSeesCompletedParent.get()) + assertTrue(completionHandlerSeesCancelledParent.get()) + } + } + + @Test + fun testNonImmediateOnCancellingRacingWithCompletion() = runTest { + testHandlerRacingWithCancellation( + onCancelling = true, + invokeImmediately = false, + parentCompletion = { complete(Unit) } + ) { + assertNull(encounteredException.get()) + assertTrue(completionHandlerSeesCompletedParent.get()) + assertFalse(completionHandlerSeesCancelledParent.get()) + } + } + + @Test + fun testNonImmediateOnCancellingRacingWithCancellation() = runTest { + testHandlerRacingWithCancellation( + onCancelling = true, + invokeImmediately = false, + parentCompletion = { completeExceptionally(TestException()) } + ) { + assertIs(encounteredException.get()) + assertTrue(completionHandlerSeesCancelledParent.get()) + } + } + + private suspend fun testHandlerRacingWithCancellation( + onCancelling: Boolean, + invokeImmediately: Boolean, + parentCompletion: CompletableDeferred.() -> Unit, + validate: () -> Unit, + ) { + repeat(N_ITERATIONS) { + val entered = Channel(1) + completionHandlerSeesCompletedParent.set(false) + completionHandlerSeesCancelledParent.set(false) + encounteredException.set(null) + val parent = createCompletableDeferredForTesting(it) + val barrier = CyclicBarrier(2) + val handlerInstallJob = coroutineScope { + launch(pool) { + barrier.await() + parent.parentCompletion() + } + async(pool) { + barrier.await() + parent.invokeOnCompletion( + onCancelling = onCancelling, + invokeImmediately = invokeImmediately, + ) { exception -> + encounteredException.set(exception) + completionHandlerSeesCompletedParent.set(parent.isCompleted) + completionHandlerSeesCancelledParent.set(parent.isCancelled) + entered.trySend(Unit) + } + } + } + if (invokeImmediately || handlerInstallJob.getCompleted() !== NonDisposableHandle) { + withTimeout(1.seconds) { + entered.receive() + } + try { + validate() + } catch (e: Throwable) { + println("Iteration $it failed") + println("invokeOnCompletion returned ${handlerInstallJob.getCompleted()}") + throw e + } + } else { + assertTrue(entered.isEmpty) + } + } + } +} + +/** + * Creates a [CompletableDeferred], optionally adding completion handlers and/or other children to the job depending + * on [iteration]. + * The purpose is to test not just attaching completion handlers to empty or one-element lists (see the [JobSupport] + * implementation for details on what this means), but also to lists with multiple elements. + */ +fun createCompletableDeferredForTesting(iteration: Int): CompletableDeferred { + val parent = CompletableDeferred() + /* We optionally add completion handlers and/or other children to the parent job + to test the scenarios where a child is placed into an empty list, a single-element list, + or a list with multiple elements. */ + if (iteration.mod(2) == 0) { + parent.invokeOnCompletion { } + } + if (iteration.mod(3) == 0) { + GlobalScope.launch(parent) { } + } + return parent +} diff --git a/kotlinx-coroutines-core/jvm/test/JobStressTest.kt b/kotlinx-coroutines-core/jvm/test/JobStressTest.kt new file mode 100644 index 0000000000..cb0274e8c0 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/JobStressTest.kt @@ -0,0 +1,14 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class JobStressTest : TestBase() { + @Test + fun testMemoryRelease() { + val job = Job() + val n = 10_000_000 * stressTestMultiplier + var fireCount = 0 + for (i in 0 until n) job.invokeOnCompletion { fireCount++ }.dispose() + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/JoinStressTest.kt b/kotlinx-coroutines-core/jvm/test/JoinStressTest.kt new file mode 100644 index 0000000000..6c3190de20 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/JoinStressTest.kt @@ -0,0 +1,90 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class JoinStressTest : TestBase() { + + private val iterations = 50_000 * stressTestMultiplier + private val pool = newFixedThreadPoolContext(3, "JoinStressTest") + + @After + fun tearDown() { + pool.close() + } + + @Test + fun testExceptionalJoinWithCancellation() = runBlocking { + val results = IntArray(2) + + repeat(iterations) { + val barrier = CyclicBarrier(3) + val exceptionalJob = async(pool + NonCancellable) { + barrier.await() + throw TestException() + } + + + val awaiterJob = async(pool) { + barrier.await() + try { + exceptionalJob.await() + } catch (e: TestException) { + 0 + } catch (e: CancellationException) { + 1 + } + } + + barrier.await() + exceptionalJob.cancel() + ++results[awaiterJob.await()] + } + + // Check that concurrent cancellation of job which throws TestException without suspends doesn't suppress TestException + assertEquals(iterations, results[0], results.toList().toString()) + assertEquals(0, results[1], results.toList().toString()) + } + + @Test + fun testExceptionalJoinWithMultipleCancellations() = runBlocking { + val results = IntArray(2) + + repeat(iterations) { + val barrier = CyclicBarrier(4) + val exceptionalJob = async(pool + NonCancellable) { + barrier.await() + throw TestException() + } + + val awaiterJob = async(pool) { + barrier.await() + try { + exceptionalJob.await() + 2 + } catch (e: TestException) { + 0 + } catch (e: TestException1) { + 1 + } + } + + val canceller = async(pool + NonCancellable) { + barrier.await() + // cast for test purposes only + (exceptionalJob as AbstractCoroutine<*>).cancelInternal(TestException1()) + } + + barrier.await() + val awaiterResult = awaiterJob.await() + canceller.await() + ++results[awaiterResult] + } + + assertTrue(results[0] > 0, results.toList().toString()) + assertTrue(results[1] > 0, results.toList().toString()) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/LimitedParallelismStressTest.kt b/kotlinx-coroutines-core/jvm/test/LimitedParallelismStressTest.kt new file mode 100644 index 0000000000..c5d0fbeef6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/LimitedParallelismStressTest.kt @@ -0,0 +1,148 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import kotlin.coroutines.CoroutineContext +import kotlin.coroutines.EmptyCoroutineContext +import kotlin.test.* + +@RunWith(Parameterized::class) +class LimitedParallelismStressTest(private val targetParallelism: Int) : TestBase() { + + companion object { + @Parameterized.Parameters(name = "{0}") + @JvmStatic + fun params(): Collection> = listOf(1, 2, 3, 4).map { arrayOf(it) } + } + + @get:Rule + val executor = ExecutorRule(targetParallelism * 2) + private val iterations = 100_000 + + private val parallelism = AtomicInteger(0) + + private fun checkParallelism() { + val value = parallelism.incrementAndGet() + Thread.yield() + assertTrue { value <= targetParallelism } + parallelism.decrementAndGet() + } + + @Test + fun testLimitedExecutor() = runTest { + val view = executor.limitedParallelism(targetParallelism) + doStress { + repeat(iterations) { + launch(view) { + checkParallelism() + } + } + } + } + + @Test + fun testLimitedDispatchersIo() = runTest { + val view = Dispatchers.IO.limitedParallelism(targetParallelism) + doStress { + repeat(iterations) { + launch(view) { + checkParallelism() + } + } + } + } + + @Test + fun testLimitedDispatchersIoDispatchYield() = runTest { + val view = Dispatchers.IO.limitedParallelism(targetParallelism) + doStress { + launch(view) { + yield() + checkParallelism() + } + } + } + + @Test + fun testLimitedExecutorReachesTargetParallelism() = runTest { + val view = executor.limitedParallelism(targetParallelism) + doStress { + repeat(iterations) { + val barrier = CyclicBarrier(targetParallelism + 1) + repeat(targetParallelism) { + launch(view) { + barrier.await() + } + } + // Successfully awaited parallelism + 1 + barrier.await() + coroutineContext.job.children.toList().joinAll() + } + } + } + + /** + * Checks that dispatcher failures during fairness redispatches don't prevent reaching the target parallelism. + */ + @Test + fun testLimitedFailingDispatcherReachesTargetParallelism() = runTest { + val keepFailing = AtomicBoolean(true) + val occasionallyFailing = object: CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + if (keepFailing.get() && ThreadLocalRandom.current().nextBoolean()) throw TestException() + executor.dispatch(context, block) + } + }.limitedParallelism(targetParallelism) + doStress { + repeat(1000) { + keepFailing.set(true) // we want the next tasks to sporadically fail + // Start some tasks to make sure redispatching for fairness is happening + repeat(targetParallelism * 16 + 1) { + // targetParallelism * 16 + 1 because we need at least one worker to go through a fairness yield + // with high probability. + try { + occasionallyFailing.dispatch(EmptyCoroutineContext, Runnable { + // do nothing. + }) + } catch (_: DispatchException) { + // ignore + } + } + keepFailing.set(false) // we want the next tasks to succeed + val barrier = CyclicBarrier(targetParallelism + 1) + repeat(targetParallelism) { + launch(occasionallyFailing) { + barrier.await() + } + } + val success = launch(Dispatchers.Default) { + // Successfully awaited parallelism + 1 + barrier.await() + } + // Feed the dispatcher with more tasks to make sure it's not stuck + while (success.isActive) { + Thread.sleep(1) + repeat(targetParallelism) { + occasionallyFailing.dispatch(EmptyCoroutineContext, Runnable { + // do nothing. + }) + } + } + coroutineContext.job.children.toList().joinAll() + } + } + } + + private suspend inline fun doStress(crossinline block: suspend CoroutineScope.() -> Unit) { + repeat(stressTestMultiplier) { + coroutineScope { + block() + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/LimitedParallelismUnhandledExceptionTest.kt b/kotlinx-coroutines-core/jvm/test/LimitedParallelismUnhandledExceptionTest.kt new file mode 100644 index 0000000000..e95255490a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/LimitedParallelismUnhandledExceptionTest.kt @@ -0,0 +1,29 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.coroutines.* +import kotlin.test.* + +class LimitedParallelismUnhandledExceptionTest : TestBase() { + + @Test + fun testUnhandledException() = runTest { + var caughtException: Throwable? = null + val executor = Executors.newFixedThreadPool( + 1 + ) { + Thread(it).also { + it.uncaughtExceptionHandler = Thread.UncaughtExceptionHandler { _, e -> caughtException = e } + } + }.asCoroutineDispatcher() + val view = executor.limitedParallelism(1) + view.dispatch(EmptyCoroutineContext, Runnable { throw TestException() }) + withContext(view) { + // Verify it is in working state and establish happens-before + } + assertTrue { caughtException is TestException } + executor.close() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/MemoryFootprintTest.kt b/kotlinx-coroutines-core/jvm/test/MemoryFootprintTest.kt new file mode 100644 index 0000000000..7a3b69fda6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/MemoryFootprintTest.kt @@ -0,0 +1,43 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import org.openjdk.jol.info.* +import kotlin.test.* + + +class MemoryFootprintTest : TestBase(true) { + + @Test + fun testJobLayout() = assertLayout(Job().javaClass, 24) + + @Test + fun testJobSize() { + assertTotalSize(jobWithChildren(1), 112) + assertTotalSize(jobWithChildren(2), 192) // + 80 + assertTotalSize(jobWithChildren(3), 248) // + 56 + assertTotalSize(jobWithChildren(4), 304) // + 56 + } + + private fun jobWithChildren(numberOfChildren: Int): Job { + val result = Job() + repeat(numberOfChildren) { + Job(result) + } + return result + } + + @Test + fun testCancellableContinuationFootprint() = assertLayout(CancellableContinuationImpl::class.java, 48) + + private fun assertLayout(clz: Class<*>, expectedSize: Int) { + val size = ClassLayout.parseClass(clz).instanceSize() +// println(ClassLayout.parseClass(clz).toPrintable()) + assertEquals(expectedSize.toLong(), size) + } + + private fun assertTotalSize(instance: Job, expectedSize: Int) { + val size = GraphLayout.parseInstance(instance).totalSize() + assertEquals(expectedSize.toLong(), size) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/MultithreadedDispatchersJvmTest.kt b/kotlinx-coroutines-core/jvm/test/MultithreadedDispatchersJvmTest.kt new file mode 100644 index 0000000000..b10ab34668 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/MultithreadedDispatchersJvmTest.kt @@ -0,0 +1,30 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.LocalAtomicInt +import kotlinx.coroutines.testing.* +import java.util.concurrent.ScheduledThreadPoolExecutor +import kotlin.coroutines.EmptyCoroutineContext +import kotlin.test.* + +class MultithreadedDispatchersJvmTest: TestBase() { + /** Tests that the executor created in [newFixedThreadPoolContext] can not leak and be reconfigured. */ + @Test + fun testExecutorReconfiguration() { + newFixedThreadPoolContext(1, "test").apply { + (executor as? ScheduledThreadPoolExecutor)?.corePoolSize = 2 + }.use { ctx -> + val atomicInt = LocalAtomicInt(0) + repeat(100) { + ctx.dispatch(EmptyCoroutineContext, Runnable { + val entered = atomicInt.incrementAndGet() + Thread.yield() // allow other tasks to run + try { + check(entered == 1) { "Expected only one thread to be used, observed $entered" } + } finally { + atomicInt.decrementAndGet() + } + }) + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/MutexCancellationStressTest.kt b/kotlinx-coroutines-core/jvm/test/MutexCancellationStressTest.kt new file mode 100644 index 0000000000..3528702a05 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/MutexCancellationStressTest.kt @@ -0,0 +1,82 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlinx.coroutines.sync.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicBoolean +import java.util.concurrent.atomic.AtomicInteger +import kotlin.test.* + +class MutexCancellationStressTest : TestBase() { + @Test + fun testStressCancellationDoesNotBreakMutex() = runTest { + val mutex = Mutex() + val mutexJobNumber = 3 + val mutexOwners = Array(mutexJobNumber) { "$it" } + val dispatcher = Executors.newFixedThreadPool(mutexJobNumber + 2).asCoroutineDispatcher() + var counter = 0 + val counterLocal = Array(mutexJobNumber) { AtomicInteger(0) } + val completed = AtomicBoolean(false) + val mutexJobLauncher: (jobNumber: Int) -> Job = { jobId -> + val coroutineName = "MutexJob-$jobId" + // ATOMIC to always have a chance to proceed + launch(dispatcher + CoroutineName(coroutineName), CoroutineStart.ATOMIC) { + while (!completed.get()) { + // Stress out holdsLock + mutex.holdsLock(mutexOwners[(jobId + 1) % mutexJobNumber]) + // Stress out lock-like primitives + if (mutex.tryLock(mutexOwners[jobId])) { + counterLocal[jobId].incrementAndGet() + counter++ + mutex.unlock(mutexOwners[jobId]) + } + mutex.withLock(mutexOwners[jobId]) { + counterLocal[jobId].incrementAndGet() + counter++ + } + select { + mutex.onLock(mutexOwners[jobId]) { + counterLocal[jobId].incrementAndGet() + counter++ + mutex.unlock(mutexOwners[jobId]) + } + } + } + } + } + val mutexJobs = (0 until mutexJobNumber).map { mutexJobLauncher(it) }.toMutableList() + val checkProgressJob = launch(dispatcher + CoroutineName("checkProgressJob")) { + var lastCounterLocalSnapshot = (0 until mutexJobNumber).map { 0 } + while (!completed.get()) { + delay(500) + // If we've caught the completion after delay, then there is a chance no progress were made whatsoever, bail out + if (completed.get()) return@launch + val c = counterLocal.map { it.get() } + for (i in 0 until mutexJobNumber) { + assert(c[i] > lastCounterLocalSnapshot[i]) { "No progress in MutexJob-$i, last observed state: ${c[i]}" } + } + lastCounterLocalSnapshot = c + } + } + val cancellationJob = launch(dispatcher + CoroutineName("cancellationJob")) { + var cancellingJobId = 0 + while (!completed.get()) { + val jobToCancel = mutexJobs.removeFirst() + jobToCancel.cancelAndJoin() + mutexJobs += mutexJobLauncher(cancellingJobId) + cancellingJobId = (cancellingJobId + 1) % mutexJobNumber + } + } + delay(2000L * stressTestMultiplier) + completed.set(true) + cancellationJob.join() + mutexJobs.forEach { it.join() } + checkProgressJob.join() + assertEquals(counter, counterLocal.sumOf { it.get() }) + dispatcher.close() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/NoParamAssertionsTest.kt b/kotlinx-coroutines-core/jvm/test/NoParamAssertionsTest.kt new file mode 100644 index 0000000000..40b614441a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/NoParamAssertionsTest.kt @@ -0,0 +1,27 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + + +class NoParamAssertionsTest : TestBase() { + // These tests verify that we haven't omitted "-Xno-param-assertions" and "-Xno-receiver-assertions" + + @Test + fun testNoReceiverAssertion() { + val function: (ThreadLocal, Int) -> ThreadContextElement = ThreadLocal::asContextElement + @Suppress("UNCHECKED_CAST") + val unsafeCasted = function as ((ThreadLocal?, Int) -> ThreadContextElement) + unsafeCasted(null, 42) + } + + @Test + fun testNoParamAssertion() { + val function: (ThreadLocal, Any) -> ThreadContextElement = ThreadLocal::asContextElement + @Suppress("UNCHECKED_CAST") + val unsafeCasted = function as ((ThreadLocal?, Any?) -> ThreadContextElement) + unsafeCasted(ThreadLocal.withInitial { Any() }, null) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/RejectedExecutionTest.kt b/kotlinx-coroutines-core/jvm/test/RejectedExecutionTest.kt new file mode 100644 index 0000000000..cf72b31481 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/RejectedExecutionTest.kt @@ -0,0 +1,166 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.scheduling.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class RejectedExecutionTest : TestBase() { + private val threadName = "RejectedExecutionTest" + private val executor = RejectingExecutor() + + @After + fun tearDown() { + executor.shutdown() + executor.awaitTermination(10, TimeUnit.SECONDS) + } + + @Test + fun testRejectOnLaunch() = runTest { + expect(1) + val job = launch(executor.asCoroutineDispatcher()) { + expectUnreached() + } + assertEquals(1, executor.submittedTasks) + assertTrue(job.isCancelled) + finish(2) + } + + @Test + fun testRejectOnLaunchAtomic() = runTest { + expect(1) + val job = launch(executor.asCoroutineDispatcher(), start = CoroutineStart.ATOMIC) { + expect(2) + assertEquals(true, coroutineContext[Job]?.isCancelled) + assertIoThread() // was rejected on start, but start was atomic + } + assertEquals(1, executor.submittedTasks) + job.join() + finish(3) + } + + @Test + fun testRejectOnWithContext() = runTest { + expect(1) + assertFailsWith { + withContext(executor.asCoroutineDispatcher()) { + expectUnreached() + } + } + assertEquals(1, executor.submittedTasks) + finish(2) + } + + @Test + fun testRejectOnResumeInContext() = runTest { + expect(1) + executor.acceptTasks = 1 // accept one task + assertFailsWith { + withContext(executor.asCoroutineDispatcher()) { + expect(2) + assertExecutorThread() + try { + withContext(Dispatchers.Default) { + expect(3) + assertDefaultDispatcherThread() + // We have to wait until caller executor thread had already suspended (if not running task), + // so that we resume back to it a new task is posted + executor.awaitNotRunningTask() + expect(4) + assertDefaultDispatcherThread() + } + // cancelled on resume back + } finally { + expect(5) + assertIoThread() + } + expectUnreached() + } + } + assertEquals(2, executor.submittedTasks) + finish(6) + } + + @Test + fun testRejectOnDelay() = runTest { + expect(1) + executor.acceptTasks = 1 // accept one task + assertFailsWith { + withContext(executor.asCoroutineDispatcher()) { + expect(2) + assertExecutorThread() + try { + delay(10) // cancelled + } finally { + // Since it was cancelled on attempt to delay, it still stays on the same thread + assertExecutorThread() + } + expectUnreached() + } + } + assertEquals(2, executor.submittedTasks) + finish(3) + } + + @Test + fun testRejectWithTimeout() = runTest { + expect(1) + executor.acceptTasks = 1 // accept one task + assertFailsWith { + withContext(executor.asCoroutineDispatcher()) { + expect(2) + assertExecutorThread() + withTimeout(1000) { + expect(3) // atomic entry into the block (legacy behavior, it seem to be Ok with way) + assertEquals(true, coroutineContext[Job]?.isCancelled) // but the job is already cancelled + } + expectUnreached() + } + } + assertEquals(2, executor.submittedTasks) + finish(4) + } + + private inner class RejectingExecutor : ScheduledThreadPoolExecutor(1, { r -> Thread(r, threadName) }) { + var acceptTasks = 0 + var submittedTasks = 0 + val runningTask = MutableStateFlow(false) + + override fun schedule(command: Runnable, delay: Long, unit: TimeUnit): ScheduledFuture<*> { + submittedTasks++ + if (submittedTasks > acceptTasks) throw RejectedExecutionException() + val wrapper = Runnable { + runningTask.value = true + try { + command.run() + } finally { + runningTask.value = false + } + } + return super.schedule(wrapper, delay, unit) + } + + suspend fun awaitNotRunningTask() = runningTask.first { !it } + } + + private fun assertExecutorThread() { + val thread = Thread.currentThread() + if (!thread.name.startsWith(threadName)) error("Not an executor thread: $thread") + } + + private fun assertDefaultDispatcherThread() { + val thread = Thread.currentThread() + if (thread !is CoroutineScheduler.Worker) error("Not a thread from Dispatchers.Default: $thread") + assertEquals(CoroutineScheduler.WorkerState.CPU_ACQUIRED, thread.state) + } + + private fun assertIoThread() { + val thread = Thread.currentThread() + if (thread !is CoroutineScheduler.Worker) error("Not a thread from Dispatchers.IO: $thread") + assertEquals(CoroutineScheduler.WorkerState.BLOCKING, thread.state) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationInvariantStressTest.kt b/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationInvariantStressTest.kt new file mode 100644 index 0000000000..ef0d146d7a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationInvariantStressTest.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import java.util.concurrent.CountDownLatch +import java.util.concurrent.atomic.AtomicReference +import kotlin.coroutines.* + +// Stresses scenario from #3613 +class ReusableCancellableContinuationInvariantStressTest : TestBase() { + + // Tests have a timeout 10 sec because the bug they catch leads to an infinite spin-loop + + @Test(timeout = 10_000) + fun testExceptionFromSuspendReusable() = doTest { /* nothing */ } + + + @Test(timeout = 10_000) + fun testExceptionFromCancelledSuspendReusable() = doTest { it.cancel() } + + + @Suppress("SuspendFunctionOnCoroutineScope") + private inline fun doTest(crossinline block: (Job) -> Unit) { + runTest { + repeat(10_000) { + val latch = CountDownLatch(1) + val continuationToResume = AtomicReference?>(null) + val j1 = launch(Dispatchers.Default) { + latch.await() + suspendCancellableCoroutineReusable { + continuationToResume.set(it) + block(coroutineContext.job) + throw CancellationException() // Don't let getResult() chance to execute + } + } + + val j2 = launch(Dispatchers.Default) { + latch.await() + while (continuationToResume.get() == null) { + // spin + } + continuationToResume.get()!!.resume(Unit) + } + + latch.countDown() + joinAll(j1, j2) + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationLeakStressTest.kt b/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationLeakStressTest.kt new file mode 100644 index 0000000000..b2ed34cd57 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationLeakStressTest.kt @@ -0,0 +1,38 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import org.junit.Test +import kotlin.test.* + +class ReusableCancellableContinuationLeakStressTest : TestBase() { + + @Suppress("UnnecessaryVariable") + private suspend fun ReceiveChannel.receiveBatch(): T { + val r = receive() // DO NOT MERGE LINES, otherwise TCE will kick in + return r + } + + private val iterations = 100_000 * stressTestMultiplier + + class Leak(val i: Int) + + @Test // Simplified version of #2564 + fun testReusableContinuationLeak() = runTest { + val channel = produce(capacity = 1) { // from the main thread + (0 until iterations).forEach { + send(Leak(it)) + } + } + + launch(Dispatchers.Default) { + repeat (iterations) { + val value = channel.receiveBatch() + assertEquals(it, value.i) + } + (channel as Job).join() + + FieldWalker.assertReachableCount(0, coroutineContext.job, false) { it is Leak } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationTest.kt b/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationTest.kt new file mode 100644 index 0000000000..5e88521f79 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ReusableCancellableContinuationTest.kt @@ -0,0 +1,218 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import org.junit.Test +import kotlin.coroutines.* +import kotlin.test.* + +class ReusableCancellableContinuationTest : TestBase() { + @Test + fun testReusable() = runTest { + testContinuationsCount(10, 1, ::suspendCancellableCoroutineReusable) + } + + @Test + fun testRegular() = runTest { + testContinuationsCount(10, 10, ::suspendCancellableCoroutine) + } + + private suspend inline fun CoroutineScope.testContinuationsCount( + iterations: Int, + expectedInstances: Int, + suspender: suspend ((CancellableContinuation) -> Unit) -> Unit + ) { + val result = mutableSetOf>() + val job = coroutineContext[Job]!! + val channel = Channel>(1) + launch { + channel.consumeEach { + val f = FieldWalker.walk(job) + result.addAll(f.filterIsInstance>()) + it.resumeWith(Result.success(Unit)) + } + } + + repeat(iterations) { + suspender { + assertTrue(channel.trySend(it).isSuccess) + } + } + channel.close() + assertEquals(expectedInstances, result.size - 1) + } + + @Test + fun testCancelledOnClaimedCancel() = runTest { + expect(1) + try { + suspendCancellableCoroutineReusable { + it.cancel() + } + expectUnreached() + } catch (e: CancellationException) { + finish(2) + } + } + + @Test + fun testNotCancelledOnClaimedResume() = runTest({ it is CancellationException }) { + expect(1) + // Bind child at first + var continuation: Continuation<*>? = null + suspendCancellableCoroutineReusable { + expect(2) + continuation = it + launch { // Attach to the parent, avoid fast path + expect(3) + it.resume(Unit) + } + } + expect(4) + ensureActive() + // Verify child was bound + FieldWalker.assertReachableCount(1, coroutineContext[Job]) { it === continuation } + try { + suspendCancellableCoroutineReusable { + expect(5) + coroutineContext[Job]!!.cancel() + it.resume(Unit) // will not dispatch, will get CancellationException + } + } catch (e: CancellationException) { + assertFalse(isActive) + finish(6) + } + } + + @Test + fun testResumeReusablePreservesReference() = runTest { + expect(1) + var cont: Continuation? = null + launch { + cont!!.resumeWith(Result.success(Unit)) + } + suspendCancellableCoroutineReusable { + cont = it + } + ensureActive() + assertTrue { FieldWalker.walk(coroutineContext[Job]).contains(cont!!) } + finish(2) + } + + @Test + fun testResumeRegularDoesntPreservesReference() = runTest { + expect(1) + var cont: Continuation? = null + launch { // Attach to the parent, avoid fast path + cont!!.resumeWith(Result.success(Unit)) + } + suspendCancellableCoroutine { + cont = it + } + ensureActive() + FieldWalker.assertReachableCount(0, coroutineContext[Job]) { it === cont } + finish(2) + } + + @Test + fun testDetachedOnCancel() = runTest { + expect(1) + var cont: Continuation<*>? = null + try { + suspendCancellableCoroutineReusable { + cont = it + it.cancel() + } + expectUnreached() + } catch (e: CancellationException) { + FieldWalker.assertReachableCount(0, coroutineContext[Job]) { it === cont } + finish(2) + } + } + + @Test + fun testPropagatedCancel() = runTest({it is CancellationException}) { + val currentJob = coroutineContext[Job]!! + expect(1) + // Bind child at first + suspendCancellableCoroutineReusable { + expect(2) + // Attach to the parent, avoid fast path + launch { + expect(3) + it.resume(Unit) + } + } + expect(4) + ensureActive() + // Verify child was bound + FieldWalker.assertReachableCount(1, currentJob) { it is CancellableContinuation<*> } + currentJob.cancel() + assertFalse(isActive) + // Child detached + FieldWalker.assertReachableCount(0, currentJob) { it is CancellableContinuation<*> } + expect(5) + try { + // Resume is non-atomic, so it throws cancellation exception + suspendCancellableCoroutineReusable { + expect(6) // but the code inside the block is executed + it.resume(Unit) + } + } catch (e: CancellationException) { + FieldWalker.assertReachableCount(0, currentJob) { it is CancellableContinuation<*> } + expect(7) + } + try { + // No resume -- still cancellation exception + suspendCancellableCoroutineReusable {} + } catch (e: CancellationException) { + FieldWalker.assertReachableCount(0, currentJob) { it is CancellableContinuation<*> } + finish(8) + } + } + + @Test + fun testChannelMemoryLeak() = runTest { + val iterations = 100 + val channel = Channel() + launch { + repeat(iterations) { + select { + channel.onSend(Unit) {} + } + } + } + + val receiver = launch { + repeat(iterations) { + channel.receive() + } + expect(2) + val job = coroutineContext[Job]!! + // 1 for reusable CC, another one for outer joiner + FieldWalker.assertReachableCount(2, job) { it is CancellableContinuation<*> } + } + expect(1) + receiver.join() + // Reference should be claimed at this point + FieldWalker.assertReachableCount(0, receiver) { it is CancellableContinuation<*> } + finish(3) + } + + @Test + fun testReusableAndRegularSuspendCancellableCoroutineMemoryLeak() = runTest { + val channel = produce { + repeat(10) { + send(Unit) + } + } + for (value in channel) { + delay(1) + } + FieldWalker.assertReachableCount(1, coroutineContext[Job]) { + // could be `it is ChildContinuation` if `ChildContinuation` wasn't private + it::class.simpleName == "ChildContinuation" + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ReusableContinuationStressTest.kt b/kotlinx-coroutines-core/jvm/test/ReusableContinuationStressTest.kt new file mode 100644 index 0000000000..489ecc3d51 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ReusableContinuationStressTest.kt @@ -0,0 +1,38 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.flow.* +import org.junit.* + +class ReusableContinuationStressTest : TestBase() { + + private val iterations = 1000 * stressTestMultiplierSqrt + + @Test // Originally reported by @denis-bezrukov in #2736 + fun testDebounceWithStateFlow() = runBlocking { + withContext(Dispatchers.Default) { + repeat(iterations) { + launch { // <- load the dispatcher and OS scheduler + runStressTestOnce(1, 1) + } + } + } + } + + private suspend fun runStressTestOnce(delay: Int, debounce: Int) = coroutineScope { + val stateFlow = MutableStateFlow(0) + val emitter = launch { + repeat(1000) { i -> + stateFlow.emit(i) + delay(delay.toLong()) + } + } + var last = 0 + stateFlow.debounce(debounce.toLong()).take(100).collect { i -> + if (i - last > 100) { + last = i + } + } + emitter.cancel() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/RunBlockingJvmTest.kt b/kotlinx-coroutines-core/jvm/test/RunBlockingJvmTest.kt new file mode 100644 index 0000000000..dcb908cc3a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/RunBlockingJvmTest.kt @@ -0,0 +1,192 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import java.util.concurrent.CountDownLatch +import java.util.concurrent.atomic.AtomicReference +import kotlin.concurrent.thread +import kotlin.test.* +import kotlin.time.Duration + +class RunBlockingJvmTest : TestBase() { + @Test + fun testContract() { + val rb: Int + runBlocking { + rb = 42 + } + rb.hashCode() // unused + } + + /** Tests that the [runBlocking] coroutine runs to completion even it was interrupted. */ + @Test + fun testFinishingWhenInterrupted() { + startInSeparateThreadAndInterrupt { mayInterrupt -> + expect(1) + try { + runBlocking { + try { + mayInterrupt() + expect(2) + delay(Duration.INFINITE) + } finally { + withContext(NonCancellable) { + expect(3) + repeat(10) { yield() } + expect(4) + } + } + } + } catch (_: InterruptedException) { + expect(5) + } + } + finish(6) + } + + /** Tests that [runBlocking] will exit if it gets interrupted. */ + @Test + fun testCancellingWhenInterrupted() { + startInSeparateThreadAndInterrupt { mayInterrupt -> + expect(1) + try { + runBlocking { + try { + mayInterrupt() + expect(2) + delay(Duration.INFINITE) + } catch (_: CancellationException) { + expect(3) + } + } + } catch (_: InterruptedException) { + expect(4) + } + } + finish(5) + } + + /** Tests that [runBlocking] does not check for interruptions before the first attempt to suspend, + * as no blocking actually happens. */ + @Test + fun testInitialPortionRunningDespiteInterruptions() { + Thread.currentThread().interrupt() + runBlocking { + expect(1) + try { + Thread.sleep(Long.MAX_VALUE) + } catch (_: InterruptedException) { + expect(2) + } + } + assertFalse(Thread.interrupted()) + finish(3) + } + + /** + * Tests that [runBlockingNonInterruptible] is going to run its job to completion even if it gets interrupted + * or if thread switches occur. + */ + @Test + fun testNonInterruptibleRunBlocking() { + startInSeparateThreadAndInterrupt { mayInterrupt -> + val v = runBlockingNonInterruptible { + mayInterrupt() + repeat(10) { + expect(it + 1) + delay(1) + } + 42 + } + assertTrue(Thread.interrupted()) + assertEquals(42, v) + expect(11) + } + finish(12) + } + + /** + * Tests that [runBlockingNonInterruptible] is going to run its job to completion even if it gets interrupted + * or if thread switches occur, and then will rethrow the exception thrown by the job. + */ + @Test + fun testNonInterruptibleRunBlockingFailure() { + val exception = AssertionError() + startInSeparateThreadAndInterrupt { mayInterrupt -> + val exception2 = assertFailsWith { + runBlockingNonInterruptible { + mayInterrupt() + repeat(10) { + expect(it + 1) + // even thread switches should not be a problem + withContext(Dispatchers.IO) { + delay(1) + } + } + throw exception + } + } + assertTrue(Thread.interrupted()) + assertSame(exception, exception2) + expect(11) + } + finish(12) + } + + + /** + * Tests that [runBlockingNonInterruptible] is going to run its job to completion even if it gets interrupted + * or if thread switches occur. + */ + @Test + fun testNonInterruptibleRunBlockingPropagatingInterruptions() { + val exception = AssertionError() + startInSeparateThreadAndInterrupt { mayInterrupt -> + runBlockingNonInterruptible { + mayInterrupt() + try { + Thread.sleep(Long.MAX_VALUE) + } catch (_: InterruptedException) { + expect(1) + } + } + expect(2) + assertFalse(Thread.interrupted()) + } + finish(3) + } + + /** + * Tests that starting [runBlockingNonInterruptible] in an interrupted thread does not affect the result. + */ + @Test + fun testNonInterruptibleRunBlockingStartingInterrupted() { + Thread.currentThread().interrupt() + val v = runBlockingNonInterruptible { 42 } + assertEquals(42, v) + assertTrue(Thread.interrupted()) + } + + private fun startInSeparateThreadAndInterrupt(action: (mayInterrupt: () -> Unit) -> Unit) { + val latch = CountDownLatch(1) + val thread = thread { + action { latch.countDown() } + } + latch.await() + thread.interrupt() + thread.join() + } + + private fun runBlockingNonInterruptible(action: suspend () -> T): T { + val result = AtomicReference>() + try { + runBlocking { + withContext(NonCancellable) { + result.set(runCatching { action() }) + } + } + } catch (_: InterruptedException) { + Thread.currentThread().interrupt() // restore the interrupted flag + } + return result.get().getOrThrow() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/RunInterruptibleStressTest.kt b/kotlinx-coroutines-core/jvm/test/RunInterruptibleStressTest.kt new file mode 100644 index 0000000000..b42bdad534 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/RunInterruptibleStressTest.kt @@ -0,0 +1,53 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.atomic.* +import kotlin.test.* + +/** + * Stress test for [runInterruptible]. + * It does not pass on JDK 1.6 on Windows: [Thread.sleep] times out without being interrupted despite the + * fact that thread interruption flag is set. + */ +class RunInterruptibleStressTest : TestBase() { + @get:Rule + val dispatcher = ExecutorRule(4) + private val repeatTimes = 1000 * stressTestMultiplier + + @Test + fun testStress() = runTest { + val enterCount = AtomicInteger(0) + val interruptedCount = AtomicInteger(0) + + repeat(repeatTimes) { + val job = launch(dispatcher) { + try { + runInterruptible { + enterCount.incrementAndGet() + try { + Thread.sleep(10_000) + error("Sleep was not interrupted, Thread.isInterrupted=${Thread.currentThread().isInterrupted}") + } catch (e: InterruptedException) { + interruptedCount.incrementAndGet() + throw e + } + } + } catch (e: CancellationException) { + // Expected + } finally { + assertFalse(Thread.currentThread().isInterrupted, "Interrupt flag should not leak") + } + } + // Add dispatch delay + val cancelJob = launch(dispatcher) { + job.cancel() + } + joinAll(job, cancelJob) + } + println("Entered runInterruptible ${enterCount.get()} times") + assertTrue(enterCount.get() > 0) // ensure timing is Ok and we don't cancel it all prematurely + assertEquals(enterCount.get(), interruptedCount.get()) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/RunInterruptibleTest.kt b/kotlinx-coroutines-core/jvm/test/RunInterruptibleTest.kt new file mode 100644 index 0000000000..bf194c0d0d --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/RunInterruptibleTest.kt @@ -0,0 +1,60 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import java.io.* +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import kotlin.test.* + +class RunInterruptibleTest : TestBase() { + + @Test + fun testNormalRun() = runTest { + val result = runInterruptible { + val x = 1 + val y = 2 + Thread.sleep(1) + x + y + } + assertEquals(3, result) + } + + @Test + fun testExceptionalRun() = runTest { + try { + runInterruptible { + expect(1) + throw TestException() + } + } catch (e: TestException) { + finish(2) + } + } + + @Test + fun testInterrupt() = runTest { + val latch = Channel(1) + val job = launch { + runInterruptible(Dispatchers.IO) { + expect(2) + latch.trySend(Unit) + try { + Thread.sleep(10_000L) + expectUnreached() + } catch (e: InterruptedException) { + expect(4) + assertFalse { Thread.currentThread().isInterrupted } + } + } + } + + launch(start = CoroutineStart.UNDISPATCHED) { + expect(1) + latch.receive() + expect(3) + job.cancelAndJoin() + }.join() + finish(5) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/TestBaseTest.kt b/kotlinx-coroutines-core/jvm/test/TestBaseTest.kt new file mode 100644 index 0000000000..49b26ab794 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/TestBaseTest.kt @@ -0,0 +1,24 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* + +class TestBaseTest : TestBase() { + @Test + fun testThreadsShutdown() { + repeat(1000 * stressTestMultiplier) { _ -> + initPoolsBeforeTest() + val threadsBefore = currentThreads() + runBlocking { + val sub = launch { + delay(10000000L) + } + sub.cancel() + sub.join() + } + shutdownPoolsAfterTest() + checkTestThreads(threadsBefore) + } + + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/ThreadContextElementRestoreTest.kt b/kotlinx-coroutines-core/jvm/test/ThreadContextElementRestoreTest.kt new file mode 100644 index 0000000000..cd3419686c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ThreadContextElementRestoreTest.kt @@ -0,0 +1,195 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import kotlin.coroutines.* +import kotlin.test.* + +class ThreadContextElementRestoreTest : TestBase() { + private val tl = ThreadLocal() + + // Checks that ThreadLocal context is properly restored after executing the given block inside + // withContext(tl.asContextElement("OK")) code running in different outer contexts + private inline fun check(crossinline block: suspend () -> Unit) = runTest { + val mainDispatcher = coroutineContext[ContinuationInterceptor] as CoroutineDispatcher + // Scenario #1: withContext(ThreadLocal) direct from runTest + withContext(tl.asContextElement("OK")) { + block() + assertEquals("OK", tl.get()) + } + assertEquals(null, tl.get()) + // Scenario #2: withContext(ThreadLocal) from coroutineScope + coroutineScope { + withContext(tl.asContextElement("OK")) { + block() + assertEquals("OK", tl.get()) + } + assertEquals(null, tl.get()) + } + // Scenario #3: withContext(ThreadLocal) from undispatched withContext + withContext(CoroutineName("NAME")) { + withContext(tl.asContextElement("OK")) { + block() + assertEquals("OK", tl.get()) + } + assertEquals(null, tl.get()) + } + // Scenario #4: withContext(ThreadLocal) from dispatched withContext + withContext(wrapperDispatcher()) { + withContext(tl.asContextElement("OK")) { + block() + assertEquals("OK", tl.get()) + } + assertEquals(null, tl.get()) + } + // Scenario #5: withContext(ThreadLocal) from withContext(ThreadLocal) + withContext(tl.asContextElement(null)) { + withContext(tl.asContextElement("OK")) { + block() + assertEquals("OK", tl.get()) + } + assertEquals(null, tl.get()) + } + // Scenario #6: withContext(ThreadLocal) from withTimeout + withTimeout(1000) { + withContext(tl.asContextElement("OK")) { + block() + assertEquals("OK", tl.get()) + } + assertEquals(null, tl.get()) + } + // Scenario #7: withContext(ThreadLocal) from withContext(Unconfined) + withContext(Dispatchers.Unconfined) { + withContext(tl.asContextElement("OK")) { + block() + assertEquals("OK", tl.get()) + } + assertEquals(null, tl.get()) + } + // Scenario #8: withContext(ThreadLocal) from withContext(Default) + withContext(Dispatchers.Default) { + withContext(tl.asContextElement("OK")) { + block() + assertEquals("OK", tl.get()) + } + assertEquals(null, tl.get()) + } + // Scenario #9: withContext(ThreadLocal) from withContext(mainDispatcher) + withContext(mainDispatcher) { + withContext(tl.asContextElement("OK")) { + block() + assertEquals("OK", tl.get()) + } + assertEquals(null, tl.get()) + } + } + + @Test + fun testSimpleNoSuspend() = + check {} + + @Test + fun testSimpleDelay() = check { + delay(1) + } + + @Test + fun testSimpleYield() = check { + yield() + } + + private suspend fun deepDelay() { + deepDelay2(); deepDelay2() + } + + private suspend fun deepDelay2() { + delay(1); delay(1) + } + + @Test + fun testDeepDelay() = check { + deepDelay() + } + + private suspend fun deepYield() { + deepYield2(); deepYield2() + } + + private suspend fun deepYield2() { + yield(); yield() + } + + @Test + fun testDeepYield() = check { + deepYield() + } + + @Test + fun testCoroutineScopeDelay() = check { + coroutineScope { + delay(1) + } + } + + @Test + fun testCoroutineScopeYield() = check { + coroutineScope { + yield() + } + } + + @Test + fun testWithContextUndispatchedDelay() = check { + withContext(CoroutineName("INNER")) { + delay(1) + } + } + + @Test + fun testWithContextUndispatchedYield() = check { + withContext(CoroutineName("INNER")) { + yield() + } + } + + @Test + fun testWithContextDispatchedDelay() = check { + withContext(wrapperDispatcher()) { + delay(1) + } + } + + @Test + fun testWithContextDispatchedYield() = check { + withContext(wrapperDispatcher()) { + yield() + } + } + + @Test + fun testWithTimeoutDelay() = check { + withTimeout(1000) { + delay(1) + } + } + + @Test + fun testWithTimeoutYield() = check { + withTimeout(1000) { + yield() + } + } + + @Test + fun testWithUnconfinedContextDelay() = check { + withContext(Dispatchers.Unconfined) { + delay(1) + } + } + @Test + fun testWithUnconfinedContextYield() = check { + withContext(Dispatchers.Unconfined) { + yield() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ThreadContextElementTest.kt b/kotlinx-coroutines-core/jvm/test/ThreadContextElementTest.kt new file mode 100644 index 0000000000..54e88677e1 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ThreadContextElementTest.kt @@ -0,0 +1,341 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.testing.* +import org.junit.Test +import java.util.concurrent.CopyOnWriteArrayList +import java.util.concurrent.ExecutorService +import java.util.concurrent.Executors +import kotlin.coroutines.* +import kotlin.test.* + +class ThreadContextElementTest : TestBase() { + + @Test + fun testExample() = runTest { + val exceptionHandler = coroutineContext[CoroutineExceptionHandler]!! + val mainDispatcher = coroutineContext[ContinuationInterceptor]!! + val mainThread = Thread.currentThread() + val data = MyData() + val element = MyElement(data) + assertNull(myThreadLocal.get()) + val job = GlobalScope.launch(element + exceptionHandler) { + assertTrue(mainThread != Thread.currentThread()) + assertSame(element, coroutineContext[MyElement]) + assertSame(data, myThreadLocal.get()) + withContext(mainDispatcher) { + assertSame(mainThread, Thread.currentThread()) + assertSame(element, coroutineContext[MyElement]) + assertSame(data, myThreadLocal.get()) + } + assertTrue(mainThread != Thread.currentThread()) + assertSame(element, coroutineContext[MyElement]) + assertSame(data, myThreadLocal.get()) + } + assertNull(myThreadLocal.get()) + job.join() + assertNull(myThreadLocal.get()) + } + + @Test + fun testUndispatched() = runTest { + val exceptionHandler = coroutineContext[CoroutineExceptionHandler]!! + val data = MyData() + val element = MyElement(data) + val job = GlobalScope.launch( + context = Dispatchers.Default + exceptionHandler + element, + start = CoroutineStart.UNDISPATCHED + ) { + assertSame(data, myThreadLocal.get()) + yield() + assertSame(data, myThreadLocal.get()) + } + assertNull(myThreadLocal.get()) + job.join() + assertNull(myThreadLocal.get()) + } + + @Test + fun testWithContext() = runTest { + expect(1) + newSingleThreadContext("withContext").use { + val data = MyData() + GlobalScope.async(Dispatchers.Default + MyElement(data)) { + assertSame(data, myThreadLocal.get()) + expect(2) + + val newData = MyData() + GlobalScope.async(it + MyElement(newData)) { + assertSame(newData, myThreadLocal.get()) + expect(3) + }.await() + + withContext(it + MyElement(newData)) { + assertSame(newData, myThreadLocal.get()) + expect(4) + } + + GlobalScope.async(it) { + assertNull(myThreadLocal.get()) + expect(5) + }.await() + + expect(6) + }.await() + } + + finish(7) + } + + @Test + fun testNonCopyableElementReferenceInheritedOnLaunch() = runTest { + var parentElement: MyElement? = null + var inheritedElement: MyElement? = null + + newSingleThreadContext("withContext").use { + withContext(it + MyElement(MyData())) { + parentElement = coroutineContext[MyElement.Key] + launch { + inheritedElement = coroutineContext[MyElement.Key] + } + } + } + + assertSame(inheritedElement, parentElement, + "Inner and outer coroutines did not have the same object reference to a" + + " ThreadContextElement that did not override `copyForChildCoroutine()`") + } + + @Test + fun testCopyableElementCopiedOnLaunch() = runTest { + var parentElement: CopyForChildCoroutineElement? = null + var inheritedElement: CopyForChildCoroutineElement? = null + + newSingleThreadContext("withContext").use { + withContext(it + CopyForChildCoroutineElement(MyData())) { + parentElement = coroutineContext[CopyForChildCoroutineElement.Key] + launch { + inheritedElement = coroutineContext[CopyForChildCoroutineElement.Key] + } + } + } + + assertNotSame(inheritedElement, parentElement, + "Inner coroutine did not copy its copyable ThreadContextElement.") + } + + @Test + fun testCopyableThreadContextElementImplementsWriteVisibility() = runTest { + newFixedThreadPoolContext(nThreads = 4, name = "withContext").use { + withContext(it + CopyForChildCoroutineElement(MyData())) { + val forBlockData = MyData() + myThreadLocal.setForBlock(forBlockData) { + assertSame(myThreadLocal.get(), forBlockData) + launch { + assertSame(myThreadLocal.get(), forBlockData) + } + launch { + assertSame(myThreadLocal.get(), forBlockData) + // Modify value in child coroutine. Writes to the ThreadLocal and + // the (copied) ThreadLocalElement's memory are not visible to peer or + // ancestor coroutines, so this write is both threadsafe and coroutinesafe. + val innerCoroutineData = MyData() + myThreadLocal.setForBlock(innerCoroutineData) { + assertSame(myThreadLocal.get(), innerCoroutineData) + } + assertSame(myThreadLocal.get(), forBlockData) // Asserts value was restored. + } + launch { + val innerCoroutineData = MyData() + myThreadLocal.setForBlock(innerCoroutineData) { + assertSame(myThreadLocal.get(), innerCoroutineData) + } + assertSame(myThreadLocal.get(), forBlockData) + } + } + assertNull(myThreadLocal.get()) // Asserts value was restored to its origin + } + } + } + + class JobCaptor(val capturees: MutableList = CopyOnWriteArrayList()) : ThreadContextElement { + + companion object Key : CoroutineContext.Key + + override val key: CoroutineContext.Key<*> get() = Key + + override fun updateThreadContext(context: CoroutineContext) { + capturees.add("Update: ${context.job}") + } + + override fun restoreThreadContext(context: CoroutineContext, oldState: Unit) { + capturees.add("Restore: ${context.job}") + } + } + + /** + * For stability of the test, it is important to make sure that + * the parent job actually suspends when calling + * `withContext(dispatcher2 + CoroutineName("dispatched"))`. + * + * Here this requirement is fulfilled by forcing execution on a single thread. + * However, dispatching is performed with two non-equal dispatchers to force dispatching. + * + * Suspend of the parent coroutine [kotlinx.coroutines.DispatchedCoroutine.trySuspend] is out of the control of the test, + * while being executed concurrently with resume of the child coroutine [kotlinx.coroutines.DispatchedCoroutine.tryResume]. + */ + @Test + fun testWithContextJobAccess() = runTest { + val executor = Executors.newSingleThreadExecutor() + // Emulate non-equal dispatchers + val executor1 = object : ExecutorService by executor {} + val executor2 = object : ExecutorService by executor {} + val dispatcher1 = executor1.asCoroutineDispatcher() + val dispatcher2 = executor2.asCoroutineDispatcher() + val captor = JobCaptor() + val manuallyCaptured = mutableListOf() + + fun registerUpdate(job: Job?) = manuallyCaptured.add("Update: $job") + fun registerRestore(job: Job?) = manuallyCaptured.add("Restore: $job") + + var rootJob: Job? = null + runBlocking(captor + dispatcher1) { + rootJob = coroutineContext.job + registerUpdate(rootJob) + var undispatchedJob: Job? = null + withContext(CoroutineName("undispatched")) { + undispatchedJob = coroutineContext.job + registerUpdate(undispatchedJob) + // These 2 restores and the corresponding next 2 updates happen only if the following `withContext` + // call actually suspends. + registerRestore(undispatchedJob) + registerRestore(rootJob) + // Without forcing of single backing thread the code inside `withContext` + // may already complete at the moment when the parent coroutine decides + // whether it needs to suspend or not. + var dispatchedJob: Job? = null + withContext(dispatcher2 + CoroutineName("dispatched")) { + dispatchedJob = coroutineContext.job + registerUpdate(dispatchedJob) + } + registerRestore(dispatchedJob) + // Context restored, captured again + registerUpdate(undispatchedJob) + } + registerRestore(undispatchedJob) + // Context restored, captured again + registerUpdate(rootJob) + } + registerRestore(rootJob) + + // Restores may be called concurrently to the update calls in other threads, so their order is not checked. + val expected = manuallyCaptured.filter { it.startsWith("Update: ") }.joinToString(separator = "\n") + val actual = captor.capturees.filter { it.startsWith("Update: ") }.joinToString(separator = "\n") + assertEquals(expected, actual) + executor.shutdownNow() + } + + @Test + fun testThreadLocalFlowOn() = runTest { + val myData = MyData() + myThreadLocal.set(myData) + expect(1) + flow { + assertEquals(myData, myThreadLocal.get()) + emit(1) + } + .flowOn(myThreadLocal.asContextElement() + Dispatchers.Default) + .single() + myThreadLocal.set(null) + finish(2) + } +} + +class MyData + +// declare thread local variable holding MyData +private val myThreadLocal = ThreadLocal() + +// declare context element holding MyData +class MyElement(val data: MyData) : ThreadContextElement { + // declare companion object for a key of this element in coroutine context + companion object Key : CoroutineContext.Key + + // provide the key of the corresponding context element + override val key: CoroutineContext.Key + get() = Key + + // this is invoked before coroutine is resumed on current thread + override fun updateThreadContext(context: CoroutineContext): MyData? { + val oldState = myThreadLocal.get() + myThreadLocal.set(data) + return oldState + } + + // this is invoked after coroutine has suspended on current thread + override fun restoreThreadContext(context: CoroutineContext, oldState: MyData?) { + myThreadLocal.set(oldState) + } +} + +/** + * A [ThreadContextElement] that implements copy semantics in [copyForChild]. + */ +class CopyForChildCoroutineElement(val data: MyData?) : CopyableThreadContextElement { + companion object Key : CoroutineContext.Key + + override val key: CoroutineContext.Key + get() = Key + + override fun updateThreadContext(context: CoroutineContext): MyData? { + val oldState = myThreadLocal.get() + myThreadLocal.set(data) + return oldState + } + + override fun mergeForChild(overwritingElement: CoroutineContext.Element): CopyForChildCoroutineElement { + TODO("Not used in tests") + } + + override fun restoreThreadContext(context: CoroutineContext, oldState: MyData?) { + myThreadLocal.set(oldState) + } + + /** + * At coroutine launch time, the _current value of the ThreadLocal_ is inherited by the new + * child coroutine, and that value is copied to a new, unique, ThreadContextElement memory + * reference for the child coroutine to use uniquely. + * + * n.b. the value copied to the child must be the __current value of the ThreadLocal__ and not + * the value initially passed to the ThreadContextElement in order to reflect writes made to the + * ThreadLocal between coroutine resumption and the child coroutine launch point. Those writes + * will be reflected in the parent coroutine's [CopyForChildCoroutineElement] when it yields the + * thread and calls [restoreThreadContext]. + */ + override fun copyForChild(): CopyForChildCoroutineElement { + return CopyForChildCoroutineElement(myThreadLocal.get()) + } +} + + +/** + * Calls [block], setting the value of [this] [ThreadLocal] for the duration of [block]. + * + * When a [CopyForChildCoroutineElement] for `this` [ThreadLocal] is used within a + * [CoroutineContext], a ThreadLocal set this way will have the "correct" value expected lexically + * at every statement reached, whether that statement is reached immediately, across suspend and + * redispatch within one coroutine, or within a child coroutine. Writes made to the `ThreadLocal` + * by child coroutines will not be visible to the parent coroutine. Writes made to the `ThreadLocal` + * by the parent coroutine _after_ launching a child coroutine will not be visible to that child + * coroutine. + */ +private inline fun ThreadLocal.setForBlock( + value: ThreadLocalT, + crossinline block: () -> OutputT +) { + val priorValue = get() + set(value) + block() + set(priorValue) +} + diff --git a/kotlinx-coroutines-core/jvm/test/ThreadContextMutableCopiesTest.kt b/kotlinx-coroutines-core/jvm/test/ThreadContextMutableCopiesTest.kt new file mode 100644 index 0000000000..0b174ec539 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ThreadContextMutableCopiesTest.kt @@ -0,0 +1,160 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.flow.* +import kotlin.coroutines.* +import kotlin.test.* + +class ThreadContextMutableCopiesTest : TestBase() { + companion object { + val threadLocalData: ThreadLocal> = ThreadLocal.withInitial { ArrayList() } + } + + class MyMutableElement( + val mutableData: MutableList + ) : CopyableThreadContextElement> { + + companion object Key : CoroutineContext.Key + + override val key: CoroutineContext.Key<*> + get() = Key + + override fun updateThreadContext(context: CoroutineContext): MutableList { + val st = threadLocalData.get() + threadLocalData.set(mutableData) + return st + } + + override fun restoreThreadContext(context: CoroutineContext, oldState: MutableList) { + threadLocalData.set(oldState) + } + + override fun copyForChild(): MyMutableElement { + return MyMutableElement(ArrayList(mutableData)) + } + + override fun mergeForChild(overwritingElement: CoroutineContext.Element): MyMutableElement { + overwritingElement as MyMutableElement // <- app-specific, may be another subtype + return MyMutableElement((mutableData.toSet() + overwritingElement.mutableData).toMutableList()) + } + } + + @Test + fun testDataIsCopied() = runTest { + val root = MyMutableElement(ArrayList()) + runBlocking(root) { + val data = threadLocalData.get() + expect(1) + launch(root) { + assertNotSame(data, threadLocalData.get()) + assertEquals(data, threadLocalData.get()) + finish(2) + } + } + } + + @Test + fun testDataIsNotOverwritten() = runTest { + val root = MyMutableElement(ArrayList()) + runBlocking(root) { + expect(1) + val originalData = threadLocalData.get() + threadLocalData.get().add("X") + launch { + threadLocalData.get().add("Y") + // Note here, +root overwrites the data + launch(Dispatchers.Default + root) { + assertEquals(listOf("X", "Y"), threadLocalData.get()) + assertNotSame(originalData, threadLocalData.get()) + finish(2) + } + } + } + } + + @Test + fun testDataIsMerged() = runTest { + val root = MyMutableElement(ArrayList()) + runBlocking(root) { + expect(1) + val originalData = threadLocalData.get() + threadLocalData.get().add("X") + launch { + threadLocalData.get().add("Y") + // Note here, +root overwrites the data + launch(Dispatchers.Default + MyMutableElement(mutableListOf("Z"))) { + assertEquals(listOf("X", "Y", "Z"), threadLocalData.get()) + assertNotSame(originalData, threadLocalData.get()) + finish(2) + } + } + } + } + + @Test + fun testDataIsNotOverwrittenWithContext() = runTest { + val root = MyMutableElement(ArrayList()) + runBlocking(root) { + val originalData = threadLocalData.get() + threadLocalData.get().add("X") + expect(1) + launch { + threadLocalData.get().add("Y") + // Note here, +root overwrites the data + withContext(Dispatchers.Default + root) { + assertEquals(listOf("X", "Y"), threadLocalData.get()) + assertNotSame(originalData, threadLocalData.get()) + finish(2) + } + } + } + } + + @Test + fun testDataIsCopiedForRunBlocking() = runTest { + val root = MyMutableElement(ArrayList()) + val originalData = root.mutableData + runBlocking(root) { + assertNotSame(originalData, threadLocalData.get()) + } + } + + @Test + fun testDataIsCopiedForCoroutine() = runTest { + val root = MyMutableElement(ArrayList()) + val originalData = root.mutableData + expect(1) + launch(root) { + assertNotSame(originalData, threadLocalData.get()) + finish(2) + } + } + + @Test + fun testDataIsCopiedThroughFlowOnUndispatched() = runTest { + expect(1) + val root = MyMutableElement(ArrayList()) + val originalData = root.mutableData + flow { + assertNotSame(originalData, threadLocalData.get()) + emit(1) + } + .flowOn(root) + .single() + finish(2) + } + + @Test + fun testDataIsCopiedThroughFlowOnDispatched() = runTest { + expect(1) + val root = MyMutableElement(ArrayList()) + val originalData = root.mutableData + flow { + assertNotSame(originalData, threadLocalData.get()) + emit(1) + } + .flowOn(root + Dispatchers.Default) + .single() + finish(2) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ThreadContextOrderTest.kt b/kotlinx-coroutines-core/jvm/test/ThreadContextOrderTest.kt new file mode 100644 index 0000000000..877a4ca2d6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ThreadContextOrderTest.kt @@ -0,0 +1,62 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.internal.* +import org.junit.Test +import kotlin.coroutines.* +import kotlin.test.* + +class ThreadContextOrderTest : TestBase() { + /* + * The test verifies that two thread context elements are correctly nested: + * The restoration order is the reverse of update order. + */ + private val transactionalContext = ThreadLocal() + private val loggingContext = ThreadLocal() + + private val transactionalElement = object : ThreadContextElement { + override val key = ThreadLocalKey(transactionalContext) + + override fun updateThreadContext(context: CoroutineContext): String { + assertEquals("test", loggingContext.get()) + val previous = transactionalContext.get() + transactionalContext.set("tr coroutine") + return previous + } + + override fun restoreThreadContext(context: CoroutineContext, oldState: String) { + assertEquals("test", loggingContext.get()) + assertEquals("tr coroutine", transactionalContext.get()) + transactionalContext.set(oldState) + } + } + + private val loggingElement = object : ThreadContextElement { + override val key = ThreadLocalKey(loggingContext) + + override fun updateThreadContext(context: CoroutineContext): String { + val previous = loggingContext.get() + loggingContext.set("log coroutine") + return previous + } + + override fun restoreThreadContext(context: CoroutineContext, oldState: String) { + assertEquals("log coroutine", loggingContext.get()) + assertEquals("tr coroutine", transactionalContext.get()) + loggingContext.set(oldState) + } + } + + @Test + fun testCorrectOrder() = runTest { + transactionalContext.set("test") + loggingContext.set("test") + launch(transactionalElement + loggingElement) { + assertEquals("log coroutine", loggingContext.get()) + assertEquals("tr coroutine", transactionalContext.get()) + } + assertEquals("test", loggingContext.get()) + assertEquals("test", transactionalContext.get()) + + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ThreadLocalStressTest.kt b/kotlinx-coroutines-core/jvm/test/ThreadLocalStressTest.kt new file mode 100644 index 0000000000..63ed3f2300 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ThreadLocalStressTest.kt @@ -0,0 +1,162 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.sync.* +import java.util.concurrent.* +import kotlin.coroutines.* +import kotlin.coroutines.intrinsics.* +import kotlin.test.* + + +class ThreadLocalStressTest : TestBase() { + + private val threadLocal = ThreadLocal() + + // See the comment in doStress for the machinery + @Test + fun testStress() = runTest { + repeat (100 * stressTestMultiplierSqrt) { + withContext(Dispatchers.Default) { + repeat(100) { + launch { + doStress(null) + } + } + } + } + } + + @Test + fun testStressWithOuterValue() = runTest { + repeat (100 * stressTestMultiplierSqrt) { + withContext(Dispatchers.Default + threadLocal.asContextElement("bar")) { + repeat(100) { + launch { + doStress("bar") + } + } + } + } + } + + private suspend fun doStress(expectedValue: String?) { + assertEquals(expectedValue, threadLocal.get()) + try { + /* + * Here we are using very specific code-path to trigger the execution we want to. + * The bug, in general, has a larger impact, but this particular code pinpoints it: + * + * 1) We use _undispatched_ withContext with thread element + * 2) We cancel the coroutine + * 3) We use 'suspendCancellableCoroutineReusable' that does _postponed_ cancellation check + * which makes the reproduction of this race pretty reliable. + * + * Now the following code path is likely to be triggered: + * + * T1 from within 'withContinuationContext' method: + * Finds 'oldValue', finds undispatched completion, invokes its 'block' argument. + * 'block' is this coroutine, it goes to 'trySuspend', checks for postponed cancellation and *dispatches* it. + * The execution stops _right_ before 'undispatchedCompletion.clearThreadContext()'. + * + * T2 now executes the dispatched cancellation and concurrently mutates the state of the undispatched completion. + * All bets are off, now both threads can leave the thread locals state inconsistent. + */ + withContext(threadLocal.asContextElement("foo")) { + yield() + cancel() + suspendCancellableCoroutineReusable { } + } + } finally { + assertEquals(expectedValue, threadLocal.get()) + } + } + + /* + * Another set of tests for undispatcheable continuations that do not require stress test multiplier. + * Also note that `uncaughtExceptionHandler` is used as the only available mechanism to propagate error from + * `resumeWith` + */ + + @Test + fun testNonDispatcheableLeak() { + repeat(100) { + doTestWithPreparation( + ::doTest, + { threadLocal.set(null) }) { threadLocal.get() == null } + assertNull(threadLocal.get()) + } + } + + @Test + fun testNonDispatcheableLeakWithInitial() { + repeat(100) { + doTestWithPreparation(::doTest, { threadLocal.set("initial") }) { threadLocal.get() == "initial" } + assertEquals("initial", threadLocal.get()) + } + } + + @Test + fun testNonDispatcheableLeakWithContextSwitch() { + repeat(100) { + doTestWithPreparation( + ::doTestWithContextSwitch, + { threadLocal.set(null) }) { threadLocal.get() == null } + assertNull(threadLocal.get()) + } + } + + @Test + fun testNonDispatcheableLeakWithInitialWithContextSwitch() { + repeat(100) { + doTestWithPreparation( + ::doTestWithContextSwitch, + { threadLocal.set("initial") }) { true /* can randomly wake up on the non-main thread */ } + // Here we are always on the main thread + assertEquals("initial", threadLocal.get()) + } + } + + private fun doTestWithPreparation(testBody: suspend () -> Unit, setup: () -> Unit, isValid: () -> Boolean) { + setup() + val latch = CountDownLatch(1) + testBody.startCoroutineUninterceptedOrReturn(Continuation(EmptyCoroutineContext) { + if (!isValid()) { + Thread.currentThread().uncaughtExceptionHandler.uncaughtException( + Thread.currentThread(), + IllegalStateException("Unexpected error: thread local was not cleaned") + ) + } + latch.countDown() + }) + latch.await() + } + + private suspend fun doTest() { + withContext(threadLocal.asContextElement("foo")) { + try { + coroutineScope { + val semaphore = Semaphore(1, 1) + cancel() + semaphore.acquire() + } + } catch (e: CancellationException) { + // Ignore cancellation + } + } + } + + private suspend fun doTestWithContextSwitch() { + withContext(threadLocal.asContextElement("foo")) { + try { + coroutineScope { + val semaphore = Semaphore(1, 1) + GlobalScope.launch { }.join() + cancel() + semaphore.acquire() + } + } catch (e: CancellationException) { + // Ignore cancellation + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ThreadLocalTest.kt b/kotlinx-coroutines-core/jvm/test/ThreadLocalTest.kt new file mode 100644 index 0000000000..79a2490fc5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ThreadLocalTest.kt @@ -0,0 +1,230 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import org.junit.Test +import java.lang.IllegalStateException +import kotlin.test.* + +@Suppress("RedundantAsync") +class ThreadLocalTest : TestBase() { + private val stringThreadLocal = ThreadLocal() + private val intThreadLocal = ThreadLocal() + private val executor = newFixedThreadPoolContext(1, "threadLocalTest") + + @After + fun tearDown() { + executor.close() + } + + @Test + fun testThreadLocal() = runTest { + assertNull(stringThreadLocal.get()) + assertFalse(stringThreadLocal.isPresent()) + val deferred = async(Dispatchers.Default + stringThreadLocal.asContextElement("value")) { + assertEquals("value", stringThreadLocal.get()) + assertTrue(stringThreadLocal.isPresent()) + withContext(executor) { + assertTrue(stringThreadLocal.isPresent()) + assertFailsWith { intThreadLocal.ensurePresent() } + assertEquals("value", stringThreadLocal.get()) + } + assertTrue(stringThreadLocal.isPresent()) + assertEquals("value", stringThreadLocal.get()) + } + + assertNull(stringThreadLocal.get()) + deferred.await() + assertNull(stringThreadLocal.get()) + assertFalse(stringThreadLocal.isPresent()) + } + + @Test + fun testThreadLocalInitialValue() = runTest { + intThreadLocal.set(42) + assertFalse(intThreadLocal.isPresent()) + val deferred = async(Dispatchers.Default + intThreadLocal.asContextElement(239)) { + assertEquals(239, intThreadLocal.get()) + withContext(executor) { + intThreadLocal.ensurePresent() + assertEquals(239, intThreadLocal.get()) + } + assertEquals(239, intThreadLocal.get()) + } + + deferred.await() + assertEquals(42, intThreadLocal.get()) + } + + @Test + fun testMultipleThreadLocals() = runTest { + stringThreadLocal.set("test") + intThreadLocal.set(314) + + val deferred = async(Dispatchers.Default + + intThreadLocal.asContextElement(value = 239) + stringThreadLocal.asContextElement(value = "pew")) { + assertEquals(239, intThreadLocal.get()) + assertEquals("pew", stringThreadLocal.get()) + + withContext(executor) { + assertEquals(239, intThreadLocal.get()) + assertEquals("pew", stringThreadLocal.get()) + intThreadLocal.ensurePresent() + stringThreadLocal.ensurePresent() + } + + assertEquals(239, intThreadLocal.get()) + assertEquals("pew", stringThreadLocal.get()) + } + + deferred.await() + assertEquals(314, intThreadLocal.get()) + assertEquals("test", stringThreadLocal.get()) + } + + @Test + fun testConflictingThreadLocals() = runTest { + intThreadLocal.set(42) + + val deferred = GlobalScope.async(intThreadLocal.asContextElement(1)) { + assertEquals(1, intThreadLocal.get()) + + withContext(executor + intThreadLocal.asContextElement(42)) { + assertEquals(42, intThreadLocal.get()) + } + + assertEquals(1, intThreadLocal.get()) + + val deferred = async(intThreadLocal.asContextElement(53)) { + assertEquals(53, intThreadLocal.get()) + } + + deferred.await() + assertEquals(1, intThreadLocal.get()) + + val deferred2 = GlobalScope.async(executor) { + assertNull(intThreadLocal.get()) + } + + deferred2.await() + assertEquals(1, intThreadLocal.get()) + } + + deferred.await() + assertEquals(42, intThreadLocal.get()) + } + + @Test + fun testThreadLocalModification() = runTest { + stringThreadLocal.set("main") + + val deferred = async(Dispatchers.Default + + stringThreadLocal.asContextElement("initial")) { + assertEquals("initial", stringThreadLocal.get()) + + stringThreadLocal.set("overridden") // <- this value is not reflected in the context, so it's not restored + + withContext(executor + stringThreadLocal.asContextElement("ctx")) { + assertEquals("ctx", stringThreadLocal.get()) + } + + val deferred = async(stringThreadLocal.asContextElement("async")) { + assertEquals("async", stringThreadLocal.get()) + } + + deferred.await() + assertEquals("initial", stringThreadLocal.get()) // <- not restored + } + + deferred.await() + assertFalse(stringThreadLocal.isPresent()) + assertEquals("main", stringThreadLocal.get()) + } + + + + private data class Counter(var cnt: Int) + private val myCounterLocal = ThreadLocal() + + @Test + fun testThreadLocalModificationMutableBox() = runTest { + myCounterLocal.set(Counter(42)) + + val deferred = async(Dispatchers.Default + + myCounterLocal.asContextElement(Counter(0))) { + assertEquals(0, myCounterLocal.get().cnt) + + // Mutate + myCounterLocal.get().cnt = 71 + + withContext(executor + myCounterLocal.asContextElement(Counter(-1))) { + assertEquals(-1, myCounterLocal.get().cnt) + ++myCounterLocal.get().cnt + } + + val deferred = async(myCounterLocal.asContextElement(Counter(31))) { + assertEquals(31, myCounterLocal.get().cnt) + ++myCounterLocal.get().cnt + } + + deferred.await() + assertEquals(71, myCounterLocal.get().cnt) + } + + deferred.await() + assertEquals(42, myCounterLocal.get().cnt) + } + + @Test + fun testWithContext() = runTest { + expect(1) + newSingleThreadContext("withContext").use { + val data = 42 + GlobalScope.async(Dispatchers.Default + intThreadLocal.asContextElement(42)) { + + assertEquals(data, intThreadLocal.get()) + expect(2) + + GlobalScope.async(it + intThreadLocal.asContextElement(31)) { + assertEquals(31, intThreadLocal.get()) + expect(3) + }.await() + + withContext(it + intThreadLocal.asContextElement(2)) { + assertEquals(2, intThreadLocal.get()) + expect(4) + } + + GlobalScope.async(it) { + assertNull(intThreadLocal.get()) + expect(5) + }.await() + + expect(6) + }.await() + } + + finish(7) + } + + @Test + fun testScope() = runTest { + intThreadLocal.set(42) + val mainThread = Thread.currentThread() + GlobalScope.async { + assertNull(intThreadLocal.get()) + assertNotSame(mainThread, Thread.currentThread()) + }.await() + + GlobalScope.async(intThreadLocal.asContextElement()) { + assertEquals(42, intThreadLocal.get()) + assertNotSame(mainThread, Thread.currentThread()) + }.await() + } + + @Test + fun testMissingThreadLocal() = runTest { + assertFailsWith { stringThreadLocal.ensurePresent() } + assertFailsWith { intThreadLocal.ensurePresent() } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/ThreadLocalsLeaksTest.kt b/kotlinx-coroutines-core/jvm/test/ThreadLocalsLeaksTest.kt new file mode 100644 index 0000000000..f7a015a13e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/ThreadLocalsLeaksTest.kt @@ -0,0 +1,106 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.TestBase +import kotlinx.coroutines.testing.isJavaAndWindows +import java.lang.ref.WeakReference +import kotlin.coroutines.AbstractCoroutineContextElement +import kotlin.coroutines.Continuation +import kotlin.coroutines.ContinuationInterceptor +import kotlin.coroutines.CoroutineContext +import kotlin.test.* + +/* + * This is an adapted verion of test from #4296. + * + * qwwdfsad: the test relies on System.gc() actually collecting the garbage. + * If these tests flake on CI, first check that JDK/GC setup in not an issue. + */ +@Ignore +class ThreadLocalCustomContinuationInterceptorTest : TestBase() { + + private class CustomContinuationInterceptor(private val delegate: ContinuationInterceptor) : + AbstractCoroutineContextElement(ContinuationInterceptor), ContinuationInterceptor { + + override fun interceptContinuation(continuation: Continuation): Continuation { + return delegate.interceptContinuation(continuation) + } + } + + private class CustomNeverEqualContinuationInterceptor(private val delegate: ContinuationInterceptor) : + AbstractCoroutineContextElement(ContinuationInterceptor), ContinuationInterceptor { + + override fun interceptContinuation(continuation: Continuation): Continuation { + return delegate.interceptContinuation(continuation) + } + + override fun equals(other: Any?) = false + } + + @Test(timeout = 20_000L) + fun testDefaultDispatcherNoSuspension() = ensureCoroutineContextGCed(Dispatchers.Default, suspend = false) + + @Test(timeout = 20_000L) + fun testDefaultDispatcher() = ensureCoroutineContextGCed(Dispatchers.Default, suspend = true) + + @Test(timeout = 20_000L) + fun testNonCoroutineDispatcher() = ensureCoroutineContextGCed( + CustomContinuationInterceptor(Dispatchers.Default), + suspend = true + ) + + @Test(timeout = 20_000L) + fun testNonCoroutineDispatcherSuspension() = ensureCoroutineContextGCed( + CustomContinuationInterceptor(Dispatchers.Default), + suspend = false + ) + + // Note asymmetric equals codepath never goes through the undispatched withContext, thus the separate test case + + @Test(timeout = 20_000L) + fun testNonCoroutineDispatcherAsymmetricEquals() = + ensureCoroutineContextGCed( + CustomNeverEqualContinuationInterceptor(Dispatchers.Default), + suspend = true + ) + + @Test(timeout = 20_000L) + fun testNonCoroutineDispatcherAsymmetricEqualsSuspension() = + ensureCoroutineContextGCed( + CustomNeverEqualContinuationInterceptor(Dispatchers.Default), + suspend = false + ) + + + @Volatile + private var letThatSinkIn: Any = "What is my purpose? To frag the garbage collctor" + + private fun ensureCoroutineContextGCed(coroutineContext: CoroutineContext, suspend: Boolean) { + // Tests are pretty timing-sensitive and flake ehavily on our virtualized Windows environment + if (isJavaAndWindows) { + return + } + + fun forceGcUntilRefIsCleaned(ref: WeakReference) { + while (ref.get() != null) { + System.gc() + letThatSinkIn = LongArray(1024 * 1024) + } + } + + runTest { + lateinit var ref: WeakReference + val job = GlobalScope.launch(coroutineContext) { + val coroutineName = CoroutineName("Yo") + ref = WeakReference(coroutineName) + withContext(coroutineName) { + if (suspend) { + delay(1) + } + } + } + job.join() + + forceGcUntilRefIsCleaned(ref) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/UnconfinedConcurrentStressTest.kt b/kotlinx-coroutines-core/jvm/test/UnconfinedConcurrentStressTest.kt new file mode 100644 index 0000000000..61aa7daacc --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/UnconfinedConcurrentStressTest.kt @@ -0,0 +1,47 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class UnconfinedConcurrentStressTest : TestBase() { + private val threads = 4 + private val executor = newFixedThreadPoolContext(threads, "UnconfinedConcurrentStressTest") + private val threadLocal = ThreadLocal() + + @After + fun tearDown() { + executor.close() + } + + @Test + fun testConcurrent() = runTest { + val iterations = 1_000 * stressTestMultiplier + val startBarrier = CyclicBarrier(threads + 1) + val finishLatch = CountDownLatch(threads) + + repeat(threads) { id -> + launch(executor) { + startBarrier.await() + repeat(iterations) { + threadLocal.set(0) + launch(Dispatchers.Unconfined) { + assertEquals(0, threadLocal.get()) + launch(Dispatchers.Unconfined) { + assertEquals(id, threadLocal.get()) + } + + threadLocal.set(id) + } + } + + finishLatch.countDown() + } + } + + startBarrier.await() + finishLatch.await() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/VirtualTimeSource.kt b/kotlinx-coroutines-core/jvm/test/VirtualTimeSource.kt new file mode 100644 index 0000000000..8a461087c3 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/VirtualTimeSource.kt @@ -0,0 +1,154 @@ +package kotlinx.coroutines + +import java.io.* +import java.util.concurrent.* +import java.util.concurrent.locks.* + +private const val SHUTDOWN_TIMEOUT = 1000L + +internal inline fun withVirtualTimeSource(log: PrintStream? = null, block: () -> Unit) { + DefaultExecutor.shutdownForTests(SHUTDOWN_TIMEOUT) // shutdown execution with old time source (in case it was working) + val testTimeSource = VirtualTimeSource(log) + mockTimeSource(testTimeSource) + DefaultExecutor.ensureStarted() // should start with new time source + try { + block() + } finally { + DefaultExecutor.shutdownForTests(SHUTDOWN_TIMEOUT) + testTimeSource.shutdown() + mockTimeSource(null) // restore time source + } +} + +private const val NOT_PARKED = -1L + +private class ThreadStatus { + @Volatile @JvmField + var parkedTill = NOT_PARKED + @Volatile @JvmField + var permit = false + var registered = 0 + override fun toString(): String = "parkedTill = ${TimeUnit.NANOSECONDS.toMillis(parkedTill)} ms, permit = $permit" +} + +private const val MAX_WAIT_NANOS = 10_000_000_000L // 10s +private const val REAL_TIME_STEP_NANOS = 200_000_000L // 200 ms +private const val REAL_PARK_NANOS = 10_000_000L // 10 ms -- park for a little to better track real-time + +@Suppress("PLATFORM_CLASS_MAPPED_TO_KOTLIN") +internal class VirtualTimeSource( + private val log: PrintStream? +) : AbstractTimeSource() { + private val mainThread: Thread = Thread.currentThread() + private var checkpointNanos: Long = System.nanoTime() + + @Volatile + private var isShutdown = false + + @Volatile + private var time: Long = 0 + + private var trackedTasks = 0 + + private val threads = ConcurrentHashMap() + + override fun currentTimeMillis(): Long = TimeUnit.NANOSECONDS.toMillis(time) + override fun nanoTime(): Long = time + + override fun wrapTask(block: Runnable): Runnable { + trackTask() + return Runnable { + try { block.run() } + finally { unTrackTask() } + } + } + + @Synchronized + override fun trackTask() { + trackedTasks++ + } + + @Synchronized + override fun unTrackTask() { + assert(trackedTasks > 0) + trackedTasks-- + } + + @Synchronized + override fun registerTimeLoopThread() { + val status = threads.getOrPut(Thread.currentThread()) { ThreadStatus() }!! + status.registered++ + } + + @Synchronized + override fun unregisterTimeLoopThread() { + val currentThread = Thread.currentThread() + val status = threads[currentThread]!! + if (--status.registered == 0) { + threads.remove(currentThread) + wakeupAll() + } + } + + override fun parkNanos(blocker: Any, nanos: Long) { + if (nanos <= 0) return + val status = threads[Thread.currentThread()]!! + assert(status.parkedTill == NOT_PARKED) + status.parkedTill = time + nanos.coerceAtMost(MAX_WAIT_NANOS) + while (true) { + checkAdvanceTime() + if (isShutdown || time >= status.parkedTill || status.permit) { + status.parkedTill = NOT_PARKED + status.permit = false + break + } + LockSupport.parkNanos(blocker, REAL_PARK_NANOS) + } + } + + override fun unpark(thread: Thread) { + val status = threads[thread] ?: return + status.permit = true + LockSupport.unpark(thread) + } + + @Synchronized + private fun checkAdvanceTime() { + if (isShutdown) return + val realNanos = System.nanoTime() + if (realNanos > checkpointNanos + REAL_TIME_STEP_NANOS) { + checkpointNanos = realNanos + val minParkedTill = minParkedTill() + time = (time + REAL_TIME_STEP_NANOS).coerceAtMost(if (minParkedTill < 0) Long.MAX_VALUE else minParkedTill) + logTime("R") + wakeupAll() + return + } + if (threads[mainThread] == null) return + if (trackedTasks != 0) return + val minParkedTill = minParkedTill() + if (minParkedTill <= time) return + time = minParkedTill + logTime("V") + wakeupAll() + } + + private fun logTime(s: String) { + log?.println("[$s: Time = ${TimeUnit.NANOSECONDS.toMillis(time)} ms]") + } + + private fun minParkedTill(): Long = + threads.values.map { if (it.permit) NOT_PARKED else it.parkedTill }.minOrNull() ?: NOT_PARKED + + @Synchronized + fun shutdown() { + isShutdown = true + wakeupAll() + while (!threads.isEmpty()) (this as Object).wait() + } + + private fun wakeupAll() { + threads.keys.forEach { LockSupport.unpark(it) } + (this as Object).notifyAll() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/WithDefaultContextTest.kt b/kotlinx-coroutines-core/jvm/test/WithDefaultContextTest.kt new file mode 100644 index 0000000000..44b31275e9 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/WithDefaultContextTest.kt @@ -0,0 +1,30 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class WithDefaultContextTest : TestBase() { + @Test + fun testNoSuspend() = runTest { + expect(1) + val result = withContext(Dispatchers.Default) { + expect(2) + "OK" + } + assertEquals("OK", result) + finish(3) + } + + @Test + fun testWithSuspend() = runTest { + expect(1) + val result = withContext(Dispatchers.Default) { + expect(2) + delay(100) + expect(3) + "OK" + } + assertEquals("OK", result) + finish(4) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/WithTimeoutChildDispatchStressTest.kt b/kotlinx-coroutines-core/jvm/test/WithTimeoutChildDispatchStressTest.kt new file mode 100644 index 0000000000..cc5e1d86e1 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/WithTimeoutChildDispatchStressTest.kt @@ -0,0 +1,29 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import org.junit.Test +import kotlin.test.* + +class WithTimeoutChildDispatchStressTest : TestBase() { + private val N_REPEATS = 10_000 * stressTestMultiplier + + /** + * This stress-test makes sure that dispatching resumption from within withTimeout + * works appropriately (without additional dispatch) despite the presence of + * children coroutine in a different dispatcher. + */ + @Test + fun testChildDispatch() = runBlocking { + repeat(N_REPEATS) { + val result = withTimeout(5000) { + // child in different dispatcher + val job = launch(Dispatchers.Default) { + // done nothing, but dispatches to join from another thread + } + job.join() + "DONE" + } + assertEquals("DONE", result) + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/WithTimeoutOrNullJvmTest.kt b/kotlinx-coroutines-core/jvm/test/WithTimeoutOrNullJvmTest.kt new file mode 100644 index 0000000000..a0a29f0de2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/WithTimeoutOrNullJvmTest.kt @@ -0,0 +1,64 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class WithTimeoutOrNullJvmTest : TestBase() { + @Test + fun testOuterTimeoutFiredBeforeInner() = runTest { + val result = withTimeoutOrNull(100) { + Thread.sleep(200) // wait enough for outer timeout to fire + withContext(NonCancellable) { yield() } // give an event loop a chance to run and process that cancellation + withTimeoutOrNull(100) { + yield() // will cancel because of outer timeout + expectUnreached() + } + expectUnreached() // should not be reached, because it is outer timeout + } + // outer timeout results in null + assertNull(result) + } + + @Test + fun testIgnoredTimeout() = runTest { + val value = withTimeout(1) { + Thread.sleep(10) + 42 + } + + assertEquals(42, value) + } + + @Test + fun testIgnoredTimeoutOnNull() = runTest { + val value = withTimeoutOrNull(1) { + Thread.sleep(10) + 42 + } + + assertEquals(42, value) + } + + @Test + fun testIgnoredTimeoutOnNullThrowsCancellation() = runTest { + try { + withTimeoutOrNull(1) { + expect(1) + Thread.sleep(10) + throw CancellationException() + } + expectUnreached() + } catch (e: CancellationException) { + finish(2) + } + } + + @Test + fun testIgnoredTimeoutOnNullThrowsOnYield() = runTest { + val value = withTimeoutOrNull(1) { + Thread.sleep(75) + yield() + } + assertNull(value) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/WithTimeoutOrNullThreadDispatchTest.kt b/kotlinx-coroutines-core/jvm/test/WithTimeoutOrNullThreadDispatchTest.kt new file mode 100644 index 0000000000..a5aeefdae9 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/WithTimeoutOrNullThreadDispatchTest.kt @@ -0,0 +1,79 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* +import java.util.concurrent.ExecutorService +import java.util.concurrent.Executors +import java.util.concurrent.ThreadFactory +import java.util.concurrent.atomic.AtomicInteger +import kotlin.coroutines.CoroutineContext + +class WithTimeoutOrNullThreadDispatchTest : TestBase() { + var executor: ExecutorService? = null + + @AfterTest + fun tearDown() { + executor?.shutdown() + } + + @Test + fun testCancellationDispatchScheduled() { + checkCancellationDispatch { + executor = Executors.newScheduledThreadPool(1, it) + executor!!.asCoroutineDispatcher() + } + } + + @Test + fun testCancellationDispatchNonScheduled() { + checkCancellationDispatch { + executor = Executors.newSingleThreadExecutor(it) + executor!!.asCoroutineDispatcher() + } + } + + @Test + fun testCancellationDispatchCustomNoDelay() { + // it also checks that there is at most once scheduled request in flight (no spurious concurrency) + var error: String? = null + checkCancellationDispatch { + executor = Executors.newSingleThreadExecutor(it) + val scheduled = AtomicInteger(0) + object : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + if (scheduled.incrementAndGet() > 1) error = "Two requests are scheduled concurrently" + executor!!.execute { + scheduled.decrementAndGet() + block.run() + } + } + } + } + error?.let { error(it) } + } + + private fun checkCancellationDispatch(factory: (ThreadFactory) -> CoroutineDispatcher) = runBlocking { + expect(1) + var thread: Thread? = null + val dispatcher = factory(ThreadFactory { Thread(it).also { thread = it } }) + withContext(dispatcher) { + expect(2) + assertEquals(thread, Thread.currentThread()) + val result = withTimeoutOrNull(100) { + try { + expect(3) + delay(1000) + expectUnreached() + } catch (e: CancellationException) { + expect(4) + assertEquals(thread, Thread.currentThread()) + throw e // rethrow + } + } + assertEquals(thread, Thread.currentThread()) + assertNull(result) + expect(5) + } + finish(6) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/WithTimeoutThreadDispatchTest.kt b/kotlinx-coroutines-core/jvm/test/WithTimeoutThreadDispatchTest.kt new file mode 100644 index 0000000000..58c7336dc8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/WithTimeoutThreadDispatchTest.kt @@ -0,0 +1,82 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.test.* +import java.util.concurrent.ExecutorService +import java.util.concurrent.Executors +import java.util.concurrent.ThreadFactory +import java.util.concurrent.atomic.AtomicInteger +import kotlin.coroutines.CoroutineContext + +class WithTimeoutThreadDispatchTest : TestBase() { + var executor: ExecutorService? = null + + @AfterTest + fun tearDown() { + executor?.shutdown() + } + + @Test + fun testCancellationDispatchScheduled() { + checkCancellationDispatch { + executor = Executors.newScheduledThreadPool(1, it) + executor!!.asCoroutineDispatcher() + } + } + + @Test + fun testCancellationDispatchNonScheduled() { + checkCancellationDispatch { + executor = Executors.newSingleThreadExecutor(it) + executor!!.asCoroutineDispatcher() + } + } + + @Test + fun testCancellationDispatchCustomNoDelay() { + // it also checks that there is at most once scheduled request in flight (no spurious concurrency) + var error: String? = null + checkCancellationDispatch { + executor = Executors.newSingleThreadExecutor(it) + val scheduled = AtomicInteger(0) + object : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + if (scheduled.incrementAndGet() > 1) error = "Two requests are scheduled concurrently" + executor!!.execute { + scheduled.decrementAndGet() + block.run() + } + } + } + } + error?.let { error(it) } + } + + private fun checkCancellationDispatch(factory: (ThreadFactory) -> CoroutineDispatcher) = runBlocking { + expect(1) + var thread: Thread? = null + val dispatcher = factory(ThreadFactory { Thread(it).also { thread = it } }) + withContext(dispatcher) { + expect(2) + assertEquals(thread, Thread.currentThread()) + try { + withTimeout(100) { + try { + expect(3) + delay(1000) + expectUnreached() + } catch (e: CancellationException) { + expect(4) + assertEquals(thread, Thread.currentThread()) + throw e // rethrow + } + } + } catch (e: CancellationException) { + expect(5) + assertEquals(thread, Thread.currentThread()) + } + expect(6) + } + finish(7) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/channels/ActorLazyTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ActorLazyTest.kt new file mode 100644 index 0000000000..7ed5d4ebe6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ActorLazyTest.kt @@ -0,0 +1,88 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class ActorLazyTest : TestBase() { + @Test + fun testEmptyStart() = runBlocking { + expect(1) + val actor = actor(start = CoroutineStart.LAZY) { + expect(5) + } + actor as Job // type assertion + assertFalse(actor.isActive) + assertFalse(actor.isCompleted) + assertFalse(actor.isClosedForSend) + expect(2) + yield() // to actor code --> nothing happens (not started!) + assertFalse(actor.isActive) + assertFalse(actor.isCompleted) + assertFalse(actor.isClosedForSend) + expect(3) + // start actor explicitly + actor.start() + expect(4) + yield() // to started actor + assertFalse(actor.isActive) + assertTrue(actor.isCompleted) + assertTrue(actor.isClosedForSend) + finish(6) + } + + @Test + fun testOne() = runBlocking { + expect(1) + val actor = actor(start = CoroutineStart.LAZY) { + expect(4) + assertEquals("OK", receive()) + expect(5) + } + actor as Job // type assertion + assertFalse(actor.isActive) + assertFalse(actor.isCompleted) + assertFalse(actor.isClosedForSend) + expect(2) + yield() // to actor code --> nothing happens (not started!) + assertFalse(actor.isActive) + assertFalse(actor.isCompleted) + assertFalse(actor.isClosedForSend) + expect(3) + // send message to actor --> should start it + actor.send("OK") + assertFalse(actor.isActive) + assertTrue(actor.isCompleted) + assertTrue(actor.isClosedForSend) + finish(6) + } + + @Test + fun testCloseFreshActor() = runTest { + val job = launch { + expect(2) + val actor = actor(start = CoroutineStart.LAZY) { + expect(3) + for (i in channel) { } + expect(4) + } + + actor.close() + } + + expect(1) + job.join() + finish(5) + } + + @Test + fun testCancelledParent() = runTest({ it is CancellationException }) { + cancel() + expect(1) + actor(start = CoroutineStart.LAZY) { + expectUnreached() + } + finish(2) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/ActorTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ActorTest.kt new file mode 100644 index 0000000000..8898158773 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ActorTest.kt @@ -0,0 +1,193 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import java.io.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class ActorTest(private val capacity: Int) : TestBase() { + + companion object { + @Parameterized.Parameters(name = "Capacity: {0}") + @JvmStatic + fun params(): Collection> = listOf(0, 1, Channel.UNLIMITED, Channel.CONFLATED).map { arrayOf(it) } + } + + @Test + fun testEmpty() = runBlocking { + expect(1) + val actor = actor(capacity = capacity) { + expect(3) + } + actor as Job // type assertion + assertTrue(actor.isActive) + assertFalse(actor.isCompleted) + assertFalse(actor.isClosedForSend) + expect(2) + yield() // to actor code + assertFalse(actor.isActive) + assertTrue(actor.isCompleted) + assertTrue(actor.isClosedForSend) + finish(4) + } + + @Test + fun testOne() = runBlocking { + expect(1) + val actor = actor(capacity = capacity) { + expect(3) + assertEquals("OK", receive()) + expect(6) + } + actor as Job // type assertion + assertTrue(actor.isActive) + assertFalse(actor.isCompleted) + assertFalse(actor.isClosedForSend) + expect(2) + yield() // to actor code + assertTrue(actor.isActive) + assertFalse(actor.isCompleted) + assertFalse(actor.isClosedForSend) + expect(4) + // send message to actor + actor.send("OK") + expect(5) + yield() // to actor code + assertFalse(actor.isActive) + assertTrue(actor.isCompleted) + assertTrue(actor.isClosedForSend) + finish(7) + } + + @Test + fun testCloseWithoutCause() = runTest { + val actor = actor(capacity = capacity) { + val element = channel.receive() + expect(2) + assertEquals(42, element) + val next = channel.receiveCatching() + assertNull(next.exceptionOrNull()) + expect(3) + } + + expect(1) + actor.send(42) + yield() + actor.close() + yield() + finish(4) + } + + @Test + fun testCloseWithCause() = runTest { + val actor = actor(capacity = capacity) { + val element = channel.receive() + expect(2) + require(element == 42) + try { + channel.receive() + } catch (e: IOException) { + expect(3) + } + } + + expect(1) + actor.send(42) + yield() + actor.close(IOException()) + yield() + finish(4) + } + + @Test + fun testCancelEnclosingJob() = runTest { + val job = async { + actor(capacity = capacity) { + expect(1) + channel.receive() + expectUnreached() + } + } + + yield() + yield() + + expect(2) + yield() + job.cancel() + + try { + job.await() + expectUnreached() + } catch (e: CancellationException) { + assertTrue(e.message?.contains("DeferredCoroutine was cancelled") ?: false) + } + + finish(3) + } + + @Test + fun testThrowingActor() = runTest(unhandled = listOf({e -> e is IllegalArgumentException})) { + val parent = Job() + val actor = actor(parent) { + channel.consumeEach { + expect(1) + throw IllegalArgumentException() + } + } + + actor.send(1) + parent.cancel() + parent.join() + finish(2) + } + + @Test + fun testChildJob() = runTest { + val parent = Job() + actor(parent) { + launch { + try { + delay(Long.MAX_VALUE) + } finally { + expect(1) + } + } + } + + yield() + yield() + parent.cancel() + parent.join() + finish(2) + } + + @Test + fun testCloseFreshActor() = runTest { + for (start in CoroutineStart.values()) { + val job = launch { + val actor = actor(start = start) { + for (i in channel) { + } + } + actor.close() + } + + job.join() + } + } + + @Test + fun testCancelledParent() = runTest({ it is CancellationException }) { + cancel() + expect(1) + actor { + expectUnreached() + } + finish(2) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelLeakTest.kt b/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelLeakTest.kt new file mode 100644 index 0000000000..7cd1487011 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelLeakTest.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import org.junit.Test +import kotlin.test.* + +@Suppress("DEPRECATION_ERROR") +class BroadcastChannelLeakTest : TestBase() { + @Test + fun testBufferedBroadcastChannelSubscriptionLeak() { + checkLeak { BroadcastChannelImpl(1) } + } + + @Test + fun testConflatedBroadcastChannelSubscriptionLeak() { + checkLeak { ConflatedBroadcastChannel() } + } + + enum class TestKind { BROADCAST_CLOSE, SUB_CANCEL, BOTH } + + private fun checkLeak(factory: () -> BroadcastChannel) = runTest { + for (kind in TestKind.entries) { + val broadcast = factory() + val sub = broadcast.openSubscription() + broadcast.send("OK") + assertEquals("OK", sub.receive()) + // now close broadcast + if (kind != TestKind.SUB_CANCEL) broadcast.close() + // and then cancel subscription + if (kind != TestKind.BROADCAST_CLOSE) sub.cancel() + // subscription should not be reachable from the channel anymore + FieldWalker.assertReachableCount(0, broadcast) { it === sub } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelMultiReceiveStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelMultiReceiveStressTest.kt new file mode 100644 index 0000000000..671f5d879e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/BroadcastChannelMultiReceiveStressTest.kt @@ -0,0 +1,166 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.* +import org.junit.runner.* +import org.junit.runners.* +import java.util.concurrent.atomic.* + +/** + * Tests delivery of events to multiple broadcast channel subscribers. + */ +@RunWith(Parameterized::class) +class BroadcastChannelMultiReceiveStressTest( + private val kind: TestBroadcastChannelKind +) : TestBase() { + + // Stressed by lincheck + companion object { + @Parameterized.Parameters(name = "{0}") + @JvmStatic + fun params(): Collection> = + TestBroadcastChannelKind.entries.map { arrayOf(it) } + } + + private val nReceivers = if (isStressTest) 10 else 5 + private val nSeconds = 3 * stressTestMultiplierSqrt + + private val broadcast = kind.create() + private val pool = newFixedThreadPoolContext(nReceivers + 1, "BroadcastChannelMultiReceiveStressTest") + + private val sentTotal = AtomicLong() + private val receivedTotal = AtomicLong() + private val stopOnReceive = AtomicLong(-1) + private val lastReceived = Array(nReceivers) { AtomicLong(-1) } + + @After + fun tearDown() { + pool.close() + } + + @Test + fun testStress() = runBlocking { + println("--- BroadcastChannelMultiReceiveStressTest $kind with nReceivers=$nReceivers") + val sender = + launch(pool + CoroutineName("Sender")) { + var i = 0L + while (isActive) { + i++ + broadcast.send(i) // could be cancelled + sentTotal.set(i) // only was for it if it was not cancelled + } + } + val receivers = mutableListOf() + fun printProgress() { + println("Sent ${sentTotal.get()}, received ${receivedTotal.get()}, receivers=${receivers.size}") + } + // ramp up receivers + repeat(nReceivers) { + delay(100) // wait 0.1 sec + val receiverIndex = receivers.size + val name = "Receiver$receiverIndex" + println("Launching $name") + receivers += launch(pool + CoroutineName(name)) { + val channel = broadcast.openSubscription() + when (receiverIndex % 5) { + 0 -> doReceive(channel, receiverIndex) + 1 -> doReceiveCatching(channel, receiverIndex) + 2 -> doIterator(channel, receiverIndex) + 3 -> doReceiveSelect(channel, receiverIndex) + 4 -> doReceiveCatchingSelect(channel, receiverIndex) + } + channel.cancel() + } + printProgress() + } + // wait + repeat(nSeconds) { _ -> + delay(1000) + printProgress() + } + sender.cancelAndJoin() + println("Tested $kind with nReceivers=$nReceivers") + val total = sentTotal.get() + println(" Sent $total events, waiting for receivers") + stopOnReceive.set(total) + try { + withTimeout(5000) { + receivers.forEachIndexed { index, receiver -> + if (lastReceived[index].get() >= total) receiver.cancel() + receiver.join() + } + } + } catch (e: Exception) { + println("Failed: $e") + pool.dumpThreads("Threads in pool") + receivers.indices.forEach { index -> + println("lastReceived[$index] = ${lastReceived[index].get()}") + } + throw e + } + println(" Received ${receivedTotal.get()} events") + } + + private fun doReceived(receiverIndex: Int, i: Long): Boolean { + val last = lastReceived[receiverIndex].get() + check(i > last) { "Last was $last, got $i" } + if (last != -1L && !kind.isConflated) + check(i == last + 1) { "Last was $last, got $i" } + receivedTotal.incrementAndGet() + lastReceived[receiverIndex].set(i) + return i >= stopOnReceive.get() + } + + private suspend fun doReceive(channel: ReceiveChannel, receiverIndex: Int) { + while (true) { + try { + val stop = doReceived(receiverIndex, channel.receive()) + if (stop) break + } catch (_: ClosedReceiveChannelException) { + break + } + } + } + + private suspend fun doReceiveCatching(channel: ReceiveChannel, receiverIndex: Int) { + while (true) { + val stop = doReceived(receiverIndex, channel.receiveCatching().getOrNull() ?: break) + if (stop) break + } + } + + private suspend fun doIterator(channel: ReceiveChannel, receiverIndex: Int) { + for (event in channel) { + val stop = doReceived(receiverIndex, event) + if (stop) break + } + } + + private suspend fun doReceiveSelect(channel: ReceiveChannel, receiverIndex: Int) { + while (true) { + try { + val event = select { channel.onReceive { it } } + val stop = doReceived(receiverIndex, event) + if (stop) break + } catch (_: ClosedReceiveChannelException) { + break + } + } + } + + private suspend fun doReceiveCatchingSelect(channel: ReceiveChannel, receiverIndex: Int) { + while (true) { + val event = select { channel.onReceiveCatching { it.getOrNull() } } ?: break + val stop = doReceived(receiverIndex, event) + if (stop) break + } + } + + @Suppress("UNUSED_PARAMETER") + private fun println(debugMessage: String) { + // Uncomment for local debugging + //println(debugMessage as Any?) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/BufferedChannelStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/BufferedChannelStressTest.kt new file mode 100644 index 0000000000..87754b359a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/BufferedChannelStressTest.kt @@ -0,0 +1,61 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.runner.* +import org.junit.runners.* + +@RunWith(Parameterized::class) +class BufferedChannelStressTest(private val capacity: Int) : TestBase() { + + companion object { + @Parameterized.Parameters(name = "{0}, nSenders={1}, nReceivers={2}") + @JvmStatic + fun params(): Collection> = listOf(1, 10, 100, 100_000, 1_000_000).map { arrayOf(it) } + } + + @Test + fun testStress() = runTest { + val n = 100_000 * stressTestMultiplier + val q = Channel(capacity) + val sender = launch { + for (i in 1..n) { + q.send(i) + } + expect(2) + } + val receiver = launch { + for (i in 1..n) { + val next = q.receive() + check(next == i) + } + expect(3) + } + expect(1) + sender.join() + receiver.join() + finish(4) + } + + @Test + fun testBurst() = runTest { + Assume.assumeTrue(capacity < 100_000) + repeat(10_000 * stressTestMultiplier) { + val channel = Channel(capacity) + val sender = launch(Dispatchers.Default) { + for (i in 1..capacity * 2) { + channel.send(i) + } + } + val receiver = launch(Dispatchers.Default) { + for (i in 1..capacity * 2) { + val next = channel.receive() + check(next == i) + } + } + sender.join() + receiver.join() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/CancelledChannelLeakTest.kt b/kotlinx-coroutines-core/jvm/test/channels/CancelledChannelLeakTest.kt new file mode 100644 index 0000000000..06108a6fe6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/CancelledChannelLeakTest.kt @@ -0,0 +1,27 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class CancelledChannelLeakTest : TestBase() { + /** + * Tests that cancellation removes the elements from the channel's buffer. + */ + @Test + fun testBufferedChannelLeak() = runTest { + for (capacity in listOf(Channel.CONFLATED, Channel.RENDEZVOUS, 1, 2, 5, 10)) { + val channel = Channel(capacity) + val value = X() + launch(start = CoroutineStart.UNDISPATCHED) { + channel.send(value) + } + FieldWalker.assertReachableCount(1, channel) { it === value } + channel.cancel() + // the element must be removed so that there is no memory leak + FieldWalker.assertReachableCount(0, channel) { it === value } + } + } + + class X +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/channels/ChannelMemoryLeakStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ChannelMemoryLeakStressTest.kt new file mode 100644 index 0000000000..4c239a3f01 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ChannelMemoryLeakStressTest.kt @@ -0,0 +1,21 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test + +class ChannelMemoryLeakStressTest : TestBase() { + private val nRepeat = 1_000_000 * stressTestMultiplier + + @Test + fun test() = runTest { + val c = Channel(1) + repeat(nRepeat) { + c.send(bigValue()) + c.receive() + } + } + + // capture big value for fast OOM in case of a bug + private fun bigValue(): ByteArray = ByteArray(4096) +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/ChannelSelectStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ChannelSelectStressTest.kt new file mode 100644 index 0000000000..f7a12c603f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ChannelSelectStressTest.kt @@ -0,0 +1,75 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.After +import org.junit.Test +import java.util.concurrent.atomic.AtomicLongArray +import kotlin.test.* + +class ChannelSelectStressTest : TestBase() { + private val pairedCoroutines = 3 + private val dispatcher = newFixedThreadPoolContext(pairedCoroutines * 2, "ChannelSelectStressTest") + private val elementsToSend = 20_000 * Long.SIZE_BITS * stressTestMultiplier + private val sent = atomic(0) + private val received = atomic(0) + private val receivedArray = AtomicLongArray(elementsToSend / Long.SIZE_BITS) + private val channel = Channel() + + @After + fun tearDown() { + dispatcher.close() + } + + @Test + fun testAtomicCancelStress() = runTest { + withContext(dispatcher) { + repeat(pairedCoroutines) { launchSender() } + repeat(pairedCoroutines) { launchReceiver() } + } + val missing = ArrayList() + for (i in 0 until receivedArray.length()) { + val bits = receivedArray[i] + if (bits != 0L.inv()) { + for (j in 0 until Long.SIZE_BITS) { + val mask = 1L shl j + if (bits and mask == 0L) missing += i * Long.SIZE_BITS + j + } + } + } + if (missing.isNotEmpty()) { + fail("Missed ${missing.size} out of $elementsToSend: $missing") + } + } + + private fun CoroutineScope.launchSender() { + launch { + while (sent.value < elementsToSend) { + val element = sent.getAndIncrement() + if (element >= elementsToSend) break + select { channel.onSend(element) {} } + } + channel.close(CancellationException()) + } + } + + private fun CoroutineScope.launchReceiver() { + launch { + while (received.value != elementsToSend) { + val element = select { channel.onReceive { it } } + received.incrementAndGet() + val index = (element / Long.SIZE_BITS) + val mask = 1L shl (element % Long.SIZE_BITS.toLong()).toInt() + while (true) { + val bits = receivedArray.get(index) + if (bits and mask != 0L) { + error("Detected duplicate") + } + if (receivedArray.compareAndSet(index, bits, bits or mask)) break + } + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/ChannelSendReceiveStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ChannelSendReceiveStressTest.kt new file mode 100644 index 0000000000..e9c087a79f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ChannelSendReceiveStressTest.kt @@ -0,0 +1,182 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.* +import org.junit.Ignore +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import java.util.concurrent.atomic.* +import kotlin.test.* + +@Ignore +@RunWith(Parameterized::class) +class ChannelSendReceiveStressTest( + private val kind: TestChannelKind, + private val nSenders: Int, + private val nReceivers: Int +) : TestBase() { + companion object { + @Parameterized.Parameters(name = "{0}, nSenders={1}, nReceivers={2}") + @JvmStatic + fun params(): Collection> = + listOf(1, 2, 10).flatMap { nSenders -> + listOf(1, 10).flatMap { nReceivers -> + TestChannelKind.values().map { arrayOf(it, nSenders, nReceivers) } + } + } + } + + private val timeLimit = 30_000L * stressTestMultiplier // 30 sec + private val nEvents = 200_000 * stressTestMultiplier + + private val maxBuffer = 10_000 // artificial limit for unlimited channel + + val channel = kind.create() + private val sendersCompleted = AtomicInteger() + private val receiversCompleted = AtomicInteger() + private val dupes = AtomicInteger() + private val sentTotal = AtomicInteger() + val received = AtomicIntegerArray(nEvents) + private val receivedTotal = AtomicInteger() + private val receivedBy = IntArray(nReceivers) + + private val pool = + newFixedThreadPoolContext(nSenders + nReceivers, "ChannelSendReceiveStressTest") + + @After + fun tearDown() { + pool.close() + } + + @Test + fun testSendReceiveStress() = runBlocking { + println("--- ChannelSendReceiveStressTest $kind with nSenders=$nSenders, nReceivers=$nReceivers") + val receivers = List(nReceivers) { receiverIndex -> + // different event receivers use different code + launch(pool + CoroutineName("receiver$receiverIndex")) { + when (receiverIndex % 5) { + 0 -> doReceive(receiverIndex) + 1 -> doReceiveCatching(receiverIndex) + 2 -> doIterator(receiverIndex) + 3 -> doReceiveSelect(receiverIndex) + 4 -> doReceiveCatchingSelect(receiverIndex) + } + receiversCompleted.incrementAndGet() + } + } + val senders = List(nSenders) { senderIndex -> + launch(pool + CoroutineName("sender$senderIndex")) { + when (senderIndex % 2) { + 0 -> doSend(senderIndex) + 1 -> doSendSelect(senderIndex) + } + sendersCompleted.incrementAndGet() + } + } + // print progress + val progressJob = launch { + var seconds = 0 + while (true) { + delay(1000) + println("${++seconds}: Sent ${sentTotal.get()}, received ${receivedTotal.get()}") + } + } + try { + withTimeout(timeLimit) { + senders.forEach { it.join() } + channel.close() + receivers.forEach { it.join() } + } + } catch (e: CancellationException) { + println("!!! Test timed out $e") + } + progressJob.cancel() + println("Tested $kind with nSenders=$nSenders, nReceivers=$nReceivers") + println("Completed successfully ${sendersCompleted.get()} sender coroutines") + println("Completed successfully ${receiversCompleted.get()} receiver coroutines") + println(" Sent ${sentTotal.get()} events") + println(" Received ${receivedTotal.get()} events") + println(" Received dupes ${dupes.get()}") + repeat(nReceivers) { receiveIndex -> + println(" Received by #$receiveIndex ${receivedBy[receiveIndex]}") + } + (channel as? BufferedChannel<*>)?.checkSegmentStructureInvariants() + assertEquals(nSenders, sendersCompleted.get()) + assertEquals(nReceivers, receiversCompleted.get()) + assertEquals(0, dupes.get()) + assertEquals(nEvents, sentTotal.get()) + if (!kind.isConflated) assertEquals(nEvents, receivedTotal.get()) + repeat(nReceivers) { receiveIndex -> + assertTrue(receivedBy[receiveIndex] > 0, "Each receiver should have received something") + } + } + + private suspend fun doSent() { + sentTotal.incrementAndGet() + if (!kind.isConflated) { + while (sentTotal.get() > receivedTotal.get() + maxBuffer) + yield() // throttle fast senders to prevent OOM with an unlimited channel + } + } + + private suspend fun doSend(senderIndex: Int) { + for (i in senderIndex until nEvents step nSenders) { + channel.send(i) + doSent() + } + } + + private suspend fun doSendSelect(senderIndex: Int) { + for (i in senderIndex until nEvents step nSenders) { + select { channel.onSend(i) { Unit } } + doSent() + } + } + + private fun doReceived(receiverIndex: Int, event: Int) { + if (!received.compareAndSet(event, 0, 1)) { + println("Duplicate event $event at $receiverIndex") + dupes.incrementAndGet() + } + receivedTotal.incrementAndGet() + receivedBy[receiverIndex]++ + } + + private suspend fun doReceive(receiverIndex: Int) { + while (true) { + try { doReceived(receiverIndex, channel.receive()) } + catch (ex: ClosedReceiveChannelException) { break } + } + } + + private suspend fun doReceiveCatching(receiverIndex: Int) { + while (true) { + doReceived(receiverIndex, channel.receiveCatching().getOrNull() ?: break) + } + } + + private suspend fun doIterator(receiverIndex: Int) { + for (event in channel) { + doReceived(receiverIndex, event) + } + } + + private suspend fun doReceiveSelect(receiverIndex: Int) { + while (true) { + try { + val event = select { channel.onReceive { it } } + doReceived(receiverIndex, event) + } catch (ex: ClosedReceiveChannelException) { break } + } + } + + private suspend fun doReceiveCatchingSelect(receiverIndex: Int) { + while (true) { + val event = select { channel.onReceiveCatching { it.getOrNull() } } ?: break + doReceived(receiverIndex, event) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementSelectOldStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementSelectOldStressTest.kt new file mode 100644 index 0000000000..fc5c3845f7 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementSelectOldStressTest.kt @@ -0,0 +1,251 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.After +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.random.Random +import kotlin.test.* + +/** + * Tests resource transfer via channel send & receive operations, including their select versions, + * using `onUndeliveredElement` to detect lost resources and close them properly. + */ +@RunWith(Parameterized::class) +class ChannelUndeliveredElementSelectOldStressTest(private val kind: TestChannelKind) : TestBase() { + companion object { + @Parameterized.Parameters(name = "{0}") + @JvmStatic + fun params(): Collection> = + TestChannelKind.values() + .filter { !it.viaBroadcast } + .map { arrayOf(it) } + } + + private val iterationDurationMs = 100L + private val testIterations = 20 * stressTestMultiplier // 2 sec + + private val dispatcher = newFixedThreadPoolContext(2, "ChannelAtomicCancelStressTest") + private val scope = CoroutineScope(dispatcher) + + private val channel = kind.create { it.failedToDeliver() } + private val senderDone = Channel(1) + private val receiverDone = Channel(1) + + @Volatile + private var lastReceived = -1L + + private var stoppedSender = 0L + private var stoppedReceiver = 0L + + private var sentCnt = 0L // total number of send attempts + private var receivedCnt = 0L // actually received successfully + private var dupCnt = 0L // duplicates (should never happen) + private val failedToDeliverCnt = atomic(0L) // out of sent + + private val modulo = 1 shl 25 + private val mask = (modulo - 1).toLong() + private val sentStatus = ItemStatus() // 1 - send norm, 2 - send select, +2 - did not throw exception + private val receivedStatus = ItemStatus() // 1-6 received + private val failedStatus = ItemStatus() // 1 - failed + + lateinit var sender: Job + lateinit var receiver: Job + + @After + fun tearDown() { + dispatcher.close() + } + + private inline fun cancellable(done: Channel, block: () -> Unit) { + try { + block() + } finally { + if (!done.trySend(true).isSuccess) + error(IllegalStateException("failed to offer to done channel")) + } + } + + @Test + fun testAtomicCancelStress() = runBlocking { + println("=== ChannelAtomicCancelStressTest $kind") + var nextIterationTime = System.currentTimeMillis() + iterationDurationMs + var iteration = 0 + launchSender() + launchReceiver() + while (!hasError()) { + if (System.currentTimeMillis() >= nextIterationTime) { + nextIterationTime += iterationDurationMs + iteration++ + verify(iteration) + if (iteration % 10 == 0) printProgressSummary(iteration) + if (iteration >= testIterations) break + launchSender() + launchReceiver() + } + when (Random.nextInt(3)) { + 0 -> { // cancel & restart sender + stopSender() + launchSender() + } + 1 -> { // cancel & restart receiver + stopReceiver() + launchReceiver() + } + 2 -> yield() // just yield (burn a little time) + } + } + } + + private suspend fun verify(iteration: Int) { + stopSender() + drainReceiver() + stopReceiver() + try { + assertEquals(0, dupCnt) + assertEquals(sentCnt - failedToDeliverCnt.value, receivedCnt) + } catch (e: Throwable) { + printProgressSummary(iteration) + printErrorDetails() + throw e + } + sentStatus.clear() + receivedStatus.clear() + failedStatus.clear() + } + + private fun printProgressSummary(iteration: Int) { + println("--- ChannelAtomicCancelStressTest $kind -- $iteration of $testIterations") + println(" Sent $sentCnt times to channel") + println(" Received $receivedCnt times from channel") + println(" Failed to deliver ${failedToDeliverCnt.value} times") + println(" Stopped sender $stoppedSender times") + println(" Stopped receiver $stoppedReceiver times") + println(" Duplicated $dupCnt deliveries") + } + + private fun printErrorDetails() { + val min = minOf(sentStatus.min, receivedStatus.min, failedStatus.min) + val max = maxOf(sentStatus.max, receivedStatus.max, failedStatus.max) + for (x in min..max) { + val sentCnt = if (sentStatus[x] != 0) 1 else 0 + val receivedCnt = if (receivedStatus[x] != 0) 1 else 0 + val failedToDeliverCnt = failedStatus[x] + if (sentCnt - failedToDeliverCnt != receivedCnt) { + println("!!! Error for value $x: " + + "sentStatus=${sentStatus[x]}, " + + "receivedStatus=${receivedStatus[x]}, " + + "failedStatus=${failedStatus[x]}" + ) + } + } + } + + + private fun launchSender() { + sender = scope.launch(start = CoroutineStart.ATOMIC) { + cancellable(senderDone) { + var counter = 0 + while (true) { + val trySendData = Data(sentCnt++) + sentStatus[trySendData.x] = 1 + selectOld { channel.onSend(trySendData) {} } + sentStatus[trySendData.x] = 3 + when { + // must artificially slow down LINKED_LIST sender to avoid overwhelming receiver and going OOM + kind == TestChannelKind.UNLIMITED -> while (sentCnt > lastReceived + 100) yield() + // yield periodically to check cancellation on conflated channels + kind.isConflated -> if (counter++ % 100 == 0) yield() + } + } + } + } + } + + private suspend fun stopSender() { + stoppedSender++ + sender.cancelAndJoin() + senderDone.receive() + } + + private fun launchReceiver() { + receiver = scope.launch(start = CoroutineStart.ATOMIC) { + cancellable(receiverDone) { + while (true) { + selectOld { + channel.onReceive { receivedData -> + receivedData.onReceived() + receivedCnt++ + val received = receivedData.x + if (received <= lastReceived) + dupCnt++ + lastReceived = received + receivedStatus[received] = 1 + } + } + } + } + } + } + + private suspend fun drainReceiver() { + while (!channel.isEmpty) yield() // burn time until receiver gets it all + } + + private suspend fun stopReceiver() { + stoppedReceiver++ + receiver.cancelAndJoin() + receiverDone.receive() + } + + private inner class Data(val x: Long) { + private val firstFailedToDeliverOrReceivedCallTrace = atomic(null) + + fun failedToDeliver() { + val trace = if (TRACING_ENABLED) Exception("First onUndeliveredElement() call") else DUMMY_TRACE_EXCEPTION + if (firstFailedToDeliverOrReceivedCallTrace.compareAndSet(null, trace)) { + failedToDeliverCnt.incrementAndGet() + failedStatus[x] = 1 + return + } + throw IllegalStateException("onUndeliveredElement()/onReceived() notified twice", firstFailedToDeliverOrReceivedCallTrace.value!!) + } + + fun onReceived() { + val trace = if (TRACING_ENABLED) Exception("First onReceived() call") else DUMMY_TRACE_EXCEPTION + if (firstFailedToDeliverOrReceivedCallTrace.compareAndSet(null, trace)) return + throw IllegalStateException("onUndeliveredElement()/onReceived() notified twice", firstFailedToDeliverOrReceivedCallTrace.value!!) + } + } + + inner class ItemStatus { + private val a = ByteArray(modulo) + private val _min = atomic(Long.MAX_VALUE) + private val _max = atomic(-1L) + + val min: Long get() = _min.value + val max: Long get() = _max.value + + operator fun set(x: Long, value: Int) { + a[(x and mask).toInt()] = value.toByte() + _min.update { y -> minOf(x, y) } + _max.update { y -> maxOf(x, y) } + } + + operator fun get(x: Long): Int = a[(x and mask).toInt()].toInt() + + fun clear() { + if (_max.value < 0) return + for (x in _min.value.._max.value) a[(x and mask).toInt()] = 0 + _min.value = Long.MAX_VALUE + _max.value = -1L + } + } +} + +private const val TRACING_ENABLED = false // Change to `true` to enable the tracing +private val DUMMY_TRACE_EXCEPTION = Exception("The tracing is disabled; please enable it by changing the `TRACING_ENABLED` constant to `true`.") diff --git a/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementStressTest.kt new file mode 100644 index 0000000000..ec123f15fb --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ChannelUndeliveredElementStressTest.kt @@ -0,0 +1,267 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.After +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.random.Random +import kotlin.test.* + +/** + * Tests resource transfer via channel send & receive operations, including their select versions, + * using `onUndeliveredElement` to detect lost resources and close them properly. + */ +@RunWith(Parameterized::class) +class ChannelUndeliveredElementStressTest(private val kind: TestChannelKind) : TestBase() { + companion object { + @Parameterized.Parameters(name = "{0}") + @JvmStatic + fun params(): Collection> = + TestChannelKind.values() + .filter { !it.viaBroadcast } + .map { arrayOf(it) } + } + + private val iterationDurationMs = 100L + private val testIterations = 20 * stressTestMultiplier // 2 sec + + private val dispatcher = newFixedThreadPoolContext(2, "ChannelAtomicCancelStressTest") + private val scope = CoroutineScope(dispatcher) + + private val channel = kind.create { it.failedToDeliver() } + private val senderDone = Channel(1) + private val receiverDone = Channel(1) + + @Volatile + private var lastReceived = -1L + + private var stoppedSender = 0L + private var stoppedReceiver = 0L + + private var sentCnt = 0L // total number of send attempts + private var receivedCnt = 0L // actually received successfully + private var dupCnt = 0L // duplicates (should never happen) + private val failedToDeliverCnt = atomic(0L) // out of sent + + private val modulo = 1 shl 25 + private val mask = (modulo - 1).toLong() + private val sentStatus = ItemStatus() // 1 - send norm, 2 - send select, +2 - did not throw exception + private val receivedStatus = ItemStatus() // 1-6 received + private val failedStatus = ItemStatus() // 1 - failed + + lateinit var sender: Job + lateinit var receiver: Job + + @After + fun tearDown() { + dispatcher.close() + } + + private inline fun cancellable(done: Channel, block: () -> Unit) { + try { + block() + } finally { + if (!done.trySend(true).isSuccess) + error(IllegalStateException("failed to offer to done channel")) + } + } + + @Test + fun testAtomicCancelStress() = runBlocking { + println("=== ChannelAtomicCancelStressTest $kind") + var nextIterationTime = System.currentTimeMillis() + iterationDurationMs + var iteration = 0 + launchSender() + launchReceiver() + while (!hasError()) { + if (System.currentTimeMillis() >= nextIterationTime) { + nextIterationTime += iterationDurationMs + iteration++ + verify(iteration) + if (iteration % 10 == 0) printProgressSummary(iteration) + if (iteration >= testIterations) break + launchSender() + launchReceiver() + } + when (Random.nextInt(3)) { + 0 -> { // cancel & restart sender + stopSender() + launchSender() + } + 1 -> { // cancel & restart receiver + stopReceiver() + launchReceiver() + } + 2 -> yield() // just yield (burn a little time) + } + } + } + + private suspend fun verify(iteration: Int) { + stopSender() + drainReceiver() + stopReceiver() + try { + assertEquals(0, dupCnt) + assertEquals(sentCnt - failedToDeliverCnt.value, receivedCnt) + } catch (e: Throwable) { + printProgressSummary(iteration) + printErrorDetails() + throw e + } + (channel as? BufferedChannel<*>)?.checkSegmentStructureInvariants() + sentStatus.clear() + receivedStatus.clear() + failedStatus.clear() + } + + private fun printProgressSummary(iteration: Int) { + println("--- ChannelAtomicCancelStressTest $kind -- $iteration of $testIterations") + println(" Sent $sentCnt times to channel") + println(" Received $receivedCnt times from channel") + println(" Failed to deliver ${failedToDeliverCnt.value} times") + println(" Stopped sender $stoppedSender times") + println(" Stopped receiver $stoppedReceiver times") + println(" Duplicated $dupCnt deliveries") + } + + private fun printErrorDetails() { + val min = minOf(sentStatus.min, receivedStatus.min, failedStatus.min) + val max = maxOf(sentStatus.max, receivedStatus.max, failedStatus.max) + for (x in min..max) { + val sentCnt = if (sentStatus[x] != 0) 1 else 0 + val receivedCnt = if (receivedStatus[x] != 0) 1 else 0 + val failedToDeliverCnt = failedStatus[x] + if (sentCnt - failedToDeliverCnt != receivedCnt) { + println("!!! Error for value $x: " + + "sentStatus=${sentStatus[x]}, " + + "receivedStatus=${receivedStatus[x]}, " + + "failedStatus=${failedStatus[x]}" + ) + } + } + } + + + private fun launchSender() { + sender = scope.launch(start = CoroutineStart.ATOMIC) { + cancellable(senderDone) { + var counter = 0 + while (true) { + val trySendData = Data(sentCnt++) + val sendMode = Random.nextInt(2) + 1 + sentStatus[trySendData.x] = sendMode + when (sendMode) { + 1 -> channel.send(trySendData) + 2 -> select { channel.onSend(trySendData) {} } + else -> error("cannot happen") + } + sentStatus[trySendData.x] = sendMode + 2 + when { + // must artificially slow down LINKED_LIST sender to avoid overwhelming receiver and going OOM + kind == TestChannelKind.UNLIMITED -> while (sentCnt > lastReceived + 100) yield() + // yield periodically to check cancellation on conflated channels + kind.isConflated -> if (counter++ % 100 == 0) yield() + } + } + } + } + } + + private suspend fun stopSender() { + stoppedSender++ + sender.cancelAndJoin() + senderDone.receive() + } + + private fun launchReceiver() { + receiver = scope.launch(start = CoroutineStart.ATOMIC) { + cancellable(receiverDone) { + while (true) { + val receiveMode = Random.nextInt(6) + 1 + val receivedData = when (receiveMode) { + 1 -> channel.receive() + 2 -> select { channel.onReceive { it } } + 3 -> channel.receiveCatching().getOrElse { error("Should not be closed") } + 4 -> select { channel.onReceiveCatching { it.getOrElse { error("Should not be closed") } } } + 5 -> channel.receiveCatching().getOrThrow() + 6 -> { + val iterator = channel.iterator() + check(iterator.hasNext()) { "Should not be closed" } + iterator.next() + } + else -> error("cannot happen") + } + receivedData.onReceived() + receivedCnt++ + val received = receivedData.x + if (received <= lastReceived) + dupCnt++ + lastReceived = received + receivedStatus[received] = receiveMode + } + } + } + } + + private suspend fun drainReceiver() { + while (!channel.isEmpty) yield() // burn time until receiver gets it all + } + + private suspend fun stopReceiver() { + stoppedReceiver++ + receiver.cancel() + receiverDone.receive() + } + + private inner class Data(val x: Long) { + private val firstFailedToDeliverOrReceivedCallTrace = atomic(null) + + fun failedToDeliver() { + val trace = if (TRACING_ENABLED) Exception("First onUndeliveredElement() call") else DUMMY_TRACE_EXCEPTION + if (firstFailedToDeliverOrReceivedCallTrace.compareAndSet(null, trace)) { + failedToDeliverCnt.incrementAndGet() + failedStatus[x] = 1 + return + } + throw IllegalStateException("onUndeliveredElement()/onReceived() notified twice", firstFailedToDeliverOrReceivedCallTrace.value!!) + } + + fun onReceived() { + val trace = if (TRACING_ENABLED) Exception("First onReceived() call") else DUMMY_TRACE_EXCEPTION + if (firstFailedToDeliverOrReceivedCallTrace.compareAndSet(null, trace)) return + throw IllegalStateException("onUndeliveredElement()/onReceived() notified twice", firstFailedToDeliverOrReceivedCallTrace.value!!) + } + } + + inner class ItemStatus { + private val a = ByteArray(modulo) + private val _min = atomic(Long.MAX_VALUE) + private val _max = atomic(-1L) + + val min: Long get() = _min.value + val max: Long get() = _max.value + + operator fun set(x: Long, value: Int) { + a[(x and mask).toInt()] = value.toByte() + _min.update { y -> minOf(x, y) } + _max.update { y -> maxOf(x, y) } + } + + operator fun get(x: Long): Int = a[(x and mask).toInt()].toInt() + + fun clear() { + if (_max.value < 0) return + for (x in _min.value.._max.value) a[(x and mask).toInt()] = 0 + _min.value = Long.MAX_VALUE + _max.value = -1L + } + } +} + +private const val TRACING_ENABLED = false // Change to `true` to enable the tracing +private val DUMMY_TRACE_EXCEPTION = Exception("The tracing is disabled; please enable it by changing the `TRACING_ENABLED` constant to `true`.") diff --git a/kotlinx-coroutines-core/jvm/test/channels/ConflatedChannelCloseStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ConflatedChannelCloseStressTest.kt new file mode 100644 index 0000000000..e36cafe4bd --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ConflatedChannelCloseStressTest.kt @@ -0,0 +1,106 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.concurrent.atomic.* + +class ConflatedChannelCloseStressTest : TestBase() { + + private val nSenders = 2 + private val testSeconds = 3 * stressTestMultiplier + + private val curChannel = AtomicReference>(Channel(Channel.CONFLATED)) + private val sent = AtomicInteger() + private val closed = AtomicInteger() + val received = AtomicInteger() + + val pool = newFixedThreadPoolContext(nSenders + 2, "TestStressClose") + + @After + fun tearDown() { + pool.close() + } + + @Test + fun testStressClose() = runBlocking { + println("--- ConflatedChannelCloseStressTest with nSenders=$nSenders") + val senderJobs = List(nSenders) { Job() } + val senders = List(nSenders) { senderId -> + launch(pool) { + var x = senderId + try { + while (isActive) { + curChannel.get().trySend(x).onSuccess { + x += nSenders + sent.incrementAndGet() + } + } + } finally { + senderJobs[senderId].cancel() + } + } + } + val closerJob = Job() + val closer = launch(pool) { + try { + while (isActive) { + flipChannel() + closed.incrementAndGet() + yield() + } + } finally { + closerJob.cancel() + } + } + val receiver = async(pool + NonCancellable) { + while (isActive) { + curChannel.get().receiveCatching().getOrElse { + it?.let { throw it } + } + received.incrementAndGet() + } + } + // print stats while running + repeat(testSeconds) { + delay(1000) + printStats() + } + println("Stopping") + senders.forEach { it.cancel() } + closer.cancel() + // wait them to complete + println("waiting for senders...") + senderJobs.forEach { it.join() } + println("waiting for closer...") + closerJob.join() + // close cur channel + println("Closing channel and signalling receiver...") + flipChannel() + curChannel.get().close(StopException()) + /// wait for receiver do complete + println("Waiting for receiver...") + try { + receiver.await() + error("Receiver should not complete normally") + } catch (e: StopException) { + // ok + } + // print stats + println("--- done") + printStats() + } + + private fun flipChannel() { + val oldChannel = curChannel.get() + val newChannel = Channel(Channel.CONFLATED) + curChannel.set(newChannel) + check(oldChannel.close()) + } + + private fun printStats() { + println("sent ${sent.get()}, closed ${closed.get()}, received ${received.get()}") + } + + class StopException : Exception() +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/DoubleChannelCloseStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/DoubleChannelCloseStressTest.kt new file mode 100644 index 0000000000..247b077ae7 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/DoubleChannelCloseStressTest.kt @@ -0,0 +1,27 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* + +class DoubleChannelCloseStressTest : TestBase() { + private val nTimes = 1000 * stressTestMultiplier + + @Test + fun testDoubleCloseStress() { + repeat(nTimes) { + val actor = GlobalScope.actor(CoroutineName("actor"), start = CoroutineStart.LAZY) { + // empty -- just closes channel + } + GlobalScope.launch(CoroutineName("sender")) { + try { + actor.send(1) + } catch (e: ClosedSendChannelException) { + // ok -- closed before send + } + } + Thread.sleep(1) + actor.close() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/InvokeOnCloseStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/InvokeOnCloseStressTest.kt new file mode 100644 index 0000000000..19aa0402e2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/InvokeOnCloseStressTest.kt @@ -0,0 +1,61 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import kotlin.coroutines.* +import kotlin.test.* + +class InvokeOnCloseStressTest : TestBase(), CoroutineScope { + + private val iterations = 1000 * stressTestMultiplier + + private val pool = newFixedThreadPoolContext(3, "InvokeOnCloseStressTest") + override val coroutineContext: CoroutineContext + get() = pool + + @After + fun tearDown() { + pool.close() + } + + @Test + fun testInvokedExactlyOnce() = runBlocking { + runStressTest(TestChannelKind.BUFFERED_1) + } + + @Test + fun testInvokedExactlyOnceBroadcast() = runBlocking { + runStressTest(TestChannelKind.CONFLATED_BROADCAST) + } + + private suspend fun runStressTest(kind: TestChannelKind) { + repeat(iterations) { + val counter = AtomicInteger(0) + val channel = kind.create() + + val latch = CountDownLatch(1) + val j1 = async { + latch.await() + channel.close() + } + + val j2 = async { + latch.await() + channel.invokeOnClose { counter.incrementAndGet() } + } + + val j3 = async { + latch.await() + channel.invokeOnClose { counter.incrementAndGet() } + } + + latch.countDown() + joinAll(j1, j2, j3) + assertEquals(1, counter.get()) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/ProduceConsumeJvmTest.kt b/kotlinx-coroutines-core/jvm/test/channels/ProduceConsumeJvmTest.kt new file mode 100644 index 0000000000..6b5f2bfa31 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/ProduceConsumeJvmTest.kt @@ -0,0 +1,59 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class ProduceConsumeJvmTest( + private val capacity: Int, + private val number: Int +) : TestBase() { + companion object { + @Parameterized.Parameters(name = "capacity={0}, number={1}") + @JvmStatic + fun params(): Collection> = + listOf(0, 1, 10, 1000, Channel.UNLIMITED).flatMap { capacity -> + listOf(1, 10, 1000).map { number -> + arrayOf(capacity, number) + } + } + } + + @Test + fun testProducer() = runTest { + var sentAll = false + val producer = produce(capacity = capacity) { + for (i in 1..number) { + send(i) + } + sentAll = true + } + var consumed = 0 + for (x in producer) { + consumed++ + } + assertTrue(sentAll) + assertEquals(number, consumed) + } + + @Test + fun testActor() = runTest { + val received = CompletableDeferred() + val actor = actor(capacity = capacity) { + var n = 0 + for(i in channel) { + n++ + } + received.complete(n) + } + for(i in 1..number) { + actor.send(i) + } + actor.close() + assertEquals(number, received.await()) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/SendReceiveJvmStressTest.kt b/kotlinx-coroutines-core/jvm/test/channels/SendReceiveJvmStressTest.kt new file mode 100644 index 0000000000..865f2ab827 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/SendReceiveJvmStressTest.kt @@ -0,0 +1,45 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.runner.* +import org.junit.runners.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class SendReceiveJvmStressTest(private val channel: Channel) : TestBase() { + + companion object { + @Parameterized.Parameters(name = "{0}") + @JvmStatic + fun params(): Collection> = listOf( + Channel(1), + Channel (10), + Channel(1_000_000), + Channel(Channel.UNLIMITED), + Channel(Channel.RENDEZVOUS) + ).map { arrayOf(it) } + } + + @Test + fun testStress() = runTest { + val n = 100_000 * stressTestMultiplier + val sender = launch { + for (i in 1..n) { + channel.send(i) + } + expect(2) + } + val receiver = launch { + for (i in 1..n) { + val next = channel.receive() + check(next == i) + } + expect(3) + } + expect(1) + sender.join() + receiver.join() + finish(4) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/SimpleSendReceiveJvmTest.kt b/kotlinx-coroutines-core/jvm/test/channels/SimpleSendReceiveJvmTest.kt new file mode 100644 index 0000000000..37082513fe --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/SimpleSendReceiveJvmTest.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class SimpleSendReceiveJvmTest( + private val kind: TestChannelKind, + val n: Int, + val concurrent: Boolean +) : TestBase() { + companion object { + @Parameterized.Parameters(name = "{0}, n={1}, concurrent={2}") + @JvmStatic + fun params(): Collection> = TestChannelKind.values().flatMap { kind -> + listOf(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 100, 1000).flatMap { n -> + listOf(false, true).map { concurrent -> + arrayOf(kind, n, concurrent) + } + } + } + } + + val channel = kind.create() + + @Test + fun testSimpleSendReceive() = runBlocking { + val ctx = if (concurrent) Dispatchers.Default else coroutineContext + launch(ctx) { + repeat(n) { channel.send(it) } + channel.close() + } + var expected = 0 + for (x in channel) { + if (!kind.isConflated) { + assertEquals(expected++, x) + } else { + assertTrue(x >= expected) + expected = x + 1 + } + } + if (!kind.isConflated) { + assertEquals(n, expected) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/TickerChannelCommonTest.kt b/kotlinx-coroutines-core/jvm/test/channels/TickerChannelCommonTest.kt new file mode 100644 index 0000000000..65102095b1 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/TickerChannelCommonTest.kt @@ -0,0 +1,164 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class TickerChannelCommonTest(private val channelFactory: Channel) : TestBase() { + companion object { + @Parameterized.Parameters(name = "{0}") + @JvmStatic + fun params(): Collection> = + Channel.values().map { arrayOf(it) } + } + + enum class Channel { + FIXED_PERIOD { + override fun invoke(delay: Long, initialDelay: Long) = + ticker(delay, initialDelayMillis = initialDelay, mode = TickerMode.FIXED_PERIOD) + }, + + FIXED_DELAY { + override fun invoke(delay: Long, initialDelay: Long) = + ticker(delay, initialDelayMillis = initialDelay, mode = TickerMode.FIXED_DELAY) + }; + + abstract operator fun invoke(delay: Long, initialDelay: Long = 0): ReceiveChannel + } + + @Test + fun testDelay() = withVirtualTimeSource { + runTest { + val delayChannel = channelFactory(delay = 10000) + delayChannel.checkNotEmpty() + delayChannel.checkEmpty() + + delay(5000) + delayChannel.checkEmpty() + delay(5100) + delayChannel.checkNotEmpty() + + delayChannel.cancel() + delay(5100) + assertFailsWith { delayChannel.tryReceive().getOrThrow() } + } + } + + @Test + fun testInitialDelay() = withVirtualTimeSource { + runTest { + val delayChannel = channelFactory(initialDelay = 750, delay = 1000) + delayChannel.checkEmpty() + delay(500) + delayChannel.checkEmpty() + delay(300) + delayChannel.checkNotEmpty() + + // Regular delay + delay(750) + delayChannel.checkEmpty() + delay(260) + delayChannel.checkNotEmpty() + delayChannel.cancel() + } + } + + @Test + fun testReceive() = withVirtualTimeSource { + runTest { + val delayChannel = channelFactory(delay = 1000) + delayChannel.checkNotEmpty() + var value = withTimeoutOrNull(750) { + delayChannel.receive() + 1 + } + + assertNull(value) + value = withTimeoutOrNull(260) { + delayChannel.receive() + 1 + } + + assertNotNull(value) + delayChannel.cancel() + } + } + + @Test + fun testComplexOperator() = withVirtualTimeSource { + runTest { + val producer = GlobalScope.produce { + for (i in 1..7) { + send(i) + delay(1000) + } + } + + val averages = producer.averageInTimeWindow(3000).toList() + assertEquals(listOf(2.0, 5.0, 7.0), averages) + } + } + + private fun ReceiveChannel.averageInTimeWindow(timespan: Long) = GlobalScope.produce { + val delayChannel = channelFactory(delay = timespan, initialDelay = timespan) + var sum = 0 + var n = 0 + whileSelect { + this@averageInTimeWindow.onReceiveCatching { + if (it.isClosed) { + // Send leftovers and bail out + if (n != 0) send(sum / n.toDouble()) + false + } else { + sum += it.getOrThrow() + ++n + true + } + } + + // Timeout, send aggregated average and reset counters + delayChannel.onReceive { + send(sum / n.toDouble()) + sum = 0 + n = 0 + true + } + } + + delayChannel.cancel() + } + + @Test + fun testStress() = runTest { + // No OOM/SOE + val iterations = 100_000 * stressTestMultiplier + val delayChannel = channelFactory(0) + repeat(iterations) { + delayChannel.receive() + } + + delayChannel.cancel() + } + + @Test(expected = IllegalArgumentException::class) + fun testNegativeDelay() { + channelFactory(-1) + } + + @Test(expected = IllegalArgumentException::class) + fun testNegativeInitialDelay() { + channelFactory(initialDelay = -1, delay = 100) + } +} + +fun ReceiveChannel.checkEmpty() = assertNull(tryReceive().getOrNull()) + +fun ReceiveChannel.checkNotEmpty() { + assertNotNull(tryReceive().getOrNull()) + assertNull(tryReceive().getOrNull()) +} diff --git a/kotlinx-coroutines-core/jvm/test/channels/TickerChannelTest.kt b/kotlinx-coroutines-core/jvm/test/channels/TickerChannelTest.kt new file mode 100644 index 0000000000..051d670743 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/channels/TickerChannelTest.kt @@ -0,0 +1,62 @@ +package kotlinx.coroutines.channels + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* + +class TickerChannelTest : TestBase() { + @Test + fun testFixedDelayChannelBackpressure() = withVirtualTimeSource { + runTest { + val delayChannel = ticker(delayMillis = 1000, initialDelayMillis = 0, mode = TickerMode.FIXED_DELAY) + delayChannel.checkNotEmpty() + delayChannel.checkEmpty() + + delay(1500) + delayChannel.checkNotEmpty() + delay(500) + delayChannel.checkEmpty() + delay(520) + delayChannel.checkNotEmpty() + delayChannel.cancel() + } + } + + @Test + fun testDelayChannelBackpressure() = withVirtualTimeSource { + runTest { + val delayChannel = ticker(delayMillis = 1000, initialDelayMillis = 0) + delayChannel.checkNotEmpty() + delayChannel.checkEmpty() + + delay(1500) + delayChannel.checkNotEmpty() + delay(520) + delayChannel.checkNotEmpty() + delay(500) + delayChannel.checkEmpty() + delay(520) + delayChannel.checkNotEmpty() + delayChannel.cancel() + } + } + + @Test + fun testDelayChannelBackpressure2() = withVirtualTimeSource { + runTest { + val delayChannel = ticker(delayMillis = 200, initialDelayMillis = 0) + delayChannel.checkNotEmpty() + delayChannel.checkEmpty() + + delay(500) + delayChannel.checkNotEmpty() + delay(110) + delayChannel.checkNotEmpty() + delay(110) + delayChannel.checkEmpty() + delay(110) + delayChannel.checkNotEmpty() + delayChannel.cancel() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-delay-01.kt b/kotlinx-coroutines-core/jvm/test/examples/example-delay-01.kt new file mode 100644 index 0000000000..63dcc081a0 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/examples/example-delay-01.kt @@ -0,0 +1,21 @@ +// This file was automatically generated from Delay.kt by Knit tool. Do not edit. +package kotlinx.coroutines.examples.exampleDelay01 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.time.Duration.Companion.milliseconds + +fun main() = runBlocking { + +flow { + emit(1) + delay(90) + emit(2) + delay(90) + emit(3) + delay(1010) + emit(4) + delay(1010) + emit(5) +}.debounce(1000) +.toList().joinToString().let { println(it) } } diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-delay-02.kt b/kotlinx-coroutines-core/jvm/test/examples/example-delay-02.kt new file mode 100644 index 0000000000..498ce33072 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/examples/example-delay-02.kt @@ -0,0 +1,27 @@ +// This file was automatically generated from Delay.kt by Knit tool. Do not edit. +package kotlinx.coroutines.examples.exampleDelay02 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.time.Duration.Companion.milliseconds + +fun main() = runBlocking { + +flow { + emit(1) + delay(90) + emit(2) + delay(90) + emit(3) + delay(1010) + emit(4) + delay(1010) + emit(5) +}.debounce { + if (it == 1) { + 0L + } else { + 1000L + } +} +.toList().joinToString().let { println(it) } } diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-delay-03.kt b/kotlinx-coroutines-core/jvm/test/examples/example-delay-03.kt new file mode 100644 index 0000000000..f9c44c3d89 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/examples/example-delay-03.kt @@ -0,0 +1,16 @@ +// This file was automatically generated from Delay.kt by Knit tool. Do not edit. +package kotlinx.coroutines.examples.exampleDelay03 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.time.Duration.Companion.milliseconds + +fun main() = runBlocking { + +flow { + repeat(10) { + emit(it) + delay(110) + } +}.sample(200) +.toList().joinToString().let { println(it) } } diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-01.kt b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-01.kt new file mode 100644 index 0000000000..00c613fd1c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-01.kt @@ -0,0 +1,21 @@ +// This file was automatically generated from Delay.kt by Knit tool. Do not edit. +package kotlinx.coroutines.examples.exampleDelayDuration01 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.time.Duration.Companion.milliseconds + +fun main() = runBlocking { + +flow { + emit(1) + delay(90.milliseconds) + emit(2) + delay(90.milliseconds) + emit(3) + delay(1010.milliseconds) + emit(4) + delay(1010.milliseconds) + emit(5) +}.debounce(1000.milliseconds) +.toList().joinToString().let { println(it) } } diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-02.kt b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-02.kt new file mode 100644 index 0000000000..a75f251706 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-02.kt @@ -0,0 +1,27 @@ +// This file was automatically generated from Delay.kt by Knit tool. Do not edit. +package kotlinx.coroutines.examples.exampleDelayDuration02 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.time.Duration.Companion.milliseconds + +fun main() = runBlocking { + +flow { + emit(1) + delay(90.milliseconds) + emit(2) + delay(90.milliseconds) + emit(3) + delay(1010.milliseconds) + emit(4) + delay(1010.milliseconds) + emit(5) +}.debounce { + if (it == 1) { + 0.milliseconds + } else { + 1000.milliseconds + } +} +.toList().joinToString().let { println(it) } } diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-03.kt b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-03.kt new file mode 100644 index 0000000000..dbd4842a0a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/examples/example-delay-duration-03.kt @@ -0,0 +1,16 @@ +// This file was automatically generated from Delay.kt by Knit tool. Do not edit. +package kotlinx.coroutines.examples.exampleDelayDuration03 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.time.Duration.Companion.milliseconds + +fun main() = runBlocking { + +flow { + repeat(10) { + emit(it) + delay(110.milliseconds) + } +}.sample(200.milliseconds) +.toList().joinToString().let { println(it) } } diff --git a/kotlinx-coroutines-core/jvm/test/examples/example-timeout-duration-01.kt b/kotlinx-coroutines-core/jvm/test/examples/example-timeout-duration-01.kt new file mode 100644 index 0000000000..c336528c8b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/examples/example-timeout-duration-01.kt @@ -0,0 +1,30 @@ +// This file was automatically generated from Delay.kt by Knit tool. Do not edit. +package kotlinx.coroutines.examples.exampleTimeoutDuration01 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.time.Duration.Companion.milliseconds + +fun main() = runBlocking { + +flow { + emit(1) + delay(100) + emit(2) + delay(100) + emit(3) + delay(1000) + emit(4) +}.timeout(100.milliseconds).catch { exception -> + if (exception is TimeoutCancellationException) { + // Catch the TimeoutCancellationException emitted above. + // Emit desired item on timeout. + emit(-1) + } else { + // Throw other exceptions. + throw exception + } +}.onEach { + delay(300) // This will not cause a timeout +} +.toList().joinToString().let { println(it) } } diff --git a/kotlinx-coroutines-core/jvm/test/examples/test/FlowDelayTest.kt b/kotlinx-coroutines-core/jvm/test/examples/test/FlowDelayTest.kt new file mode 100644 index 0000000000..b0bc05fc8b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/examples/test/FlowDelayTest.kt @@ -0,0 +1,56 @@ +// This file was automatically generated from Delay.kt by Knit tool. Do not edit. +package kotlinx.coroutines.examples.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class FlowDelayTest { + @Test + fun testExampleDelay01() { + test("ExampleDelay01") { kotlinx.coroutines.examples.exampleDelay01.main() }.verifyLines( + "3, 4, 5" + ) + } + + @Test + fun testExampleDelay02() { + test("ExampleDelay02") { kotlinx.coroutines.examples.exampleDelay02.main() }.verifyLines( + "1, 3, 4, 5" + ) + } + + @Test + fun testExampleDelayDuration01() { + test("ExampleDelayDuration01") { kotlinx.coroutines.examples.exampleDelayDuration01.main() }.verifyLines( + "3, 4, 5" + ) + } + + @Test + fun testExampleDelayDuration02() { + test("ExampleDelayDuration02") { kotlinx.coroutines.examples.exampleDelayDuration02.main() }.verifyLines( + "1, 3, 4, 5" + ) + } + + @Test + fun testExampleDelay03() { + test("ExampleDelay03") { kotlinx.coroutines.examples.exampleDelay03.main() }.verifyLines( + "1, 3, 5, 7, 9" + ) + } + + @Test + fun testExampleDelayDuration03() { + test("ExampleDelayDuration03") { kotlinx.coroutines.examples.exampleDelayDuration03.main() }.verifyLines( + "1, 3, 5, 7, 9" + ) + } + + @Test + fun testExampleTimeoutDuration01() { + test("ExampleTimeoutDuration01") { kotlinx.coroutines.examples.exampleTimeoutDuration01.main() }.verifyLines( + "1, 2, 3, -1" + ) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/CoroutineExceptionHandlerJvmTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/CoroutineExceptionHandlerJvmTest.kt new file mode 100644 index 0000000000..2e3519902c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/CoroutineExceptionHandlerJvmTest.kt @@ -0,0 +1,51 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class CoroutineExceptionHandlerJvmTest : TestBase() { + + private val exceptionHandler = Thread.getDefaultUncaughtExceptionHandler() + private lateinit var caughtException: Throwable + + @Before + fun setUp() { + Thread.setDefaultUncaughtExceptionHandler({ _, e -> caughtException = e }) + } + + @After + fun tearDown() { + Thread.setDefaultUncaughtExceptionHandler(exceptionHandler) + } + + @Test + fun testFailingHandler() = runBlocking { + expect(1) + val job = GlobalScope.launch(CoroutineExceptionHandler { _, _ -> throw AssertionError() }) { + expect(2) + throw TestException() + } + + job.join() + assertIs(caughtException) + assertIs(caughtException.cause) + assertIs(caughtException.suppressed[0]) + + finish(3) + } + + @Test + fun testLastDitchHandlerContainsContextualInformation() = runBlocking { + expect(1) + GlobalScope.launch(CoroutineName("last-ditch")) { + expect(2) + throw TestException() + }.join() + assertIs(caughtException) + assertContains(caughtException.suppressed[0].toString(), "last-ditch") + finish(3) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/FlowSuppressionTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/FlowSuppressionTest.kt new file mode 100644 index 0000000000..8af4937e15 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/FlowSuppressionTest.kt @@ -0,0 +1,71 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class FlowSuppressionTest : TestBase() { + @Test + fun testSuppressionForPrimaryException() = runTest { + val flow = flow { + try { + emit(1) + } finally { + throw TestException() + } + }.catch { expectUnreached() }.onEach { throw TestException2() } + + try { + flow.collect() + } catch (e: Throwable) { + assertIs(e) + assertIs(e.suppressed[0]) + } + } + + @Test + fun testSuppressionForPrimaryExceptionRetry() = runTest { + val flow = flow { + try { + emit(1) + } finally { + throw TestException() + } + }.retry { expectUnreached(); true }.onEach { throw TestException2() } + + try { + flow.collect() + } catch (e: Throwable) { + assertIs(e) + assertIs(e.suppressed[0]) + + } + } + + @Test + fun testCancellationSuppression() = runTest { + val flow = flow { + try { + expect(1) + emit(1) + } finally { + expect(3) + throw CancellationException("") + } + }.catch { expectUnreached() }.onEach { + expect(2) + throw TestException("") + } + + try { + flow.collect() + } catch (e: Throwable) { + assertIs(e) + assertIs(e.suppressed[0]) + } + finish(4) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/JobBasicCancellationTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/JobBasicCancellationTest.kt new file mode 100644 index 0000000000..7ea6c4a898 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/JobBasicCancellationTest.kt @@ -0,0 +1,156 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import java.io.* +import kotlin.test.* + +/* + * Basic checks that check that cancellation more or less works, + * parent is not cancelled on child cancellation and launch {}, Job(), async {} and + * CompletableDeferred behave properly + */ + +@Suppress("DEPRECATION") // cancel(cause) +class JobBasicCancellationTest : TestBase() { + + @Test + fun testJobCancelChild() = runTest { + val parent = launch { + expect(1) + val child = launch { + expect(2) + } + + yield() + expect(3) + child.cancel() + child.join() + expect(4) + } + + parent.join() + finish(5) + } + + @Test + fun testJobCancelChildAtomic() = runTest { + val parent = launch { + expect(1) + val child = launch(start = CoroutineStart.ATOMIC) { + expect(3) + } + + expect(2) + child.cancel() + child.join() + yield() + expect(4) + } + + parent.join() + assertTrue(parent.isCompleted) + assertFalse(parent.isCancelled) + finish(5) + } + + @Test + fun testAsyncCancelChild() = runTest { + val parent = async { + expect(1) + val child = async { + expect(2) + } + + yield() + expect(3) + child.cancel() + child.await() + expect(4) + } + + parent.await() + finish(5) + } + + @Test + fun testAsyncCancelChildAtomic() = runTest { + val parent = async { + expect(1) + val child = async(start = CoroutineStart.ATOMIC) { + expect(3) + } + + expect(2) + child.cancel() + child.join() + expect(4) + } + + parent.await() + finish(5) + } + + @Test + fun testNestedAsyncFailure() = runTest { + val deferred = async(NonCancellable) { + val nested = async(NonCancellable) { + expect(3) + throw IOException() + } + + expect(2) + yield() + expect(4) + nested.await() + } + + expect(1) + try { + deferred.await() + } catch (e: IOException) { + finish(5) + } + } + + @Test + fun testCancelJobImpl() = runTest { + val parent = launch { + expect(1) + val child = Job(coroutineContext[Job]) + expect(2) + child.cancel() // cancel without cause -- should not cancel us (parent) + child.join() + expect(3) + } + parent.join() + finish(4) + } + + @Test + fun cancelCompletableDeferred() = runTest { + val parent = launch { + expect(1) + val child = CompletableDeferred(coroutineContext[Job]) + expect(2) + child.cancel() // cancel without cause -- should not cancel us (parent) + child.join() + expect(3) + } + + parent.join() + finish(4) + } + + @Test + fun testConsecutiveCancellation() { + val deferred = CompletableDeferred() + assertTrue(deferred.completeExceptionally(IndexOutOfBoundsException())) + assertFalse(deferred.completeExceptionally(AssertionError())) // second is too late + val cause = deferred.getCancellationException().cause!! + assertIs(cause) + assertNull(cause.cause) + assertTrue(cause.suppressed.isEmpty()) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/JobExceptionHandlingTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/JobExceptionHandlingTest.kt new file mode 100644 index 0000000000..d0310204b8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/JobExceptionHandlingTest.kt @@ -0,0 +1,377 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CoroutineStart.* +import kotlinx.coroutines.testing.exceptions.* +import org.junit.Test +import java.io.* +import kotlin.test.* + +@Suppress("DEPRECATION") // cancel(cause) +class JobExceptionHandlingTest : TestBase() { + + @Test + fun testChildException() { + /* + * Root parent: JobImpl() + * Child: throws ISE + * Result: ISE in exception handler + */ + val exception = captureExceptionsRun { + val job = Job() + launch(job, start = ATOMIC) { + expect(2) + throw IllegalStateException() + } + + expect(1) + job.join() + finish(3) + } + + checkException(exception) + } + + @Test + fun testAsyncCancellationWithCauseAndParent() = runTest { + val parent = Job() + val deferred = async(parent) { + expect(2) + delay(Long.MAX_VALUE) + } + + expect(1) + yield() + parent.completeExceptionally(IOException()) + try { + deferred.await() + expectUnreached() + } catch (e: CancellationException) { + assertTrue(e.suppressed.isEmpty()) + assertTrue(e.cause?.suppressed?.isEmpty() ?: false) + finish(3) + } + } + + @Test + fun testAsyncCancellationWithCauseAndParentDoesNotTriggerHandling() = runTest { + val parent = Job() + val job = launch(parent) { + expect(2) + delay(Long.MAX_VALUE) + } + + expect(1) + yield() + parent.completeExceptionally(IOException()) + job.join() + finish(3) + } + + @Test + fun testExceptionDuringCancellation() { + /* + * Root parent: JobImpl() + * Launcher: cancels job + * Child: throws ISE + * Result: ISE in exception handler + * + * Github issue #354 + */ + val exception = captureExceptionsRun { + val job = Job() + val child = launch(job, start = ATOMIC) { + expect(2) + throw IllegalStateException() + } + + expect(1) + job.cancelAndJoin() + assert(child.isCompleted && !child.isActive) + finish(3) + } + + checkException(exception) + } + + @Test + fun testExceptionOnChildCancellation() { + /* + * Root parent: JobImpl() + * Child: launch inner child and cancels parent + * Inner child: throws AE + * Result: AE in exception handler + */ + val exception = captureExceptionsRun { + val job = Job() + launch(job) { + expect(2) // <- child is launched successfully + + launch { + expect(3) // <- child's child is launched successfully + try { + yield() + } catch (e: CancellationException) { + throw ArithmeticException() + } + } + + yield() + expect(4) + job.cancel() + } + + expect(1) + job.join() + finish(5) + } + + checkException(exception) + } + + @Test + fun testInnerChildException() { + /* + * Root parent: JobImpl() + * Launcher: launch child and cancel root + * Child: launch nested child atomically and yields + * Inner child: throws AE + * Result: AE + */ + val exception = captureExceptionsRun { + val job = Job() + launch(job, start = ATOMIC) { + expect(2) + launch(start = ATOMIC) { + expect(3) // <- child's child is launched successfully + throw ArithmeticException() + } + + yield() // will throw cancellation exception + } + + expect(1) + job.cancelAndJoin() + finish(4) + } + + checkException(exception) + } + + @Test + fun testExceptionOnChildCancellationWithCause() { + /* + * Root parent: JobImpl() + * Child: launch inner child and cancels parent with IOE + * Inner child: throws AE + * Result: IOE with suppressed AE + */ + val exception = captureExceptionsRun { + val job = Job() + launch(job) { + expect(2) // <- child is launched successfully + launch { + expect(3) // <- child's child is launched successfully + try { + yield() + } catch (e: CancellationException) { + throw ArithmeticException() + } + } + + yield() + expect(4) + job.completeExceptionally(IOException()) + } + + expect(1) + job.join() + finish(5) + } + + checkException(exception) + } + + @Test + fun testMultipleChildrenThrowAtomically() { + /* + * Root parent: JobImpl() + * Launcher: launches child + * Child: launch 3 children, each of them throws an exception (AE, IOE, IAE) and calls delay() + * Result: AE with suppressed IOE and IAE + */ + val exception = captureExceptionsRun { + val job = Job() + launch(job, start = ATOMIC) { + expect(2) + launch(start = ATOMIC) { + expect(3) + throw ArithmeticException() + } + + launch(start = ATOMIC) { + expect(4) + throw IOException() + } + + launch(start = ATOMIC) { + expect(5) + throw IllegalArgumentException() + } + + delay(Long.MAX_VALUE) + } + + expect(1) + job.join() + finish(6) + } + + assertIs(exception) + assertNull(exception.cause) + val suppressed = exception.suppressed + assertEquals(2, suppressed.size) + assertIs(suppressed[0]) + assertIs(suppressed[1]) + } + + @Test + fun testMultipleChildrenAndParentThrowsAtomic() { + /* + * Root parent: JobImpl() + * Launcher: launches child + * Child: launch 2 children (each of them throws an exception (IOE, IAE)), throws AE + * Result: AE with suppressed IOE and IAE + */ + val exception = captureExceptionsRun { + val job = Job() + launch(job, start = ATOMIC) { + expect(2) + launch(start = ATOMIC) { + expect(3) + throw IOException() + } + + launch(start = ATOMIC) { + expect(4) + throw IllegalArgumentException() + } + + throw AssertionError() + } + + expect(1) + job.join() + finish(5) + } + + assertIs(exception) + val suppressed = exception.suppressed + assertEquals(2, suppressed.size) + assertIs(suppressed[0]) + assertIs(suppressed[1]) + } + + @Test + fun testExceptionIsHandledOnce() = runTest(unhandled = listOf { e -> e is TestException }) { + val job = Job() + val j1 = launch(job) { + expect(1) + delay(Long.MAX_VALUE) + } + + val j2 = launch(job) { + expect(2) + throw TestException() + } + + joinAll(j1 ,j2) + finish(3) + } + + @Test + fun testCancelledParent() = runTest { + expect(1) + val parent = Job() + parent.completeExceptionally(TestException()) + launch(parent) { + expectUnreached() + }.join() + finish(2) + } + + @Test + fun testExceptionIsNotReported() = runTest { + try { + expect(1) + coroutineScope { + val job = Job(coroutineContext[Job]) + launch(job) { + throw TestException() + } + } + expectUnreached() + } catch (e: TestException) { + finish(2) + } + } + + @Test + fun testExceptionIsNotReportedTripleChain() = runTest { + try { + expect(1) + coroutineScope { + val job = Job(Job(Job(coroutineContext[Job]))) + launch(job) { + throw TestException() + } + } + expectUnreached() + } catch (e: TestException) { + finish(2) + } + } + + @Test + fun testAttachToCancelledJob() = runTest(unhandled = listOf({ e -> e is TestException })) { + val parent = launch(Job()) { + throw TestException() + }.apply { join() } + + launch(parent) { expectUnreached() } + launch(Job(parent)) { expectUnreached() } + } + + @Test + fun testBadException() = runTest(unhandled = listOf({e -> e is BadException})) { + val job = launch(Job()) { + expect(2) + launch { + expect(3) + throw BadException() + } + + launch(start = ATOMIC) { + expect(4) + throw BadException() + } + + yield() + BadException() + } + + expect(1) + yield() + yield() + expect(5) + job.join() + finish(6) + } + + private class BadException : Exception() { + override fun hashCode(): Int { + throw AssertionError() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/JobExceptionsStressTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/JobExceptionsStressTest.kt new file mode 100644 index 0000000000..4634c4f775 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/JobExceptionsStressTest.kt @@ -0,0 +1,64 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.exceptions.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class JobExceptionsStressTest : TestBase() { + + private val executor = newFixedThreadPoolContext(5, "JobExceptionsStressTest") + + @After + fun tearDown() { + executor.close() + } + + @Test + fun testMultipleChildrenThrows() { + /* + * Root parent: launched job + * Owner: launch 3 children, every of it throws an exception, and then call delay() + * Result: one of the exceptions with the rest two as suppressed + */ + repeat(1000 * stressTestMultiplier) { + val exception = captureExceptionsRun(executor) { + val barrier = CyclicBarrier(4) + val job = launch(NonCancellable) { + launch(start = CoroutineStart.ATOMIC) { + barrier.await() + throw TestException1() + } + launch(start = CoroutineStart.ATOMIC) { + barrier.await() + throw TestException2() + } + launch(start = CoroutineStart.ATOMIC) { + barrier.await() + throw TestException3() + } + delay(1000) // to avoid OutOfMemory errors.... + } + barrier.await() + job.join() + } + val classes = mutableSetOf( + TestException1::class, + TestException2::class, + TestException3::class + ) + val suppressedExceptions = exception.suppressed.toSet() + assertTrue(classes.remove(exception::class), + "Failed to remove ${exception::class} from $suppressedExceptions" + ) + for (throwable in suppressedExceptions.toSet()) { // defensive copy + assertTrue(classes.remove(throwable::class), + "Failed to remove ${throwable::class} from $suppressedExceptions") + } + assertTrue(classes.isEmpty(), "Expected all exception to be present, but following exceptions are missing: $classes") + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/JobNestedExceptionsTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/JobNestedExceptionsTest.kt new file mode 100644 index 0000000000..8b0a32b3bc --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/JobNestedExceptionsTest.kt @@ -0,0 +1,116 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.exceptions.* +import org.junit.Test +import java.io.* +import kotlin.test.* + +class JobNestedExceptionsTest : TestBase() { + + @Test + fun testExceptionUnwrapping() { + val exception = captureExceptionsRun { + val job = Job() + launch(job) { + expect(2) + launch { + launch { + launch { + throw IllegalStateException() + } + } + } + } + + expect(1) + job.join() + finish(3) + } + + checkException(exception) + checkCycles(exception) + } + + @Test + fun testExceptionUnwrappingWithSuspensions() { + val exception = captureExceptionsRun { + val job = Job() + launch(job) { + expect(2) + launch { + launch { + launch { + launch { + throw IOException() + } + yield() + } + delay(Long.MAX_VALUE) + } + delay(Long.MAX_VALUE) + } + delay(Long.MAX_VALUE) + } + + expect(1) + job.join() + finish(3) + } + + assertIs(exception) + } + + @Test + fun testNestedAtomicThrow() { + val exception = captureExceptionsRun { + expect(1) + val job = launch(NonCancellable + CoroutineName("outer"), start = CoroutineStart.ATOMIC) { + expect(2) + launch(CoroutineName("nested"), start = CoroutineStart.ATOMIC) { + expect(4) + throw IOException() + } + expect(3) + throw ArithmeticException() + } + job.join() + finish(5) + } + assertIs(exception, "Found $exception") + checkException(exception.suppressed[0]) + } + + @Test + fun testChildThrowsDuringCompletion() { + val exception = captureExceptionsRun { + expect(1) + val job = launch(NonCancellable + CoroutineName("outer"), start = CoroutineStart.ATOMIC) { + expect(2) + launch(CoroutineName("nested"), start = CoroutineStart.ATOMIC) { + expect(4) + launch(CoroutineName("nested2"), start = CoroutineStart.ATOMIC) { + // This child attaches to the parent and throws after parent completion + expect(6) + throw NullPointerException() + } + expect(5) + throw IOException() + } + expect(3) + throw ArithmeticException() + } + + job.join() + finish(7) + } + + assertIs(exception, "Exception is $exception") + val suppressed = exception.suppressed + val ioe = suppressed[0] + assertIs(ioe) + checkException(ioe.suppressed[0]) + checkCycles(exception) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/ProduceExceptionsTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/ProduceExceptionsTest.kt new file mode 100644 index 0000000000..98f080e66f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/ProduceExceptionsTest.kt @@ -0,0 +1,166 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.junit.Test +import kotlin.test.* + +class ProduceExceptionsTest : TestBase() { + + @Test + fun testFailingProduce() = runTest(unhandled = listOf({ e -> e is TestException })) { + expect(1) + val producer = produce(Job()) { + expect(2) + try { + yield() + } finally { + expect(3) + throw TestException() + + } + } + + yield() + producer.cancel() + yield() + finish(4) + } + + @Test + fun testSuppressedExceptionUncaught() = + runTest(unhandled = listOf({ e -> e is TestException && e.suppressed[0] is TestException2 })) { + val produce = produce(Job()) { + launch(start = CoroutineStart.ATOMIC) { + throw TestException() + } + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() + } + } + + yield() + produce.cancel() + } + + @Test + fun testSuppressedException() = runTest { + val produce = produce(NonCancellable) { + launch(start = CoroutineStart.ATOMIC) { + throw TestException() // child coroutine fails + } + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() // but parent throws another exception while cleaning up + } + } + try { + produce.receive() + expectUnreached() + } catch (e: TestException) { + assertIs(e.suppressed[0]) + } + } + + @Test + fun testCancelProduceChannel() = runTest { + var channel: ReceiveChannel? = null + channel = produce { + expect(2) + channel!!.cancel() + try { + send(1) + } catch (e: CancellationException) { + expect(3) + throw e + } + } + + expect(1) + yield() + try { + channel.receive() + } catch (e: CancellationException) { + assertTrue(e.suppressed.isEmpty()) + finish(4) + } + } + + @Test + fun testCancelProduceChannelWithException() = runTest { + var channel: ReceiveChannel? = null + channel = produce(NonCancellable) { + expect(2) + channel!!.cancel(TestCancellationException()) + try { + send(1) + // Not a ClosedForSendException + } catch (e: TestCancellationException) { + expect(3) + throw e + } + } + + expect(1) + yield() + try { + channel.receive() + } catch (e: TestCancellationException) { + assertTrue(e.suppressed.isEmpty()) + finish(4) + } + } + + @Test + fun testCancelChannelWithJob() = runTest { + val job = Job() + val channel = produce(job) { + expect(2) + job.cancel() + try { + send(1) + } catch (e: CancellationException) { + expect(3) + throw e + } + } + + expect(1) + yield() + try { + channel.receive() + } catch (e: CancellationException) { + assertTrue(e.suppressed.isEmpty()) + finish(4) + } + } + + @Test + fun testCancelChannelWithJobWithException() = runTest { + val job = Job() + val channel = produce(job) { + expect(2) + job.completeExceptionally(TestException2()) + try { + send(1) + } catch (e: CancellationException) { // Not a TestException2 + expect(3) + throw e + } + } + + expect(1) + yield() + try { + channel.receive() + } catch (e: CancellationException) { + // RECOVER_STACK_TRACES + assertIs(e.cause?.cause) + finish(4) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryChannelsTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryChannelsTest.kt new file mode 100644 index 0000000000..f6d1986bf1 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryChannelsTest.kt @@ -0,0 +1,150 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.junit.* +import org.junit.rules.* +import kotlin.coroutines.* + +class StackTraceRecoveryChannelsTest : TestBase() { + + @get:Rule + val name = TestName() + + @Test + fun testReceiveFromChannel() = runTest { + val channel = Channel() + val job = launch { + expect(2) + channel.close(RecoverableTestException()) + } + + expect(1) + channelReceive(channel) + expect(3) + job.join() + finish(4) + } + + @Test + fun testReceiveFromClosedChannel() = runTest { + val channel = Channel() + channel.close(RecoverableTestException()) + channelReceive(channel) + } + + @Test + fun testSendToClosedChannel() = runTest { + val channel = Channel() + channel.close(RecoverableTestException()) + channelSend(channel) + } + + @Test + fun testSendToChannel() = runTest { + val channel = Channel() + val job = launch { + expect(2) + channel.cancel() + } + + expect(1) + channelSend(channel) + expect(3) + job.join() + finish(4) + } + + private suspend fun channelReceive(channel: Channel) = channelOp { channel.receive() } + + private suspend inline fun channelOp(block: () -> Unit) { + try { + yield() + block() + expectUnreached() + } catch (e: RecoverableTestException) { + verifyStackTrace("channels/${name.methodName}", e) + } + } + + private suspend fun channelSend(channel: Channel) { + try { + yield() + channel.send(1) + expectUnreached() + } catch (e: Exception) { + verifyStackTrace("channels/${name.methodName}", e) + } + } + + @Test + fun testOfferWithCurrentContext() = runTest { + val channel = Channel() + channel.close(RecoverableTestException()) + + try { + channel.sendWithContext(coroutineContext) + } catch (e: RecoverableTestException) { + verifyStackTrace("channels/${name.methodName}", e) + } + } + + @Test + fun testOfferWithContextWrapped() = runTest { + val channel = Channel() + channel.close(RecoverableTestException()) + try { + channel.sendWithContext(wrapperDispatcher(coroutineContext)) + } catch (e: Exception) { + verifyStackTrace("channels/${name.methodName}", e) + } + } + + @Test + fun testOfferFromScope() = runTest { + val channel = Channel() + channel.close(RecoverableTestException()) + + try { + channel.sendFromScope() + } catch (e: Exception) { + verifyStackTrace("channels/${name.methodName}", e) + } + } + + // Slow path via suspending send + @Test + fun testSendFromScope() = runTest { + val channel = Channel() + val deferred = async { + try { + expect(1) + channel.sendFromScope() + } catch (e: Exception) { + verifyStackTrace("channels/${name.methodName}", e) + } + } + + yield() + expect(2) + // Cancel is an analogue of `produce` failure, just a shorthand + channel.cancel(RecoverableTestCancellationException()) + finish(3) + deferred.await() + } + + private suspend fun Channel.sendWithContext(ctx: CoroutineContext) = withContext(ctx) { + sendInChannel() + yield() // TCE + } + + private suspend fun Channel.sendInChannel() { + send(42) + yield() // TCE + } + + private suspend fun Channel.sendFromScope() = coroutineScope { + sendWithContext(wrapperDispatcher(coroutineContext)) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryCustomExceptionsTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryCustomExceptionsTest.kt new file mode 100644 index 0000000000..ea0a8681f8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryCustomExceptionsTest.kt @@ -0,0 +1,162 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.junit.Test +import kotlin.test.* + +@Suppress("UNREACHABLE_CODE", "UNUSED", "UNUSED_PARAMETER") +class StackTraceRecoveryCustomExceptionsTest : TestBase() { + + internal class NonCopyable(val customData: Int) : Throwable() { + // Bait + public constructor(cause: Throwable) : this(42) + } + + internal class Copyable(val customData: Int) : Throwable(), CopyableThrowable { + // Bait + public constructor(cause: Throwable) : this(42) + + override fun createCopy(): Copyable { + val copy = Copyable(customData) + copy.initCause(this) + return copy + } + } + + @Test + fun testStackTraceNotRecovered() = runTest { + try { + withContext(wrapperDispatcher(coroutineContext)) { + throw NonCopyable(239) + } + expectUnreached() + } catch (e: NonCopyable) { + assertEquals(239, e.customData) + assertNull(e.cause) + } + } + + @Test + fun testStackTraceRecovered() = runTest { + try { + withContext(wrapperDispatcher(coroutineContext)) { + throw Copyable(239) + } + expectUnreached() + } catch (e: Copyable) { + assertEquals(239, e.customData) + val cause = e.cause + assertIs(cause) + assertEquals(239, cause.customData) + } + } + + internal class WithDefault(message: String = "default") : Exception(message) + + @Test + fun testStackTraceRecoveredWithCustomMessage() = runTest { + try { + withContext(wrapperDispatcher(coroutineContext)) { + throw WithDefault("custom") + } + expectUnreached() + } catch (e: WithDefault) { + assertEquals("custom", e.message) + val cause = e.cause + assertIs(cause) + assertEquals("custom", cause.message) + } + } + + class WrongMessageException(token: String) : RuntimeException("Token $token") + + @Test + fun testWrongMessageException() = runTest { + val result = runCatching { + coroutineScope { + throw WrongMessageException("OK") + } + } + val ex = result.exceptionOrNull() ?: error("Expected to fail") + assertIs(ex) + assertEquals("Token OK", ex.message) + } + + @Test + fun testNestedExceptionWithCause() = runTest { + val result = runCatching { + coroutineScope { + throw NestedException(IllegalStateException("ERROR")) + } + } + val ex = result.exceptionOrNull() ?: error("Expected to fail") + assertIs(ex) + assertIs(ex.cause) + val originalCause = ex.cause?.cause + assertIs(originalCause) + assertEquals("ERROR", originalCause.message) + } + + class NestedException : RuntimeException { + constructor(cause: Throwable) : super(cause) + constructor() : super() + } + + @Test + fun testWrongMessageExceptionInChannel() = runTest { + val result = produce(SupervisorJob() + Dispatchers.Unconfined) { + throw WrongMessageException("OK") + } + val ex = runCatching { + @Suppress("ControlFlowWithEmptyBody") + for (unit in result) { + // Iterator has a special code path + } + }.exceptionOrNull() ?: error("Expected to fail") + assertIs(ex) + assertEquals("Token OK", ex.message) + } + + class CopyableWithCustomMessage( + message: String?, + cause: Throwable? = null + ) : RuntimeException(message, cause), + CopyableThrowable { + + override fun createCopy(): CopyableWithCustomMessage { + return CopyableWithCustomMessage("Recovered: [$message]", cause) + } + } + + @Test + fun testCustomCopyableMessage() = runTest { + val result = runCatching { + coroutineScope { + throw CopyableWithCustomMessage("OK") + } + } + val ex = result.exceptionOrNull() ?: error("Expected to fail") + assertIs(ex) + assertEquals("Recovered: [OK]", ex.message) + } + + @Test + fun testTryCopyThrows() = runTest { + class FailingException : Exception(), CopyableThrowable { + override fun createCopy(): FailingException? { + TODO("Not yet implemented") + } + } + + val e = FailingException() + val result = runCatching { + coroutineScope { + throw e + } + } + + assertSame(e, result.exceptionOrNull()) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedScopesTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedScopesTest.kt new file mode 100644 index 0000000000..0422df98ec --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedScopesTest.kt @@ -0,0 +1,99 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import kotlin.coroutines.* + +class StackTraceRecoveryNestedScopesTest : TestBase() { + + private val TEST_MACROS = "TEST_NAME" + + private val expectedTrace = "kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.failure(StackTraceRecoveryNestedScopesTest.kt:9)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.access\$failure(StackTraceRecoveryNestedScopesTest.kt:7)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$createFailingAsync\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:12)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$callWithTimeout\$2.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:23)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$callCoroutineScope\$2.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:29)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$$TEST_MACROS\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:36)\n" + + "Caused by: kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.failure(StackTraceRecoveryNestedScopesTest.kt:9)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.access\$failure(StackTraceRecoveryNestedScopesTest.kt:7)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$createFailingAsync\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:12)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)" + + private fun failure(): String = throw RecoverableTestException() + + private fun CoroutineScope.createFailingAsync() = async { + failure() + } + + private suspend fun callWithContext(doYield: Boolean) = withContext(wrapperDispatcher(coroutineContext)) { + if (doYield) yield() + createFailingAsync().await() + yield() + } + + private suspend fun callWithTimeout(doYield: Boolean) = withTimeout(Long.MAX_VALUE) { + if (doYield) yield() + callWithContext(doYield) + yield() + } + + private suspend fun callCoroutineScope(doYield: Boolean) = coroutineScope { + if (doYield) yield() + callWithTimeout(doYield) + yield() + } + + @Test + fun testNestedScopes() = runTest { + try { + callCoroutineScope(false) + } catch (e: Exception) { + verifyStackTrace(e, expectedTrace.replace(TEST_MACROS, "testNestedScopes")) + } + } + + @Test + fun testNestedScopesYield() = runTest { + try { + callCoroutineScope(true) + } catch (e: Exception) { + verifyStackTrace(e, expectedTrace.replace(TEST_MACROS, "testNestedScopesYield")) + } + } + + @Test + fun testAwaitNestedScopes() = runTest { + val deferred = async(NonCancellable) { + callCoroutineScope(false) + } + + verifyAwait(deferred) + } + + private suspend fun verifyAwait(deferred: Deferred) { + try { + deferred.await() + } catch (e: Exception) { + verifyStackTrace(e, + "kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.failure(StackTraceRecoveryNestedScopesTest.kt:23)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.access\$failure(StackTraceRecoveryNestedScopesTest.kt:7)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$createFailingAsync\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:26)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$callWithTimeout\$2.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:37)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$callCoroutineScope\$2.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:43)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$testAwaitNestedScopes\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:68)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.verifyAwait(StackTraceRecoveryNestedScopesTest.kt:76)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$testAwaitNestedScopes\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:71)\n" + + "Caused by: kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.failure(StackTraceRecoveryNestedScopesTest.kt:23)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest.access\$failure(StackTraceRecoveryNestedScopesTest.kt:7)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryNestedScopesTest\$createFailingAsync\$1.invokeSuspend(StackTraceRecoveryNestedScopesTest.kt:26)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)") + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedTest.kt new file mode 100644 index 0000000000..28875c9a30 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryNestedTest.kt @@ -0,0 +1,84 @@ +@file:Suppress("DeferredResultUnused") + +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class StackTraceRecoveryNestedTest : TestBase() { + + @Test + fun testNestedAsync() = runTest { + val rootAsync = async(NonCancellable) { + expect(1) + + // Just a noise for unwrapping + async { + expect(2) + delay(Long.MAX_VALUE) + } + + // Do not catch, fail on cancellation + async { + expect(3) + async { + expect(4) + delay(Long.MAX_VALUE) + } + + async { + expect(5) + // 1) await(), catch, verify and rethrow + try { + val nested = async { + expect(6) + throw RecoverableTestException() + } + + nested.awaitNested() + } catch (e: RecoverableTestException) { + expect(7) + e.verifyException( + "await\$suspendImpl", + "awaitNested", + "\$testNestedAsync\$1\$rootAsync\$1\$2\$2.invokeSuspend" + ) + // Just rethrow it + throw e + } + } + } + } + + try { + rootAsync.awaitRootLevel() + } catch (e: RecoverableTestException) { + e.verifyException("awaitRootLevel") + finish(8) + } + } + + private suspend fun Deferred<*>.awaitRootLevel() { + await() + assertTrue(true) + } + + private suspend fun Deferred<*>.awaitNested() { + await() + assertTrue(true) + } + + private fun RecoverableTestException.verifyException(vararg expectedTraceElements: String) { + // It is "recovered" only once + assertEquals(1, depth()) + val stacktrace = stackTrace.map { it.methodName }.toSet() + assertTrue(expectedTraceElements.all { stacktrace.contains(it) }) + } + + private fun Throwable.depth(): Int { + val cause = cause ?: return 0 + return 1 + cause.depth() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryResumeModeTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryResumeModeTest.kt new file mode 100644 index 0000000000..1039d2951d --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryResumeModeTest.kt @@ -0,0 +1,146 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.junit.* +import kotlin.coroutines.* +import org.junit.rules.TestName +import org.junit.Rule + +class StackTraceRecoveryResumeModeTest : TestBase() { + + @get:Rule + val testName = TestName() + + @Test + fun testUnconfined() = runTest { + testResumeModeFastPath(Dispatchers.Unconfined) + } + + @Test + fun testNestedUnconfined() = runTest { + withContext(Dispatchers.Unconfined) { + testResumeModeFastPath(Dispatchers.Unconfined) + } + } + + @Test + fun testNestedUnconfinedChangedContext() = runTest { + withContext(Dispatchers.Unconfined) { + testResumeModeFastPath(CoroutineName("Test")) + } + } + + @Test + fun testEventLoopDispatcher() = runTest { + testResumeModeFastPath(wrapperDispatcher()) + } + + @Test + fun testNestedEventLoopDispatcher() = runTest { + val dispatcher = wrapperDispatcher() + withContext(dispatcher) { + testResumeModeFastPath(dispatcher) + } + } + + @Test + fun testNestedEventLoopChangedContext() = runTest { + withContext(wrapperDispatcher()) { + testResumeModeFastPath(CoroutineName("Test")) + } + } + + private suspend fun testResumeModeFastPath(context: CoroutineContext) { + try { + val channel = Channel() + channel.close(RecoverableTestException()) + doFastPath(context, channel) + } catch (e: Throwable) { + verifyStackTrace("resume-mode/${testName.methodName}", e) + } + } + + private suspend fun doFastPath(context: CoroutineContext, channel: Channel) { + yield() + withContext(context, channel) + } + + private suspend fun withContext(context: CoroutineContext, channel: Channel) { + withContext(context) { + channel.receive() + yield() + } + } + + @Test + fun testUnconfinedSuspending() = runTest { + testResumeModeSuspending(Dispatchers.Unconfined) + } + + @Test + fun testNestedUnconfinedSuspending() = runTest { + withContext(Dispatchers.Unconfined) { + testResumeModeSuspending(Dispatchers.Unconfined) + } + } + + @Test + fun testNestedUnconfinedChangedContextSuspending() = runTest { + withContext(Dispatchers.Unconfined) { + testResumeModeSuspending(CoroutineName("Test")) + } + } + + @Test + fun testEventLoopDispatcherSuspending() = runTest { + testResumeModeSuspending(wrapperDispatcher()) + } + + @Test + fun testNestedEventLoopDispatcherSuspending() = runTest { + val dispatcher = wrapperDispatcher() + withContext(dispatcher) { + testResumeModeSuspending(dispatcher) + } + } + + @Test + fun testNestedEventLoopChangedContextSuspending() = runTest { + withContext(wrapperDispatcher()) { + testResumeModeSuspending(CoroutineName("Test")) + } + } + + private suspend fun testResumeModeSuspending(context: CoroutineContext) { + try { + val channel = Channel() + val latch = Channel() + GlobalScope.launch(coroutineContext) { + latch.receive() + expect(3) + channel.close(RecoverableTestException()) + } + doSuspendingPath(context, channel, latch) + } catch (e: Throwable) { + finish(4) + verifyStackTrace("resume-mode/${testName.methodName}", e) + } + } + + private suspend fun doSuspendingPath(context: CoroutineContext, channel: Channel, latch: Channel) { + yield() + withContext(context, channel, latch) + } + + private suspend fun withContext(context: CoroutineContext, channel: Channel, latch: Channel) { + withContext(context) { + expect(1) + latch.send(1) + expect(2) + channel.receive() + yield() + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoverySelectTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoverySelectTest.kt new file mode 100644 index 0000000000..0ac808483b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoverySelectTest.kt @@ -0,0 +1,68 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import org.junit.* +import org.junit.rules.* + +class StackTraceRecoverySelectTest : TestBase() { + + @get:Rule + val name = TestName() + + @Test + fun testSelectJoin() = runTest { + expect(1) + val result = runCatching { doSelect() } + expect(3) + verifyStackTrace("select/${name.methodName}", result.exceptionOrNull()!!) + finish(4) + } + + private suspend fun doSelect(): Int { + val job = CompletableDeferred(Unit) + return select { + job.onJoin { + yield() // Hide the stacktrace + expect(2) + throw RecoverableTestException() + } + } + } + + @Test + fun testSelectCompletedAwait() = runTest { + val deferred = CompletableDeferred() + deferred.completeExceptionally(RecoverableTestException()) + val result = runCatching { doSelectAwait(deferred) } + verifyStackTrace("select/${name.methodName}", result.exceptionOrNull()!!) + } + + private suspend fun doSelectAwait(deferred: Deferred): Int { + return select { + deferred.onAwait { + yield() // Hide the frame + 42 + } + } + } + + @Test + fun testSelectOnReceive() = runTest { + val c = Channel() + c.close() + val result = kotlin.runCatching { doSelectOnReceive(c) } + verifyStackTrace("select/${name.methodName}", result.exceptionOrNull()!!) + } + + private suspend fun doSelectOnReceive(c: Channel) { + // The channel is closed, should throw an exception + select { + c.onReceive { + expectUnreached() + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryTest.kt new file mode 100644 index 0000000000..169654d1a8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryTest.kt @@ -0,0 +1,262 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.intrinsics.* +import kotlinx.coroutines.testing.exceptions.* +import org.junit.Test +import java.lang.RuntimeException +import java.util.concurrent.* +import kotlin.concurrent.* +import kotlin.coroutines.* +import kotlin.test.* + +/* + * All stacktrace validation skips line numbers + */ +class StackTraceRecoveryTest : TestBase() { + + @Test + fun testAsync() = runTest { + fun createDeferred(depth: Int): Deferred<*> { + return if (depth == 0) { + async(coroutineContext + NonCancellable) { + throw ExecutionException(null) + } + } else { + createDeferred(depth - 1) + } + } + + val deferred = createDeferred(3) + val traces = listOf( + "java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testAsync\$1\$createDeferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:99)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.oneMoreNestedMethod(StackTraceRecoveryTest.kt:49)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.nestedMethod(StackTraceRecoveryTest.kt:44)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testAsync\$1.invokeSuspend(StackTraceRecoveryTest.kt:17)\n", + "Caused by: java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testAsync\$1\$createDeferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:21)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)\n" + ) + nestedMethod(deferred, *traces.toTypedArray()) + deferred.join() + } + + @Test + fun testCompletedAsync() = runTest { + val deferred = async(coroutineContext + NonCancellable) { + throw ExecutionException(null) + } + + deferred.join() + val stacktrace = listOf( + "java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCompletedAsync\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:44)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.oneMoreNestedMethod(StackTraceRecoveryTest.kt:81)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.nestedMethod(StackTraceRecoveryTest.kt:75)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCompletedAsync\$1.invokeSuspend(StackTraceRecoveryTest.kt:71)", + "Caused by: java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCompletedAsync\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:44)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)" + ) + nestedMethod(deferred, *stacktrace.toTypedArray()) + } + + private suspend fun nestedMethod(deferred: Deferred<*>, vararg traces: String) { + oneMoreNestedMethod(deferred, *traces) + assertTrue(true) // Prevent tail-call optimization + } + + private suspend fun oneMoreNestedMethod(deferred: Deferred<*>, vararg traces: String) { + try { + deferred.await() + expectUnreached() + } catch (e: ExecutionException) { + verifyStackTrace(e, *traces) + } + } + + @Test + fun testWithContext() = runTest { + val deferred = async(NonCancellable, start = CoroutineStart.LAZY) { + throw RecoverableTestException() + } + + outerMethod(deferred, + "kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testWithContext\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:143)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.innerMethod(StackTraceRecoveryTest.kt:158)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$outerMethod\$2.invokeSuspend(StackTraceRecoveryTest.kt:151)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.outerMethod(StackTraceRecoveryTest.kt:150)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testWithContext\$1.invokeSuspend(StackTraceRecoveryTest.kt:141)\n", + "Caused by: kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testWithContext\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:143)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)\n") + deferred.join() + } + + private suspend fun outerMethod(deferred: Deferred<*>, vararg traces: String) { + withContext(Dispatchers.IO) { + innerMethod(deferred, *traces) + } + + assertTrue(true) + } + + private suspend fun innerMethod(deferred: Deferred<*>, vararg traces: String) { + try { + deferred.await() + expectUnreached() + } catch (e: RecoverableTestException) { + verifyStackTrace(e, *traces) + } + } + + @Test + fun testCoroutineScope() = runTest { + val deferred = async(NonCancellable, start = CoroutineStart.LAZY) { + throw RecoverableTestException() + } + + outerScopedMethod(deferred, + "kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCoroutineScope\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:143)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.innerMethod(StackTraceRecoveryTest.kt:158)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$outerScopedMethod\$2\$1.invokeSuspend(StackTraceRecoveryTest.kt:193)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$outerScopedMethod\$2.invokeSuspend(StackTraceRecoveryTest.kt:151)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCoroutineScope\$1.invokeSuspend(StackTraceRecoveryTest.kt:141)\n", + "Caused by: kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCoroutineScope\$1\$deferred\$1.invokeSuspend(StackTraceRecoveryTest.kt:143)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)\n") + deferred.join() + } + + public class TrickyException() : Throwable() { + // To be sure ctor is never invoked + @Suppress("UNUSED", "UNUSED_PARAMETER") + private constructor(message: String, cause: Throwable): this() { + error("Should never be called") + } + + override fun initCause(cause: Throwable?): Throwable { + error("Can't call initCause") + } + } + + @Test + fun testThrowingInitCause() = runTest { + val deferred = async(NonCancellable) { + expect(2) + throw TrickyException() + } + + try { + expect(1) + deferred.await() + } catch (e: TrickyException) { + assertNull(e.cause) + finish(3) + } + } + + private suspend fun outerScopedMethod(deferred: Deferred<*>, vararg traces: String) = coroutineScope { + supervisorScope { + innerMethod(deferred, *traces) + assertTrue(true) + } + assertTrue(true) + } + + @Test + fun testSelfSuppression() = runTest { + try { + runBlocking { + val job = launch { + coroutineScope { + throw RecoverableTestException() + } + } + + job.join() + expectUnreached() + } + expectUnreached() + } catch (e: RecoverableTestException) { + checkCycles(e) + } + } + + + private suspend fun throws() { + yield() // TCE + throw RecoverableTestException() + } + + private suspend fun awaiter() { + val task = GlobalScope.async(Dispatchers.Default, start = CoroutineStart.LAZY) { throws() } + task.await() + yield() // TCE + } + + @Test + fun testNonDispatchedRecovery() { + val await = suspend { awaiter() } + + val barrier = CyclicBarrier(2) + var exception: Throwable? = null + + thread { + await.startCoroutineUnintercepted(Continuation(EmptyCoroutineContext) { + exception = it.exceptionOrNull() + barrier.await() + }) + } + + barrier.await() + val e = exception + assertNotNull(e) + verifyStackTrace(e, "kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.throws(StackTraceRecoveryTest.kt:280)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.access\$throws(StackTraceRecoveryTest.kt:20)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$throws\$1.invokeSuspend(StackTraceRecoveryTest.kt)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.awaiter(StackTraceRecoveryTest.kt:285)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testNonDispatchedRecovery\$await\$1.invokeSuspend(StackTraceRecoveryTest.kt:291)\n" + + "Caused by: kotlinx.coroutines.testing.RecoverableTestException") + } + + private class Callback(val cont: CancellableContinuation<*>) + + @Test + fun testCancellableContinuation() = runTest { + val channel = Channel(1) + launch { + try { + awaitCallback(channel) + } catch (e: Throwable) { + verifyStackTrace(e, "kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCancellableContinuation\$1.invokeSuspend(StackTraceRecoveryTest.kt:329)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest.awaitCallback(StackTraceRecoveryTest.kt:348)\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCancellableContinuation\$1\$1.invokeSuspend(StackTraceRecoveryTest.kt:322)\n" + + "Caused by: kotlinx.coroutines.testing.RecoverableTestException\n" + + "\tat kotlinx.coroutines.exceptions.StackTraceRecoveryTest\$testCancellableContinuation\$1.invokeSuspend(StackTraceRecoveryTest.kt:329)") + } + } + val callback = channel.receive() + callback.cont.resumeWithException(RecoverableTestException()) + } + + private suspend fun awaitCallback(channel: Channel) { + suspendCancellableCoroutine { cont -> + channel.trySend(Callback(cont)) + } + yield() // nop to make sure it is not a tail call + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryWithTimeoutTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryWithTimeoutTest.kt new file mode 100644 index 0000000000..6d573669bc --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/StackTraceRecoveryWithTimeoutTest.kt @@ -0,0 +1,85 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.rules.* + +class StackTraceRecoveryWithTimeoutTest : TestBase() { + + @get:Rule + val name = TestName() + + @Test + fun testStacktraceIsRecoveredFromSuspensionPoint() = runTest { + try { + outerWithTimeout() + } catch (e: TimeoutCancellationException) { + verifyStackTrace("timeout/${name.methodName}", e) + } + } + + private suspend fun outerWithTimeout() { + withTimeout(200) { + suspendForever() + } + expectUnreached() + } + + private suspend fun suspendForever() { + hang { } + expectUnreached() + } + + @Test + fun testStacktraceIsRecoveredFromLexicalBlockWhenTriggeredByChild() = runTest { + try { + outerChildWithTimeout() + } catch (e: TimeoutCancellationException) { + verifyStackTrace("timeout/${name.methodName}", e) + } + } + + private suspend fun outerChildWithTimeout() { + withTimeout(200) { + launch { + withTimeoutInChild() + } + yield() + } + expectUnreached() + } + + private suspend fun withTimeoutInChild() { + withTimeout(300) { + hang { } + } + expectUnreached() + } + + @Test + fun testStacktraceIsRecoveredFromSuspensionPointWithChild() = runTest { + try { + outerChild() + } catch (e: TimeoutCancellationException) { + verifyStackTrace("timeout/${name.methodName}", e) + } + } + + private suspend fun outerChild() { + withTimeout(200) { + launch { + smallWithTimeout() + } + suspendForever() + } + expectUnreached() + } + + private suspend fun smallWithTimeout() { + withTimeout(100) { + suspendForever() + } + expectUnreached() + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/Stacktraces.kt b/kotlinx-coroutines-core/jvm/test/exceptions/Stacktraces.kt new file mode 100644 index 0000000000..5d85c9c9f2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/Stacktraces.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.* +import java.io.* +import kotlin.test.* + +public fun verifyStackTrace(e: Throwable, vararg traces: String) { + val stacktrace = toStackTrace(e) + val normalizedActual = stacktrace.normalizeStackTrace() + traces.forEach { + val normalizedExpected = it.normalizeStackTrace() + if (!normalizedActual.contains(normalizedExpected)) { + // A more readable error message would be produced by assertEquals + assertEquals(normalizedExpected, normalizedActual, "Actual trace does not contain expected one") + } + } + // Check "Caused by" counts + val causes = stacktrace.count("Caused by") + assertNotEquals(0, causes) + assertEquals(traces.map { it.count("Caused by") }.sum(), causes) +} + +public fun verifyStackTrace(path: String, e: Throwable) { + val resource = Job::class.java.classLoader.getResourceAsStream("stacktraces/$path.txt") + val lines = resource.reader().readLines() + verifyStackTrace(e, *lines.toTypedArray()) +} + +public fun toStackTrace(t: Throwable): String { + val sw = StringWriter() as Writer + t.printStackTrace(PrintWriter(sw)) + return sw.toString() +} + +public fun String.normalizeStackTrace(): String = + replace(Regex(":[0-9]+"), "") // remove line numbers + .replace("kotlinx_coroutines_core_main", "") // yay source sets + .replace("kotlinx_coroutines_core", "") + .replace(Regex("@[0-9a-f]+"), "") // remove hex addresses in debug toStrings + .lines().joinToString("\n") // normalize line separators + +public fun String.count(substring: String): Int = split(substring).size - 1 \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/SuppressionTests.kt b/kotlinx-coroutines-core/jvm/test/exceptions/SuppressionTests.kt new file mode 100644 index 0000000000..33469573d0 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/SuppressionTests.kt @@ -0,0 +1,83 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.testing.exceptions.* +import java.io.* +import kotlin.coroutines.* +import kotlin.test.* + +@Suppress("DEPRECATION") +class SuppressionTests : TestBase() { + @Test + fun testNotificationsWithException() = runTest { + expect(1) + val coroutineContext = kotlin.coroutines.coroutineContext + NonCancellable // workaround for KT-22984 + val coroutine = object : AbstractCoroutine(coroutineContext, true, false) { + override fun onStart() { + expect(3) + } + + override fun onCancelling(cause: Throwable?) { + assertIs(cause) + assertTrue(cause.suppressed.isEmpty()) + expect(5) + } + + override fun onCompleted(value: String) { + expectUnreached() + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + assertIs(cause) + checkException(cause.suppressed[0]) + expect(8) + } + } + + coroutine.invokeOnCompletion(onCancelling = true) { + assertIs(it) + assertTrue(it.suppressed.isEmpty()) + expect(6) + } + + coroutine.invokeOnCompletion { + assertIs(it) + checkException(it.suppressed[0]) + expect(9) + } + + expect(2) + coroutine.start() + expect(4) + coroutine.cancelInternal(ArithmeticException()) + expect(7) + coroutine.resumeWithException(IOException()) + finish(10) + } + + @Test + fun testExceptionUnwrapping() = runTest { + val channel = Channel() + + val deferred = async(NonCancellable) { + launch { + while (true) channel.send(1) + } + + launch { + val exception = RecoverableTestCancellationException() + channel.cancel(exception) + throw exception + } + } + + try { + deferred.await() + } catch (e: RecoverableTestException) { + assertTrue(e.suppressed.isEmpty()) + assertTrue(e.cause!!.suppressed.isEmpty()) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/WithContextCancellationStressTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/WithContextCancellationStressTest.kt new file mode 100644 index 0000000000..ad353340f8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/WithContextCancellationStressTest.kt @@ -0,0 +1,104 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.coroutines.* +import kotlin.test.* +import kotlin.time.Duration.Companion.minutes + +class WithContextCancellationStressTest : TestBase() { + + private val timeoutAfter = 1.minutes + private val pool = newFixedThreadPoolContext(3, "WithContextCancellationStressTest") + + @After + fun tearDown() { + pool.close() + } + + @Test + @Suppress("DEPRECATION") + fun testConcurrentFailure() = runBlocking { + var eCnt = 0 + var e1Cnt = 0 + var e2Cnt = 0 + + withTimeout(timeoutAfter) { + while (eCnt == 0 || e1Cnt == 0 || e2Cnt == 0) { + val barrier = CyclicBarrier(4) + val ctx = pool + NonCancellable + var e1 = false + var e2 = false + val jobWithContext = async(ctx) { + withContext(wrapperDispatcher(coroutineContext)) { + launch { + barrier.await() + e1 = true + throw TestException1() + } + + launch { + barrier.await() + e2 = true + throw TestException2() + } + + barrier.await() + throw TestException() + } + } + + barrier.await() + + try { + jobWithContext.await() + } catch (e: Throwable) { + when (e) { + is TestException -> { + eCnt++ + e.checkSuppressed(e1 = e1, e2 = e2) + } + is TestException1 -> { + e1Cnt++ + e.checkSuppressed(ex = true, e2 = e2) + } + is TestException2 -> { + e2Cnt++ + e.checkSuppressed(ex = true, e1 = e1) + } + else -> error("Unexpected exception $e") + } + } + } + } + } + + private fun wrapperDispatcher(context: CoroutineContext): CoroutineContext { + val dispatcher = context[ContinuationInterceptor] as CoroutineDispatcher + return object : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + dispatcher.dispatch(context, block) + } + } + } + + private fun Throwable.checkSuppressed( + ex: Boolean = false, + e1: Boolean = false, + e2: Boolean = false + ) { + val suppressed: Array = suppressed + if (ex) { + assertTrue(suppressed.any { it is TestException }, "TestException should be present: $this") + } + if (e1) { + assertTrue(suppressed.any { it is TestException1 }, "TestException1 should be present: $this") + } + if (e2) { + assertTrue(suppressed.any { it is TestException2 }, "TestException2 should be present: $this") + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/exceptions/WithContextExceptionHandlingTest.kt b/kotlinx-coroutines-core/jvm/test/exceptions/WithContextExceptionHandlingTest.kt new file mode 100644 index 0000000000..11809ce5c5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/exceptions/WithContextExceptionHandlingTest.kt @@ -0,0 +1,278 @@ +package kotlinx.coroutines.exceptions + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.coroutines.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class WithContextExceptionHandlingTest(private val mode: Mode) : TestBase() { + enum class Mode { WITH_CONTEXT, ASYNC_AWAIT } + + companion object { + @Parameterized.Parameters(name = "mode={0}") + @JvmStatic + fun params(): Collection> = Mode.values().map { arrayOf(it) } + } + + @Test + fun testCancellation() = runTest { + /* + * context cancelled without cause + * code itself throws TE2 + * Result: TE2 + */ + runCancellation(null, TestException2()) { e -> + assertIs(e) + assertNull(e.cause) + val suppressed = e.suppressed + assertTrue(suppressed.isEmpty()) + } + } + + @Test + fun testCancellationWithException() = runTest { + /* + * context cancelled with TCE + * block itself throws TE2 + * Result: TE (CancellationException is always ignored) + */ + val cancellationCause = TestCancellationException() + runCancellation(cancellationCause, TestException2()) { e -> + assertIs(e) + assertNull(e.cause) + val suppressed = e.suppressed + assertTrue(suppressed.isEmpty()) + } + } + + @Test + fun testSameException() = runTest { + /* + * context cancelled with TCE + * block itself throws the same TCE + * Result: TCE + */ + val cancellationCause = TestCancellationException() + runCancellation(cancellationCause, cancellationCause) { e -> + assertIs(e) + assertNull(e.cause) + val suppressed = e.suppressed + assertTrue(suppressed.isEmpty()) + } + } + + @Test + fun testSameCancellation() = runTest { + /* + * context cancelled with TestCancellationException + * block itself throws the same TCE + * Result: TCE + */ + val cancellationCause = TestCancellationException() + runCancellation(cancellationCause, cancellationCause) { e -> + assertSame(e, cancellationCause) + assertNull(e.cause) + val suppressed = e.suppressed + assertTrue(suppressed.isEmpty()) + } + } + + @Test + fun testSameCancellationWithException() = runTest { + /* + * context cancelled with CancellationException(TE) + * block itself throws the same TE + * Result: TE + */ + val cancellationCause = CancellationException() + val exception = TestException() + cancellationCause.initCause(exception) + runCancellation(cancellationCause, exception) { e -> + assertSame(exception, e) + assertNull(e.cause) + assertTrue(e.suppressed.isEmpty()) + } + } + + @Test + fun testConflictingCancellation() = runTest { + /* + * context cancelled with TCE + * block itself throws CE(TE) + * Result: TE (because cancellation exception is always ignored and not handled) + */ + val cancellationCause = TestCancellationException() + val thrown = CancellationException() + thrown.initCause(TestException()) + runCancellation(cancellationCause, thrown) { e -> + assertSame(cancellationCause, e) + assertTrue(e.suppressed.isEmpty()) + } + } + + @Test + fun testConflictingCancellation2() = runTest { + /* + * context cancelled with TE + * block itself throws CE + * Result: TE + */ + val cancellationCause = TestCancellationException() + val thrown = CancellationException() + runCancellation(cancellationCause, thrown) { e -> + assertSame(cancellationCause, e) + val suppressed = e.suppressed + assertTrue(suppressed.isEmpty()) + } + } + + @Test + fun testConflictingCancellation3() = runTest { + /* + * context cancelled with TCE + * block itself throws TCE + * Result: TCE + */ + val cancellationCause = TestCancellationException() + val thrown = TestCancellationException() + runCancellation(cancellationCause, thrown) { e -> + assertSame(cancellationCause, e) + assertNull(e.cause) + assertTrue(e.suppressed.isEmpty()) + } + } + + @Test + fun testThrowingCancellation() = runTest { + val thrown = TestCancellationException() + runThrowing(thrown) { e -> + assertSame(thrown, e) + } + } + + @Test + fun testThrowingCancellationWithCause() = runTest { + // Exception are never unwrapped, so if CE(TE) is thrown then it is the cancellation cause + val thrown = TestCancellationException() + thrown.initCause(TestException()) + runThrowing(thrown) { e -> + assertSame(thrown, e) + } + } + + @Test + fun testCancel() = runTest { + runOnlyCancellation(null) { e -> + val cause = e.cause as JobCancellationException // shall be recovered JCE + assertNull(cause.cause) + assertTrue(e.suppressed.isEmpty()) + assertTrue(cause.suppressed.isEmpty()) + } + } + + @Test + fun testCancelWithCause() = runTest { + val cause = TestCancellationException() + runOnlyCancellation(cause) { e -> + assertSame(cause, e) + assertTrue(e.suppressed.isEmpty()) + } + } + + @Test + fun testCancelWithCancellationException() = runTest { + val cause = TestCancellationException() + runThrowing(cause) { e -> + assertSame(cause, e) + assertNull(e.cause) + assertTrue(e.suppressed.isEmpty()) + } + } + + private fun wrapperDispatcher(context: CoroutineContext): CoroutineContext { + val dispatcher = context[ContinuationInterceptor] as CoroutineDispatcher + return object : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + dispatcher.dispatch(context, block) + } + } + } + + private suspend fun runCancellation( + cancellationCause: CancellationException?, + thrownException: Throwable, + exceptionChecker: (Throwable) -> Unit + ) { + expect(1) + + try { + withCtx(wrapperDispatcher(coroutineContext)) { job -> + require(isActive) // not cancelled yet + job.cancel(cancellationCause) + require(!isActive) // now cancelled + expect(2) + throw thrownException + } + } catch (e: Throwable) { + exceptionChecker(e) + finish(3) + return + } + fail() + } + + private suspend fun runThrowing( + thrownException: Throwable, + exceptionChecker: (Throwable) -> Unit + ) { + expect(1) + try { + withCtx(wrapperDispatcher(coroutineContext).minusKey(Job)) { + require(isActive) + expect(2) + throw thrownException + } + } catch (e: Throwable) { + exceptionChecker(e) + finish(3) + return + } + fail() + } + + private suspend fun withCtx(context: CoroutineContext, job: Job = Job(), block: suspend CoroutineScope.(Job) -> Nothing) { + when (mode) { + Mode.WITH_CONTEXT -> withContext(context + job) { + block(job) + } + Mode.ASYNC_AWAIT -> CoroutineScope(coroutineContext).async(context + job) { + block(job) + }.await() + } + } + + private suspend fun runOnlyCancellation( + cancellationCause: CancellationException?, + exceptionChecker: (Throwable) -> Unit + ) { + expect(1) + val job = Job() + try { + withContext(wrapperDispatcher(coroutineContext) + job) { + require(isActive) // still active + job.cancel(cancellationCause) + require(!isActive) // is already cancelled + expect(2) + } + } catch (e: Throwable) { + exceptionChecker(e) + finish(3) + return + } + fail() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/flow/CallbackFlowTest.kt b/kotlinx-coroutines-core/jvm/test/flow/CallbackFlowTest.kt new file mode 100644 index 0000000000..f59bc1b2ca --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/CallbackFlowTest.kt @@ -0,0 +1,125 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.testing.flow.* +import org.junit.Test +import kotlin.concurrent.* +import kotlin.test.* + +class CallbackFlowTest : TestBase() { + + private class CallbackApi(val block: (SendChannel) -> Unit) { + var started = false + @Volatile + var stopped = false + lateinit var thread: Thread + + fun start(sink: SendChannel) { + started = true + thread = thread { + while (!stopped) { + block(sink) + } + } + } + + fun stop() { + stopped = true + } + } + + @Test(timeout = 5_000L) + fun testThrowingConsumer() = runTest { + var i = 0 + val api = CallbackApi { + it.trySend(++i) + } + + val flow = callbackFlow { + api.start(channel) + awaitClose { + api.stop() + } + } + + var receivedConsensus = 0 + var isDone = false + var exception: Throwable? = null + val job = flow + .filter { it > 10 } + .launchIn(this) { + onEach { + if (it == 11) { + ++receivedConsensus + } else { + receivedConsensus = 42 + } + throw RuntimeException() + } + catch { exception = it } + finally { isDone = true } + } + job.join() + assertEquals(1, receivedConsensus) + assertTrue(isDone) + assertTrue { exception is RuntimeException } + api.thread.join() + assertTrue(api.started) + assertTrue(api.stopped) + } + + @Test(timeout = 5_000L) + fun testThrowingSource() = runBlocking { + var i = 0 + val api = CallbackApi { + if (i < 5) { + it.trySend(++i) + } else { + it.close(RuntimeException()) + } + } + + val flow = callbackFlow { + api.start(channel) + awaitClose { + api.stop() + } + } + + var received = 0 + var isDone = false + var exception: Throwable? = null + val job = flow.launchIn(this) { + onEach { ++received } + catch { exception = it } + finally { isDone = true } + } + + job.join() + assertTrue(isDone) + assertTrue { exception is RuntimeException } + api.thread.join() + assertTrue(api.started) + assertTrue(api.stopped) + } + + + @Test + fun testMergeExample() = runTest { + // Too slow on JS + withContext(Dispatchers.Default) { + val f1 = (1..10_000).asFlow() + val f2 = (10_001..20_000).asFlow() + assertEquals((1..20_000).toSet(), f1.merge(f2).toSet()) + } + } + + private fun Flow.merge(other: Flow): Flow = channelFlow { + launch { + collect { send(it) } + } + other.collect { send(it) } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/flow/ExceptionTransparencyTest.kt b/kotlinx-coroutines-core/jvm/test/flow/ExceptionTransparencyTest.kt new file mode 100644 index 0000000000..63875087a2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/ExceptionTransparencyTest.kt @@ -0,0 +1,74 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class ExceptionTransparencyTest : TestBase() { + + @Test + fun testViolation() = runTest { + val flow = flow { + try { + expect(1) + emit(1) + expectUnreached() + } catch (e: CancellationException) { + expect(3) + emit(2) + } + }.take(1) + + assertFailsWith { flow.collect { expect(2) } } + finish(4) + } + + @Test + fun testViolationResumeWith() = runTest { + val flow = flow { + try { + expect(1) + emit(1) + yield() + expectUnreached() + } catch (e: CancellationException) { + expect(3) + emit(2) + } + }.take(1) + + assertFailsWith { + flow.collect { + yield() + expect(2) + } + } + finish(4) + } + + @Test + fun testViolationAfterInvariantVariation() = runTest { + val flow = flow { + coroutineScope { + try { + expect(1) + launch { + expect(2) + emit(1) + }.join() + expectUnreached() + } catch (e: Throwable) { + try { + emit(2) + } catch (e: IllegalStateException) { + assertTrue { e.message!!.contains("exception transparency") } + emit(3) + } + } + } + } + val e = assertFailsWith { flow.collect { expectUnreached() } } + assertTrue { e.message!!.contains("channelFlow") } + finish(3) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/flow/FirstJvmTest.kt b/kotlinx-coroutines-core/jvm/test/flow/FirstJvmTest.kt new file mode 100644 index 0000000000..1ac9bd2c9c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/FirstJvmTest.kt @@ -0,0 +1,25 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class FirstJvmTest : TestBase() { + + @Test + fun testTakeInterference() = runBlocking(Dispatchers.Default) { + /* + * This test tests a racy situation when outer channelFlow is being cancelled, + * inner flow starts atomically in "CANCELLING" state, sends one element and completes + * (=> cancels and drops element away), triggering NSEE in Flow.first operator + */ + val values = (0..10000).asFlow().flatMapMerge(Int.MAX_VALUE) { + channelFlow { + val value = channelFlow { send(1) }.first() + send(value) + } + }.take(1).toList() + assertEquals(listOf(1), values) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/flow/FlatMapStressTest.kt b/kotlinx-coroutines-core/jvm/test/flow/FlatMapStressTest.kt new file mode 100644 index 0000000000..01201e092b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/FlatMapStressTest.kt @@ -0,0 +1,99 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.internal.* +import kotlinx.coroutines.scheduling.* +import org.junit.Assume.* +import org.junit.Test +import java.util.concurrent.atomic.* +import kotlin.test.* + +class FlatMapStressTest : TestBase() { + + private val iterations = 2000 * stressTestMultiplier + private val expectedSum = iterations.toLong() * (iterations + 1) / 2 + + @Test + fun testConcurrencyLevel() = runTest { + withContext(Dispatchers.Default) { + testConcurrencyLevel(2) + } + } + + @Test + fun testConcurrencyLevel2() = runTest { + withContext(Dispatchers.Default) { + testConcurrencyLevel(3) + } + } + + @Test + fun testBufferSize() = runTest { + val bufferSize = 5 + withContext(Dispatchers.Default) { + val inFlightElements = AtomicLong(0L) + var result = 0L + (1..iterations step 4).asFlow().flatMapMerge { value -> + unsafeFlow { + repeat(4) { + emit(value + it) + inFlightElements.incrementAndGet() + } + } + }.buffer(bufferSize).collect { value -> + val inFlight = inFlightElements.get() + assertTrue(inFlight <= bufferSize + 1, + "Expected less in flight elements than ${bufferSize + 1}, but had $inFlight") + inFlightElements.decrementAndGet() + result += value + } + + assertEquals(0, inFlightElements.get()) + assertEquals(expectedSum, result) + } + } + + @Test + fun testDelivery() = runTest { + withContext(Dispatchers.Default) { + val result = (1L..iterations step 4).asFlow().flatMapMerge { value -> + unsafeFlow { + repeat(4) { emit(value + it) } + } + }.longSum() + assertEquals(expectedSum, result) + } + } + + @Test + fun testIndependentShortBursts() = runTest { + withContext(Dispatchers.Default) { + repeat(iterations) { + val result = (1L..4L).asFlow().flatMapMerge { value -> + unsafeFlow { + emit(value) + emit(value) + } + }.longSum() + assertEquals(20, result) + } + } + } + + private suspend fun testConcurrencyLevel(maxConcurrency: Int) { + assumeTrue(maxConcurrency <= CORE_POOL_SIZE) + val concurrency = AtomicLong() + val result = (1L..iterations).asFlow().flatMapMerge(concurrency = maxConcurrency) { value -> + unsafeFlow { + val current = concurrency.incrementAndGet() + assertTrue(current in 1..maxConcurrency) + emit(value) + concurrency.decrementAndGet() + } + }.longSum() + + assertEquals(0, concurrency.get()) + assertEquals(expectedSum, result) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/flow/OnCompletionInterceptedReleaseTest.kt b/kotlinx-coroutines-core/jvm/test/flow/OnCompletionInterceptedReleaseTest.kt new file mode 100644 index 0000000000..5d332bf3ad --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/OnCompletionInterceptedReleaseTest.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.coroutines.* +import kotlin.test.* + +class OnCompletionInterceptedReleaseTest : TestBase() { + @Test + fun testLeak() = runTest { + expect(1) + var cont: Continuation? = null + val interceptor = CountingInterceptor() + val job = launch(interceptor, start = CoroutineStart.UNDISPATCHED) { + emptyFlow() + .onCompletion { emit(1) } + .collect { value -> + expect(2) + assertEquals(1, value) + suspendCoroutine { cont = it } + } + } + cont!!.resume(Unit) + assertTrue(job.isCompleted) + assertEquals(interceptor.intercepted, interceptor.released) + finish(3) + } + + class CountingInterceptor : AbstractCoroutineContextElement(ContinuationInterceptor), ContinuationInterceptor { + var intercepted = 0 + var released = 0 + override fun interceptContinuation(continuation: Continuation): Continuation { + intercepted++ + return Continuation(continuation.context) { continuation.resumeWith(it) } + } + + override fun releaseInterceptedContinuation(continuation: Continuation<*>) { + released++ + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/flow/SafeCollectorMemoryLeakTest.kt b/kotlinx-coroutines-core/jvm/test/flow/SafeCollectorMemoryLeakTest.kt new file mode 100644 index 0000000000..094f8cca33 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/SafeCollectorMemoryLeakTest.kt @@ -0,0 +1,45 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* + +class SafeCollectorMemoryLeakTest : TestBase() { + // custom List.forEach impl to avoid using iterator (FieldWalker cannot scan it) + private inline fun List.listForEach(action: (T) -> Unit) { + for (i in indices) action(get(i)) + } + + @Test + fun testCompletionIsProperlyCleanedUp() = runBlocking { + val job = flow { + emit(listOf(239)) + expect(2) + hang {} + }.transform { l -> l.listForEach { _ -> emit(42) } } + .onEach { expect(1) } + .launchIn(this) + yield() + expect(3) + FieldWalker.assertReachableCount(0, job) { it == 239 } + job.cancelAndJoin() + finish(4) + } + + @Test + fun testCompletionIsNotCleanedUp() = runBlocking { + val job = flow { + emit(listOf(239)) + hang {} + }.transform { l -> l.listForEach { _ -> emit(42) } } + .onEach { + expect(1) + hang { finish(3) } + } + .launchIn(this) + yield() + expect(2) + FieldWalker.assertReachableCount(1, job) { it == 239 } + job.cancelAndJoin() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/flow/SharedFlowStressTest.kt b/kotlinx-coroutines-core/jvm/test/flow/SharedFlowStressTest.kt new file mode 100644 index 0000000000..712f2bd005 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/SharedFlowStressTest.kt @@ -0,0 +1,83 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import kotlin.collections.ArrayList +import kotlin.test.* +import kotlin.time.Duration.Companion.seconds + +class SharedFlowStressTest : TestBase() { + private val nProducers = 5 + private val nConsumers = 3 + private val nSeconds = 3 * stressTestMultiplier + + private lateinit var sf: MutableSharedFlow + private lateinit var view: SharedFlow + + @get:Rule + val producerDispatcher = ExecutorRule(nProducers) + @get:Rule + val consumerDispatcher = ExecutorRule(nConsumers) + + private val totalProduced = atomic(0L) + private val totalConsumed = atomic(0L) + + @Test + fun testStressReplay1() = + testStress(1, 0) + + @Test + fun testStressReplay1ExtraBuffer1() = + testStress(1, 1) + + @Test + fun testStressReplay2ExtraBuffer1() = + testStress(2, 1) + + private fun testStress(replay: Int, extraBufferCapacity: Int) = runTest { + sf = MutableSharedFlow(replay, extraBufferCapacity) + view = sf.asSharedFlow() + val jobs = ArrayList() + jobs += List(nProducers) { producerIndex -> + launch(producerDispatcher) { + var cur = producerIndex.toLong() + while (isActive) { + sf.emit(cur) + totalProduced.incrementAndGet() + cur += nProducers + } + } + } + jobs += List(nConsumers) { consumerIndex -> + launch(consumerDispatcher) { + while (isActive) { + view + .dropWhile { it % nConsumers != consumerIndex.toLong() } + .take(1) + .collect { + check(it % nConsumers == consumerIndex.toLong()) + totalConsumed.incrementAndGet() + } + } + } + } + var lastProduced = 0L + var lastConsumed = 0L + for (sec in 1..nSeconds) { + delay(1.seconds) + val produced = totalProduced.value + val consumed = totalConsumed.value + println("$sec sec: produced = $produced; consumed = $consumed") + assertNotEquals(lastProduced, produced) + assertNotEquals(lastConsumed, consumed) + lastProduced = produced + lastConsumed = consumed + } + jobs.forEach { it.cancel() } + jobs.forEach { it.join() } + println("total: produced = ${totalProduced.value}; consumed = ${totalConsumed.value}") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/flow/SharingReferenceTest.kt b/kotlinx-coroutines-core/jvm/test/flow/SharingReferenceTest.kt new file mode 100644 index 0000000000..f1e15be176 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/SharingReferenceTest.kt @@ -0,0 +1,57 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import org.junit.* + +/** + * Tests that shared flows keep strong reference to their source flows. + * See https://github.com/Kotlin/kotlinx.coroutines/issues/2557 + */ +class SharingReferenceTest : TestBase() { + private val token = object {} + + /* + * Single-threaded executor that we are using to ensure that the flow being sharing actually + * suspended (spilled its locals, attached to parent), so we can verify reachability. + * Without that, it's possible to have a situation where target flow is still + * being strongly referenced (by its dispatcher), but the test already tries to test reachability and fails. + */ + @get:Rule + val executor = ExecutorRule(1) + + private val weakEmitter = flow { + emit(null) + // suspend forever without keeping a strong reference to continuation -- this is a model of + // a callback API that does not keep a strong reference it is listeners, but works + suspendCancellableCoroutine { } + // using the token here to make it easily traceable + emit(token) + } + + @Test + fun testShareInReference() { + val flow = weakEmitter.shareIn(ContextScope(executor), SharingStarted.Eagerly, 0) + linearize() + FieldWalker.assertReachableCount(1, flow) { it === token } + } + + @Test + fun testStateInReference() { + val flow = weakEmitter.stateIn(ContextScope(executor), SharingStarted.Eagerly, null) + linearize() + FieldWalker.assertReachableCount(1, flow) { it === token } + } + + @Test + fun testStateInSuspendingReference() = runTest { + val flow = weakEmitter.stateIn(ContextScope(executor)) + linearize() + FieldWalker.assertReachableCount(1, flow) { it === token } + } + + private fun linearize() { + runBlocking(executor) { } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/flow/SharingStressTest.kt b/kotlinx-coroutines-core/jvm/test/flow/SharingStressTest.kt new file mode 100644 index 0000000000..0d160e6a70 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/SharingStressTest.kt @@ -0,0 +1,193 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import java.util.* +import java.util.concurrent.atomic.* +import kotlin.random.* +import kotlin.test.* +import kotlin.time.* +import kotlin.time.TimeSource + +class SharingStressTest : TestBase() { + private val testDuration = 1000L * stressTestMultiplier + private val nSubscribers = 5 + private val testStarted = TimeSource.Monotonic.markNow() + + @get:Rule + val emitterDispatcher = ExecutorRule(1) + + @get:Rule + val subscriberDispatcher = ExecutorRule(nSubscribers) + + @Test + public fun testNoReplayLazy() = + testStress(0, started = SharingStarted.Lazily) + + @Test + public fun testNoReplayWhileSubscribed() = + testStress(0, started = SharingStarted.WhileSubscribed()) + + @Test + public fun testNoReplayWhileSubscribedTimeout() = + testStress(0, started = SharingStarted.WhileSubscribed(stopTimeoutMillis = 50L)) + + @Test + public fun testReplay100WhileSubscribed() = + testStress(100, started = SharingStarted.WhileSubscribed()) + + @Test + public fun testReplay100WhileSubscribedReset() = + testStress(100, started = SharingStarted.WhileSubscribed(replayExpirationMillis = 0L)) + + @Test + public fun testReplay100WhileSubscribedTimeout() = + testStress(100, started = SharingStarted.WhileSubscribed(stopTimeoutMillis = 50L)) + + @Test + public fun testStateLazy() = + testStress(1, started = SharingStarted.Lazily) + + @Test + public fun testStateWhileSubscribed() = + testStress(1, started = SharingStarted.WhileSubscribed()) + + @Test + public fun testStateWhileSubscribedReset() = + testStress(1, started = SharingStarted.WhileSubscribed(replayExpirationMillis = 0L)) + + private fun testStress(replay: Int, started: SharingStarted) = runTest { + log("-- Stress with replay=$replay, started=$started") + val random = Random(1) + val emitIndex = AtomicLong() + val cancelledEmits = HashSet() + val missingCollects = Collections.synchronizedSet(LinkedHashSet()) + // at most one copy of upstream can be running at any time + val isRunning = AtomicInteger(0) + val upstream = flow { + assertEquals(0, isRunning.getAndIncrement()) + try { + while (true) { + val value = emitIndex.getAndIncrement() + try { + emit(value) + } catch (e: CancellationException) { + // emission was cancelled -> could be missing + cancelledEmits.add(value) + throw e + } + } + } finally { + assertEquals(1, isRunning.getAndDecrement()) + } + } + val subCount = MutableStateFlow(0) + val sharingJob = Job() + val sharingScope = this + emitterDispatcher + sharingJob + val usingStateFlow = replay == 1 + val sharedFlow = if (usingStateFlow) + upstream.stateIn(sharingScope, started, 0L) + else + upstream.shareIn(sharingScope, started, replay) + try { + val subscribers = ArrayList() + withTimeoutOrNull(testDuration) { + // start and stop subscribers + while (true) { + log("Staring $nSubscribers subscribers") + repeat(nSubscribers) { + subscribers += launchSubscriber(sharedFlow, usingStateFlow, subCount, missingCollects) + } + // wait until they all subscribed + subCount.first { it == nSubscribers } + // let them work a bit more & make sure emitter did not hang + val fromEmitIndex = emitIndex.get() + val waitEmitIndex = fromEmitIndex + 100 // wait until 100 emitted + withTimeout(10000) { // wait for at most 10s for something to be emitted + do { + delay(random.nextLong(50L..100L)) + } while (emitIndex.get() < waitEmitIndex) // Ok, enough was emitted, wait more if not + } + // Stop all subscribers and ensure they collected something + log("Stopping subscribers (emitted = ${emitIndex.get() - fromEmitIndex})") + subscribers.forEach { + it.job.cancelAndJoin() + assertTrue { it.count > 0 } // something must be collected too + } + subscribers.clear() + log("Intermission") + delay(random.nextLong(10L..100L)) // wait a bit before starting them again + } + } + if (!subscribers.isEmpty()) { + log("Stopping subscribers") + subscribers.forEach { it.job.cancelAndJoin() } + } + } finally { + log("--- Finally: Cancelling sharing job") + sharingJob.cancel() + } + sharingJob.join() // make sure sharing job did not hang + log("Emitter was cancelled ${cancelledEmits.size} times") + log("Collectors missed ${missingCollects.size} values") + for (value in missingCollects) { + assertTrue(value in cancelledEmits, "Value $value is missing for no apparent reason") + } + } + + private fun CoroutineScope.launchSubscriber( + sharedFlow: SharedFlow, + usingStateFlow: Boolean, + subCount: MutableStateFlow, + missingCollects: MutableSet + ): SubJob { + val subJob = SubJob() + subJob.job = launch(subscriberDispatcher) { + var last = -1L + sharedFlow + .onSubscription { + subCount.increment(1) + } + .onCompletion { + subCount.increment(-1) + } + .collect { j -> + subJob.count++ + // last must grow sequentially, no jumping or losses + if (last == -1L) { + last = j + } else { + val expected = last + 1 + if (usingStateFlow) + assertTrue(expected <= j) + else { + if (expected != j) { + if (j == expected + 1) { + // if missing just one -- could be race with cancelled emit + missingCollects.add(expected) + } else { + // broken otherwise + assertEquals(expected, j) + } + } + } + last = j + } + } + } + return subJob + } + + private class SubJob { + lateinit var job: Job + var count = 0L + } + + private fun log(msg: String) = println("${testStarted.elapsedNow().inWholeMilliseconds} ms: $msg") + + private fun MutableStateFlow.increment(delta: Int) { + update { it + delta } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/flow/StateFlowCancellabilityTest.kt b/kotlinx-coroutines-core/jvm/test/flow/StateFlowCancellabilityTest.kt new file mode 100644 index 0000000000..cb91dd6dbe --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/StateFlowCancellabilityTest.kt @@ -0,0 +1,53 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import java.util.concurrent.* +import kotlin.test.* + +@Suppress("BlockingMethodInNonBlockingContext") +class StateFlowCancellabilityTest : TestBase() { + @Test + fun testCancellabilityNoConflation() = runTest { + expect(1) + val state = MutableStateFlow(0) + var subscribed = true + var lastReceived = -1 + val barrier = CyclicBarrier(2) + val job = state + .onSubscription { + subscribed = true + barrier.await() + } + .onEach { i -> + when (i) { + 0 -> expect(2) // initial value + 1 -> expect(3) + 2 -> { + expect(4) + currentCoroutineContext().cancel() + } + else -> expectUnreached() // shall check for cancellation + } + lastReceived = i + barrier.await() + barrier.await() + } + .launchIn(this + Dispatchers.Default) + barrier.await() + assertTrue(subscribed) // should have subscribed in the first barrier + barrier.await() + assertEquals(0, lastReceived) // should get initial value, too + for (i in 1..3) { // emit after subscription + state.value = i + barrier.await() // let it go + if (i < 3) { + barrier.await() // wait for receive + assertEquals(i, lastReceived) // shall receive it + } + } + job.join() + finish(5) + } +} + diff --git a/kotlinx-coroutines-core/jvm/test/flow/StateFlowStressTest.kt b/kotlinx-coroutines-core/jvm/test/flow/StateFlowStressTest.kt new file mode 100644 index 0000000000..3739aef978 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/StateFlowStressTest.kt @@ -0,0 +1,77 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import kotlin.random.* + +class StateFlowStressTest : TestBase() { + private val nSeconds = 3 * stressTestMultiplier + private val state = MutableStateFlow(0) + private lateinit var pool: ExecutorCoroutineDispatcher + + @After + fun tearDown() { + pool.close() + } + + fun stress(nEmitters: Int, nCollectors: Int) = runTest { + pool = newFixedThreadPoolContext(nEmitters + nCollectors, "StateFlowStressTest") + val collected = Array(nCollectors) { LongArray(nEmitters) } + val collectors = launch { + repeat(nCollectors) { collector -> + launch(pool) { + val c = collected[collector] + // collect, but abort and collect again after every 1000 values to stress allocation/deallocation + do { + val batchSize = Random.nextInt(1..1000) + var index = 0 + val cnt = state.onEach { value -> + val emitter = (value % nEmitters).toInt() + val current = value / nEmitters + // the first value in batch is allowed to repeat, but cannot go back + val ok = if (index++ == 0) current >= c[emitter] else current > c[emitter] + check(ok) { + "Values must be monotonic, but $current is not, " + + "was ${c[emitter]} in collector #$collector from emitter #$emitter" + } + c[emitter] = current + + }.take(batchSize).map { 1 }.sum() + } while (cnt == batchSize) + } + } + } + val emitted = LongArray(nEmitters) + val emitters = launch { + repeat(nEmitters) { emitter -> + launch(pool) { + var current = 1L + while (true) { + state.value = current * nEmitters + emitter + emitted[emitter] = current + current++ + if (current % 1000 == 0L) yield() // make it cancellable + } + } + } + } + for (second in 1..nSeconds) { + delay(1000) + val cs = collected.map { it.sum() } + println("$second: emitted=${emitted.sum()}, collected=${cs.minOrNull()}..${cs.maxOrNull()}") + } + emitters.cancelAndJoin() + collectors.cancelAndJoin() + // make sure nothing hanged up + require(collected.all { c -> + c.withIndex().all { (emitter, current) -> current > emitted[emitter] / 2 } + }) + } + + @Test + fun testSingleEmitterAndCollector() = stress(1, 1) + + @Test + fun testTenEmittersAndCollectors() = stress(10, 10) +} diff --git a/kotlinx-coroutines-core/jvm/test/flow/StateFlowUpdateStressTest.kt b/kotlinx-coroutines-core/jvm/test/flow/StateFlowUpdateStressTest.kt new file mode 100644 index 0000000000..adc6610f25 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/flow/StateFlowUpdateStressTest.kt @@ -0,0 +1,41 @@ +package kotlinx.coroutines.flow + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import kotlin.test.* +import kotlin.test.Test + +class StateFlowUpdateStressTest : TestBase() { + private val iterations = 1_000_000 * stressTestMultiplier + + @get:Rule + public val executor = ExecutorRule(2) + + @Test + fun testUpdate() = doTest { update { it + 1 } } + + @Test + fun testUpdateAndGet() = doTest { updateAndGet { it + 1 } } + + @Test + fun testGetAndUpdate() = doTest { getAndUpdate { it + 1 } } + + private fun doTest(increment: MutableStateFlow.() -> Unit) = runTest { + val flow = MutableStateFlow(0) + val j1 = launch(Dispatchers.Default) { + repeat(iterations / 2) { + flow.increment() + } + } + + val j2 = launch(Dispatchers.Default) { + repeat(iterations / 2) { + flow.increment() + } + } + + joinAll(j1, j2) + assertEquals(iterations, flow.value) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-basic-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-basic-01.kt new file mode 100644 index 0000000000..725beac447 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-basic-01.kt @@ -0,0 +1,12 @@ +// This file was automatically generated from coroutines-basics.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleBasic01 + +import kotlinx.coroutines.* + +fun main() = runBlocking { // this: CoroutineScope + launch { // launch a new coroutine and continue + delay(1000L) // non-blocking delay for 1 second (default time unit is ms) + println("World!") // print after delay + } + println("Hello") // main coroutine continues while a previous one is delayed +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-basic-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-basic-02.kt new file mode 100644 index 0000000000..b0102060ed --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-basic-02.kt @@ -0,0 +1,15 @@ +// This file was automatically generated from coroutines-basics.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleBasic02 + +import kotlinx.coroutines.* + +fun main() = runBlocking { // this: CoroutineScope + launch { doWorld() } + println("Hello") +} + +// this is your first suspending function +suspend fun doWorld() { + delay(1000L) + println("World!") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-basic-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-basic-03.kt new file mode 100644 index 0000000000..37ad624a96 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-basic-03.kt @@ -0,0 +1,16 @@ +// This file was automatically generated from coroutines-basics.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleBasic03 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + doWorld() +} + +suspend fun doWorld() = coroutineScope { // this: CoroutineScope + launch { + delay(1000L) + println("World!") + } + println("Hello") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-basic-04.kt b/kotlinx-coroutines-core/jvm/test/guide/example-basic-04.kt new file mode 100644 index 0000000000..bee0331ad2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-basic-04.kt @@ -0,0 +1,23 @@ +// This file was automatically generated from coroutines-basics.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleBasic04 + +import kotlinx.coroutines.* + +// Sequentially executes doWorld followed by "Done" +fun main() = runBlocking { + doWorld() + println("Done") +} + +// Concurrently executes both sections +suspend fun doWorld() = coroutineScope { // this: CoroutineScope + launch { + delay(2000L) + println("World 2") + } + launch { + delay(1000L) + println("World 1") + } + println("Hello") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-basic-05.kt b/kotlinx-coroutines-core/jvm/test/guide/example-basic-05.kt new file mode 100644 index 0000000000..d9a29ac2dc --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-basic-05.kt @@ -0,0 +1,14 @@ +// This file was automatically generated from coroutines-basics.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleBasic05 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val job = launch { // launch a new coroutine and keep a reference to its Job + delay(1000L) + println("World!") + } + println("Hello") + job.join() // wait until child coroutine completes + println("Done") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-basic-06.kt b/kotlinx-coroutines-core/jvm/test/guide/example-basic-06.kt new file mode 100644 index 0000000000..c621cfb259 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-basic-06.kt @@ -0,0 +1,13 @@ +// This file was automatically generated from coroutines-basics.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleBasic06 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + repeat(50_000) { // launch a lot of coroutines + launch { + delay(5000L) + print(".") + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-01.kt new file mode 100644 index 0000000000..5a0d5a8b1a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-01.kt @@ -0,0 +1,18 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel01 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val job = launch { + repeat(1000) { i -> + println("job: I'm sleeping $i ...") + delay(500L) + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancel() // cancels the job + job.join() // waits for job's completion + println("main: Now I can quit.") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-02.kt new file mode 100644 index 0000000000..3704051697 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-02.kt @@ -0,0 +1,23 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel02 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val startTime = currentTimeMillis() + val job = launch(Dispatchers.Default) { + var nextPrintTime = startTime + var i = 0 + while (i < 5) { // computation loop, just wastes CPU + // print a message twice a second + if (currentTimeMillis() >= nextPrintTime) { + println("job: I'm sleeping ${i++} ...") + nextPrintTime += 500L + } + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-03.kt new file mode 100644 index 0000000000..a950f80d80 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-03.kt @@ -0,0 +1,23 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel03 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val job = launch(Dispatchers.Default) { + repeat(5) { i -> + try { + // print a message twice a second + println("job: I'm sleeping $i ...") + delay(500) + } catch (e: Exception) { + // log the exception + println(e) + } + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-04.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-04.kt new file mode 100644 index 0000000000..f3027a6ef0 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-04.kt @@ -0,0 +1,23 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel04 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val startTime = currentTimeMillis() + val job = launch(Dispatchers.Default) { + var nextPrintTime = startTime + var i = 0 + while (isActive) { // cancellable computation loop + // prints a message twice a second + if (currentTimeMillis() >= nextPrintTime) { + println("job: I'm sleeping ${i++} ...") + nextPrintTime += 500L + } + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-05.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-05.kt new file mode 100644 index 0000000000..c82fb4ad0f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-05.kt @@ -0,0 +1,21 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel05 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val job = launch { + try { + repeat(1000) { i -> + println("job: I'm sleeping $i ...") + delay(500L) + } + } finally { + println("job: I'm running finally") + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-06.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-06.kt new file mode 100644 index 0000000000..844db9879e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-06.kt @@ -0,0 +1,25 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel06 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val job = launch { + try { + repeat(1000) { i -> + println("job: I'm sleeping $i ...") + delay(500L) + } + } finally { + withContext(NonCancellable) { + println("job: I'm running finally") + delay(1000L) + println("job: And I've just delayed for 1 sec because I'm non-cancellable") + } + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-07.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-07.kt new file mode 100644 index 0000000000..73d25748fe --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-07.kt @@ -0,0 +1,13 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel07 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + withTimeout(1300L) { + repeat(1000) { i -> + println("I'm sleeping $i ...") + delay(500L) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-08.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-08.kt new file mode 100644 index 0000000000..29f3e10c7f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-08.kt @@ -0,0 +1,15 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel08 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val result = withTimeoutOrNull(1300L) { + repeat(1000) { i -> + println("I'm sleeping $i ...") + delay(500L) + } + "Done" // will get cancelled before it produces this result + } + println("Result is $result") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-09.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-09.kt new file mode 100644 index 0000000000..5baa99f37d --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-09.kt @@ -0,0 +1,27 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel09 + +import kotlinx.coroutines.* + +var acquired = 0 + +class Resource { + init { acquired++ } // Acquire the resource + fun close() { acquired-- } // Release the resource +} + +fun main() { + runBlocking { + repeat(10_000) { // Launch 10K coroutines + launch { + val resource = withTimeout(60) { // Timeout of 60 ms + delay(50) // Delay for 50 ms + Resource() // Acquire a resource and return it from withTimeout block + } + resource.close() // Release the resource + } + } + } + // Outside of runBlocking all coroutines have completed + println(acquired) // Print the number of resources still acquired +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-cancel-10.kt b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-10.kt new file mode 100644 index 0000000000..6cbc2e579a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-cancel-10.kt @@ -0,0 +1,32 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCancel10 + +import kotlinx.coroutines.* + +var acquired = 0 + +class Resource { + init { acquired++ } // Acquire the resource + fun close() { acquired-- } // Release the resource +} + +fun main() { + runBlocking { + repeat(10_000) { // Launch 10K coroutines + launch { + var resource: Resource? = null // Not acquired yet + try { + withTimeout(60) { // Timeout of 60 ms + delay(50) // Delay for 50 ms + resource = Resource() // Store a resource to the variable if acquired + } + // We can do something else with the resource here + } finally { + resource?.close() // Release the resource if it was acquired + } + } + } + } + // Outside of runBlocking all coroutines have completed + println(acquired) // Print the number of resources still acquired +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-01.kt new file mode 100644 index 0000000000..8c9845afa5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-01.kt @@ -0,0 +1,17 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel01 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { + val channel = Channel() + launch { + // this might be heavy CPU-consuming computation or async logic, + // we'll just send five squares + for (x in 1..5) channel.send(x * x) + } + // here we print five received integers: + repeat(5) { println(channel.receive()) } + println("Done!") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-02.kt new file mode 100644 index 0000000000..5a5a7547bd --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-02.kt @@ -0,0 +1,16 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel02 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { + val channel = Channel() + launch { + for (x in 1..5) channel.send(x * x) + channel.close() // we're done sending + } + // here we print received values using `for` loop (until the channel is closed) + for (y in channel) println(y) + println("Done!") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-03.kt new file mode 100644 index 0000000000..9c69c62fff --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-03.kt @@ -0,0 +1,15 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel03 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun CoroutineScope.produceSquares(): ReceiveChannel = produce { + for (x in 1..5) send(x * x) +} + +fun main() = runBlocking { + val squares = produceSquares() + squares.consumeEach { println(it) } + println("Done!") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-04.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-04.kt new file mode 100644 index 0000000000..da61e14c94 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-04.kt @@ -0,0 +1,24 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel04 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { + val numbers = produceNumbers() // produces integers from 1 and on + val squares = square(numbers) // squares integers + repeat(5) { + println(squares.receive()) // print first five + } + println("Done!") // we are done + coroutineContext.cancelChildren() // cancel children coroutines +} + +fun CoroutineScope.produceNumbers() = produce { + var x = 1 + while (true) send(x++) // infinite stream of integers starting from 1 +} + +fun CoroutineScope.square(numbers: ReceiveChannel): ReceiveChannel = produce { + for (x in numbers) send(x * x) +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-05.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-05.kt new file mode 100644 index 0000000000..f8b2f369f5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-05.kt @@ -0,0 +1,24 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel05 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { + var cur = numbersFrom(2) + repeat(10) { + val prime = cur.receive() + println(prime) + cur = filter(cur, prime) + } + coroutineContext.cancelChildren() // cancel all children to let main finish +} + +fun CoroutineScope.numbersFrom(start: Int) = produce { + var x = start + while (true) send(x++) // infinite stream of integers from start +} + +fun CoroutineScope.filter(numbers: ReceiveChannel, prime: Int) = produce { + for (x in numbers) if (x % prime != 0) send(x) +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-06.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-06.kt new file mode 100644 index 0000000000..1d61cea8cf --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-06.kt @@ -0,0 +1,26 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel06 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { + val producer = produceNumbers() + repeat(5) { launchProcessor(it, producer) } + delay(950) + producer.cancel() // cancel producer coroutine and thus kill them all +} + +fun CoroutineScope.produceNumbers() = produce { + var x = 1 // start from 1 + while (true) { + send(x++) // produce next + delay(100) // wait 0.1s + } +} + +fun CoroutineScope.launchProcessor(id: Int, channel: ReceiveChannel) = launch { + for (msg in channel) { + println("Processor #$id received $msg") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-07.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-07.kt new file mode 100644 index 0000000000..f21274f11a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-07.kt @@ -0,0 +1,22 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel07 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { + val channel = Channel() + launch { sendString(channel, "foo", 200L) } + launch { sendString(channel, "BAR!", 500L) } + repeat(6) { // receive first six + println(channel.receive()) + } + coroutineContext.cancelChildren() // cancel all children to let main finish +} + +suspend fun sendString(channel: SendChannel, s: String, time: Long) { + while (true) { + delay(time) + channel.send(s) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-08.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-08.kt new file mode 100644 index 0000000000..f5549cad4e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-08.kt @@ -0,0 +1,18 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel08 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { + val channel = Channel(4) // create buffered channel + val sender = launch { // launch sender coroutine + repeat(10) { + println("Sending $it") // print before sending each element + channel.send(it) // will suspend when buffer is full + } + } + // don't receive anything... just wait.... + delay(1000) + sender.cancel() // cancel sender coroutine +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-09.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-09.kt new file mode 100644 index 0000000000..d93bc0e81e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-09.kt @@ -0,0 +1,25 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel09 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +data class Ball(var hits: Int) + +fun main() = runBlocking { + val table = Channel() // a shared table + launch { player("ping", table) } + launch { player("pong", table) } + table.send(Ball(0)) // serve the ball + delay(1000) // delay 1 second + coroutineContext.cancelChildren() // game over, cancel them +} + +suspend fun player(name: String, table: Channel) { + for (ball in table) { // receive the ball in a loop + ball.hits++ + println("$name $ball") + delay(300) // wait a bit + table.send(ball) // send the ball back + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-channel-10.kt b/kotlinx-coroutines-core/jvm/test/guide/example-channel-10.kt new file mode 100644 index 0000000000..8867e949ab --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-channel-10.kt @@ -0,0 +1,29 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleChannel10 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* + +fun main() = runBlocking { + val tickerChannel = ticker(delayMillis = 200, initialDelayMillis = 0) // create a ticker channel + var nextElement = withTimeoutOrNull(1) { tickerChannel.receive() } + println("Initial element is available immediately: $nextElement") // no initial delay + + nextElement = withTimeoutOrNull(100) { tickerChannel.receive() } // all subsequent elements have 200ms delay + println("Next element is not ready in 100 ms: $nextElement") + + nextElement = withTimeoutOrNull(120) { tickerChannel.receive() } + println("Next element is ready in 200 ms: $nextElement") + + // Emulate large consumption delays + println("Consumer pauses for 300ms") + delay(300) + // Next element is available immediately + nextElement = withTimeoutOrNull(1) { tickerChannel.receive() } + println("Next element is available immediately after large consumer delay: $nextElement") + // Note that the pause between `receive` calls is taken into account and next element arrives faster + nextElement = withTimeoutOrNull(120) { tickerChannel.receive() } + println("Next element is ready in 100ms after consumer pause in 300ms: $nextElement") + + tickerChannel.cancel() // indicate that no more elements are needed +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-compose-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-compose-01.kt new file mode 100644 index 0000000000..7194b64a26 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-compose-01.kt @@ -0,0 +1,24 @@ +// This file was automatically generated from composing-suspending-functions.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCompose01 + +import kotlinx.coroutines.* +import kotlin.system.* + +fun main() = runBlocking { + val time = measureTimeMillis { + val one = doSomethingUsefulOne() + val two = doSomethingUsefulTwo() + println("The answer is ${one + two}") + } + println("Completed in $time ms") +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-compose-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-compose-02.kt new file mode 100644 index 0000000000..126004ca9b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-compose-02.kt @@ -0,0 +1,24 @@ +// This file was automatically generated from composing-suspending-functions.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCompose02 + +import kotlinx.coroutines.* +import kotlin.system.* + +fun main() = runBlocking { + val time = measureTimeMillis { + val one = async { doSomethingUsefulOne() } + val two = async { doSomethingUsefulTwo() } + println("The answer is ${one.await() + two.await()}") + } + println("Completed in $time ms") +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-compose-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-compose-03.kt new file mode 100644 index 0000000000..04801c6ae4 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-compose-03.kt @@ -0,0 +1,27 @@ +// This file was automatically generated from composing-suspending-functions.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCompose03 + +import kotlinx.coroutines.* +import kotlin.system.* + +fun main() = runBlocking { + val time = measureTimeMillis { + val one = async(start = CoroutineStart.LAZY) { doSomethingUsefulOne() } + val two = async(start = CoroutineStart.LAZY) { doSomethingUsefulTwo() } + // some computation + one.start() // start the first one + two.start() // start the second one + println("The answer is ${one.await() + two.await()}") + } + println("Completed in $time ms") +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-compose-04.kt b/kotlinx-coroutines-core/jvm/test/guide/example-compose-04.kt new file mode 100644 index 0000000000..0d770452dc --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-compose-04.kt @@ -0,0 +1,40 @@ +// This file was automatically generated from composing-suspending-functions.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCompose04 + +import kotlinx.coroutines.* +import kotlin.system.* + +// note that we don't have `runBlocking` to the right of `main` in this example +fun main() { + val time = measureTimeMillis { + // we can initiate async actions outside of a coroutine + val one = somethingUsefulOneAsync() + val two = somethingUsefulTwoAsync() + // but waiting for a result must involve either suspending or blocking. + // here we use `runBlocking { ... }` to block the main thread while waiting for the result + runBlocking { + println("The answer is ${one.await() + two.await()}") + } + } + println("Completed in $time ms") +} + +@OptIn(DelicateCoroutinesApi::class) +fun somethingUsefulOneAsync() = GlobalScope.async { + doSomethingUsefulOne() +} + +@OptIn(DelicateCoroutinesApi::class) +fun somethingUsefulTwoAsync() = GlobalScope.async { + doSomethingUsefulTwo() +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-compose-05.kt b/kotlinx-coroutines-core/jvm/test/guide/example-compose-05.kt new file mode 100644 index 0000000000..d32b5efb60 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-compose-05.kt @@ -0,0 +1,28 @@ +// This file was automatically generated from composing-suspending-functions.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCompose05 + +import kotlinx.coroutines.* +import kotlin.system.* + +fun main() = runBlocking { + val time = measureTimeMillis { + println("The answer is ${concurrentSum()}") + } + println("Completed in $time ms") +} + +suspend fun concurrentSum(): Int = coroutineScope { + val one = async { doSomethingUsefulOne() } + val two = async { doSomethingUsefulTwo() } + one.await() + two.await() +} + +suspend fun doSomethingUsefulOne(): Int { + delay(1000L) // pretend we are doing something useful here + return 13 +} + +suspend fun doSomethingUsefulTwo(): Int { + delay(1000L) // pretend we are doing something useful here, too + return 29 +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-compose-06.kt b/kotlinx-coroutines-core/jvm/test/guide/example-compose-06.kt new file mode 100644 index 0000000000..1dbbc80c93 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-compose-06.kt @@ -0,0 +1,28 @@ +// This file was automatically generated from composing-suspending-functions.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleCompose06 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + try { + failedConcurrentSum() + } catch(e: ArithmeticException) { + println("Computation failed with ArithmeticException") + } +} + +suspend fun failedConcurrentSum(): Int = coroutineScope { + val one = async { + try { + delay(Long.MAX_VALUE) // Emulates very long computation + 42 + } finally { + println("First child was cancelled") + } + } + val two = async { + println("Second child throws an exception") + throw ArithmeticException() + } + one.await() + two.await() +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-01.kt new file mode 100644 index 0000000000..1c6a362a96 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-01.kt @@ -0,0 +1,19 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext01 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + launch { // context of the parent, main runBlocking coroutine + println("main runBlocking : I'm working in thread ${Thread.currentThread().name}") + } + launch(Dispatchers.Unconfined) { // not confined -- will work with main thread + println("Unconfined : I'm working in thread ${Thread.currentThread().name}") + } + launch(Dispatchers.Default) { // will get dispatched to DefaultDispatcher + println("Default : I'm working in thread ${Thread.currentThread().name}") + } + launch(newSingleThreadContext("MyOwnThread")) { // will get its own new thread + println("newSingleThreadContext: I'm working in thread ${Thread.currentThread().name}") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-02.kt new file mode 100644 index 0000000000..3aa64e9725 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-02.kt @@ -0,0 +1,17 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext02 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + launch(Dispatchers.Unconfined) { // not confined -- will work with main thread + println("Unconfined : I'm working in thread ${Thread.currentThread().name}") + delay(500) + println("Unconfined : After delay in thread ${Thread.currentThread().name}") + } + launch { // context of the parent, main runBlocking coroutine + println("main runBlocking: I'm working in thread ${Thread.currentThread().name}") + delay(1000) + println("main runBlocking: After delay in thread ${Thread.currentThread().name}") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-03.kt new file mode 100644 index 0000000000..696852cbee --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-03.kt @@ -0,0 +1,18 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext03 + +import kotlinx.coroutines.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +fun main() = runBlocking { + val a = async { + log("I'm computing a piece of the answer") + 6 + } + val b = async { + log("I'm computing another piece of the answer") + 7 + } + log("The answer is ${a.await() * b.await()}") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-04.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-04.kt new file mode 100644 index 0000000000..9609df83a2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-04.kt @@ -0,0 +1,20 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext04 + +import kotlinx.coroutines.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +fun main() { + newSingleThreadContext("Ctx1").use { ctx1 -> + newSingleThreadContext("Ctx2").use { ctx2 -> + runBlocking(ctx1) { + log("Started in ctx1") + withContext(ctx2) { + log("Working in ctx2") + } + log("Back to ctx1") + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-05.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-05.kt new file mode 100644 index 0000000000..e459561472 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-05.kt @@ -0,0 +1,8 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext05 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + println("My job is ${coroutineContext[Job]}") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-06.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-06.kt new file mode 100644 index 0000000000..a90e5216eb --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-06.kt @@ -0,0 +1,27 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext06 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + // launch a coroutine to process some kind of incoming request + val request = launch { + // it spawns two other jobs + launch(Job()) { + println("job1: I run in my own Job and execute independently!") + delay(1000) + println("job1: I am not affected by cancellation of the request") + } + // and the other inherits the parent context + launch { + delay(100) + println("job2: I am a child of the request coroutine") + delay(1000) + println("job2: I will not execute this line if my parent request is cancelled") + } + } + delay(500) + request.cancel() // cancel processing of the request + println("main: Who has survived request cancellation?") + delay(1000) // delay the main thread for a second to see what happens +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-07.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-07.kt new file mode 100644 index 0000000000..847009671d --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-07.kt @@ -0,0 +1,19 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext07 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + // launch a coroutine to process some kind of incoming request + val request = launch { + repeat(3) { i -> // launch a few children jobs + launch { + delay((i + 1) * 200L) // variable delay 200ms, 400ms, 600ms + println("Coroutine $i is done") + } + } + println("request: I'm done and I don't explicitly join my children that are still active") + } + request.join() // wait for completion of the request, including all its children + println("Now processing of the request is complete") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-08.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-08.kt new file mode 100644 index 0000000000..bb871662b6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-08.kt @@ -0,0 +1,22 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext08 + +import kotlinx.coroutines.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +fun main() = runBlocking(CoroutineName("main")) { + log("Started main coroutine") + // run two background value computations + val v1 = async(CoroutineName("v1coroutine")) { + delay(500) + log("Computing v1") + 6 + } + val v2 = async(CoroutineName("v2coroutine")) { + delay(1000) + log("Computing v2") + 7 + } + log("The answer for v1 * v2 = ${v1.await() * v2.await()}") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-09.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-09.kt new file mode 100644 index 0000000000..a4f730b70a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-09.kt @@ -0,0 +1,10 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext09 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + launch(Dispatchers.Default + CoroutineName("test")) { + println("I'm working in thread ${Thread.currentThread().name}") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-10.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-10.kt new file mode 100644 index 0000000000..d1385a993f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-10.kt @@ -0,0 +1,32 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext10 + +import kotlinx.coroutines.* + +class Activity { + private val mainScope = CoroutineScope(Dispatchers.Default) // use Default for test purposes + + fun destroy() { + mainScope.cancel() + } + + fun doSomething() { + // launch ten coroutines for a demo, each working for a different time + repeat(10) { i -> + mainScope.launch { + delay((i + 1) * 200L) // variable delay 200ms, 400ms, ... etc + println("Coroutine $i is done") + } + } + } +} // class Activity ends + +fun main() = runBlocking { + val activity = Activity() + activity.doSomething() // run test function + println("Launched coroutines") + delay(500L) // delay for half a second + println("Destroying activity!") + activity.destroy() // cancels all coroutines + delay(1000) // visually confirm that they don't work +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-context-11.kt b/kotlinx-coroutines-core/jvm/test/guide/example-context-11.kt new file mode 100644 index 0000000000..2b0a521812 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-context-11.kt @@ -0,0 +1,18 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleContext11 + +import kotlinx.coroutines.* + +val threadLocal = ThreadLocal() // declare thread-local variable + +fun main() = runBlocking { + threadLocal.set("main") + println("Pre-main, current thread: ${Thread.currentThread()}, thread local value: '${threadLocal.get()}'") + val job = launch(Dispatchers.Default + threadLocal.asContextElement(value = "launch")) { + println("Launch start, current thread: ${Thread.currentThread()}, thread local value: '${threadLocal.get()}'") + yield() + println("After yield, current thread: ${Thread.currentThread()}, thread local value: '${threadLocal.get()}'") + } + job.join() + println("Post-main, current thread: ${Thread.currentThread()}, thread local value: '${threadLocal.get()}'") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-01.kt new file mode 100644 index 0000000000..a6ae6cac0f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-01.kt @@ -0,0 +1,24 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleExceptions01 + +import kotlinx.coroutines.* + +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { + val job = GlobalScope.launch { // root coroutine with launch + println("Throwing exception from launch") + throw IndexOutOfBoundsException() // Will be printed to the console by Thread.defaultUncaughtExceptionHandler + } + job.join() + println("Joined failed job") + val deferred = GlobalScope.async { // root coroutine with async + println("Throwing exception from async") + throw ArithmeticException() // Nothing is printed, relying on user to call await + } + try { + deferred.await() + println("Unreached") + } catch (e: ArithmeticException) { + println("Caught ArithmeticException") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-02.kt new file mode 100644 index 0000000000..c619d7998d --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-02.kt @@ -0,0 +1,18 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleExceptions02 + +import kotlinx.coroutines.* + +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception") + } + val job = GlobalScope.launch(handler) { // root coroutine, running in GlobalScope + throw AssertionError() + } + val deferred = GlobalScope.async(handler) { // also root, but async instead of launch + throw ArithmeticException() // Nothing will be printed, relying on user to call deferred.await() + } + joinAll(job, deferred) +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-03.kt new file mode 100644 index 0000000000..02bd6b8721 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-03.kt @@ -0,0 +1,23 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleExceptions03 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val job = launch { + val child = launch { + try { + delay(Long.MAX_VALUE) + } finally { + println("Child is cancelled") + } + } + yield() + println("Cancelling child") + child.cancel() + child.join() + yield() + println("Parent is not cancelled") + } + job.join() +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-04.kt b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-04.kt new file mode 100644 index 0000000000..cb8b4ff6ad --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-04.kt @@ -0,0 +1,30 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleExceptions04 + +import kotlinx.coroutines.* + +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception") + } + val job = GlobalScope.launch(handler) { + launch { // the first child + try { + delay(Long.MAX_VALUE) + } finally { + withContext(NonCancellable) { + println("Children are cancelled, but exception is not handled until all children terminate") + delay(100) + println("The first child finished its non cancellable block") + } + } + } + launch { // the second child + delay(10) + println("Second child throws an exception") + throw ArithmeticException() + } + } + job.join() +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-05.kt b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-05.kt new file mode 100644 index 0000000000..c7e543d25f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-05.kt @@ -0,0 +1,29 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleExceptions05 + +import kotlinx.coroutines.exceptions.* + +import kotlinx.coroutines.* +import java.io.* + +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception with suppressed ${exception.suppressed.contentToString()}") + } + val job = GlobalScope.launch(handler) { + launch { + try { + delay(Long.MAX_VALUE) // it gets cancelled when another sibling fails with IOException + } finally { + throw ArithmeticException() // the second exception + } + } + launch { + delay(100) + throw IOException() // the first exception + } + delay(Long.MAX_VALUE) + } + job.join() +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-06.kt b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-06.kt new file mode 100644 index 0000000000..5d88a1ed1b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-exceptions-06.kt @@ -0,0 +1,28 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleExceptions06 + +import kotlinx.coroutines.* +import java.io.* + +@OptIn(DelicateCoroutinesApi::class) +fun main() = runBlocking { + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception") + } + val job = GlobalScope.launch(handler) { + val innerJob = launch { // all this stack of coroutines will get cancelled + launch { + launch { + throw IOException() // the original exception + } + } + } + try { + innerJob.join() + } catch (e: CancellationException) { + println("Rethrowing CancellationException with original cause") + throw e // cancellation exception is rethrown, yet the original IOException gets to the handler + } + } + job.join() +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-01.kt new file mode 100644 index 0000000000..8db2596892 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-01.kt @@ -0,0 +1,8 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow01 + +fun simple(): List = listOf(1, 2, 3) + +fun main() { + simple().forEach { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-02.kt new file mode 100644 index 0000000000..1162b1eb9c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-02.kt @@ -0,0 +1,13 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow02 + +fun simple(): Sequence = sequence { // sequence builder + for (i in 1..3) { + Thread.sleep(100) // pretend we are computing it + yield(i) // yield next value + } +} + +fun main() { + simple().forEach { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-03.kt new file mode 100644 index 0000000000..57e466fdf6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-03.kt @@ -0,0 +1,13 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow03 + +import kotlinx.coroutines.* + +suspend fun simple(): List { + delay(1000) // pretend we are doing something asynchronous here + return listOf(1, 2, 3) +} + +fun main() = runBlocking { + simple().forEach { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-04.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-04.kt new file mode 100644 index 0000000000..9efed63b34 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-04.kt @@ -0,0 +1,24 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow04 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = flow { // flow builder + for (i in 1..3) { + delay(100) // pretend we are doing something useful here + emit(i) // emit next value + } +} + +fun main() = runBlocking { + // Launch a concurrent coroutine to check if the main thread is blocked + launch { + for (k in 1..3) { + println("I'm not blocked $k") + delay(100) + } + } + // Collect the flow + simple().collect { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-05.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-05.kt new file mode 100644 index 0000000000..93fcb194f4 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-05.kt @@ -0,0 +1,22 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow05 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = flow { + println("Flow started") + for (i in 1..3) { + delay(100) + emit(i) + } +} + +fun main() = runBlocking { + println("Calling simple function...") + val flow = simple() + println("Calling collect...") + flow.collect { value -> println(value) } + println("Calling collect again...") + flow.collect { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-06.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-06.kt new file mode 100644 index 0000000000..8abc161340 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-06.kt @@ -0,0 +1,20 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow06 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) + println("Emitting $i") + emit(i) + } +} + +fun main() = runBlocking { + withTimeoutOrNull(250) { // Timeout after 250ms + simple().collect { value -> println(value) } + } + println("Done") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-07.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-07.kt new file mode 100644 index 0000000000..03e2d2e860 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-07.kt @@ -0,0 +1,10 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow07 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { + // Convert an integer range to a flow + (1..3).asFlow().collect { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-08.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-08.kt new file mode 100644 index 0000000000..c834439d69 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-08.kt @@ -0,0 +1,16 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow08 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +suspend fun performRequest(request: Int): String { + delay(1000) // imitate long-running asynchronous work + return "response $request" +} + +fun main() = runBlocking { + (1..3).asFlow() // a flow of requests + .map { request -> performRequest(request) } + .collect { response -> println(response) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-09.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-09.kt new file mode 100644 index 0000000000..a151946239 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-09.kt @@ -0,0 +1,19 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow09 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +suspend fun performRequest(request: Int): String { + delay(1000) // imitate long-running asynchronous work + return "response $request" +} + +fun main() = runBlocking { + (1..3).asFlow() // a flow of requests + .transform { request -> + emit("Making request $request") + emit(performRequest(request)) + } + .collect { response -> println(response) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-10.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-10.kt new file mode 100644 index 0000000000..03981b6612 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-10.kt @@ -0,0 +1,22 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow10 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun numbers(): Flow = flow { + try { + emit(1) + emit(2) + println("This line will not execute") + emit(3) + } finally { + println("Finally in numbers") + } +} + +fun main() = runBlocking { + numbers() + .take(2) // take only the first two + .collect { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-11.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-11.kt new file mode 100644 index 0000000000..74359206cd --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-11.kt @@ -0,0 +1,12 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow11 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { + val sum = (1..5).asFlow() + .map { it * it } // squares of numbers from 1 to 5 + .reduce { a, b -> a + b } // sum them (terminal operator) + println(sum) +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-12.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-12.kt new file mode 100644 index 0000000000..806d2b8760 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-12.kt @@ -0,0 +1,19 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow12 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { + (1..5).asFlow() + .filter { + println("Filter $it") + it % 2 == 0 + } + .map { + println("Map $it") + "string $it" + }.collect { + println("Collect $it") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-13.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-13.kt new file mode 100644 index 0000000000..2ff0d084eb --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-13.kt @@ -0,0 +1,18 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow13 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +fun simple(): Flow = flow { + log("Started simple flow") + for (i in 1..3) { + emit(i) + } +} + +fun main() = runBlocking { + simple().collect { value -> log("Collected $value") } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-14.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-14.kt new file mode 100644 index 0000000000..26d2b4956e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-14.kt @@ -0,0 +1,19 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow14 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = flow { + // The WRONG way to change context for CPU-consuming code in flow builder + kotlinx.coroutines.withContext(Dispatchers.Default) { + for (i in 1..3) { + Thread.sleep(100) // pretend we are computing it in CPU-consuming way + emit(i) // emit next value + } + } +} + +fun main() = runBlocking { + simple().collect { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-15.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-15.kt new file mode 100644 index 0000000000..1a6e0fd374 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-15.kt @@ -0,0 +1,21 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow15 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") + +fun simple(): Flow = flow { + for (i in 1..3) { + Thread.sleep(100) // pretend we are computing it in CPU-consuming way + log("Emitting $i") + emit(i) // emit next value + } +}.flowOn(Dispatchers.Default) // RIGHT way to change context for CPU-consuming code in flow builder + +fun main() = runBlocking { + simple().collect { value -> + log("Collected $value") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-16.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-16.kt new file mode 100644 index 0000000000..35ec7a27d3 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-16.kt @@ -0,0 +1,23 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow16 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.system.* + +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) // pretend we are asynchronously waiting 100 ms + emit(i) // emit next value + } +} + +fun main() = runBlocking { + val time = measureTimeMillis { + simple().collect { value -> + delay(300) // pretend we are processing it for 300 ms + println(value) + } + } + println("Collected in $time ms") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-17.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-17.kt new file mode 100644 index 0000000000..4048f437f3 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-17.kt @@ -0,0 +1,25 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow17 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.system.* + +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) // pretend we are asynchronously waiting 100 ms + emit(i) // emit next value + } +} + +fun main() = runBlocking { + val time = measureTimeMillis { + simple() + .buffer() // buffer emissions, don't wait + .collect { value -> + delay(300) // pretend we are processing it for 300 ms + println(value) + } + } + println("Collected in $time ms") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-18.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-18.kt new file mode 100644 index 0000000000..61779b3809 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-18.kt @@ -0,0 +1,25 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow18 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.system.* + +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) // pretend we are asynchronously waiting 100 ms + emit(i) // emit next value + } +} + +fun main() = runBlocking { + val time = measureTimeMillis { + simple() + .conflate() // conflate emissions, don't process each one + .collect { value -> + delay(300) // pretend we are processing it for 300 ms + println(value) + } + } + println("Collected in $time ms") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-19.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-19.kt new file mode 100644 index 0000000000..82b02699fa --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-19.kt @@ -0,0 +1,25 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow19 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.system.* + +fun simple(): Flow = flow { + for (i in 1..3) { + delay(100) // pretend we are asynchronously waiting 100 ms + emit(i) // emit next value + } +} + +fun main() = runBlocking { + val time = measureTimeMillis { + simple() + .collectLatest { value -> // cancel & restart on the latest value + println("Collecting $value") + delay(300) // pretend we are processing it for 300 ms + println("Done $value") + } + } + println("Collected in $time ms") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-20.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-20.kt new file mode 100644 index 0000000000..f0d68019c7 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-20.kt @@ -0,0 +1,12 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow20 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { + val nums = (1..3).asFlow() // numbers 1..3 + val strs = flowOf("one", "two", "three") // strings + nums.zip(strs) { a, b -> "$a -> $b" } // compose a single string + .collect { println(it) } // collect and print +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-21.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-21.kt new file mode 100644 index 0000000000..9c3c22d16b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-21.kt @@ -0,0 +1,15 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow21 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { + val nums = (1..3).asFlow().onEach { delay(300) } // numbers 1..3 every 300 ms + val strs = flowOf("one", "two", "three").onEach { delay(400) } // strings every 400 ms + val startTime = currentTimeMillis() // remember the start time + nums.zip(strs) { a, b -> "$a -> $b" } // compose a single string with "zip" + .collect { value -> // collect and print + println("$value at ${currentTimeMillis() - startTime} ms from start") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-22.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-22.kt new file mode 100644 index 0000000000..9cf2734b60 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-22.kt @@ -0,0 +1,15 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow22 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { + val nums = (1..3).asFlow().onEach { delay(300) } // numbers 1..3 every 300 ms + val strs = flowOf("one", "two", "three").onEach { delay(400) } // strings every 400 ms + val startTime = currentTimeMillis() // remember the start time + nums.combine(strs) { a, b -> "$a -> $b" } // compose a single string with "combine" + .collect { value -> // collect and print + println("$value at ${currentTimeMillis() - startTime} ms from start") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-23.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-23.kt new file mode 100644 index 0000000000..4ff44881cb --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-23.kt @@ -0,0 +1,20 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow23 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun requestFlow(i: Int): Flow = flow { + emit("$i: First") + delay(500) // wait 500 ms + emit("$i: Second") +} + +fun main() = runBlocking { + val startTime = currentTimeMillis() // remember the start time + (1..3).asFlow().onEach { delay(100) } // emit a number every 100 ms + .flatMapConcat { requestFlow(it) } + .collect { value -> // collect and print + println("$value at ${currentTimeMillis() - startTime} ms from start") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-24.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-24.kt new file mode 100644 index 0000000000..f4132be50b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-24.kt @@ -0,0 +1,20 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow24 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun requestFlow(i: Int): Flow = flow { + emit("$i: First") + delay(500) // wait 500 ms + emit("$i: Second") +} + +fun main() = runBlocking { + val startTime = currentTimeMillis() // remember the start time + (1..3).asFlow().onEach { delay(100) } // a number every 100 ms + .flatMapMerge { requestFlow(it) } + .collect { value -> // collect and print + println("$value at ${currentTimeMillis() - startTime} ms from start") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-25.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-25.kt new file mode 100644 index 0000000000..a09def0018 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-25.kt @@ -0,0 +1,20 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow25 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun requestFlow(i: Int): Flow = flow { + emit("$i: First") + delay(500) // wait 500 ms + emit("$i: Second") +} + +fun main() = runBlocking { + val startTime = currentTimeMillis() // remember the start time + (1..3).asFlow().onEach { delay(100) } // a number every 100 ms + .flatMapLatest { requestFlow(it) } + .collect { value -> // collect and print + println("$value at ${currentTimeMillis() - startTime} ms from start") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-26.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-26.kt new file mode 100644 index 0000000000..1297f3f02c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-26.kt @@ -0,0 +1,23 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow26 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) // emit next value + } +} + +fun main() = runBlocking { + try { + simple().collect { value -> + println(value) + check(value <= 1) { "Collected $value" } + } + } catch (e: Throwable) { + println("Caught $e") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-27.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-27.kt new file mode 100644 index 0000000000..29ed17cf13 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-27.kt @@ -0,0 +1,25 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow27 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = + flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) // emit next value + } + } + .map { value -> + check(value <= 1) { "Crashed on $value" } + "string $value" + } + +fun main() = runBlocking { + try { + simple().collect { value -> println(value) } + } catch (e: Throwable) { + println("Caught $e") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-28.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-28.kt new file mode 100644 index 0000000000..6514f157da --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-28.kt @@ -0,0 +1,23 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow28 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = + flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) // emit next value + } + } + .map { value -> + check(value <= 1) { "Crashed on $value" } + "string $value" + } + +fun main() = runBlocking { + simple() + .catch { e -> emit("Caught $e") } // emit on exception + .collect { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-29.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-29.kt new file mode 100644 index 0000000000..536660af19 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-29.kt @@ -0,0 +1,21 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow29 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) + } +} + +fun main() = runBlocking { + simple() + .catch { e -> println("Caught $e") } // does not catch downstream exceptions + .collect { value -> + check(value <= 1) { "Collected $value" } + println(value) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-30.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-30.kt new file mode 100644 index 0000000000..0cff2ab142 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-30.kt @@ -0,0 +1,22 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow30 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = flow { + for (i in 1..3) { + println("Emitting $i") + emit(i) + } +} + +fun main() = runBlocking { + simple() + .onEach { value -> + check(value <= 1) { "Collected $value" } + println(value) + } + .catch { e -> println("Caught $e") } + .collect() +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-31.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-31.kt new file mode 100644 index 0000000000..4f833f5bf2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-31.kt @@ -0,0 +1,15 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow31 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = (1..3).asFlow() + +fun main() = runBlocking { + try { + simple().collect { value -> println(value) } + } finally { + println("Done") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-32.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-32.kt new file mode 100644 index 0000000000..d0079934f1 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-32.kt @@ -0,0 +1,13 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow32 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = (1..3).asFlow() + +fun main() = runBlocking { + simple() + .onCompletion { println("Done") } + .collect { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-33.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-33.kt new file mode 100644 index 0000000000..0d4f7f0544 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-33.kt @@ -0,0 +1,17 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow33 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = flow { + emit(1) + throw RuntimeException() +} + +fun main() = runBlocking { + simple() + .onCompletion { cause -> if (cause != null) println("Flow completed exceptionally") } + .catch { cause -> println("Caught exception") } + .collect { value -> println(value) } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-34.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-34.kt new file mode 100644 index 0000000000..f27545dd9d --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-34.kt @@ -0,0 +1,16 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow34 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun simple(): Flow = (1..3).asFlow() + +fun main() = runBlocking { + simple() + .onCompletion { cause -> println("Flow completed with $cause") } + .collect { value -> + check(value <= 1) { "Collected $value" } + println(value) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-35.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-35.kt new file mode 100644 index 0000000000..a2b15e87df --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-35.kt @@ -0,0 +1,15 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow35 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +// Imitate a flow of events +fun events(): Flow = (1..3).asFlow().onEach { delay(100) } + +fun main() = runBlocking { + events() + .onEach { event -> println("Event: $event") } + .collect() // <--- Collecting the flow waits + println("Done") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-36.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-36.kt new file mode 100644 index 0000000000..c58035bcdc --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-36.kt @@ -0,0 +1,15 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow36 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +// Imitate a flow of events +fun events(): Flow = (1..3).asFlow().onEach { delay(100) } + +fun main() = runBlocking { + events() + .onEach { event -> println("Event: $event") } + .launchIn(this) // <--- Launching the flow in a separate coroutine + println("Done") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-37.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-37.kt new file mode 100644 index 0000000000..6d35f6ff79 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-37.kt @@ -0,0 +1,19 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow37 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun foo(): Flow = flow { + for (i in 1..5) { + println("Emitting $i") + emit(i) + } +} + +fun main() = runBlocking { + foo().collect { value -> + if (value == 3) cancel() + println(value) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-38.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-38.kt new file mode 100644 index 0000000000..169a3fde7b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-38.kt @@ -0,0 +1,12 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow38 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { + (1..5).asFlow().collect { value -> + if (value == 3) cancel() + println(value) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-flow-39.kt b/kotlinx-coroutines-core/jvm/test/guide/example-flow-39.kt new file mode 100644 index 0000000000..a79e8a55c5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-flow-39.kt @@ -0,0 +1,12 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleFlow39 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +fun main() = runBlocking { + (1..5).asFlow().cancellable().collect { value -> + if (value == 3) cancel() + println(value) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-select-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-select-01.kt new file mode 100644 index 0000000000..5bc6710817 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-select-01.kt @@ -0,0 +1,40 @@ +// This file was automatically generated from select-expression.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSelect01 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* + +fun CoroutineScope.fizz() = produce { + while (true) { // sends "Fizz" every 500 ms + delay(500) + send("Fizz") + } +} + +fun CoroutineScope.buzz() = produce { + while (true) { // sends "Buzz!" every 1000 ms + delay(1000) + send("Buzz!") + } +} + +suspend fun selectFizzBuzz(fizz: ReceiveChannel, buzz: ReceiveChannel) { + select { // means that this select expression does not produce any result + fizz.onReceive { value -> // this is the first select clause + println("fizz -> '$value'") + } + buzz.onReceive { value -> // this is the second select clause + println("buzz -> '$value'") + } + } +} + +fun main() = runBlocking { + val fizz = fizz() + val buzz = buzz() + repeat(7) { + selectFizzBuzz(fizz, buzz) + } + coroutineContext.cancelChildren() // cancel fizz & buzz coroutines +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-select-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-select-02.kt new file mode 100644 index 0000000000..3aa101a7c1 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-select-02.kt @@ -0,0 +1,39 @@ +// This file was automatically generated from select-expression.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSelect02 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* + +suspend fun selectAorB(a: ReceiveChannel, b: ReceiveChannel): String = + select { + a.onReceiveCatching { it -> + val value = it.getOrNull() + if (value != null) { + "a -> '$value'" + } else { + "Channel 'a' is closed" + } + } + b.onReceiveCatching { it -> + val value = it.getOrNull() + if (value != null) { + "b -> '$value'" + } else { + "Channel 'b' is closed" + } + } + } + +fun main() = runBlocking { + val a = produce { + repeat(4) { send("Hello $it") } + } + val b = produce { + repeat(4) { send("World $it") } + } + repeat(8) { // print first eight results + println(selectAorB(a, b)) + } + coroutineContext.cancelChildren() +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-select-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-select-03.kt new file mode 100644 index 0000000000..c9f3d37e74 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-select-03.kt @@ -0,0 +1,29 @@ +// This file was automatically generated from select-expression.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSelect03 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* + +fun CoroutineScope.produceNumbers(side: SendChannel) = produce { + for (num in 1..10) { // produce 10 numbers from 1 to 10 + delay(100) // every 100 ms + select { + onSend(num) {} // Send to the primary channel + side.onSend(num) {} // or to the side channel + } + } +} + +fun main() = runBlocking { + val side = Channel() // allocate side channel + launch { // this is a very fast consumer for the side channel + side.consumeEach { println("Side channel has $it") } + } + produceNumbers(side).consumeEach { + println("Consuming $it") + delay(250) // let us digest the consumed number properly, do not hurry + } + println("Done consuming") + coroutineContext.cancelChildren() +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-select-04.kt b/kotlinx-coroutines-core/jvm/test/guide/example-select-04.kt new file mode 100644 index 0000000000..4837957b7f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-select-04.kt @@ -0,0 +1,30 @@ +// This file was automatically generated from select-expression.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSelect04 + +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import java.util.* + +fun CoroutineScope.asyncString(time: Int) = async { + delay(time.toLong()) + "Waited for $time ms" +} + +fun CoroutineScope.asyncStringsList(): List> { + val random = Random(3) + return List(12) { asyncString(random.nextInt(1000)) } +} + +fun main() = runBlocking { + val list = asyncStringsList() + val result = select { + list.withIndex().forEach { (index, deferred) -> + deferred.onAwait { answer -> + "Deferred $index produced answer '$answer'" + } + } + } + println(result) + val countActive = list.count { it.isActive } + println("$countActive coroutines are still active") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-select-05.kt b/kotlinx-coroutines-core/jvm/test/guide/example-select-05.kt new file mode 100644 index 0000000000..c8b2e9987b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-select-05.kt @@ -0,0 +1,50 @@ +// This file was automatically generated from select-expression.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSelect05 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* + +fun CoroutineScope.switchMapDeferreds(input: ReceiveChannel>) = produce { + var current = input.receive() // start with first received deferred value + while (isActive) { // loop while not cancelled/closed + val next = select?> { // return next deferred value from this select or null + input.onReceiveCatching { update -> + update.getOrNull() + } + current.onAwait { value -> + send(value) // send value that current deferred has produced + input.receiveCatching().getOrNull() // and use the next deferred from the input channel + } + } + if (next == null) { + println("Channel was closed") + break // out of loop + } else { + current = next + } + } +} + +fun CoroutineScope.asyncString(str: String, time: Long) = async { + delay(time) + str +} + +fun main() = runBlocking { + val chan = Channel>() // the channel for test + launch { // launch printing coroutine + for (s in switchMapDeferreds(chan)) + println(s) // print each received string + } + chan.send(asyncString("BEGIN", 100)) + delay(200) // enough time for "BEGIN" to be produced + chan.send(asyncString("Slow", 500)) + delay(100) // not enough time to produce slow + chan.send(asyncString("Replace", 100)) + delay(500) // give it time before the last one + chan.send(asyncString("END", 500)) + delay(1000) // give it time to process + chan.close() // close the channel ... + delay(500) // and wait some time to let it finish +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-supervision-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-supervision-01.kt new file mode 100644 index 0000000000..a5cd1fd13f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-supervision-01.kt @@ -0,0 +1,32 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSupervision01 + +import kotlinx.coroutines.* + +fun main() = runBlocking { + val supervisor = SupervisorJob() + with(CoroutineScope(coroutineContext + supervisor)) { + // launch the first child -- its exception is ignored for this example (don't do this in practice!) + val firstChild = launch(CoroutineExceptionHandler { _, _ -> }) { + println("The first child is failing") + throw AssertionError("The first child is cancelled") + } + // launch the second child + val secondChild = launch { + firstChild.join() + // Cancellation of the first child is not propagated to the second child + println("The first child is cancelled: ${firstChild.isCancelled}, but the second one is still active") + try { + delay(Long.MAX_VALUE) + } finally { + // But cancellation of the supervisor is propagated + println("The second child is cancelled because the supervisor was cancelled") + } + } + // wait until the first child fails & completes + firstChild.join() + println("Cancelling the supervisor") + supervisor.cancel() + secondChild.join() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-supervision-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-supervision-02.kt new file mode 100644 index 0000000000..3c82527b8f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-supervision-02.kt @@ -0,0 +1,26 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSupervision02 + +import kotlin.coroutines.* +import kotlinx.coroutines.* + +fun main() = runBlocking { + try { + supervisorScope { + val child = launch { + try { + println("The child is sleeping") + delay(Long.MAX_VALUE) + } finally { + println("The child is cancelled") + } + } + // Give our child a chance to execute and print using yield + yield() + println("Throwing an exception from the scope") + throw AssertionError() + } + } catch(e: AssertionError) { + println("Caught an assertion error") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-supervision-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-supervision-03.kt new file mode 100644 index 0000000000..e23fe7f2db --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-supervision-03.kt @@ -0,0 +1,19 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSupervision03 + +import kotlin.coroutines.* +import kotlinx.coroutines.* + +fun main() = runBlocking { + val handler = CoroutineExceptionHandler { _, exception -> + println("CoroutineExceptionHandler got $exception") + } + supervisorScope { + val child = launch(handler) { + println("The child throws an exception") + throw AssertionError() + } + println("The scope is completing") + } + println("The scope is completed") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-sync-01.kt b/kotlinx-coroutines-core/jvm/test/guide/example-sync-01.kt new file mode 100644 index 0000000000..69c01b7cb8 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-sync-01.kt @@ -0,0 +1,31 @@ +// This file was automatically generated from shared-mutable-state-and-concurrency.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSync01 + +import kotlinx.coroutines.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +var counter = 0 + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + counter++ + } + } + println("Counter = $counter") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-sync-02.kt b/kotlinx-coroutines-core/jvm/test/guide/example-sync-02.kt new file mode 100644 index 0000000000..971c6e52e5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-sync-02.kt @@ -0,0 +1,32 @@ +// This file was automatically generated from shared-mutable-state-and-concurrency.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSync02 + +import kotlinx.coroutines.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +@Volatile // in Kotlin `volatile` is an annotation +var counter = 0 + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + counter++ + } + } + println("Counter = $counter") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-sync-03.kt b/kotlinx-coroutines-core/jvm/test/guide/example-sync-03.kt new file mode 100644 index 0000000000..e3b3b94814 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-sync-03.kt @@ -0,0 +1,32 @@ +// This file was automatically generated from shared-mutable-state-and-concurrency.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSync03 + +import kotlinx.coroutines.* +import java.util.concurrent.atomic.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +val counter = AtomicInteger() + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + counter.incrementAndGet() + } + } + println("Counter = $counter") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-sync-04.kt b/kotlinx-coroutines-core/jvm/test/guide/example-sync-04.kt new file mode 100644 index 0000000000..382a834793 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-sync-04.kt @@ -0,0 +1,35 @@ +// This file was automatically generated from shared-mutable-state-and-concurrency.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSync04 + +import kotlinx.coroutines.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +val counterContext = newSingleThreadContext("CounterContext") +var counter = 0 + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + // confine each increment to a single-threaded context + withContext(counterContext) { + counter++ + } + } + } + println("Counter = $counter") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-sync-05.kt b/kotlinx-coroutines-core/jvm/test/guide/example-sync-05.kt new file mode 100644 index 0000000000..597674ae2f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-sync-05.kt @@ -0,0 +1,33 @@ +// This file was automatically generated from shared-mutable-state-and-concurrency.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSync05 + +import kotlinx.coroutines.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +val counterContext = newSingleThreadContext("CounterContext") +var counter = 0 + +fun main() = runBlocking { + // confine everything to a single-threaded context + withContext(counterContext) { + massiveRun { + counter++ + } + } + println("Counter = $counter") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-sync-06.kt b/kotlinx-coroutines-core/jvm/test/guide/example-sync-06.kt new file mode 100644 index 0000000000..4fc77676bf --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-sync-06.kt @@ -0,0 +1,36 @@ +// This file was automatically generated from shared-mutable-state-and-concurrency.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSync06 + +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +val mutex = Mutex() +var counter = 0 + +fun main() = runBlocking { + withContext(Dispatchers.Default) { + massiveRun { + // protect each increment with lock + mutex.withLock { + counter++ + } + } + } + println("Counter = $counter") +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/example-sync-07.kt b/kotlinx-coroutines-core/jvm/test/guide/example-sync-07.kt new file mode 100644 index 0000000000..14c452c2f7 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/example-sync-07.kt @@ -0,0 +1,51 @@ +// This file was automatically generated from shared-mutable-state-and-concurrency.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.exampleSync07 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.system.* + +suspend fun massiveRun(action: suspend () -> Unit) { + val n = 100 // number of coroutines to launch + val k = 1000 // times an action is repeated by each coroutine + val time = measureTimeMillis { + coroutineScope { // scope for coroutines + repeat(n) { + launch { + repeat(k) { action() } + } + } + } + } + println("Completed ${n * k} actions in $time ms") +} + +// Message types for counterActor +sealed class CounterMsg +object IncCounter : CounterMsg() // one-way message to increment counter +class GetCounter(val response: CompletableDeferred) : CounterMsg() // a request with reply + +// This function launches a new counter actor +fun CoroutineScope.counterActor() = actor { + var counter = 0 // actor state + for (msg in channel) { // iterate over incoming messages + when (msg) { + is IncCounter -> counter++ + is GetCounter -> msg.response.complete(counter) + } + } +} + +fun main() = runBlocking { + val counter = counterActor() // create the actor + withContext(Dispatchers.Default) { + massiveRun { + counter.send(IncCounter) + } + } + // send a message to get a counter value from an actor + val response = CompletableDeferred() + counter.send(GetCounter(response)) + println("Counter = ${response.await()}") + counter.close() // shutdown the actor +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/BasicsGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/BasicsGuideTest.kt new file mode 100644 index 0000000000..8452f4344e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/test/BasicsGuideTest.kt @@ -0,0 +1,57 @@ +// This file was automatically generated from coroutines-basics.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class BasicsGuideTest { + @Test + fun testExampleBasic01() { + test("ExampleBasic01") { kotlinx.coroutines.guide.exampleBasic01.main() }.verifyLines( + "Hello", + "World!" + ) + } + + @Test + fun testExampleBasic02() { + test("ExampleBasic02") { kotlinx.coroutines.guide.exampleBasic02.main() }.verifyLines( + "Hello", + "World!" + ) + } + + @Test + fun testExampleBasic03() { + test("ExampleBasic03") { kotlinx.coroutines.guide.exampleBasic03.main() }.verifyLines( + "Hello", + "World!" + ) + } + + @Test + fun testExampleBasic04() { + test("ExampleBasic04") { kotlinx.coroutines.guide.exampleBasic04.main() }.verifyLines( + "Hello", + "World 1", + "World 2", + "Done" + ) + } + + @Test + fun testExampleBasic05() { + test("ExampleBasic05") { kotlinx.coroutines.guide.exampleBasic05.main() }.verifyLines( + "Hello", + "World!", + "Done" + ) + } + + @Test + fun testExampleBasic06() { + test("ExampleBasic06") { kotlinx.coroutines.guide.exampleBasic06.main() }.also { lines -> + check(lines.size == 1 && lines[0] == ".".repeat(50_000)) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/CancellationGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/CancellationGuideTest.kt new file mode 100644 index 0000000000..f09b762288 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/test/CancellationGuideTest.kt @@ -0,0 +1,94 @@ +// This file was automatically generated from cancellation-and-timeouts.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class CancellationGuideTest { + @Test + fun testExampleCancel01() { + test("ExampleCancel01") { kotlinx.coroutines.guide.exampleCancel01.main() }.verifyLines( + "job: I'm sleeping 0 ...", + "job: I'm sleeping 1 ...", + "job: I'm sleeping 2 ...", + "main: I'm tired of waiting!", + "main: Now I can quit." + ) + } + + @Test + fun testExampleCancel02() { + test("ExampleCancel02") { kotlinx.coroutines.guide.exampleCancel02.main() }.verifyLines( + "job: I'm sleeping 0 ...", + "job: I'm sleeping 1 ...", + "job: I'm sleeping 2 ...", + "main: I'm tired of waiting!", + "job: I'm sleeping 3 ...", + "job: I'm sleeping 4 ...", + "main: Now I can quit." + ) + } + + @Test + fun testExampleCancel04() { + test("ExampleCancel04") { kotlinx.coroutines.guide.exampleCancel04.main() }.verifyLines( + "job: I'm sleeping 0 ...", + "job: I'm sleeping 1 ...", + "job: I'm sleeping 2 ...", + "main: I'm tired of waiting!", + "main: Now I can quit." + ) + } + + @Test + fun testExampleCancel05() { + test("ExampleCancel05") { kotlinx.coroutines.guide.exampleCancel05.main() }.verifyLines( + "job: I'm sleeping 0 ...", + "job: I'm sleeping 1 ...", + "job: I'm sleeping 2 ...", + "main: I'm tired of waiting!", + "job: I'm running finally", + "main: Now I can quit." + ) + } + + @Test + fun testExampleCancel06() { + test("ExampleCancel06") { kotlinx.coroutines.guide.exampleCancel06.main() }.verifyLines( + "job: I'm sleeping 0 ...", + "job: I'm sleeping 1 ...", + "job: I'm sleeping 2 ...", + "main: I'm tired of waiting!", + "job: I'm running finally", + "job: And I've just delayed for 1 sec because I'm non-cancellable", + "main: Now I can quit." + ) + } + + @Test + fun testExampleCancel07() { + test("ExampleCancel07") { kotlinx.coroutines.guide.exampleCancel07.main() }.verifyLinesStartWith( + "I'm sleeping 0 ...", + "I'm sleeping 1 ...", + "I'm sleeping 2 ...", + "Exception in thread \"main\" kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 1300 ms" + ) + } + + @Test + fun testExampleCancel08() { + test("ExampleCancel08") { kotlinx.coroutines.guide.exampleCancel08.main() }.verifyLines( + "I'm sleeping 0 ...", + "I'm sleeping 1 ...", + "I'm sleeping 2 ...", + "Result is null" + ) + } + + @Test + fun testExampleCancel10() { + test("ExampleCancel10") { kotlinx.coroutines.guide.exampleCancel10.main() }.verifyLines( + "0" + ) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/ChannelsGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/ChannelsGuideTest.kt new file mode 100644 index 0000000000..97aa1da836 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/test/ChannelsGuideTest.kt @@ -0,0 +1,123 @@ +// This file was automatically generated from channels.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class ChannelsGuideTest { + @Test + fun testExampleChannel01() { + test("ExampleChannel01") { kotlinx.coroutines.guide.exampleChannel01.main() }.verifyLines( + "1", + "4", + "9", + "16", + "25", + "Done!" + ) + } + + @Test + fun testExampleChannel02() { + test("ExampleChannel02") { kotlinx.coroutines.guide.exampleChannel02.main() }.verifyLines( + "1", + "4", + "9", + "16", + "25", + "Done!" + ) + } + + @Test + fun testExampleChannel03() { + test("ExampleChannel03") { kotlinx.coroutines.guide.exampleChannel03.main() }.verifyLines( + "1", + "4", + "9", + "16", + "25", + "Done!" + ) + } + + @Test + fun testExampleChannel04() { + test("ExampleChannel04") { kotlinx.coroutines.guide.exampleChannel04.main() }.verifyLines( + "1", + "4", + "9", + "16", + "25", + "Done!" + ) + } + + @Test + fun testExampleChannel05() { + test("ExampleChannel05") { kotlinx.coroutines.guide.exampleChannel05.main() }.verifyLines( + "2", + "3", + "5", + "7", + "11", + "13", + "17", + "19", + "23", + "29" + ) + } + + @Test + fun testExampleChannel06() { + test("ExampleChannel06") { kotlinx.coroutines.guide.exampleChannel06.main() }.also { lines -> + check(lines.size == 10 && lines.withIndex().all { (i, line) -> line.startsWith("Processor #") && line.endsWith(" received ${i + 1}") }) + } + } + + @Test + fun testExampleChannel07() { + test("ExampleChannel07") { kotlinx.coroutines.guide.exampleChannel07.main() }.verifyLines( + "foo", + "foo", + "BAR!", + "foo", + "foo", + "BAR!" + ) + } + + @Test + fun testExampleChannel08() { + test("ExampleChannel08") { kotlinx.coroutines.guide.exampleChannel08.main() }.verifyLines( + "Sending 0", + "Sending 1", + "Sending 2", + "Sending 3", + "Sending 4" + ) + } + + @Test + fun testExampleChannel09() { + test("ExampleChannel09") { kotlinx.coroutines.guide.exampleChannel09.main() }.verifyLines( + "ping Ball(hits=1)", + "pong Ball(hits=2)", + "ping Ball(hits=3)", + "pong Ball(hits=4)" + ) + } + + @Test + fun testExampleChannel10() { + test("ExampleChannel10") { kotlinx.coroutines.guide.exampleChannel10.main() }.verifyLines( + "Initial element is available immediately: kotlin.Unit", + "Next element is not ready in 100 ms: null", + "Next element is ready in 200 ms: kotlin.Unit", + "Consumer pauses for 300ms", + "Next element is available immediately after large consumer delay: kotlin.Unit", + "Next element is ready in 100ms after consumer pause in 300ms: kotlin.Unit" + ) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/ComposingGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/ComposingGuideTest.kt new file mode 100644 index 0000000000..26682765e7 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/test/ComposingGuideTest.kt @@ -0,0 +1,56 @@ +// This file was automatically generated from composing-suspending-functions.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class ComposingGuideTest { + @Test + fun testExampleCompose01() { + test("ExampleCompose01") { kotlinx.coroutines.guide.exampleCompose01.main() }.verifyLinesArbitraryTime( + "The answer is 42", + "Completed in 2017 ms" + ) + } + + @Test + fun testExampleCompose02() { + test("ExampleCompose02") { kotlinx.coroutines.guide.exampleCompose02.main() }.verifyLinesArbitraryTime( + "The answer is 42", + "Completed in 1017 ms" + ) + } + + @Test + fun testExampleCompose03() { + test("ExampleCompose03") { kotlinx.coroutines.guide.exampleCompose03.main() }.verifyLinesArbitraryTime( + "The answer is 42", + "Completed in 1017 ms" + ) + } + + @Test + fun testExampleCompose04() { + test("ExampleCompose04") { kotlinx.coroutines.guide.exampleCompose04.main() }.verifyLinesArbitraryTime( + "The answer is 42", + "Completed in 1085 ms" + ) + } + + @Test + fun testExampleCompose05() { + test("ExampleCompose05") { kotlinx.coroutines.guide.exampleCompose05.main() }.verifyLinesArbitraryTime( + "The answer is 42", + "Completed in 1017 ms" + ) + } + + @Test + fun testExampleCompose06() { + test("ExampleCompose06") { kotlinx.coroutines.guide.exampleCompose06.main() }.verifyLines( + "Second child throws an exception", + "First child was cancelled", + "Computation failed with ArithmeticException" + ) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/DispatcherGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/DispatcherGuideTest.kt new file mode 100644 index 0000000000..1cf6c2bd4d --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/test/DispatcherGuideTest.kt @@ -0,0 +1,110 @@ +// This file was automatically generated from coroutine-context-and-dispatchers.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class DispatcherGuideTest { + @Test + fun testExampleContext01() { + test("ExampleContext01") { kotlinx.coroutines.guide.exampleContext01.main() }.verifyLinesStartUnordered( + "Unconfined : I'm working in thread main", + "Default : I'm working in thread DefaultDispatcher-worker-1", + "newSingleThreadContext: I'm working in thread MyOwnThread", + "main runBlocking : I'm working in thread main" + ) + } + + @Test + fun testExampleContext02() { + test("ExampleContext02") { kotlinx.coroutines.guide.exampleContext02.main() }.verifyLinesStart( + "Unconfined : I'm working in thread main", + "main runBlocking: I'm working in thread main", + "Unconfined : After delay in thread kotlinx.coroutines.DefaultExecutor", + "main runBlocking: After delay in thread main" + ) + } + + @Test + fun testExampleContext03() { + test("ExampleContext03") { kotlinx.coroutines.guide.exampleContext03.main() }.verifyLinesFlexibleThread( + "[main @coroutine#2] I'm computing a piece of the answer", + "[main @coroutine#3] I'm computing another piece of the answer", + "[main @coroutine#1] The answer is 42" + ) + } + + @Test + fun testExampleContext04() { + test("ExampleContext04") { kotlinx.coroutines.guide.exampleContext04.main() }.verifyLines( + "[Ctx1 @coroutine#1] Started in ctx1", + "[Ctx2 @coroutine#1] Working in ctx2", + "[Ctx1 @coroutine#1] Back to ctx1" + ) + } + + @Test + fun testExampleContext05() { + test("ExampleContext05") { kotlinx.coroutines.guide.exampleContext05.main() }.also { lines -> + check(lines.size == 1 && lines[0].startsWith("My job is \"coroutine#1\":BlockingCoroutine{Active}@")) + } + } + + @Test + fun testExampleContext06() { + test("ExampleContext06") { kotlinx.coroutines.guide.exampleContext06.main() }.verifyLines( + "job1: I run in my own Job and execute independently!", + "job2: I am a child of the request coroutine", + "main: Who has survived request cancellation?", + "job1: I am not affected by cancellation of the request" + ) + } + + @Test + fun testExampleContext07() { + test("ExampleContext07") { kotlinx.coroutines.guide.exampleContext07.main() }.verifyLines( + "request: I'm done and I don't explicitly join my children that are still active", + "Coroutine 0 is done", + "Coroutine 1 is done", + "Coroutine 2 is done", + "Now processing of the request is complete" + ) + } + + @Test + fun testExampleContext08() { + test("ExampleContext08") { kotlinx.coroutines.guide.exampleContext08.main() }.verifyLinesFlexibleThread( + "[main @main#1] Started main coroutine", + "[main @v1coroutine#2] Computing v1", + "[main @v2coroutine#3] Computing v2", + "[main @main#1] The answer for v1 * v2 = 42" + ) + } + + @Test + fun testExampleContext09() { + test("ExampleContext09") { kotlinx.coroutines.guide.exampleContext09.main() }.verifyLinesFlexibleThread( + "I'm working in thread DefaultDispatcher-worker-1 @test#2" + ) + } + + @Test + fun testExampleContext10() { + test("ExampleContext10") { kotlinx.coroutines.guide.exampleContext10.main() }.verifyLines( + "Launched coroutines", + "Coroutine 0 is done", + "Coroutine 1 is done", + "Destroying activity!" + ) + } + + @Test + fun testExampleContext11() { + test("ExampleContext11") { kotlinx.coroutines.guide.exampleContext11.main() }.verifyLinesFlexibleThread( + "Pre-main, current thread: Thread[main @coroutine#1,5,main], thread local value: 'main'", + "Launch start, current thread: Thread[DefaultDispatcher-worker-1 @coroutine#2,5,main], thread local value: 'launch'", + "After yield, current thread: Thread[DefaultDispatcher-worker-2 @coroutine#2,5,main], thread local value: 'launch'", + "Post-main, current thread: Thread[main @coroutine#1,5,main], thread local value: 'main'" + ) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/ExceptionsGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/ExceptionsGuideTest.kt new file mode 100644 index 0000000000..dc600616e1 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/test/ExceptionsGuideTest.kt @@ -0,0 +1,89 @@ +// This file was automatically generated from exception-handling.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class ExceptionsGuideTest { + @Test + fun testExampleExceptions01() { + test("ExampleExceptions01") { kotlinx.coroutines.guide.exampleExceptions01.main() }.verifyExceptions( + "Throwing exception from launch", + "Exception in thread \"DefaultDispatcher-worker-1 @coroutine#2\" java.lang.IndexOutOfBoundsException", + "Joined failed job", + "Throwing exception from async", + "Caught ArithmeticException" + ) + } + + @Test + fun testExampleExceptions02() { + test("ExampleExceptions02") { kotlinx.coroutines.guide.exampleExceptions02.main() }.verifyLines( + "CoroutineExceptionHandler got java.lang.AssertionError" + ) + } + + @Test + fun testExampleExceptions03() { + test("ExampleExceptions03") { kotlinx.coroutines.guide.exampleExceptions03.main() }.verifyLines( + "Cancelling child", + "Child is cancelled", + "Parent is not cancelled" + ) + } + + @Test + fun testExampleExceptions04() { + test("ExampleExceptions04") { kotlinx.coroutines.guide.exampleExceptions04.main() }.verifyLines( + "Second child throws an exception", + "Children are cancelled, but exception is not handled until all children terminate", + "The first child finished its non cancellable block", + "CoroutineExceptionHandler got java.lang.ArithmeticException" + ) + } + + @Test + fun testExampleExceptions05() { + test("ExampleExceptions05") { kotlinx.coroutines.guide.exampleExceptions05.main() }.verifyLines( + "CoroutineExceptionHandler got java.io.IOException with suppressed [java.lang.ArithmeticException]" + ) + } + + @Test + fun testExampleExceptions06() { + test("ExampleExceptions06") { kotlinx.coroutines.guide.exampleExceptions06.main() }.verifyLines( + "Rethrowing CancellationException with original cause", + "CoroutineExceptionHandler got java.io.IOException" + ) + } + + @Test + fun testExampleSupervision01() { + test("ExampleSupervision01") { kotlinx.coroutines.guide.exampleSupervision01.main() }.verifyLines( + "The first child is failing", + "The first child is cancelled: true, but the second one is still active", + "Cancelling the supervisor", + "The second child is cancelled because the supervisor was cancelled" + ) + } + + @Test + fun testExampleSupervision02() { + test("ExampleSupervision02") { kotlinx.coroutines.guide.exampleSupervision02.main() }.verifyLines( + "The child is sleeping", + "Throwing an exception from the scope", + "The child is cancelled", + "Caught an assertion error" + ) + } + + @Test + fun testExampleSupervision03() { + test("ExampleSupervision03") { kotlinx.coroutines.guide.exampleSupervision03.main() }.verifyLines( + "The scope is completing", + "The child throws an exception", + "CoroutineExceptionHandler got java.lang.AssertionError", + "The scope is completed" + ) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/FlowGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/FlowGuideTest.kt new file mode 100644 index 0000000000..4735a9ce19 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/test/FlowGuideTest.kt @@ -0,0 +1,417 @@ +// This file was automatically generated from flow.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class FlowGuideTest { + @Test + fun testExampleFlow01() { + test("ExampleFlow01") { kotlinx.coroutines.guide.exampleFlow01.main() }.verifyLines( + "1", + "2", + "3" + ) + } + + @Test + fun testExampleFlow02() { + test("ExampleFlow02") { kotlinx.coroutines.guide.exampleFlow02.main() }.verifyLines( + "1", + "2", + "3" + ) + } + + @Test + fun testExampleFlow03() { + test("ExampleFlow03") { kotlinx.coroutines.guide.exampleFlow03.main() }.verifyLines( + "1", + "2", + "3" + ) + } + + @Test + fun testExampleFlow04() { + test("ExampleFlow04") { kotlinx.coroutines.guide.exampleFlow04.main() }.verifyLines( + "I'm not blocked 1", + "1", + "I'm not blocked 2", + "2", + "I'm not blocked 3", + "3" + ) + } + + @Test + fun testExampleFlow05() { + test("ExampleFlow05") { kotlinx.coroutines.guide.exampleFlow05.main() }.verifyLines( + "Calling simple function...", + "Calling collect...", + "Flow started", + "1", + "2", + "3", + "Calling collect again...", + "Flow started", + "1", + "2", + "3" + ) + } + + @Test + fun testExampleFlow06() { + test("ExampleFlow06") { kotlinx.coroutines.guide.exampleFlow06.main() }.verifyLines( + "Emitting 1", + "1", + "Emitting 2", + "2", + "Done" + ) + } + + @Test + fun testExampleFlow07() { + test("ExampleFlow07") { kotlinx.coroutines.guide.exampleFlow07.main() }.verifyLines( + "1", + "2", + "3" + ) + } + + @Test + fun testExampleFlow08() { + test("ExampleFlow08") { kotlinx.coroutines.guide.exampleFlow08.main() }.verifyLines( + "response 1", + "response 2", + "response 3" + ) + } + + @Test + fun testExampleFlow09() { + test("ExampleFlow09") { kotlinx.coroutines.guide.exampleFlow09.main() }.verifyLines( + "Making request 1", + "response 1", + "Making request 2", + "response 2", + "Making request 3", + "response 3" + ) + } + + @Test + fun testExampleFlow10() { + test("ExampleFlow10") { kotlinx.coroutines.guide.exampleFlow10.main() }.verifyLines( + "1", + "2", + "Finally in numbers" + ) + } + + @Test + fun testExampleFlow11() { + test("ExampleFlow11") { kotlinx.coroutines.guide.exampleFlow11.main() }.verifyLines( + "55" + ) + } + + @Test + fun testExampleFlow12() { + test("ExampleFlow12") { kotlinx.coroutines.guide.exampleFlow12.main() }.verifyLines( + "Filter 1", + "Filter 2", + "Map 2", + "Collect string 2", + "Filter 3", + "Filter 4", + "Map 4", + "Collect string 4", + "Filter 5" + ) + } + + @Test + fun testExampleFlow13() { + test("ExampleFlow13") { kotlinx.coroutines.guide.exampleFlow13.main() }.verifyLinesFlexibleThread( + "[main @coroutine#1] Started simple flow", + "[main @coroutine#1] Collected 1", + "[main @coroutine#1] Collected 2", + "[main @coroutine#1] Collected 3" + ) + } + + @Test + fun testExampleFlow14() { + test("ExampleFlow14") { kotlinx.coroutines.guide.exampleFlow14.main() }.verifyExceptions( + "Exception in thread \"main\" java.lang.IllegalStateException: Flow invariant is violated:", + "\t\tFlow was collected in [CoroutineId(1), \"coroutine#1\":BlockingCoroutine{Active}@5511c7f8, BlockingEventLoop@2eac3323],", + "\t\tbut emission happened in [CoroutineId(1), \"coroutine#1\":DispatchedCoroutine{Active}@2dae0000, Dispatchers.Default].", + "\t\tPlease refer to 'flow' documentation or use 'flowOn' instead", + "\tat ..." + ) + } + + @Test + fun testExampleFlow15() { + test("ExampleFlow15") { kotlinx.coroutines.guide.exampleFlow15.main() }.verifyLinesFlexibleThread( + "[DefaultDispatcher-worker-1 @coroutine#2] Emitting 1", + "[main @coroutine#1] Collected 1", + "[DefaultDispatcher-worker-1 @coroutine#2] Emitting 2", + "[main @coroutine#1] Collected 2", + "[DefaultDispatcher-worker-1 @coroutine#2] Emitting 3", + "[main @coroutine#1] Collected 3" + ) + } + + @Test + fun testExampleFlow16() { + test("ExampleFlow16") { kotlinx.coroutines.guide.exampleFlow16.main() }.verifyLinesArbitraryTime( + "1", + "2", + "3", + "Collected in 1220 ms" + ) + } + + @Test + fun testExampleFlow17() { + test("ExampleFlow17") { kotlinx.coroutines.guide.exampleFlow17.main() }.verifyLinesArbitraryTime( + "1", + "2", + "3", + "Collected in 1071 ms" + ) + } + + @Test + fun testExampleFlow18() { + test("ExampleFlow18") { kotlinx.coroutines.guide.exampleFlow18.main() }.verifyLinesArbitraryTime( + "1", + "3", + "Collected in 758 ms" + ) + } + + @Test + fun testExampleFlow19() { + test("ExampleFlow19") { kotlinx.coroutines.guide.exampleFlow19.main() }.verifyLinesArbitraryTime( + "Collecting 1", + "Collecting 2", + "Collecting 3", + "Done 3", + "Collected in 741 ms" + ) + } + + @Test + fun testExampleFlow20() { + test("ExampleFlow20") { kotlinx.coroutines.guide.exampleFlow20.main() }.verifyLines( + "1 -> one", + "2 -> two", + "3 -> three" + ) + } + + @Test + fun testExampleFlow21() { + test("ExampleFlow21") { kotlinx.coroutines.guide.exampleFlow21.main() }.verifyLinesArbitraryTime( + "1 -> one at 437 ms from start", + "2 -> two at 837 ms from start", + "3 -> three at 1243 ms from start" + ) + } + + @Test + fun testExampleFlow22() { + test("ExampleFlow22") { kotlinx.coroutines.guide.exampleFlow22.main() }.verifyLinesArbitraryTime( + "1 -> one at 452 ms from start", + "2 -> one at 651 ms from start", + "2 -> two at 854 ms from start", + "3 -> two at 952 ms from start", + "3 -> three at 1256 ms from start" + ) + } + + @Test + fun testExampleFlow23() { + test("ExampleFlow23") { kotlinx.coroutines.guide.exampleFlow23.main() }.verifyLinesArbitraryTime( + "1: First at 121 ms from start", + "1: Second at 622 ms from start", + "2: First at 727 ms from start", + "2: Second at 1227 ms from start", + "3: First at 1328 ms from start", + "3: Second at 1829 ms from start" + ) + } + + @Test + fun testExampleFlow24() { + test("ExampleFlow24") { kotlinx.coroutines.guide.exampleFlow24.main() }.verifyLinesArbitraryTime( + "1: First at 136 ms from start", + "2: First at 231 ms from start", + "3: First at 333 ms from start", + "1: Second at 639 ms from start", + "2: Second at 732 ms from start", + "3: Second at 833 ms from start" + ) + } + + @Test + fun testExampleFlow25() { + test("ExampleFlow25") { kotlinx.coroutines.guide.exampleFlow25.main() }.verifyLinesArbitraryTime( + "1: First at 142 ms from start", + "2: First at 322 ms from start", + "3: First at 425 ms from start", + "3: Second at 931 ms from start" + ) + } + + @Test + fun testExampleFlow26() { + test("ExampleFlow26") { kotlinx.coroutines.guide.exampleFlow26.main() }.verifyLines( + "Emitting 1", + "1", + "Emitting 2", + "2", + "Caught java.lang.IllegalStateException: Collected 2" + ) + } + + @Test + fun testExampleFlow27() { + test("ExampleFlow27") { kotlinx.coroutines.guide.exampleFlow27.main() }.verifyLines( + "Emitting 1", + "string 1", + "Emitting 2", + "Caught java.lang.IllegalStateException: Crashed on 2" + ) + } + + @Test + fun testExampleFlow28() { + test("ExampleFlow28") { kotlinx.coroutines.guide.exampleFlow28.main() }.verifyLines( + "Emitting 1", + "string 1", + "Emitting 2", + "Caught java.lang.IllegalStateException: Crashed on 2" + ) + } + + @Test + fun testExampleFlow29() { + test("ExampleFlow29") { kotlinx.coroutines.guide.exampleFlow29.main() }.verifyExceptions( + "Emitting 1", + "1", + "Emitting 2", + "Exception in thread \"main\" java.lang.IllegalStateException: Collected 2", + "\tat ..." + ) + } + + @Test + fun testExampleFlow30() { + test("ExampleFlow30") { kotlinx.coroutines.guide.exampleFlow30.main() }.verifyExceptions( + "Emitting 1", + "1", + "Emitting 2", + "Caught java.lang.IllegalStateException: Collected 2" + ) + } + + @Test + fun testExampleFlow31() { + test("ExampleFlow31") { kotlinx.coroutines.guide.exampleFlow31.main() }.verifyLines( + "1", + "2", + "3", + "Done" + ) + } + + @Test + fun testExampleFlow32() { + test("ExampleFlow32") { kotlinx.coroutines.guide.exampleFlow32.main() }.verifyLines( + "1", + "2", + "3", + "Done" + ) + } + + @Test + fun testExampleFlow33() { + test("ExampleFlow33") { kotlinx.coroutines.guide.exampleFlow33.main() }.verifyLines( + "1", + "Flow completed exceptionally", + "Caught exception" + ) + } + + @Test + fun testExampleFlow34() { + test("ExampleFlow34") { kotlinx.coroutines.guide.exampleFlow34.main() }.verifyExceptions( + "1", + "Flow completed with java.lang.IllegalStateException: Collected 2", + "Exception in thread \"main\" java.lang.IllegalStateException: Collected 2" + ) + } + + @Test + fun testExampleFlow35() { + test("ExampleFlow35") { kotlinx.coroutines.guide.exampleFlow35.main() }.verifyLines( + "Event: 1", + "Event: 2", + "Event: 3", + "Done" + ) + } + + @Test + fun testExampleFlow36() { + test("ExampleFlow36") { kotlinx.coroutines.guide.exampleFlow36.main() }.verifyLines( + "Done", + "Event: 1", + "Event: 2", + "Event: 3" + ) + } + + @Test + fun testExampleFlow37() { + test("ExampleFlow37") { kotlinx.coroutines.guide.exampleFlow37.main() }.verifyExceptions( + "Emitting 1", + "1", + "Emitting 2", + "2", + "Emitting 3", + "3", + "Emitting 4", + "Exception in thread \"main\" kotlinx.coroutines.JobCancellationException: BlockingCoroutine was cancelled; job=\"coroutine#1\":BlockingCoroutine{Cancelled}@6d7b4f4c" + ) + } + + @Test + fun testExampleFlow38() { + test("ExampleFlow38") { kotlinx.coroutines.guide.exampleFlow38.main() }.verifyExceptions( + "1", + "2", + "3", + "4", + "5", + "Exception in thread \"main\" kotlinx.coroutines.JobCancellationException: BlockingCoroutine was cancelled; job=\"coroutine#1\":BlockingCoroutine{Cancelled}@3327bd23" + ) + } + + @Test + fun testExampleFlow39() { + test("ExampleFlow39") { kotlinx.coroutines.guide.exampleFlow39.main() }.verifyExceptions( + "1", + "2", + "3", + "Exception in thread \"main\" kotlinx.coroutines.JobCancellationException: BlockingCoroutine was cancelled; job=\"coroutine#1\":BlockingCoroutine{Cancelled}@5ec0a365" + ) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/SelectGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/SelectGuideTest.kt new file mode 100644 index 0000000000..d0e879e01e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/test/SelectGuideTest.kt @@ -0,0 +1,69 @@ +// This file was automatically generated from select-expression.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class SelectGuideTest { + @Test + fun testExampleSelect01() { + test("ExampleSelect01") { kotlinx.coroutines.guide.exampleSelect01.main() }.verifyLines( + "fizz -> 'Fizz'", + "buzz -> 'Buzz!'", + "fizz -> 'Fizz'", + "fizz -> 'Fizz'", + "buzz -> 'Buzz!'", + "fizz -> 'Fizz'", + "fizz -> 'Fizz'" + ) + } + + @Test + fun testExampleSelect02() { + test("ExampleSelect02") { kotlinx.coroutines.guide.exampleSelect02.main() }.verifyLines( + "a -> 'Hello 0'", + "a -> 'Hello 1'", + "b -> 'World 0'", + "a -> 'Hello 2'", + "a -> 'Hello 3'", + "b -> 'World 1'", + "Channel 'a' is closed", + "Channel 'a' is closed" + ) + } + + @Test + fun testExampleSelect03() { + test("ExampleSelect03") { kotlinx.coroutines.guide.exampleSelect03.main() }.verifyLines( + "Consuming 1", + "Side channel has 2", + "Side channel has 3", + "Consuming 4", + "Side channel has 5", + "Side channel has 6", + "Consuming 7", + "Side channel has 8", + "Side channel has 9", + "Consuming 10", + "Done consuming" + ) + } + + @Test + fun testExampleSelect04() { + test("ExampleSelect04") { kotlinx.coroutines.guide.exampleSelect04.main() }.verifyLines( + "Deferred 4 produced answer 'Waited for 128 ms'", + "11 coroutines are still active" + ) + } + + @Test + fun testExampleSelect05() { + test("ExampleSelect05") { kotlinx.coroutines.guide.exampleSelect05.main() }.verifyLines( + "BEGIN", + "Replace", + "END", + "Channel was closed" + ) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/guide/test/SharedStateGuideTest.kt b/kotlinx-coroutines-core/jvm/test/guide/test/SharedStateGuideTest.kt new file mode 100644 index 0000000000..7e43650bf6 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/guide/test/SharedStateGuideTest.kt @@ -0,0 +1,55 @@ +// This file was automatically generated from shared-mutable-state-and-concurrency.md by Knit tool. Do not edit. +package kotlinx.coroutines.guide.test + +import kotlinx.coroutines.knit.* +import org.junit.Test + +class SharedStateGuideTest { + @Test + fun testExampleSync01() { + test("ExampleSync01") { kotlinx.coroutines.guide.exampleSync01.main() }.verifyLinesStart( + "Completed 100000 actions in", + "Counter =" + ) + } + + @Test + fun testExampleSync02() { + test("ExampleSync02") { kotlinx.coroutines.guide.exampleSync02.main() }.verifyLinesStart( + "Completed 100000 actions in", + "Counter =" + ) + } + + @Test + fun testExampleSync03() { + test("ExampleSync03") { kotlinx.coroutines.guide.exampleSync03.main() }.verifyLinesArbitraryTime( + "Completed 100000 actions in xxx ms", + "Counter = 100000" + ) + } + + @Test + fun testExampleSync04() { + test("ExampleSync04") { kotlinx.coroutines.guide.exampleSync04.main() }.verifyLinesArbitraryTime( + "Completed 100000 actions in xxx ms", + "Counter = 100000" + ) + } + + @Test + fun testExampleSync05() { + test("ExampleSync05") { kotlinx.coroutines.guide.exampleSync05.main() }.verifyLinesArbitraryTime( + "Completed 100000 actions in xxx ms", + "Counter = 100000" + ) + } + + @Test + fun testExampleSync06() { + test("ExampleSync06") { kotlinx.coroutines.guide.exampleSync06.main() }.verifyLinesArbitraryTime( + "Completed 100000 actions in xxx ms", + "Counter = 100000" + ) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/internal/ConcurrentWeakMapCollectionStressTest.kt b/kotlinx-coroutines-core/jvm/test/internal/ConcurrentWeakMapCollectionStressTest.kt new file mode 100644 index 0000000000..a85b3963bb --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/ConcurrentWeakMapCollectionStressTest.kt @@ -0,0 +1,29 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.testing.* +import junit.framework.Assert.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* +import org.junit.* +import kotlin.concurrent.* + +class ConcurrentWeakMapCollectionStressTest : TestBase() { + private data class Key(val i: Int) + private val nElements = 100_000 * stressTestMultiplier + private val size = 100_000 + + @Test + fun testCollected() { + // use very big arrays as values, we'll need a queue and a cleaner thread to handle them + val m = ConcurrentWeakMap(weakRefQueue = true) + val cleaner = thread(name = "ConcurrentWeakMapCollectionStressTest-Cleaner") { + m.runWeakRefQueueCleaningLoopUntilInterrupted() + } + for (i in 1..nElements) { + m.put(Key(i), ByteArray(size)) + } + assertTrue(m.size < nElements) // some of it was collected for sure + cleaner.interrupt() + cleaner.join() + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/internal/ConcurrentWeakMapOperationStressTest.kt b/kotlinx-coroutines-core/jvm/test/internal/ConcurrentWeakMapOperationStressTest.kt new file mode 100644 index 0000000000..934dbbcd3c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/ConcurrentWeakMapOperationStressTest.kt @@ -0,0 +1,70 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* +import org.junit.Test +import kotlin.concurrent.* +import kotlin.test.* + +/** + * Concurrent test for [ConcurrentWeakMap] that tests put/get/remove from concurrent threads and is + * arranged so that concurrent rehashing is also happening. + */ +class ConcurrentWeakMapOperationStressTest : TestBase() { + private val nThreads = 10 + private val batchSize = 1000 + private val nSeconds = 3 * stressTestMultiplier + + private val count = atomic(0L) + private val stop = atomic(false) + + private data class Key(val i: Long) + + @Test + fun testOperations() { + // We don't create queue here, because concurrent operations are enough to make it clean itself + val m = ConcurrentWeakMap() + val threads = Array(nThreads) { index -> + thread(start = false, name = "ConcurrentWeakMapOperationStressTest-$index") { + var generationOffset = 0L + while (!stop.value) { + val kvs = (generationOffset + batchSize * index until generationOffset + batchSize * (index + 1)) + .associateBy({ Key(it) }, { it * it }) + generationOffset += batchSize * nThreads + for ((k, v) in kvs) { + assertEquals(null, m.put(k, v)) + } + for ((k, v) in kvs) { + assertEquals(v, m[k]) + } + for ((k, v) in kvs) { + assertEquals(v, m.remove(k)) + } + for ((k, _) in kvs) { + assertEquals(null, m[k]) + } + count.incrementAndGet() + } + } + } + val uncaughtExceptionHandler = Thread.UncaughtExceptionHandler { t, ex -> + ex.printStackTrace() + error("Error in thread $t", ex) + } + threads.forEach { it.uncaughtExceptionHandler = uncaughtExceptionHandler } + threads.forEach { it.start() } + var lastCount = -1L + for (sec in 1..nSeconds) { + Thread.sleep(1000) + val count = count.value + println("$sec: done $count batches") + assertTrue(count > lastCount) // ensure progress + lastCount = count + } + stop.value = true + threads.forEach { it.join() } + assertEquals(0, m.size, "Unexpected map state: $m") + } +} diff --git a/kotlinx-coroutines-core/jvm/test/internal/ConcurrentWeakMapTest.kt b/kotlinx-coroutines-core/jvm/test/internal/ConcurrentWeakMapTest.kt new file mode 100644 index 0000000000..2394e90d79 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/ConcurrentWeakMapTest.kt @@ -0,0 +1,43 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.testing.* +import junit.framework.Assert.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* +import org.junit.* + +class ConcurrentWeakMapTest : TestBase() { + @Test + fun testSimple() { + val expect = (1..1000).associate { it.toString().let { it to it } } + val m = ConcurrentWeakMap() + // repeat adding/removing a few times + repeat(5) { + assertEquals(0, m.size) + assertEquals(emptySet(), m.keys) + assertEquals(emptyList(), m.values.toList()) + assertEquals(emptySet>(), m.entries) + for ((k, v) in expect) { + assertNull(m.put(k, v)) + } + assertEquals(expect.size, m.size) + assertEquals(expect.keys, m.keys) + assertEquals(expect.entries, m.entries) + for ((k, v) in expect) { + assertEquals(v, m[k]) + } + assertEquals(expect.size, m.size) + if (it % 2 == 0) { + for ((k, v) in expect) { + assertEquals(v, m.remove(k)) + } + } else { + m.clear() + } + assertEquals(0, m.size) + for ((k, _) in expect) { + assertNull(m[k]) + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/internal/FastServiceLoaderTest.kt b/kotlinx-coroutines-core/jvm/test/internal/FastServiceLoaderTest.kt new file mode 100644 index 0000000000..b47cd7d661 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/FastServiceLoaderTest.kt @@ -0,0 +1,17 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +class FastServiceLoaderTest : TestBase() { + @Test + fun testCrossModuleService() { + val providers = CoroutineScope::class.java.let { FastServiceLoader.loadProviders(it, it.classLoader) } + assertEquals(3, providers.size) + val className = "kotlinx.coroutines.android.EmptyCoroutineScopeImpl" + for (i in 1 .. 3) { + assert(providers[i - 1].javaClass.name == "$className$i") + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListLongStressTest.kt b/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListLongStressTest.kt new file mode 100644 index 0000000000..95be1cbe62 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/LockFreeLinkedListLongStressTest.kt @@ -0,0 +1,81 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.testing.TestBase +import org.junit.Test +import java.util.* +import java.util.concurrent.atomic.AtomicInteger +import kotlin.concurrent.thread + +/** + * This stress test has 2 threads adding on one side on list, 2 more threads adding on the other, + * and 6 threads iterating and concurrently removing items. The resulting list that is being + * stressed is long. + */ +class LockFreeLinkedListLongStressTest : TestBase() { + data class IntNode(val i: Int) : LockFreeLinkedListNode() + val list = LockFreeLinkedListHead() + + val threads = mutableListOf() + private val nAdded = 10_000_000 // should not stress more, because that'll run out of memory + private val nAddThreads = 4 // must be power of 2 (!!!) + private val nRemoveThreads = 6 + private val removeProbability = 0.2 + private val workingAdders = AtomicInteger(nAddThreads) + + private fun shallRemove(i: Int) = i and 63 != 42 + + @Test + fun testStress() { + println("--- LockFreeLinkedListLongStressTest") + for (j in 0 until nAddThreads) + threads += thread(start = false, name = "adder-$j") { + for (i in j until nAdded step nAddThreads) { + list.addLast(IntNode(i), Int.MAX_VALUE) + } + println("${Thread.currentThread().name} completed") + workingAdders.decrementAndGet() + } + for (j in 0 until nRemoveThreads) + threads += thread(start = false, name = "remover-$j") { + val rnd = Random() + do { + val lastTurn = workingAdders.get() == 0 + list.forEach { node -> + if (node is IntNode && shallRemove(node.i) && (lastTurn || rnd.nextDouble() < removeProbability)) + node.remove() + } + } while (!lastTurn) + println("${Thread.currentThread().name} completed") + } + println("Starting ${threads.size} threads") + for (thread in threads) + thread.start() + println("Joining threads") + for (thread in threads) + thread.join() + // verification + println("Verify result") + list.validate() + val expected = iterator { + for (i in 0 until nAdded) + if (!shallRemove(i)) + yield(i) + } + list.forEach { node -> + require(node !is IntNode || node.i == expected.next()) + } + require(!expected.hasNext()) + } + + private fun LockFreeLinkedListHead.validate() { + var prev: LockFreeLinkedListNode = this + var cur: LockFreeLinkedListNode = next as LockFreeLinkedListNode + while (cur != this) { + val next = cur.nextNode + cur.validateNode(prev, next) + prev = cur + cur = next + } + validateNode(prev, next as LockFreeLinkedListNode) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/internal/LockFreeTaskQueueStressTest.kt b/kotlinx-coroutines-core/jvm/test/internal/LockFreeTaskQueueStressTest.kt new file mode 100644 index 0000000000..625df7665f --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/LockFreeTaskQueueStressTest.kt @@ -0,0 +1,116 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import org.junit.runner.* +import org.junit.runners.* +import java.util.concurrent.* +import kotlin.concurrent.* +import kotlin.test.* + +// Tests many short queues to stress copy/resize +@RunWith(Parameterized::class) +class LockFreeTaskQueueStressTest( + private val nConsumers: Int +) : TestBase() { + companion object { + @Parameterized.Parameters(name = "nConsumers={0}") + @JvmStatic + fun params(): Collection = listOf(1, 3) + } + + private val singleConsumer = nConsumers == 1 + + private val nSeconds = 3 * stressTestMultiplier + private val nProducers = 4 + private val batchSize = 100 + + private val batch = atomic(0) + private val produced = atomic(0L) + private val consumed = atomic(0L) + private var expected = LongArray(nProducers) + + private val queue = atomic?>(null) + private val done = atomic(0) + private val doneProducers = atomic(0) + + private val barrier = CyclicBarrier(nProducers + nConsumers + 1) + + private class Item(val producer: Int, val index: Long) + + @Test + fun testStress() { + val threads = mutableListOf() + threads += thread(name = "Pacer", start = false) { + while (done.value == 0) { + queue.value = LockFreeTaskQueue(singleConsumer) + batch.value = 0 + doneProducers.value = 0 + barrier.await() // start consumers & producers + barrier.await() // await consumers & producers + } + queue.value = null + println("Pacer done") + barrier.await() // wakeup the rest + } + threads += List(nConsumers) { consumer -> + thread(name = "Consumer-$consumer", start = false) { + while (true) { + barrier.await() + val queue = queue.value ?: break + while (true) { + val item = queue.removeFirstOrNull() + if (item == null) { + if (doneProducers.value == nProducers && queue.isEmpty) break // that's it + continue // spin to retry + } + consumed.incrementAndGet() + if (singleConsumer) { + // This check only properly works in single-consumer case + val eItem = expected[item.producer]++ + if (eItem != item.index) error("Expected $eItem but got ${item.index} from Producer-${item.producer}") + } + } + barrier.await() + } + println("Consumer-$consumer done") + } + } + threads += List(nProducers) { producer -> + thread(name = "Producer-$producer", start = false) { + var index = 0L + while (true) { + barrier.await() + val queue = queue.value ?: break + while (true) { + if (batch.incrementAndGet() >= batchSize) break + check(queue.addLast(Item(producer, index++))) // never closed + produced.incrementAndGet() + } + doneProducers.incrementAndGet() + barrier.await() + } + println("Producer-$producer done") + } + } + threads.forEach { + it.setUncaughtExceptionHandler { t, e -> + System.err.println("Thread $t failed: $e") + e.printStackTrace() + done.value = 1 + error("Thread $t failed", e) + } + } + threads.forEach { it.start() } + for (second in 1..nSeconds) { + Thread.sleep(1000) + println("$second: produced=${produced.value}, consumed=${consumed.value}") + if (done.value == 1) break + } + done.value = 1 + threads.forEach { it.join() } + println("T: produced=${produced.value}, consumed=${consumed.value}") + assertEquals(produced.value, consumed.value) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/internal/LockFreeTaskQueueTest.kt b/kotlinx-coroutines-core/jvm/test/internal/LockFreeTaskQueueTest.kt new file mode 100644 index 0000000000..de4dff8ad5 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/LockFreeTaskQueueTest.kt @@ -0,0 +1,73 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.runner.* +import org.junit.runners.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class LockFreeTaskQueueTest( + private val singleConsumer: Boolean +) : TestBase() { + companion object { + @Parameterized.Parameters(name = "singleConsumer={0}") + @JvmStatic + fun params(): Collection> = listOf( + arrayOf(false), + arrayOf(true) + ) + } + + @Test + fun testBasic() { + val q = LockFreeTaskQueue(singleConsumer) + assertTrue(q.isEmpty) + assertEquals(0, q.size) + assertTrue(q.addLast(1)) + assertFalse(q.isEmpty) + assertEquals(1, q.size) + assertTrue(q.addLast(2)) + assertFalse(q.isEmpty) + assertEquals(2, q.size) + assertTrue(q.addLast(3)) + assertFalse(q.isEmpty) + assertEquals(3, q.size) + assertEquals(1, q.removeFirstOrNull()) + assertFalse(q.isEmpty) + assertEquals(2, q.size) + assertEquals(2, q.removeFirstOrNull()) + assertFalse(q.isEmpty) + assertEquals(1, q.size) + assertTrue(q.addLast(4)) + assertFalse(q.isEmpty) + assertEquals(2, q.size) + q.close() + assertFalse(q.isEmpty) + assertEquals(2, q.size) + assertFalse(q.addLast(5)) + assertFalse(q.isEmpty) + assertEquals(2, q.size) + assertEquals(3, q.removeFirstOrNull()) + assertFalse(q.isEmpty) + assertEquals(1, q.size) + assertEquals(4, q.removeFirstOrNull()) + assertTrue(q.isEmpty) + assertEquals(0, q.size) + } + + @Test + fun testCopyGrow() { + val n = 1000 * stressTestMultiplier + val q = LockFreeTaskQueue(singleConsumer) + assertTrue(q.isEmpty) + repeat(n) { i -> + assertTrue(q.addLast(i)) + assertFalse(q.isEmpty) + } + repeat(n) { i -> + assertEquals(i, q.removeFirstOrNull()) + } + assertTrue(q.isEmpty) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/internal/OnDemandAllocatingPoolLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/internal/OnDemandAllocatingPoolLincheckTest.kt new file mode 100644 index 0000000000..77d8179929 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/OnDemandAllocatingPoolLincheckTest.kt @@ -0,0 +1,54 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import org.jetbrains.kotlinx.lincheck.* +import org.jetbrains.kotlinx.lincheck.annotations.* + +/** + * Test that: + * - All elements allocated in [OnDemandAllocatingPool] get returned when [close] is invoked. + * - After reaching the maximum capacity, new elements are not added. + * - After [close] is invoked, [OnDemandAllocatingPool.allocate] returns `false`. + * - [OnDemandAllocatingPool.close] will return an empty list after the first invocation. + */ +abstract class OnDemandAllocatingPoolLincheckTest(maxCapacity: Int) : AbstractLincheckTest() { + private val counter = atomic(0) + private val pool = OnDemandAllocatingPool(maxCapacity = maxCapacity, create = { + counter.getAndIncrement() + }) + + @Operation + fun allocate(): Boolean = pool.allocate() + + @Operation + fun close(): String = pool.close().sorted().toString() +} + +abstract class OnDemandAllocatingSequentialPool(private val maxCapacity: Int) { + var closed = false + var elements = 0 + + fun allocate() = if (closed) { + false + } else { + if (elements < maxCapacity) { + elements++ + } + true + } + + fun close(): String = if (closed) { + emptyList() + } else { + closed = true + (0 until elements) + }.sorted().toString() +} + +class OnDemandAllocatingPool3LincheckTest : OnDemandAllocatingPoolLincheckTest(3) { + override fun > O.customize(isStressTest: Boolean): O = + this.sequentialSpecification(OnDemandAllocatingSequentialPool3::class.java) +} + +class OnDemandAllocatingSequentialPool3 : OnDemandAllocatingSequentialPool(3) diff --git a/kotlinx-coroutines-core/jvm/test/internal/ThreadSafeHeapStressTest.kt b/kotlinx-coroutines-core/jvm/test/internal/ThreadSafeHeapStressTest.kt new file mode 100644 index 0000000000..20ae9f3eef --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/ThreadSafeHeapStressTest.kt @@ -0,0 +1,63 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import java.util.concurrent.* +import java.util.concurrent.CancellationException +import kotlin.test.* + +class ThreadSafeHeapStressTest : TestBase() { + private class DisposableNode : EventLoopImplBase.DelayedTask(1L) { + override fun run() { + } + } + + @Test + fun testConcurrentRemoveDispose() = runTest { + val heap = EventLoopImplBase.DelayedTaskQueue(1) + repeat(10_000 * stressTestMultiplierSqrt) { + withContext(Dispatchers.Default) { + val node = DisposableNode() + val barrier = CyclicBarrier(2) + launch { + heap.addLast(node) + barrier.await() + heap.remove(node) + } + launch { + barrier.await() + Thread.yield() + node.dispose() + } + } + } + } + + @Test() + fun testConcurrentAddDispose() = runTest { + repeat(10_000 * stressTestMultiplierSqrt) { + val jobToCancel = Job() + val barrier = CyclicBarrier(2) + val jobToJoin = launch(Dispatchers.Default) { + barrier.await() + jobToCancel.cancelAndJoin() + } + + try { + runBlocking { // Use event loop impl + withContext(jobToCancel) { + // This one is to work around heap allocation optimization + launch(start = CoroutineStart.UNDISPATCHED) { + delay(100_000) + } + barrier.await() + delay(100_000) + } + } + } catch (e: CancellationException) { + // Expected exception + } + jobToJoin.join() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/internal/ThreadSafeHeapTest.kt b/kotlinx-coroutines-core/jvm/test/internal/ThreadSafeHeapTest.kt new file mode 100644 index 0000000000..c7f254800d --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/internal/ThreadSafeHeapTest.kt @@ -0,0 +1,93 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* +import java.util.* + +class ThreadSafeHeapTest : TestBase() { + internal class Node(val value: Int) : ThreadSafeHeapNode, Comparable { + override var heap: ThreadSafeHeap<*>? = null + override var index = -1 + override fun compareTo(other: Node): Int = value.compareTo(other.value) + override fun equals(other: Any?): Boolean = other is Node && other.value == value + override fun hashCode(): Int = value + override fun toString(): String = "$value" + } + + @Test + fun testBasic() { + val h = ThreadSafeHeap() + assertNull(h.peek()) + val n1 = Node(1) + h.addLast(n1) + assertEquals(n1, h.peek()) + val n2 = Node(2) + h.addLast(n2) + assertEquals(n1, h.peek()) + val n3 = Node(3) + h.addLast(n3) + assertEquals(n1, h.peek()) + val n4 = Node(4) + h.addLast(n4) + assertEquals(n1, h.peek()) + val n5 = Node(5) + h.addLast(n5) + assertEquals(n1, h.peek()) + assertEquals(n1, h.removeFirstOrNull()) + assertEquals(-1, n1.index) + assertEquals(n2, h.peek()) + h.remove(n2) + assertEquals(n3, h.peek()) + h.remove(n4) + assertEquals(n3, h.peek()) + h.remove(n3) + assertEquals(n5, h.peek()) + h.remove(n5) + assertNull(h.peek()) + } + + @Test + fun testRandomSort() { + val n = 1000 * stressTestMultiplier + val r = Random(1) + val h = ThreadSafeHeap() + val a = IntArray(n) { r.nextInt() } + repeat(n) { h.addLast(Node(a[it])) } + a.sort() + repeat(n) { assertEquals(Node(a[it]), h.removeFirstOrNull()) } + assertNull(h.peek()) + } + + @Test + fun testRandomRemove() { + val n = 1000 * stressTestMultiplier + check(n % 2 == 0) { "Must be even" } + val r = Random(1) + val h = ThreadSafeHeap() + val set = TreeSet() + repeat(n) { + val node = Node(r.nextInt()) + h.addLast(node) + assertTrue(set.add(node)) + } + while (!h.isEmpty) { + // pick random node to remove + val rndNode: Node + while (true) { + val tail = set.tailSet(Node(r.nextInt())) + if (!tail.isEmpty()) { + rndNode = tail.first() + break + } + } + assertTrue(set.remove(rndNode)) + assertTrue(h.remove(rndNode)) + // remove head and validate + val headNode = h.removeFirstOrNull()!! // must not be null!!! + assertSame(headNode, set.first(), "Expected ${set.first()}, but found $headNode, remaining size ${h.size}") + assertTrue(set.remove(headNode)) + assertEquals(set.size, h.size) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/jdk8/future/AsFutureTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/future/AsFutureTest.kt new file mode 100644 index 0000000000..3296072c20 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/jdk8/future/AsFutureTest.kt @@ -0,0 +1,121 @@ +package kotlinx.coroutines.future + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.CancellationException +import kotlin.test.* + +class AsFutureTest : TestBase() { + + @Test + fun testCompletedDeferredAsCompletableFuture() = runTest { + expect(1) + val deferred = async(start = CoroutineStart.UNDISPATCHED) { + expect(2) // completed right away + "OK" + } + expect(3) + val future = deferred.asCompletableFuture() + assertEquals("OK", future.await()) + finish(4) + } + + @Test + fun testCompletedJobAsCompletableFuture() = runTest { + val job = Job().apply { complete() } + val future = job.asCompletableFuture() + assertEquals(Unit, future.await()) + } + + @Test + fun testWaitForDeferredAsCompletableFuture() = runTest { + expect(1) + val deferred = async { + expect(3) // will complete later + "OK" + } + expect(2) + val future = deferred.asCompletableFuture() + assertEquals("OK", future.await()) // await yields main thread to deferred coroutine + finish(4) + } + + @Test + fun testWaitForJobAsCompletableFuture() = runTest { + val job = Job() + val future = job.asCompletableFuture() + assertTrue(job.isActive) + job.complete() + assertFalse(job.isActive) + assertEquals(Unit, future.await()) + } + + @Test + fun testAsCompletableFutureThrowable() { + val deferred = GlobalScope.async { throw OutOfMemoryError() } + val future = deferred.asCompletableFuture() + try { + expect(1) + future.get() + expectUnreached() + } catch (e: ExecutionException) { + assertTrue(future.isCompletedExceptionally) + assertIs(e.cause) + finish(2) + } + } + + @Test + fun testJobAsCompletableFutureThrowable() { + val job = Job() + CompletableDeferred(parent = job).apply { completeExceptionally(OutOfMemoryError()) } + val future = job.asCompletableFuture() + try { + expect(1) + future.get() + expectUnreached() + } catch (e: ExecutionException) { + assertTrue(future.isCompletedExceptionally) + assertIs(e.cause) + finish(2) + } + } + + @Test + fun testJobAsCompletableFutureCancellation() { + val job = Job() + val future = job.asCompletableFuture() + job.cancel() + try { + expect(1) + future.get() + expectUnreached() + } catch (e: CancellationException) { + assertTrue(future.isCompletedExceptionally) + finish(2) + } + } + + @Test + fun testJobCancellation() { + val job = Job() + val future = job.asCompletableFuture() + future.cancel(true) + assertTrue(job.isCancelled) + assertTrue(job.isCompleted) + assertFalse(job.isActive) + } + + @Test + fun testDeferredCancellation() { + val deferred = CompletableDeferred() + val future = deferred.asCompletableFuture() + future.cancel(true) + assertTrue(deferred.isCancelled) + assertTrue(deferred.isCompleted) + assertFalse(deferred.isActive) + assertIs(deferred.getCompletionExceptionOrNull()) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureAsDeferredUnhandledCompletionExceptionTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureAsDeferredUnhandledCompletionExceptionTest.kt new file mode 100644 index 0000000000..97d54847de --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureAsDeferredUnhandledCompletionExceptionTest.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.future + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.future.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class FutureAsDeferredUnhandledCompletionExceptionTest : TestBase() { + + // This is a separate test in order to avoid interference with uncaught exception handlers in other tests + private val exceptionHandler = Thread.getDefaultUncaughtExceptionHandler() + private lateinit var caughtException: Throwable + + @Before + fun setUp() { + Thread.setDefaultUncaughtExceptionHandler { _, e -> caughtException = e } + } + + @After + fun tearDown() { + Thread.setDefaultUncaughtExceptionHandler(exceptionHandler) + } + + @Test + fun testLostException() = runTest { + val future = CompletableFuture() + val deferred = future.asDeferred() + deferred.invokeOnCompletion { throw TestException() } + future.complete(1) + assertTrue { caughtException is CompletionHandlerException && caughtException.cause is TestException } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureExceptionsTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureExceptionsTest.kt new file mode 100644 index 0000000000..0fc1290a62 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureExceptionsTest.kt @@ -0,0 +1,84 @@ +package kotlinx.coroutines.future + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import java.io.* +import java.util.concurrent.* +import kotlin.test.* + +class FutureExceptionsTest : TestBase() { + + @Test + fun testAwait() { + testException(IOException(), { it is IOException }) + } + + @Test + fun testAwaitChained() { + testException(IOException(), { it is IOException }, { f -> f.thenApply { it + 1 } }) + } + + @Test + fun testAwaitDeepChain() { + testException(IOException(), { it is IOException }, + { f -> f + .thenApply { it + 1 } + .thenApply { it + 2 } }) + } + + @Test + fun testAwaitCompletionException() { + testException(CompletionException("test", IOException()), { it is IOException }) + } + + @Test + fun testAwaitChainedCompletionException() { + testException(CompletionException("test", IOException()), { it is IOException }, { f -> f.thenApply { it + 1 } }) + } + + @Test + fun testAwaitTestException() { + testException(TestException(), { it is TestException }) + } + + @Test + fun testAwaitChainedTestException() { + testException(TestException(), { it is TestException }, { f -> f.thenApply { it + 1 } }) + } + + private fun testException( + exception: Throwable, + expected: ((Throwable) -> Boolean), + transformer: (CompletableFuture) -> CompletableFuture = { it } + ) { + + // Fast path + runTest { + val future = CompletableFuture() + val chained = transformer(future) + future.completeExceptionally(exception) + try { + chained.await() + } catch (e: Throwable) { + assertTrue(expected(e)) + } + } + + // Slow path + runTest { + val future = CompletableFuture() + val chained = transformer(future) + + launch { + future.completeExceptionally(exception) + } + + try { + chained.await() + } catch (e: Throwable) { + assertTrue(expected(e)) + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureTest.kt new file mode 100644 index 0000000000..81178e193c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/jdk8/future/FutureTest.kt @@ -0,0 +1,614 @@ +package kotlinx.coroutines.future + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import org.junit.* +import org.junit.Test +import java.lang.IllegalArgumentException +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import java.util.concurrent.locks.* +import java.util.function.* +import kotlin.concurrent.withLock +import kotlin.coroutines.* +import kotlin.reflect.* +import kotlin.test.* + +class FutureTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("ForkJoinPool.commonPool-worker-") + } + + @Test + fun testSimpleAwait() { + val future = GlobalScope.future { + CompletableFuture.supplyAsync { + "O" + }.await() + "K" + } + assertEquals("OK", future.get()) + } + + @Test + fun testCompletedFuture() { + val toAwait = CompletableFuture() + toAwait.complete("O") + val future = GlobalScope.future { + toAwait.await() + "K" + } + assertEquals("OK", future.get()) + } + + @Test + fun testCompletedCompletionStage() { + val completable = CompletableFuture() + completable.complete("O") + val toAwait: CompletionStage = completable + val future = GlobalScope.future { + toAwait.await() + "K" + } + assertEquals("OK", future.get()) + } + + @Test + fun testWaitForFuture() { + val toAwait = CompletableFuture() + val future = GlobalScope.future { + toAwait.await() + "K" + } + assertFalse(future.isDone) + toAwait.complete("O") + assertEquals("OK", future.get()) + } + + @Test + fun testWaitForCompletionStage() { + val completable = CompletableFuture() + val toAwait: CompletionStage = completable + val future = GlobalScope.future { + toAwait.await() + "K" + } + assertFalse(future.isDone) + completable.complete("O") + assertEquals("OK", future.get()) + } + + @Test + fun testCompletedFutureExceptionally() { + val toAwait = CompletableFuture() + toAwait.completeExceptionally(TestException("O")) + val future = GlobalScope.future { + try { + toAwait.await() + } catch (e: TestException) { + e.message!! + } + "K" + } + assertEquals("OK", future.get()) + } + + @Test + // Test fast-path of CompletionStage.await() extension + fun testCompletedCompletionStageExceptionally() { + val completable = CompletableFuture() + val toAwait: CompletionStage = completable + completable.completeExceptionally(TestException("O")) + val future = GlobalScope.future { + try { + toAwait.await() + } catch (e: TestException) { + e.message!! + } + "K" + } + assertEquals("OK", future.get()) + } + + @Test + // Test slow-path of CompletionStage.await() extension + fun testWaitForFutureWithException() = runTest { + expect(1) + val toAwait = CompletableFuture() + val future = future(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + toAwait.await() // will suspend (slow path) + } catch (e: TestException) { + expect(4) + e.message!! + } + "K" + } + expect(3) + assertFalse(future.isDone) + toAwait.completeExceptionally(TestException("O")) + yield() // to future coroutine + assertEquals("OK", future.get()) + finish(5) + } + + @Test + fun testWaitForCompletionStageWithException() { + val completable = CompletableFuture() + val toAwait: CompletionStage = completable + val future = GlobalScope.future { + try { + toAwait.await() + } catch (e: TestException) { + e.message!! + } + "K" + } + assertFalse(future.isDone) + completable.completeExceptionally(TestException("O")) + assertEquals("OK", future.get()) + } + + @Test + fun testExceptionInsideCoroutine() { + val future = GlobalScope.future { + if (CompletableFuture.supplyAsync { true }.await()) { + throw IllegalStateException("OK") + } + "fail" + } + try { + future.get() + fail("'get' should've throw an exception") + } catch (e: ExecutionException) { + assertIs(e.cause) + assertEquals("OK", e.cause!!.message) + } + } + + @Test + fun testCancellableAwaitFuture() = runBlocking { + expect(1) + val toAwait = CompletableFuture() + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + try { + toAwait.await() // suspends + } catch (e: CancellationException) { + expect(5) // should throw cancellation exception + throw e + } + } + expect(3) + job.cancel() // cancel the job + toAwait.complete("fail") // too late, the waiting job was already cancelled + expect(4) // job processing of cancellation was scheduled, not executed yet + yield() // yield main thread to job + finish(6) + } + + @Test + fun testContinuationWrapped() { + val depth = AtomicInteger() + val future = GlobalScope.future(wrapContinuation { + depth.andIncrement + it() + depth.andDecrement + }) { + assertEquals(1, depth.get(), "Part before first suspension must be wrapped") + val result = + CompletableFuture.supplyAsync { + while (depth.get() > 0); + assertEquals(0, depth.get(), "Part inside suspension point should not be wrapped") + "OK" + }.await() + assertEquals(1, depth.get(), "Part after first suspension should be wrapped") + CompletableFuture.supplyAsync { + while (depth.get() > 0); + assertEquals(0, depth.get(), "Part inside suspension point should not be wrapped") + "ignored" + }.await() + result + } + assertEquals("OK", future.get()) + } + + @Test + fun testCompletableFutureStageAsDeferred() = runBlocking { + val lock = ReentrantLock().apply { lock() } + + val deferred: Deferred = CompletableFuture.supplyAsync { + lock.withLock { 42 } + }.asDeferred() + + assertFalse(deferred.isCompleted) + lock.unlock() + + assertEquals(42, deferred.await()) + assertTrue(deferred.isCompleted) + } + + @Test + fun testCompletedFutureAsDeferred() = runBlocking { + val deferred: Deferred = CompletableFuture.completedFuture(42).asDeferred() + assertEquals(42, deferred.await()) + } + + @Test + fun testFailedFutureAsDeferred() = runBlocking { + val future = CompletableFuture().apply { + completeExceptionally(TestException("something went wrong")) + } + val deferred = future.asDeferred() + + assertTrue(deferred.isCancelled) + val completionException = deferred.getCompletionExceptionOrNull()!! + assertIs(completionException) + assertEquals("something went wrong", completionException.message) + + try { + deferred.await() + fail("deferred.await() should throw an exception") + } catch (e: Throwable) { + assertIs(e) + assertEquals("something went wrong", e.message) + } + } + + @Test + fun testCompletableFutureWithExceptionAsDeferred() = runBlocking { + val lock = ReentrantLock().apply { lock() } + + val deferred: Deferred = CompletableFuture.supplyAsync { + lock.withLock { throw TestException("something went wrong") } + }.asDeferred() + + assertFalse(deferred.isCompleted) + lock.unlock() + try { + deferred.await() + fail("deferred.await() should throw an exception") + } catch (e: TestException) { + assertTrue(deferred.isCancelled) + assertEquals("something went wrong", e.message) + } + } + + private val threadLocal = ThreadLocal() + + @Test + fun testApiBridge() = runTest { + val result = newSingleThreadContext("ctx").use { + val future = CompletableFuture.supplyAsync(Supplier { threadLocal.set("value") }, it.executor) + val job = async(it) { + future.await() + threadLocal.get() + } + + job.await() + } + + assertEquals("value", result) + } + + @Test + fun testFutureCancellation() = runTest { + val future = awaitFutureWithCancel(true) + assertTrue(future.isCompletedExceptionally) + assertFailsWith { future.get() } + finish(4) + } + + @Test + fun testNoFutureCancellation() = runTest { + val future = awaitFutureWithCancel(false) + assertFalse(future.isCompletedExceptionally) + assertEquals(239, future.get()) + finish(4) + } + + private suspend fun CoroutineScope.awaitFutureWithCancel(cancellable: Boolean): CompletableFuture { + val latch = CountDownLatch(1) + val future = CompletableFuture.supplyAsync { + latch.await() + 239 + } + + val deferred = async { + expect(2) + if (cancellable) future.await() + else future.asDeferred().await() + } + expect(1) + yield() + deferred.cancel() + expect(3) + latch.countDown() + return future + } + + @Test + fun testStructuredException() = runTest( + expected = { it is TestException } // exception propagates to parent with structured concurrency + ) { + val result = future(Dispatchers.Unconfined) { + throw TestException("FAIL") + } + result.checkFutureException() + } + + @Test + fun testChildException() = runTest( + expected = { it is TestException } // exception propagates to parent with structured concurrency + ) { + val result = future(Dispatchers.Unconfined) { + // child crashes + launch { throw TestException("FAIL") } + 42 + } + result.checkFutureException() + } + + @Test + fun testExceptionAggregation() = runTest( + expected = { it is TestException } // exception propagates to parent with structured concurrency + ) { + val result = future(Dispatchers.Unconfined) { + // child crashes + launch(start = CoroutineStart.ATOMIC) { throw TestException1("FAIL") } + launch(start = CoroutineStart.ATOMIC) { throw TestException2("FAIL") } + throw TestException() + } + result.checkFutureException(TestException1::class, TestException2::class) + finish(1) + } + + @Test + fun testExternalCompletion() = runTest { + expect(1) + val result = future(Dispatchers.Unconfined) { + try { + delay(Long.MAX_VALUE) + } finally { + expect(2) + } + } + + result.complete(Unit) + finish(3) + } + + @Test + fun testExceptionOnExternalCompletion() = runTest( + expected = { it is TestException } // exception propagates to parent with structured concurrency + ) { + expect(1) + val result = future(Dispatchers.Unconfined) { + try { + delay(Long.MAX_VALUE) + } finally { + expect(2) + throw TestException() + } + } + result.complete(Unit) + finish(3) + } + + @Test + fun testUnhandledExceptionOnExternalCompletionIsNotReported() = runTest { + expect(1) + // No parent here (NonCancellable), so nowhere to propagate exception + val result = future(NonCancellable + Dispatchers.Unconfined) { + try { + delay(Long.MAX_VALUE) + } finally { + expect(2) + throw TestException() // this exception cannot be handled + } + } + result.complete(Unit) + finish(3) + } + + /** + * See [https://github.com/Kotlin/kotlinx.coroutines/issues/892] + */ + @Test + fun testTimeoutCancellationFailRace() { + repeat(10 * stressTestMultiplier) { + runBlocking { + withTimeoutOrNull(10) { + while (true) { + var caught = false + try { + CompletableFuture.supplyAsync { + throw TestException() + }.await() + } catch (ignored: TestException) { + caught = true + } + assertTrue(caught) // should have caught TestException or timed out + } + } + } + } + } + + /** + * Tests that both [CompletionStage.await] and [CompletionStage.asDeferred] consistently unwrap + * [CompletionException] both in their slow and fast paths. + * See [issue #1479](https://github.com/Kotlin/kotlinx.coroutines/issues/1479). + */ + @Test + fun testConsistentExceptionUnwrapping() = runTest { + expect(1) + // Check the fast path + val fFast = CompletableFuture.supplyAsync { + expect(2) + throw TestException() + } + fFast.checkFutureException() // wait until it completes + // Fast path in await and asDeferred.await() shall produce TestException + expect(3) + val dFast = fFast.asDeferred() + assertFailsWith { fFast.await() } + assertFailsWith { dFast.await() } + // Same test, but future has not completed yet, check the slow path + expect(4) + val barrier = CyclicBarrier(2) + val fSlow = CompletableFuture.supplyAsync { + barrier.await() + expect(6) + throw TestException() + } + val dSlow = fSlow.asDeferred() + launch(start = CoroutineStart.UNDISPATCHED) { + expect(5) + // Slow path on await shall produce TestException, too + assertFailsWith { fSlow.await() } // will suspend here + assertFailsWith { dSlow.await() } + finish(7) + } + barrier.await() + fSlow.checkFutureException() // now wait until it completes + } + + private inline fun CompletableFuture<*>.checkFutureException(vararg suppressed: KClass) { + val e = assertFailsWith { get() } + val cause = e.cause!! + assertIs(cause) + for ((index, clazz) in suppressed.withIndex()) { + assertTrue(clazz.isInstance(cause.suppressed[index])) + } + } + + private fun wrapContinuation(wrapper: (() -> Unit) -> Unit): CoroutineDispatcher = object : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + wrapper { + block.run() + } + } + } + + /** + * https://github.com/Kotlin/kotlinx.coroutines/issues/2456 + */ + @Test + fun testCompletedStageAwait() = runTest { + val stage = CompletableFuture.completedStage("OK") + assertEquals("OK", stage.await()) + } + + /** + * https://github.com/Kotlin/kotlinx.coroutines/issues/2456 + */ + @Test + fun testCompletedStageAsDeferredAwait() = runTest { + val stage = CompletableFuture.completedStage("OK") + val deferred = stage.asDeferred() + assertEquals("OK", deferred.await()) + } + + @Test + fun testCompletedStateThenApplyAwait() = runTest { + expect(1) + val cf = CompletableFuture() + launch { + expect(3) + cf.complete("O") + } + expect(2) + val stage = cf.thenApply { it + "K" } + assertEquals("OK", stage.await()) + finish(4) + } + + @Test + fun testCompletedStateThenApplyAwaitCancel() = runTest { + expect(1) + val cf = CompletableFuture() + launch { + expect(3) + cf.cancel(false) + } + expect(2) + val stage = cf.thenApply { it + "K" } + assertFailsWith { stage.await() } + finish(4) + } + + @Test + fun testCompletedStateThenApplyAsDeferredAwait() = runTest { + expect(1) + val cf = CompletableFuture() + launch { + expect(3) + cf.complete("O") + } + expect(2) + val stage = cf.thenApply { it + "K" } + val deferred = stage.asDeferred() + assertEquals("OK", deferred.await()) + finish(4) + } + + @Test + fun testCompletedStateThenApplyAsDeferredAwaitCancel() = runTest { + expect(1) + val cf = CompletableFuture() + expect(2) + val stage = cf.thenApply { it + "K" } + val deferred = stage.asDeferred() + launch { + expect(3) + deferred.cancel() // cancel the deferred! + } + assertFailsWith { stage.await() } + finish(4) + } + + @Test + fun testCancelledParent() = runTest({ it is java.util.concurrent.CancellationException }) { + cancel() + future { expectUnreached() } + future(start = CoroutineStart.ATOMIC) { } + future(start = CoroutineStart.UNDISPATCHED) { } + } + + @Test + fun testStackOverflow() = runTest { + val future = CompletableFuture() + val completed = AtomicLong() + val count = 10000L + val children = ArrayList() + for (i in 0 until count) { + children += launch(Dispatchers.Default) { + future.asDeferred().await() + completed.incrementAndGet() + } + } + future.complete(1) + withTimeout(60_000) { + children.forEach { it.join() } + assertEquals(count, completed.get()) + } + } + + @Test + fun testFailsIfLazy() { + assertFailsWith { + GlobalScope.future(start = CoroutineStart.LAZY) { } + } + } + + @Test + fun testStackOverflowOnExceptionalCompletion() = runTest { + val future = CompletableFuture() + val didRun = AtomicBoolean(false) + future.whenComplete { _, _ -> didRun.set(true) } + val deferreds = List(100000) { future.asDeferred() } + future.completeExceptionally(TestException()) + deferreds.forEach { + assertTrue(it.isCompleted) + val exception = it.getCompletionExceptionOrNull() + assertIs(exception) + assertTrue(exception.suppressedExceptions.isEmpty()) + } + assertTrue(didRun.get()) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/jdk8/stream/ConsumeAsFlowTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/stream/ConsumeAsFlowTest.kt new file mode 100644 index 0000000000..49e8f34c05 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/jdk8/stream/ConsumeAsFlowTest.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.stream + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.junit.Test +import java.lang.IllegalStateException +import kotlin.test.* + +class ConsumeAsFlowTest : TestBase() { + + @Test + fun testCollect() = runTest { + val list = listOf(1, 2, 3) + assertEquals(list, list.stream().consumeAsFlow().toList()) + } + + @Test + fun testCollectInvokesClose() = runTest { + val list = listOf(3, 4, 5) + expect(1) + assertEquals(list, list.stream().onClose { expect(2) }.consumeAsFlow().toList()) + finish(3) + } + + @Test + fun testCollectTwice() = runTest { + val list = listOf(2, 3, 9) + val flow = list.stream().onClose { expect(2) } .consumeAsFlow() + expect(1) + assertEquals(list, flow.toList()) + assertFailsWith { flow.collect() } + finish(3) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/jdk8/time/DurationOverflowTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/time/DurationOverflowTest.kt new file mode 100644 index 0000000000..d13e4e0536 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/jdk8/time/DurationOverflowTest.kt @@ -0,0 +1,76 @@ +package kotlinx.coroutines.time + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.Test +import java.time.* +import java.time.temporal.* +import kotlin.test.* + +class DurationOverflowTest : TestBase() { + + private val durations = ChronoUnit.values().map { it.duration } + + @Test + fun testDelay() = runTest { + var counter = 0 + for (duration in durations) { + expect(++counter) + delay(duration.negated()) // Instant bail out from negative values + launch(start = CoroutineStart.UNDISPATCHED) { + expect(++counter) + delay(duration) + }.cancelAndJoin() + expect(++counter) + } + + finish(++counter) + } + + @Test + fun testOnTimeout() = runTest { + for (duration in durations) { + // Does not crash on overflows + select { + onTimeout(duration) {} + onTimeout(duration.negated()) {} + } + } + } + + @Test + fun testWithTimeout() = runTest { + for (duration in durations) { + withTimeout(duration) {} + } + } + + @Test + fun testWithTimeoutOrNull() = runTest { + for (duration in durations) { + withTimeoutOrNull(duration) {} + } + } + + @Test + fun testWithTimeoutOrNullNegativeDuration() = runTest { + val result = withTimeoutOrNull(Duration.ofSeconds(1).negated()) { + 1 + } + + assertNull(result) + } + + @Test + fun testZeroDurationWithTimeout() = runTest { + assertFailsWith { withTimeout(0L) {} } + assertFailsWith { withTimeout(Duration.ZERO) {} } + } + + @Test + fun testZeroDurationWithTimeoutOrNull() = runTest { + assertNull(withTimeoutOrNull(0L) {}) + assertNull(withTimeoutOrNull(Duration.ZERO) {}) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowDebounceTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowDebounceTest.kt new file mode 100644 index 0000000000..7313b1a77e --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowDebounceTest.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.time + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.testing.TestBase +import kotlinx.coroutines.flow.flow +import kotlinx.coroutines.flow.toList +import kotlinx.coroutines.withVirtualTime +import org.junit.Test +import java.time.Duration +import kotlin.test.assertEquals + +class FlowDebounceTest : TestBase() { + @Test + public fun testBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(Duration.ofMillis(1500)) + emit("B") + delay(Duration.ofMillis(500)) + emit("C") + delay(Duration.ofMillis(250)) + emit("D") + delay(Duration.ofMillis(2000)) + emit("E") + expect(4) + } + + expect(2) + val result = flow.debounce(Duration.ofMillis(1000)).toList() + assertEquals(listOf("A", "D", "E"), result) + finish(5) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowSampleTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowSampleTest.kt new file mode 100644 index 0000000000..a19c4b75ec --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/jdk8/time/FlowSampleTest.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.time + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.testing.TestBase +import kotlinx.coroutines.flow.flow +import kotlinx.coroutines.flow.toList +import kotlinx.coroutines.withVirtualTime +import org.junit.Test +import java.time.Duration +import kotlin.test.assertEquals + +class FlowSampleTest : TestBase() { + @Test + public fun testBasic() = withVirtualTime { + expect(1) + val flow = flow { + expect(3) + emit("A") + delay(Duration.ofMillis(1500)) + emit("B") + delay(Duration.ofMillis(500)) + emit("C") + delay(Duration.ofMillis(250)) + emit("D") + delay(Duration.ofMillis(2000)) + emit("E") + expect(4) + } + + expect(2) + val result = flow.sample(Duration.ofMillis(1000)).toList() + assertEquals(listOf("A", "B", "D"), result) + finish(5) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/jdk8/time/WithTimeoutTest.kt b/kotlinx-coroutines-core/jvm/test/jdk8/time/WithTimeoutTest.kt new file mode 100644 index 0000000000..c34a6b1935 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/jdk8/time/WithTimeoutTest.kt @@ -0,0 +1,65 @@ +package kotlinx.coroutines.time + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import java.time.* +import kotlin.test.* + +class WithTimeoutTest : TestBase() { + + @Test + fun testWithTimeout() = runTest { + expect(1) + val result = withTimeout(Duration.ofMillis(10_000)) { + expect(2) + delay(Duration.ofNanos(1)) + expect(3) + 42 + } + + assertEquals(42, result) + finish(4) + } + + @Test + fun testWithTimeoutOrNull() = runTest { + expect(1) + val result = withTimeoutOrNull(Duration.ofMillis(10_000)) { + expect(2) + delay(Duration.ofNanos(1)) + expect(3) + 42 + } + + assertEquals(42, result) + finish(4) + } + + @Test + fun testWithTimeoutOrNullExceeded() = runTest { + expect(1) + val result = withTimeoutOrNull(Duration.ofMillis(3)) { + expect(2) + delay(Duration.ofSeconds(Long.MAX_VALUE)) + expectUnreached() + } + + assertNull(result) + finish(3) + } + + @Test + fun testWithTimeoutExceeded() = runTest { + expect(1) + try { + withTimeout(Duration.ofMillis(3)) { + expect(2) + delay(Duration.ofSeconds(Long.MAX_VALUE)) + expectUnreached() + } + } catch (e: TimeoutCancellationException) { + finish(3) + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/knit/ClosedAfterGuideTestExecutor.kt b/kotlinx-coroutines-core/jvm/test/knit/ClosedAfterGuideTestExecutor.kt new file mode 100644 index 0000000000..b98bae97aa --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/knit/ClosedAfterGuideTestExecutor.kt @@ -0,0 +1,36 @@ +package kotlinx.coroutines // Trick to make guide tests use these declarations with executors that can be closed on our side implicitly + +import kotlinx.coroutines.testing.* +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import kotlin.coroutines.* + +internal fun newSingleThreadContext(name: String): ExecutorCoroutineDispatcher = ClosedAfterGuideTestDispatcher(1, name) + +private class ClosedAfterGuideTestDispatcher( + private val nThreads: Int, + private val name: String +) : ExecutorCoroutineDispatcher() { + private val threadNo = AtomicInteger() + + override val executor: Executor = + Executors.newScheduledThreadPool(nThreads, object : ThreadFactory { + override fun newThread(target: java.lang.Runnable): Thread { + return PoolThread( + this@ClosedAfterGuideTestDispatcher, + target, + if (nThreads == 1) name else name + "-" + threadNo.incrementAndGet() + ) + } + }) + + override fun dispatch(context: CoroutineContext, block: Runnable) { + executor.execute(wrapTask(block)) + } + + override fun close() { + (executor as ExecutorService).shutdown() + } + + override fun toString(): String = "ThreadPoolDispatcher[$nThreads, $name]" +} diff --git a/kotlinx-coroutines-core/jvm/test/knit/TestUtil.kt b/kotlinx-coroutines-core/jvm/test/knit/TestUtil.kt new file mode 100644 index 0000000000..d0a5551567 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/knit/TestUtil.kt @@ -0,0 +1,171 @@ +package kotlinx.coroutines.knit + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.scheduling.* +import kotlinx.coroutines.testing.* +import kotlinx.knit.test.* +import java.util.concurrent.* +import kotlin.test.* + +// helper function to dump exception to stdout for ease of debugging failed tests +private inline fun outputException(name: String, block: () -> T): T = + try { block() } + catch (e: Throwable) { + println("--- Failed test$name") + e.printStackTrace(System.out) + throw e + } + +private const val SHUTDOWN_TIMEOUT = 5000L // 5 sec at most to wait +private val OUT_ENABLED = systemProp("guide.tests.sout", false) + +fun test(name: String, block: () -> R): List = outputException(name) { + try { + captureOutput(name, stdoutEnabled = OUT_ENABLED) { log -> + DefaultScheduler.usePrivateScheduler() + DefaultExecutor.shutdownForTests(SHUTDOWN_TIMEOUT) + resetCoroutineId() + val threadsBefore = currentThreads() + try { + withVirtualTimeSource(log) { + val result = block() + require(result === Unit) { "Test 'main' shall return Unit" } + } + } finally { + // the shutdown + log.println("--- shutting down") + DefaultScheduler.shutdown(SHUTDOWN_TIMEOUT) + shutdownDispatcherPools(SHUTDOWN_TIMEOUT) + DefaultExecutor.shutdownForTests(SHUTDOWN_TIMEOUT) // the last man standing -- cleanup all pending tasks + } + checkTestThreads(threadsBefore) // check thread if the main completed successfully + } + } finally { + DefaultScheduler.restore() + } +} + +private fun shutdownDispatcherPools(timeout: Long) { + val threads = arrayOfNulls(Thread.activeCount()) + val n = Thread.enumerate(threads) + for (i in 0 until n) { + val thread = threads[i] + if (thread is PoolThread) + (thread.dispatcher.executor as ExecutorService).apply { + shutdown() + awaitTermination(timeout, TimeUnit.MILLISECONDS) + shutdownNow().forEach { DefaultExecutor.enqueue(it) } + } + } +} + +enum class SanitizeMode { + NONE, + ARBITRARY_TIME, + FLEXIBLE_THREAD +} + +private fun sanitize(s: String, mode: SanitizeMode): String { + var res = s + when (mode) { + SanitizeMode.ARBITRARY_TIME -> { + res = res.replace(Regex(" [0-9]+ ms"), " xxx ms") + } + SanitizeMode.FLEXIBLE_THREAD -> { + res = res.replace(Regex("ForkJoinPool\\.commonPool-worker-[0-9]+"), "DefaultDispatcher") + res = res.replace(Regex("ForkJoinPool-[0-9]+-worker-[0-9]+"), "DefaultDispatcher") + res = res.replace(Regex("CommonPool-worker-[0-9]+"), "DefaultDispatcher") + res = res.replace(Regex("DefaultDispatcher-worker-[0-9]+"), "DefaultDispatcher") + res = res.replace(Regex("RxComputationThreadPool-[0-9]+"), "RxComputationThreadPool") + res = res.replace(Regex("Test( worker)?"), "main") + res = res.replace(Regex("@[0-9a-f]+"), "") // drop hex address + } + SanitizeMode.NONE -> {} + } + return res +} + +private fun List.verifyCommonLines(expected: Array, mode: SanitizeMode = SanitizeMode.NONE) { + val n = minOf(size, expected.size) + for (i in 0 until n) { + val exp = sanitize(expected[i], mode) + val act = sanitize(get(i), mode) + assertEquals(exp, act, "Line ${i + 1}") + } +} + +private fun List.checkEqualNumberOfLines(expected: Array) { + if (size > expected.size) + error("Expected ${expected.size} lines, but found $size. Unexpected line '${get(expected.size)}'") + else if (size < expected.size) + error("Expected ${expected.size} lines, but found $size") +} + +fun List.verifyLines(vararg expected: String) = verify { + verifyCommonLines(expected) + checkEqualNumberOfLines(expected) +} + +fun List.verifyLinesStartWith(vararg expected: String) = verify { + verifyCommonLines(expected) + assertTrue(expected.size <= size, "Number of lines") +} + +fun List.verifyLinesArbitraryTime(vararg expected: String) = verify { + verifyCommonLines(expected, SanitizeMode.ARBITRARY_TIME) + checkEqualNumberOfLines(expected) +} + +fun List.verifyLinesFlexibleThread(vararg expected: String) = verify { + verifyCommonLines(expected, SanitizeMode.FLEXIBLE_THREAD) + checkEqualNumberOfLines(expected) +} + +fun List.verifyLinesStartUnordered(vararg expected: String) = verify { + val expectedSorted = expected.sorted().toTypedArray() + sorted().verifyLinesStart(*expectedSorted) +} + +fun List.verifyExceptions(vararg expected: String) { + val original = this + val actual = ArrayList().apply { + var except = false + for (line in original) { + when { + !except && line.startsWith("\tat") -> except = true + except && !line.startsWith("\t") && !line.startsWith("Caused by: ") -> except = false + } + if (!except) add(line) + } + } + val n = minOf(actual.size, expected.size) + for (i in 0 until n) { + val exp = sanitize(expected[i], SanitizeMode.FLEXIBLE_THREAD) + val act = sanitize(actual[i], SanitizeMode.FLEXIBLE_THREAD) + assertEquals(exp, act, "Line ${i + 1}") + } +} + + +fun List.verifyLinesStart(vararg expected: String) = verify { + val n = minOf(size, expected.size) + for (i in 0 until n) { + val exp = sanitize(expected[i], SanitizeMode.FLEXIBLE_THREAD) + val act = sanitize(get(i), SanitizeMode.FLEXIBLE_THREAD) + assertEquals(exp, act.substring(0, minOf(act.length, exp.length)), "Line ${i + 1}") + } + checkEqualNumberOfLines(expected) +} + +private inline fun List.verify(verification: () -> Unit) { + try { + verification() + } catch (t: Throwable) { + if (!OUT_ENABLED) { + println("Printing [delayed] test output") + forEach { println(it) } + } + throw t + } +} diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/ChannelsLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/ChannelsLincheckTest.kt new file mode 100644 index 0000000000..96a8d13354 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/lincheck/ChannelsLincheckTest.kt @@ -0,0 +1,273 @@ +@file:Suppress("unused", "MemberVisibilityCanBePrivate") + +package kotlinx.coroutines.lincheck + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.channels.Channel.Factory.CONFLATED +import kotlinx.coroutines.channels.Channel.Factory.RENDEZVOUS +import kotlinx.coroutines.channels.Channel.Factory.UNLIMITED +import kotlinx.coroutines.selects.* +import org.jetbrains.kotlinx.lincheck.* +import org.jetbrains.kotlinx.lincheck.annotations.* +import org.jetbrains.kotlinx.lincheck.annotations.Operation +import org.jetbrains.kotlinx.lincheck.paramgen.* +import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* + +class RendezvousChannelLincheckTest : ChannelLincheckTestBaseWithOnSend( + c = Channel(RENDEZVOUS), + sequentialSpecification = SequentialRendezvousChannel::class.java +) +class SequentialRendezvousChannel : SequentialIntChannelBase(RENDEZVOUS) + +class Buffered1ChannelLincheckTest : ChannelLincheckTestBaseWithOnSend( + c = Channel(1), + sequentialSpecification = SequentialBuffered1Channel::class.java +) +class Buffered1BroadcastChannelLincheckTest : ChannelLincheckTestBase( + c = ChannelViaBroadcast(BroadcastChannelImpl(1)), + sequentialSpecification = SequentialBuffered1Channel::class.java, + obstructionFree = false +) +class SequentialBuffered1Channel : SequentialIntChannelBase(1) + +class Buffered2ChannelLincheckTest : ChannelLincheckTestBaseWithOnSend( + c = Channel(2), + sequentialSpecification = SequentialBuffered2Channel::class.java +) +class Buffered2BroadcastChannelLincheckTest : ChannelLincheckTestBase( + c = ChannelViaBroadcast(BroadcastChannelImpl(2)), + sequentialSpecification = SequentialBuffered2Channel::class.java, + obstructionFree = false +) +class SequentialBuffered2Channel : SequentialIntChannelBase(2) + +class UnlimitedChannelLincheckTest : ChannelLincheckTestBaseAll( + c = Channel(UNLIMITED), + sequentialSpecification = SequentialUnlimitedChannel::class.java +) +class SequentialUnlimitedChannel : SequentialIntChannelBase(UNLIMITED) + +class ConflatedChannelLincheckTest : ChannelLincheckTestBaseAll( + c = Channel(CONFLATED), + sequentialSpecification = SequentialConflatedChannel::class.java, + obstructionFree = false +) +@Suppress("DEPRECATION_ERROR") +class ConflatedBroadcastChannelLincheckTest : ChannelLincheckTestBaseAll( + c = ChannelViaBroadcast(ConflatedBroadcastChannel()), + sequentialSpecification = SequentialConflatedChannel::class.java, + obstructionFree = false +) +class SequentialConflatedChannel : SequentialIntChannelBase(CONFLATED) + +abstract class ChannelLincheckTestBaseAll( + c: Channel, + sequentialSpecification: Class<*>, + obstructionFree: Boolean = true +) : ChannelLincheckTestBaseWithOnSend(c, sequentialSpecification, obstructionFree) { + @Operation + override fun trySend(value: Int) = super.trySend(value) + @Operation + override fun isClosedForReceive() = super.isClosedForReceive() + @Operation + override fun isEmpty() = super.isEmpty() +} + +abstract class ChannelLincheckTestBaseWithOnSend( + c: Channel, + sequentialSpecification: Class<*>, + obstructionFree: Boolean = true +) : ChannelLincheckTestBase(c, sequentialSpecification, obstructionFree) { + @Operation(allowExtraSuspension = true, blocking = true) + suspend fun sendViaSelect(@Param(name = "value") value: Int): Any = try { + select { c.onSend(value) {} } + } catch (e: NumberedCancellationException) { + e.testResult + } +} + +@Param.Params( + Param(name = "value", gen = IntGen::class, conf = "1:9"), + Param(name = "closeToken", gen = IntGen::class, conf = "1:9") +) +abstract class ChannelLincheckTestBase( + protected val c: Channel, + private val sequentialSpecification: Class<*>, + private val obstructionFree: Boolean = true +) : AbstractLincheckTest() { + + @Operation(allowExtraSuspension = true, blocking = true) + suspend fun send(@Param(name = "value") value: Int): Any = try { + c.send(value) + } catch (e: NumberedCancellationException) { + e.testResult + } + + // @Operation TODO: `trySend()` is not linearizable as it can fail due to postponed buffer expansion + // TODO: or make a rendezvous with `tryReceive`, which violates the sequential specification. + open fun trySend(@Param(name = "value") value: Int): Any = c.trySend(value) + .onSuccess { return true } + .onFailure { + return if (it is NumberedCancellationException) it.testResult + else false + } + + @Operation(allowExtraSuspension = true, blocking = true) + suspend fun receive(): Any = try { + c.receive() + } catch (e: NumberedCancellationException) { + e.testResult + } + + @Operation(allowExtraSuspension = true, blocking = true) + suspend fun receiveCatching(): Any = c.receiveCatching() + .onSuccess { return it } + .onClosed { e -> return (e as NumberedCancellationException).testResult } + + @Operation(blocking = true) + fun tryReceive(): Any? = + c.tryReceive() + .onSuccess { return it } + .onFailure { return if (it is NumberedCancellationException) it.testResult else null } + + @Operation(allowExtraSuspension = true, blocking = true) + suspend fun receiveViaSelect(): Any = try { + select { c.onReceive { it } } + } catch (e: NumberedCancellationException) { + e.testResult + } + + @Operation(causesBlocking = true, blocking = true) + fun close(@Param(name = "closeToken") token: Int): Boolean = c.close(NumberedCancellationException(token)) + + @Operation(causesBlocking = true, blocking = true) + fun cancel(@Param(name = "closeToken") token: Int) = c.cancel(NumberedCancellationException(token)) + + // @Operation TODO non-linearizable in BufferedChannel + open fun isClosedForReceive() = c.isClosedForReceive + + @Operation(blocking = true) + fun isClosedForSend() = c.isClosedForSend + + // @Operation TODO non-linearizable in BufferedChannel + open fun isEmpty() = c.isEmpty + + @StateRepresentation + fun state() = (c as? BufferedChannel<*>)?.toStringDebug() ?: c.toString() + + @Validate + fun validate() { + (c as? BufferedChannel<*>)?.checkSegmentStructureInvariants() + } + + override fun > O.customize(isStressTest: Boolean) = + actorsBefore(0).sequentialSpecification(sequentialSpecification) + + override fun ModelCheckingOptions.customize(isStressTest: Boolean) = + checkObstructionFreedom(obstructionFree) +} + +private class NumberedCancellationException(number: Int) : CancellationException() { + val testResult = "Closed($number)" +} + + +abstract class SequentialIntChannelBase(private val capacity: Int) { + private val senders = ArrayList, Int>>() + private val receivers = ArrayList>() + private val buffer = ArrayList() + private var closedMessage: String? = null + + suspend fun send(x: Int): Any = when (val offerRes = trySend(x)) { + true -> Unit + false -> suspendCancellableCoroutine { cont -> + senders.add(cont to x) + } + else -> offerRes + } + + fun trySend(element: Int): Any { + if (closedMessage !== null) return closedMessage!! + if (capacity == CONFLATED) { + if (resumeFirstReceiver(element)) return true + buffer.clear() + buffer.add(element) + return true + } + if (resumeFirstReceiver(element)) return true + if (buffer.size < capacity) { + buffer.add(element) + return true + } + return false + } + + private fun resumeFirstReceiver(element: Int): Boolean { + while (receivers.isNotEmpty()) { + val r = receivers.removeAt(0) + if (r.resume(element)) return true + } + return false + } + + suspend fun receive(): Any = tryReceive() ?: suspendCancellableCoroutine { cont -> + receivers.add(cont) + } + + suspend fun receiveCatching() = receive() + + fun tryReceive(): Any? { + if (buffer.isNotEmpty()) { + val el = buffer.removeAt(0) + resumeFirstSender().also { + if (it !== null) buffer.add(it) + } + return el + } + resumeFirstSender()?.also { return it } + if (closedMessage !== null) return closedMessage + return null + } + + private fun resumeFirstSender(): Int? { + while (senders.isNotEmpty()) { + val (s, el) = senders.removeAt(0) + if (s.resume(Unit)) return el + } + return null + } + + suspend fun sendViaSelect(element: Int) = send(element) + suspend fun receiveViaSelect() = receive() + + fun close(token: Int): Boolean { + if (closedMessage !== null) return false + closedMessage = "Closed($token)" + for (r in receivers) r.resume(closedMessage!!) + receivers.clear() + return true + } + + fun cancel(token: Int) { + close(token) + for ((s, _) in senders) s.resume(closedMessage!!) + senders.clear() + buffer.clear() + } + + fun isClosedForSend(): Boolean = closedMessage !== null + fun isClosedForReceive(): Boolean = isClosedForSend() && buffer.isEmpty() && senders.isEmpty() + + fun isEmpty(): Boolean { + if (closedMessage !== null) return false + return buffer.isEmpty() && senders.isEmpty() + } +} + +private fun CancellableContinuation.resume(res: T): Boolean { + val token = tryResume(res) ?: return false + completeResume(token) + return true +} diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeTaskQueueLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeTaskQueueLincheckTest.kt new file mode 100644 index 0000000000..9a44891733 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/lincheck/LockFreeTaskQueueLincheckTest.kt @@ -0,0 +1,44 @@ +@file:Suppress("unused") + +package kotlinx.coroutines.lincheck + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import org.jetbrains.kotlinx.lincheck.* +import org.jetbrains.kotlinx.lincheck.annotations.* +import org.jetbrains.kotlinx.lincheck.annotations.Operation +import org.jetbrains.kotlinx.lincheck.paramgen.* +import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* +import org.jetbrains.kotlinx.lincheck.verifier.quiescent.* + +@Param(name = "value", gen = IntGen::class, conf = "1:3") +internal abstract class AbstractLockFreeTaskQueueWithoutRemoveLincheckTest( + val singleConsumer: Boolean +) : AbstractLincheckTest() { + @JvmField + protected val q = LockFreeTaskQueue(singleConsumer = singleConsumer) + + @Operation + fun close() = q.close() + + @Operation + fun addLast(@Param(name = "value") value: Int) = q.addLast(value) + + override fun > O.customize(isStressTest: Boolean): O = + verifier(QuiescentConsistencyVerifier::class.java) + + override fun ModelCheckingOptions.customize(isStressTest: Boolean) = + checkObstructionFreedom() +} + +internal class MCLockFreeTaskQueueWithRemoveLincheckTest : AbstractLockFreeTaskQueueWithoutRemoveLincheckTest(singleConsumer = false) { + @QuiescentConsistent + @Operation(blocking = true) + fun removeFirstOrNull() = q.removeFirstOrNull() +} + +internal class SCLockFreeTaskQueueWithRemoveLincheckTest : AbstractLockFreeTaskQueueWithoutRemoveLincheckTest(singleConsumer = true) { + @QuiescentConsistent + @Operation(nonParallelGroup = "consumer") + fun removeFirstOrNull() = q.removeFirstOrNull() +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/MutexLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/MutexLincheckTest.kt new file mode 100644 index 0000000000..37051794b0 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/lincheck/MutexLincheckTest.kt @@ -0,0 +1,41 @@ +@file:Suppress("unused") +package kotlinx.coroutines.lincheck + +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import kotlinx.coroutines.sync.* +import org.jetbrains.kotlinx.lincheck.* +import org.jetbrains.kotlinx.lincheck.annotations.* +import org.jetbrains.kotlinx.lincheck.annotations.Operation +import org.jetbrains.kotlinx.lincheck.paramgen.* + +@Param(name = "owner", gen = IntGen::class, conf = "0:2") +class MutexLincheckTest : AbstractLincheckTest() { + private val mutex = Mutex() + + @Operation(handleExceptionsAsResult = [IllegalStateException::class]) + fun tryLock(@Param(name = "owner") owner: Int) = mutex.tryLock(owner.asOwnerOrNull) + + // TODO: `lock()` with non-null owner is non-linearizable + @Operation(promptCancellation = true) + suspend fun lock() = mutex.lock(null) + + // TODO: `onLock` with non-null owner is non-linearizable + // onLock may suspend in case of clause re-registration. + @Operation(allowExtraSuspension = true, promptCancellation = true) + suspend fun onLock() = select { mutex.onLock(null) {} } + + @Operation(handleExceptionsAsResult = [IllegalStateException::class]) + fun unlock(@Param(name = "owner") owner: Int) = mutex.unlock(owner.asOwnerOrNull) + + @Operation + fun isLocked() = mutex.isLocked + + @Operation + fun holdsLock(@Param(name = "owner") owner: Int) = mutex.holdsLock(owner) + + override fun > O.customize(isStressTest: Boolean): O = + actorsBefore(0) + + private val Int.asOwnerOrNull get() = if (this == 0) null else this +} diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/ResizableAtomicArrayLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/ResizableAtomicArrayLincheckTest.kt new file mode 100644 index 0000000000..5561a7dbf3 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/lincheck/ResizableAtomicArrayLincheckTest.kt @@ -0,0 +1,20 @@ +package kotlinx.coroutines.lincheck + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import org.jetbrains.kotlinx.lincheck.annotations.* +import org.jetbrains.kotlinx.lincheck.paramgen.* + +@Param(name = "index", gen = IntGen::class, conf = "0:4") +@Param(name = "value", gen = IntGen::class, conf = "1:5") +class ResizableAtomicArrayLincheckTest : AbstractLincheckTest() { + private val a = ResizableAtomicArray(2) + + @Operation + fun get(@Param(name = "index") index: Int): Int? = a[index] + + @Operation(nonParallelGroup = "writer") + fun set(@Param(name = "index") index: Int, @Param(name = "value") value: Int) { + a.setSynchronized(index, value) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/lincheck/SemaphoreLincheckTest.kt b/kotlinx-coroutines-core/jvm/test/lincheck/SemaphoreLincheckTest.kt new file mode 100644 index 0000000000..e99e8a185c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/lincheck/SemaphoreLincheckTest.kt @@ -0,0 +1,31 @@ +@file:Suppress("unused") +package kotlinx.coroutines.lincheck + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import org.jetbrains.kotlinx.lincheck.* +import org.jetbrains.kotlinx.lincheck.annotations.Operation +import org.jetbrains.kotlinx.lincheck.strategy.managed.modelchecking.* + +abstract class SemaphoreLincheckTestBase(permits: Int) : AbstractLincheckTest() { + private val semaphore = SemaphoreAndMutexImpl(permits = permits, acquiredPermits = 0) + + @Operation + fun tryAcquire() = semaphore.tryAcquire() + + @Operation(promptCancellation = true, allowExtraSuspension = true) + suspend fun acquire() = semaphore.acquire() + + @Operation(handleExceptionsAsResult = [IllegalStateException::class]) + fun release() = semaphore.release() + + override fun > O.customize(isStressTest: Boolean): O = + actorsBefore(0) + + override fun ModelCheckingOptions.customize(isStressTest: Boolean) = + checkObstructionFreedom() +} + +class Semaphore1LincheckTest : SemaphoreLincheckTestBase(1) +class Semaphore2LincheckTest : SemaphoreLincheckTestBase(2) diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherLivenessStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherLivenessStressTest.kt new file mode 100644 index 0000000000..20748a2d34 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherLivenessStressTest.kt @@ -0,0 +1,61 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.atomic.* +import kotlin.test.* + +/** + * Test that ensures implementation correctness of [LimitingDispatcher] and + * designed to stress its particular implementation details. + */ +class BlockingCoroutineDispatcherLivenessStressTest : SchedulerTestBase() { + private val concurrentWorkers = AtomicInteger(0) + + @Before + fun setUp() { + // In case of starvation test will hang + idleWorkerKeepAliveNs = Long.MAX_VALUE + } + + @Test + fun testAddPollRace() = runBlocking { + val limitingDispatcher = blockingDispatcher(1) + val iterations = 25_000 * stressTestMultiplier + // Stress test for specific case (race #2 from LimitingDispatcher). Shouldn't hang. + for (i in 1..iterations) { + val tasks = (1..2).map { + async(limitingDispatcher) { + try { + val currentlyExecuting = concurrentWorkers.incrementAndGet() + assertEquals(1, currentlyExecuting) + } finally { + concurrentWorkers.decrementAndGet() + } + } + } + tasks.forEach { it.await() } + } + } + + @Test + fun testPingPongThreadsCount() = runBlocking { + corePoolSize = CORES_COUNT + val iterations = 100_000 * stressTestMultiplier + val completed = AtomicInteger(0) + for (i in 1..iterations) { + val tasks = (1..2).map { + async(dispatcher) { + // Useless work + concurrentWorkers.incrementAndGet() + concurrentWorkers.decrementAndGet() + completed.incrementAndGet() + } + } + tasks.forEach { it.await() } + } + assertEquals(2 * iterations, completed.get()) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherMixedStealingStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherMixedStealingStressTest.kt new file mode 100644 index 0000000000..fe0fa4a433 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherMixedStealingStressTest.kt @@ -0,0 +1,77 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.concurrent.* + +/** + * Specific test that was designed to expose inference between stealing/polling of blocking and non-blocking tasks.RunningThreadStackMergeTest + */ +class BlockingCoroutineDispatcherMixedStealingStressTest : SchedulerTestBase() { + + private val iterations = 10_000 + + @Before + fun setUp() { + idleWorkerKeepAliveNs = Long.MAX_VALUE + } + + @Test + fun testBlockingProgressPreventedInternal() { + val blocking = blockingDispatcher(corePoolSize).asExecutor() + val regular = dispatcher.asExecutor() + repeat(iterations * stressTestMultiplier) { + val cpuBlocker = CyclicBarrier(corePoolSize + 1) + val blockingBlocker = CyclicBarrier(2) + regular.execute(Runnable { + // Block all CPU cores except current one + repeat(corePoolSize - 1) { + regular.execute(Runnable { + cpuBlocker.await() + }) + } + + blocking.execute(Runnable { + blockingBlocker.await() + }) + + regular.execute(Runnable { + blockingBlocker.await() + cpuBlocker.await() + }) + }) + cpuBlocker.await() + } + } + + @Test + fun testBlockingProgressPreventedExternal() { + val blocking = blockingDispatcher(corePoolSize).asExecutor() + val regular = dispatcher.asExecutor() + repeat(iterations / 2 * stressTestMultiplier) { + val cpuBlocker = CyclicBarrier(corePoolSize + 1) + val blockingBlocker = CyclicBarrier(2) + repeat(corePoolSize) { + regular.execute(Runnable { + cpuBlocker.await() + }) + } + // Wait for all threads to park + while (true) { + val waiters = Thread.getAllStackTraces().keys.count { (it.state == Thread.State.TIMED_WAITING || it.state == Thread.State.WAITING) + && it is CoroutineScheduler.Worker } + if (waiters >= corePoolSize) break + Thread.yield() + } + blocking.execute(Runnable { + blockingBlocker.await() + }) + regular.execute(Runnable { + }) + + blockingBlocker.await() + cpuBlocker.await() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherTerminationStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherTerminationStressTest.kt new file mode 100644 index 0000000000..ccb219186b --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherTerminationStressTest.kt @@ -0,0 +1,39 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.* +import java.util.concurrent.* + +class BlockingCoroutineDispatcherTerminationStressTest : TestBase() { + private val baseDispatcher = SchedulerCoroutineDispatcher( + 2, 20, + TimeUnit.MILLISECONDS.toNanos(10) + ) + private val ioDispatcher = baseDispatcher.blocking() + private val TEST_SECONDS = 3L * stressTestMultiplier + + @After + fun tearDown() { + baseDispatcher.close() + } + + /** + * Tests that when threads are created to accommodate the new tasks, but then don't receive any tasks for the + * duration of their terminate-on-idling timeout, liveness does not suffer. + */ + @Test + fun testTermination() = runTest { + val rnd = Random() + val deadline = System.currentTimeMillis() + TimeUnit.SECONDS.toMillis(TEST_SECONDS) + while (System.currentTimeMillis() < deadline) { + Thread.sleep(rnd.nextInt(30).toLong()) + repeat(rnd.nextInt(5) + 1) { + launch(ioDispatcher) { + Thread.sleep(rnd.nextInt(5).toLong()) + } + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherTest.kt new file mode 100644 index 0000000000..49510cde06 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherTest.kt @@ -0,0 +1,174 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.rules.* +import java.util.concurrent.* + +class BlockingCoroutineDispatcherTest : SchedulerTestBase() { + + @get:Rule + val timeout = Timeout.seconds(10L)!! + + @Test + fun testNonBlockingWithBlockingExternal() = runBlocking { + val barrier = CyclicBarrier(2) + + val blockingJob = launch(blockingDispatcher.value) { + barrier.await() + } + + val nonBlockingJob = launch(dispatcher) { + barrier.await() + } + + nonBlockingJob.join() + blockingJob.join() + checkPoolThreadsCreated(2..3) + } + + @Test + fun testNonBlockingFromBlocking() = runBlocking { + val barrier = CyclicBarrier(2) + + val blocking = launch(blockingDispatcher.value) { + // This task will be stolen + launch(dispatcher) { + barrier.await() + } + + barrier.await() + } + + blocking.join() + checkPoolThreadsCreated(2..3) + } + + @Test + fun testScheduleBlockingThreadCount() = runTest { + // After first iteration pool is idle, repeat, no new threads should be created + repeat(2) { + val blocking = launch(blockingDispatcher.value) { + launch(blockingDispatcher.value) { + } + } + + blocking.join() + // Depends on how fast thread will be created + checkPoolThreadsCreated(2..3) + } + } + + @Test + fun testNoCpuStarvation() = runBlocking { + val tasksNum = 100 + val barrier = CyclicBarrier(tasksNum + 1) + val tasks = (1..tasksNum).map { launch(blockingDispatcher.value) { barrier.await() } } + + val cpuTask = launch(dispatcher) { + // Do nothing, just complete + } + + cpuTask.join() + tasks.forEach { require(it.isActive) } + barrier.await() + tasks.joinAll() + } + + @Test + fun testNoCpuStarvationWithMultipleBlockingContexts() = runBlocking { + val firstBarrier = CyclicBarrier(11) + val secondBarrier = CyclicBarrier(11) + val blockingDispatcher = blockingDispatcher(10) + val blockingDispatcher2 = blockingDispatcher(10) + + val blockingTasks = (1..10).flatMap { + listOf(launch(blockingDispatcher) { firstBarrier.await() }, launch(blockingDispatcher2) { secondBarrier.await() }) + } + + val cpuTasks = (1..100).map { + launch(dispatcher) { + // Do nothing, just complete + } + }.toList() + + cpuTasks.joinAll() + blockingTasks.forEach { require(it.isActive) } + firstBarrier.await() + secondBarrier.await() + blockingTasks.joinAll() + checkPoolThreadsCreated(21 /* blocking tasks + 1 for CPU */..20 + CORES_COUNT) + } + + @Test + fun testNoExcessThreadsCreated() = runBlocking { + corePoolSize = 4 + + val tasksNum = 100 + val barrier = CyclicBarrier(tasksNum + 1) + val blockingTasks = (1..tasksNum).map { launch(blockingDispatcher.value) { barrier.await() } } + + val nonBlockingTasks = (1..tasksNum).map { + launch(dispatcher) { + yield() + } + } + + nonBlockingTasks.joinAll() + barrier.await() + blockingTasks.joinAll() + // There may be race when multiple CPU threads are trying to lazily created one more + checkPoolThreadsCreated(101..100 + CORES_COUNT) + } + + @Test(timeout = 1_000) + fun testYield() = runBlocking { + corePoolSize = 1 + maxPoolSize = 1 + val bd = blockingDispatcher(1) + val outerJob = launch(bd) { + expect(1) + val innerJob = launch(bd) { + // Do nothing + expect(3) + } + + expect(2) + while (innerJob.isActive) { + yield() + } + + expect(4) + innerJob.join() + } + + outerJob.join() + finish(5) + } + + @Test + fun testUndispatchedYield() = runTest { + expect(1) + corePoolSize = 1 + maxPoolSize = 1 + val blockingDispatcher = blockingDispatcher(1) + val job = launch(blockingDispatcher, CoroutineStart.UNDISPATCHED) { + expect(2) + yield() + } + expect(3) + job.join() + finish(4) + } + + @Test(expected = IllegalArgumentException::class) + fun testNegativeParallelism() { + blockingDispatcher(-1) + } + + @Test(expected = IllegalArgumentException::class) + fun testZeroParallelism() { + blockingDispatcher(0) + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherThreadLimitStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherThreadLimitStressTest.kt new file mode 100644 index 0000000000..29d8ed97aa --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherThreadLimitStressTest.kt @@ -0,0 +1,58 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Ignore +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import kotlin.test.* + +class BlockingCoroutineDispatcherThreadLimitStressTest : SchedulerTestBase() { + + init { + corePoolSize = CORES_COUNT + } + + private val observedParallelism = ConcurrentHashMap().keySet(true) + private val concurrentWorkers = AtomicInteger(0) + + @Test + fun testLimitParallelismToOne() = runTest { + val limitingDispatcher = blockingDispatcher(1) + // Do in bursts to avoid OOM + repeat(100 * stressTestMultiplierSqrt) { + val iterations = 1_000 * stressTestMultiplierSqrt + val tasks = (1..iterations).map { + async(limitingDispatcher) { + try { + val currentlyExecuting = concurrentWorkers.incrementAndGet() + observedParallelism.add(currentlyExecuting) + } finally { + concurrentWorkers.decrementAndGet() + } + } + } + tasks.awaitAll() + assertEquals(1, observedParallelism.single(), "Expected parallelism should be 1, had $observedParallelism") + } + } + + @Test + fun testLimitParallelism() = runBlocking { + val limitingDispatcher = blockingDispatcher(CORES_COUNT) + val iterations = 50_000 * stressTestMultiplier + val tasks = (1..iterations).map { + async(limitingDispatcher) { + try { + val currentlyExecuting = concurrentWorkers.incrementAndGet() + observedParallelism.add(currentlyExecuting) + } finally { + concurrentWorkers.decrementAndGet() + } + } + } + tasks.awaitAll() + assertTrue(observedParallelism.max() <= CORES_COUNT, "Unexpected state: $observedParallelism") + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherWorkSignallingStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherWorkSignallingStressTest.kt new file mode 100644 index 0000000000..81a1421019 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/BlockingCoroutineDispatcherWorkSignallingStressTest.kt @@ -0,0 +1,95 @@ +@file:Suppress("DeferredResultUnused") + +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class BlockingCoroutineDispatcherWorkSignallingStressTest : SchedulerTestBase() { + + @Test + fun testCpuTasksStarvation() = runBlocking { + val iterations = 1000 * stressTestMultiplier + repeat(iterations) { + // Create a dispatcher every iteration to increase probability of race + val dispatcher = SchedulerCoroutineDispatcher(CORES_COUNT) + val blockingDispatcher = dispatcher.blocking(100) + + val blockingBarrier = CyclicBarrier(CORES_COUNT * 3 + 1) + val cpuBarrier = CyclicBarrier(CORES_COUNT + 1) + + val cpuTasks = CopyOnWriteArrayList>() + val blockingTasks = CopyOnWriteArrayList>() + + repeat(CORES_COUNT) { + async(dispatcher) { + // These two will be stolen first + blockingTasks += blockingAwait(blockingDispatcher, blockingBarrier) + blockingTasks += blockingAwait(blockingDispatcher, blockingBarrier) + // Empty on CPU job which should be executed while blocked tasks are waiting + cpuTasks += cpuAwait(dispatcher, cpuBarrier) + // Block with next task. Block cores * 3 threads in total + blockingTasks += blockingAwait(blockingDispatcher, blockingBarrier) + } + } + + cpuTasks.forEach { require(it.isActive) } + cpuBarrier.await() + cpuTasks.awaitAll() + blockingTasks.forEach { require(it.isActive) } + blockingBarrier.await() + blockingTasks.awaitAll() + dispatcher.close() + } + } + + private fun CoroutineScope.blockingAwait( + blockingDispatcher: CoroutineDispatcher, + blockingBarrier: CyclicBarrier + ) = async(blockingDispatcher) { blockingBarrier.await() } + + + private fun CoroutineScope.cpuAwait( + blockingDispatcher: CoroutineDispatcher, + blockingBarrier: CyclicBarrier + ) = async(blockingDispatcher) { blockingBarrier.await() } + + @Test + fun testBlockingTasksStarvation() = runBlocking { + corePoolSize = 2 // Easier to reproduce race with unparks + val iterations = 10_000 * stressTestMultiplier + val blockingLimit = 4 // CORES_COUNT * 3 + val blocking = blockingDispatcher(blockingLimit) + + repeat(iterations) { + val barrier = CyclicBarrier(blockingLimit + 1) + // Should eat all limit * 3 cpu without any starvation + val tasks = (1..blockingLimit).map { async(blocking) { barrier.await() } } + tasks.forEach { assertTrue(it.isActive) } + barrier.await() + tasks.joinAll() + } + } + + @Test + fun testBlockingTasksStarvationWithCpuTasks() = runBlocking { + val iterations = 1000 * stressTestMultiplier + val blockingLimit = CORES_COUNT * 2 + val blocking = blockingDispatcher(blockingLimit) + + repeat(iterations) { + // Overwhelm global queue with external CPU tasks + val cpuTasks = (1..CORES_COUNT).map { async(dispatcher) { while (true) delay(1) } } + val barrier = CyclicBarrier(blockingLimit + 1) + // Should eat all limit * 3 cpu without any starvation + val tasks = (1..blockingLimit).map { async(blocking) { barrier.await() } } + tasks.forEach { assertTrue(it.isActive) } + barrier.await() + tasks.joinAll() + cpuTasks.forEach { it.cancelAndJoin() } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineDispatcherTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineDispatcherTest.kt new file mode 100644 index 0000000000..ee21be23fc --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineDispatcherTest.kt @@ -0,0 +1,144 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.atomic.* +import kotlin.test.* + +class CoroutineDispatcherTest : SchedulerTestBase() { + + @After + fun tearDown() { + schedulerTimeSource = NanoTimeSource + } + + @Test + fun testSingleThread() = runBlocking { + corePoolSize = 1 + expect(1) + withContext(dispatcher) { + require(Thread.currentThread() is CoroutineScheduler.Worker) + expect(2) + val job = async { + expect(3) + delay(10) + expect(4) + } + + job.await() + expect(5) + } + + finish(6) + checkPoolThreadsCreated(1) + } + + @Test + fun testFairScheduling() = runBlocking { + corePoolSize = 1 + expect(1) + val outerJob = launch(dispatcher) { + val d1 = launch(dispatcher) { expect(3) } + val d2 = launch(dispatcher) { expect(4) } + val d3 = launch(dispatcher) { expect(2) } + listOf(d1, d2, d3).joinAll() + } + outerJob.join() + finish(5) + } + + @Test + fun testStealing() = runBlocking { + corePoolSize = 2 + val flag = AtomicBoolean(false) + val job = async(dispatcher) { + expect(1) + val innerJob = async { + expect(2) + flag.set(true) + } + while (!flag.get()) { + Thread.yield() // Block current thread, submitted inner job will be stolen + } + + innerJob.await() + expect(3) + } + job.await() + finish(4) + checkPoolThreadsCreated(2) + } + + @Test + fun testDelay() = runBlocking { + corePoolSize = 2 + withContext(dispatcher) { + expect(1) + delay(10) + expect(2) + } + finish(3) + checkPoolThreadsCreated(2) + } + + @Test + fun testMaxSize() = runBlocking { + corePoolSize = 1 + maxPoolSize = 4 + (1..10).map { launch(blockingDispatcher.value) { Thread.sleep(100) } }.joinAll() + checkPoolThreadsCreated(4) + } + + @Test(timeout = 1_000) + fun testYield() = runBlocking { + corePoolSize = 1 + maxPoolSize = 1 + val outerJob = launch(dispatcher) { + expect(1) + val innerJob = launch(dispatcher) { + // Do nothing + expect(3) + } + + expect(2) + while (innerJob.isActive) { + yield() + } + + expect(4) + innerJob.join() + } + outerJob.join() + finish(5) + } + + @Test + fun testUndispatchedYield() = runTest { + expect(1) + val job = launch(dispatcher, CoroutineStart.UNDISPATCHED) { + expect(2) + yield() + } + expect(3) + job.join() + finish(4) + } + + @Test + fun testThreadName() = runBlocking { + val initialCount = Thread.getAllStackTraces().keys.asSequence() + .count { it is CoroutineScheduler.Worker && it.name.contains("SomeTestName") } + assertEquals(0, initialCount) + val dispatcher = SchedulerCoroutineDispatcher(1, 1, IDLE_WORKER_KEEP_ALIVE_NS, "SomeTestName") + dispatcher.use { + launch(dispatcher) { + }.join() + + val count = Thread.getAllStackTraces().keys.asSequence() + .count { it is CoroutineScheduler.Worker && it.name.contains("SomeTestName") } + assertEquals(1, count) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerCloseStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerCloseStressTest.kt new file mode 100644 index 0000000000..d6719aaa68 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerCloseStressTest.kt @@ -0,0 +1,76 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import java.util.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class CoroutineSchedulerCloseStressTest(private val mode: Mode) : TestBase() { + enum class Mode { CPU, BLOCKING, CPU_LIMITED } + + companion object { + @Parameterized.Parameters(name = "mode={0}") + @JvmStatic + fun params(): Collection> = Mode.values().map { arrayOf(it) } + } + + private val MAX_LEVEL = 5 + private val N_COROS = (1 shl (MAX_LEVEL + 1)) - 1 + private val N_THREADS = 4 + private val rnd = Random() + + private lateinit var closeableDispatcher: SchedulerCoroutineDispatcher + private lateinit var dispatcher: CoroutineDispatcher + + private val started = atomic(0) + private val finished = atomic(0) + + @Test + fun testNormalClose() { + try { + launchCoroutines() + } finally { + closeableDispatcher.close() + } + } + + private fun launchCoroutines() = runBlocking { + closeableDispatcher = SchedulerCoroutineDispatcher(N_THREADS) + dispatcher = when (mode) { + Mode.CPU -> closeableDispatcher + Mode.CPU_LIMITED -> closeableDispatcher.limitedParallelism(N_THREADS) + Mode.BLOCKING -> closeableDispatcher.blocking(N_THREADS) + } + started.value = 0 + finished.value = 0 + withContext(dispatcher) { + launchChild(0, 0) + } + assertEquals(N_COROS, started.value) + assertEquals(N_COROS, finished.value) + } + + // Index and level are used only for debugging purpose + private fun CoroutineScope.launchChild(index: Int, level: Int): Job = launch(start = CoroutineStart.ATOMIC) { + started.incrementAndGet() + try { + if (level < MAX_LEVEL) { + launchChild(2 * index + 1, level + 1) + launchChild(2 * index + 2, level + 1) + } else { + if (rnd.nextBoolean()) { + delay(1000) + } else { + yield() + } + } + } finally { + finished.incrementAndGet() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerInternalApiStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerInternalApiStressTest.kt new file mode 100644 index 0000000000..dbc6609101 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerInternalApiStressTest.kt @@ -0,0 +1,86 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.AVAILABLE_PROCESSORS +import org.junit.Test +import java.util.* +import java.util.concurrent.ConcurrentHashMap +import java.util.concurrent.CountDownLatch +import java.util.concurrent.CyclicBarrier +import java.util.concurrent.atomic.AtomicInteger +import kotlin.random.* +import kotlin.random.Random +import kotlin.test.* +import kotlin.time.* + +class CoroutineSchedulerInternalApiStressTest : TestBase() { + + @Test(timeout = 120_000L) + fun testHelpDefaultIoIsIsolated() = repeat(100 * stressTestMultiplierSqrt) { + val ioTaskMarker = ThreadLocal.withInitial { false } + runTest { + val jobToComplete = Job() + val expectedIterations = 100 + val completionLatch = CountDownLatch(1) + val tasksToCompleteJob = AtomicInteger(expectedIterations) + val observedIoThreads = Collections.newSetFromMap(ConcurrentHashMap()) + val observedDefaultThreads = Collections.newSetFromMap(ConcurrentHashMap()) + + val barrier = CyclicBarrier(AVAILABLE_PROCESSORS) + val spawners = ArrayList() + repeat(AVAILABLE_PROCESSORS - 1) { + // Launch CORES - 1 spawners + spawners += launch(Dispatchers.Default) { + barrier.await() + repeat(expectedIterations) { + launch { + val tasksLeft = tasksToCompleteJob.decrementAndGet() + if (tasksLeft < 0) return@launch // Leftovers are being executed all over the place + observedDefaultThreads.add(Thread.currentThread()) + if (tasksLeft == 0) { + // Verify threads first + try { + assertFalse(observedIoThreads.containsAll(observedDefaultThreads)) + } finally { + jobToComplete.complete() + } + } + } + + // Sometimes launch an IO task to mess with a scheduler + if (Random.nextInt(0..9) == 0) { + launch(Dispatchers.IO) { + ioTaskMarker.set(true) + observedIoThreads.add(Thread.currentThread()) + assertTrue(Thread.currentThread().isIoDispatcherThread()) + } + } + } + completionLatch.await() + } + } + + withContext(Dispatchers.Default) { + barrier.await() + var timesHelped = 0 + while (!jobToComplete.isCompleted) { + val result = runSingleTaskFromCurrentSystemDispatcher() + assertFalse(ioTaskMarker.get()) + if (result == 0L) { + ++timesHelped + continue + } else if (result >= 0L) { + Thread.sleep(result.toDuration(DurationUnit.NANOSECONDS).toDelayMillis()) + } else { + Thread.sleep(10) + } + } + completionLatch.countDown() + assertEquals(100, timesHelped) + assertTrue(Thread.currentThread() in observedDefaultThreads, observedDefaultThreads.toString()) + } + } + } +} + diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerLivenessStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerLivenessStressTest.kt new file mode 100644 index 0000000000..07b549f917 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerLivenessStressTest.kt @@ -0,0 +1,49 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.scheduling.CoroutineScheduler.Companion.MAX_SUPPORTED_POOL_SIZE +import org.junit.* +import java.util.concurrent.* + +class CoroutineSchedulerLivenessStressTest : TestBase() { + private val scheduler = lazy { CoroutineScheduler(CORE_POOL_SIZE, MAX_SUPPORTED_POOL_SIZE, Long.MAX_VALUE) } + private val iterations = 1000 * stressTestMultiplier + + @After + fun tearDown() { + if (scheduler.isInitialized()) { + scheduler.value.close() + } + } + + @Test + fun testInternalSubmissions() { + Assume.assumeTrue(CORE_POOL_SIZE >= 2) + repeat(iterations) { + val barrier = CyclicBarrier(CORE_POOL_SIZE + 1) + scheduler.value.execute { + repeat(CORE_POOL_SIZE) { + scheduler.value.execute { + barrier.await() + } + } + } + barrier.await() + } + } + + @Test + fun testExternalSubmissions() { + Assume.assumeTrue(CORE_POOL_SIZE >= 2) + repeat(iterations) { + val barrier = CyclicBarrier(CORE_POOL_SIZE + 1) + repeat(CORE_POOL_SIZE) { + scheduler.value.execute { + barrier.await() + } + } + barrier.await() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerOversubscriptionTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerOversubscriptionTest.kt new file mode 100644 index 0000000000..c891878ffb --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerOversubscriptionTest.kt @@ -0,0 +1,86 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicInteger + +class CoroutineSchedulerOversubscriptionTest : TestBase() { + + private val inDefault = AtomicInteger(0) + + private fun CountDownLatch.runAndCheck() { + if (inDefault.incrementAndGet() > CORE_POOL_SIZE) { + error("Oversubscription detected") + } + + await() + inDefault.decrementAndGet() + } + + @Test + fun testOverSubscriptionDeterministic() = runTest { + val barrier = CountDownLatch(1) + val threadsOccupiedBarrier = CyclicBarrier(CORE_POOL_SIZE) + // All threads but one + repeat(CORE_POOL_SIZE - 1) { + launch(Dispatchers.Default) { + threadsOccupiedBarrier.await() + barrier.runAndCheck() + } + } + threadsOccupiedBarrier.await() + withContext(Dispatchers.Default) { + // Put a task in a local queue, it will be stolen + launch(Dispatchers.Default) { + barrier.runAndCheck() + } + // Put one more task to trick the local queue check + launch(Dispatchers.Default) { + barrier.runAndCheck() + } + + withContext(Dispatchers.IO) { + try { + // Release the thread + delay(100) + } finally { + barrier.countDown() + } + } + } + } + + @Test + fun testOverSubscriptionStress() = repeat(1000 * stressTestMultiplierSqrt) { + inDefault.set(0) + runTest { + val barrier = CountDownLatch(1) + val threadsOccupiedBarrier = CyclicBarrier(CORE_POOL_SIZE) + // All threads but one + repeat(CORE_POOL_SIZE - 1) { + launch(Dispatchers.Default) { + threadsOccupiedBarrier.await() + barrier.runAndCheck() + } + } + threadsOccupiedBarrier.await() + withContext(Dispatchers.Default) { + // Put a task in a local queue + launch(Dispatchers.Default) { + barrier.runAndCheck() + } + // Put one more task to trick the local queue check + launch(Dispatchers.Default) { + barrier.runAndCheck() + } + + withContext(Dispatchers.IO) { + yield() + barrier.countDown() + } + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerStressTest.kt new file mode 100644 index 0000000000..615cdeea7c --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerStressTest.kt @@ -0,0 +1,105 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import kotlin.coroutines.* +import kotlin.test.* + +class CoroutineSchedulerStressTest : TestBase() { + private var dispatcher: SchedulerCoroutineDispatcher = SchedulerCoroutineDispatcher() + private val observedThreads = ConcurrentHashMap() + private val tasksNum = 500_000 * stressMemoryMultiplier() + + private fun stressMemoryMultiplier(): Int { + return if (isStressTest) { + AVAILABLE_PROCESSORS * 4 + } else { + 1 + } + } + + private val processed = AtomicInteger(0) + private val finishLatch = CountDownLatch(1) + + @After + fun tearDown() { + dispatcher.close() + } + + @Test + fun testInternalTasksSubmissionProgress() { + /* + * Run a lot of tasks and validate that + * 1) All of them are completed successfully + * 2) Every thread executed task at least once + */ + dispatcher.dispatch(EmptyCoroutineContext, Runnable { + for (i in 1..tasksNum) { + dispatcher.dispatch(EmptyCoroutineContext, ValidatingRunnable()) + } + }) + + finishLatch.await() + val observed = observedThreads.size + // on slow machines not all threads can be observed + assertTrue(observed in (AVAILABLE_PROCESSORS - 1)..(AVAILABLE_PROCESSORS + 1), "Observed $observed threads with $AVAILABLE_PROCESSORS available processors") + validateResults() + } + + @Test + fun testStealingFromNonProgressing() { + /* + * Work-stealing stress test, + * one thread submits pack of tasks, waits until they are completed (to avoid work offloading) + * and then repeats, thus never executing its own tasks and relying only on work stealing. + */ + var blockingThread: Thread? = null + dispatcher.dispatch(EmptyCoroutineContext, Runnable { + // Submit million tasks + blockingThread = Thread.currentThread() + var submittedTasks = 0 + while (submittedTasks < tasksNum) { + + ++submittedTasks + dispatcher.dispatch(EmptyCoroutineContext, ValidatingRunnable()) + while (submittedTasks - processed.get() > 100) { + Thread.yield() + } + } + // Block current thread + finishLatch.await() + }) + + finishLatch.await() + + assertFalse(observedThreads.containsKey(blockingThread!!)) + validateResults() + } + + private fun processTask() { + val counter = observedThreads[Thread.currentThread()] ?: 0L + observedThreads[Thread.currentThread()] = counter + 1 + if (processed.incrementAndGet() == tasksNum) { + finishLatch.countDown() + } + } + + private fun validateResults() { + val result = observedThreads.values.sum() + assertEquals(tasksNum.toLong(), result) + } + + private inner class ValidatingRunnable : Runnable { + private val invoked = atomic(false) + override fun run() { + if (!invoked.compareAndSet(false, true)) error("The same runnable was invoked twice") + processTask() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerTest.kt new file mode 100644 index 0000000000..fe09090362 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/CoroutineSchedulerTest.kt @@ -0,0 +1,166 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import org.junit.Test +import java.lang.Runnable +import java.util.concurrent.* +import kotlin.coroutines.* +import kotlin.test.* + +class CoroutineSchedulerTest : TestBase() { + private val contexts = listOf(NonBlockingContext, BlockingContext) + + @Test + fun testModesExternalSubmission() { // Smoke + CoroutineScheduler(1, 1).use { + for (context in contexts) { + val latch = CountDownLatch(1) + it.dispatch(Runnable { + latch.countDown() + }, context) + + latch.await() + } + } + } + + @Test + fun testModesInternalSubmission() { // Smoke + CoroutineScheduler(2, 2).use { + val latch = CountDownLatch(contexts.size) + it.dispatch(Runnable { + for (context in contexts) { + it.dispatch(Runnable { + latch.countDown() + }, context) + } + }) + + latch.await() + } + } + + @Test + fun testNonFairSubmission() { + CoroutineScheduler(1, 1).use { + val startLatch = CountDownLatch(1) + val finishLatch = CountDownLatch(2) + + it.dispatch(Runnable { + it.dispatch(Runnable { + expect(2) + finishLatch.countDown() + }) + + it.dispatch(Runnable { + expect(1) + finishLatch.countDown() + }) + }) + + startLatch.countDown() + finishLatch.await() + finish(3) + } + } + + @Test + fun testFairSubmission() { + CoroutineScheduler(1, 1).use { + val startLatch = CountDownLatch(1) + val finishLatch = CountDownLatch(2) + + it.dispatch(Runnable { + it.dispatch(Runnable { + expect(1) + finishLatch.countDown() + }) + + it.dispatch(Runnable { + expect(2) + finishLatch.countDown() + }, fair = true) + }) + + startLatch.countDown() + finishLatch.await() + finish(3) + } + } + + @Test + fun testRngUniformDistribution() { + CoroutineScheduler(1, 128).use { scheduler -> + val worker = scheduler.Worker(1) + testUniformDistribution(worker, 2) + testUniformDistribution(worker, 4) + testUniformDistribution(worker, 8) + testUniformDistribution(worker, 12) + testUniformDistribution(worker, 16) + } + } + + @Test(expected = IllegalArgumentException::class) + fun testNegativeCorePoolSize() { + SchedulerCoroutineDispatcher(-1, 4) + } + + @Test(expected = IllegalArgumentException::class) + fun testNegativeMaxPoolSize() { + SchedulerCoroutineDispatcher(1, -4) + } + + @Test(expected = IllegalArgumentException::class) + fun testCorePoolSizeGreaterThanMaxPoolSize() { + SchedulerCoroutineDispatcher(4, 1) + } + + @Test + fun testSelfClose() { + val dispatcher = SchedulerCoroutineDispatcher(1, 1) + val latch = CountDownLatch(1) + dispatcher.dispatch(EmptyCoroutineContext, Runnable { + dispatcher.close(); latch.countDown() + }) + latch.await() + } + + @Test + fun testInterruptionCleanup() { + SchedulerCoroutineDispatcher(1, 1).use { + val executor = it.executor + var latch = CountDownLatch(1) + executor.execute { + Thread.currentThread().interrupt() + latch.countDown() + } + latch.await() + Thread.sleep(100) // I am really sorry + latch = CountDownLatch(1) + executor.execute { + try { + assertFalse(Thread.currentThread().isInterrupted) + } finally { + latch.countDown() + } + } + latch.await() + } + } + + private fun testUniformDistribution(worker: CoroutineScheduler.Worker, bound: Int) { + val result = IntArray(bound) + val iterations = 10_000_000 + repeat(iterations) { + ++result[worker.nextInt(bound)] + } + + val bucketSize = iterations / bound + for (i in result) { + val ratio = i.toDouble() / bucketSize + // 10% deviation + check(ratio <= 1.1) + check(ratio >= 0.9) + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/DefaultDispatchersTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/DefaultDispatchersTest.kt new file mode 100644 index 0000000000..f06f7cbe88 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/DefaultDispatchersTest.kt @@ -0,0 +1,72 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import kotlin.test.* + +class DefaultDispatchersTest : TestBase() { + + private /*const*/ val EXPECTED_PARALLELISM = 64 + + @Test(timeout = 10_000L) + fun testLimitedParallelismIsSeparatedFromDefaultIo() = runTest { + val barrier = CyclicBarrier(EXPECTED_PARALLELISM + 1) + val ioBlocker = CountDownLatch(1) + repeat(EXPECTED_PARALLELISM) { + launch(Dispatchers.IO) { + barrier.await() + ioBlocker.await() + } + } + + barrier.await() // Ensure all threads are occupied + barrier.reset() + val limited = Dispatchers.IO.limitedParallelism(EXPECTED_PARALLELISM) + repeat(EXPECTED_PARALLELISM) { + launch(limited) { + barrier.await() + } + } + barrier.await() + ioBlocker.countDown() + } + + @Test(timeout = 10_000L) + fun testDefaultDispatcherIsSeparateFromIO() = runTest { + val ioBarrier = CyclicBarrier(EXPECTED_PARALLELISM + 1) + val ioBlocker = CountDownLatch(1) + repeat(EXPECTED_PARALLELISM) { + launch(Dispatchers.IO) { + ioBarrier.await() + ioBlocker.await() + } + } + + ioBarrier.await() // Ensure all threads are occupied + val parallelism = Runtime.getRuntime().availableProcessors() + val defaultBarrier = CyclicBarrier(parallelism + 1) + repeat(parallelism) { + launch(Dispatchers.Default) { + defaultBarrier.await() + } + } + defaultBarrier.await() + ioBlocker.countDown() + } + + @Test + fun testHardCapOnParallelism() = runTest { + val iterations = 100_000 * stressTestMultiplierSqrt + val concurrency = AtomicInteger() + repeat(iterations) { + launch(Dispatchers.IO) { + val c = concurrency.incrementAndGet() + assertTrue("Got: $c") { c <= EXPECTED_PARALLELISM } + concurrency.decrementAndGet() + } + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/LimitingCoroutineDispatcherStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/LimitingCoroutineDispatcherStressTest.kt new file mode 100644 index 0000000000..6a11444d53 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/LimitingCoroutineDispatcherStressTest.kt @@ -0,0 +1,52 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.coroutines.* +import kotlin.test.* + +class LimitingCoroutineDispatcherStressTest : SchedulerTestBase() { + + init { + corePoolSize = 3 + } + + private val blocking = blockingDispatcher(2) + private val cpuView = view(2) + private val cpuView2 = view(2) + private val concurrentWorkers = atomic(0) + private val iterations = 25_000 * stressTestMultiplierSqrt + + @Test + fun testCpuLimitNotExtended() = runBlocking { + val tasks = ArrayList>(iterations * 2) + repeat(iterations) { + tasks += task(cpuView, 3) + tasks += task(cpuView2, 3) + } + + tasks.awaitAll() + } + + @Test + fun testCpuLimitWithBlocking() = runBlocking { + val tasks = ArrayList>(iterations * 2) + repeat(iterations) { + tasks += task(cpuView, 4) + tasks += task(blocking, 4) + } + + tasks.awaitAll() + } + + private fun task(ctx: CoroutineContext, maxLimit: Int): Deferred = GlobalScope.async(ctx) { + try { + val currentlyExecuting = concurrentWorkers.incrementAndGet() + assertTrue(currentlyExecuting <= maxLimit, "Executing: $currentlyExecuting, max limit: $maxLimit") + } finally { + concurrentWorkers.decrementAndGet() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/LimitingDispatcherTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/LimitingDispatcherTest.kt new file mode 100644 index 0000000000..1ba11200f3 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/LimitingDispatcherTest.kt @@ -0,0 +1,41 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.concurrent.* + +class LimitingDispatcherTest : SchedulerTestBase() { + + @Test(expected = IllegalArgumentException::class) + fun testNegativeView() { + view(-1) + } + + @Test(expected = IllegalArgumentException::class) + fun testZeroView() { + view(0) + } + + @Test(timeout = 10_000) + fun testBlockingInterleave() = runBlocking { + corePoolSize = 3 + val view = view(2) + val blocking = blockingDispatcher(4) + val barrier = CyclicBarrier(6) + val tasks = ArrayList(6) + repeat(2) { + tasks += async(view) { + barrier.await() + } + + repeat(2) { + tasks += async(blocking) { + barrier.await() + } + } + } + + tasks.joinAll() + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/SchedulerTestBase.kt b/kotlinx-coroutines-core/jvm/test/scheduling/SchedulerTestBase.kt new file mode 100644 index 0000000000..33e32838da --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/SchedulerTestBase.kt @@ -0,0 +1,116 @@ +@file:Suppress("UNUSED_VARIABLE") + +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import org.junit.* +import kotlin.coroutines.* +import kotlin.test.* + +abstract class SchedulerTestBase : TestBase() { + companion object { + val CORES_COUNT = AVAILABLE_PROCESSORS + + /** + * Asserts that [expectedThreadsCount] pool worker threads were created. + * Note that 'created' doesn't mean 'exists' because pool supports dynamic shrinking + */ + fun checkPoolThreadsCreated(expectedThreadsCount: Int = CORES_COUNT) { + val threadsCount = maxSequenceNumber()!! + assertEquals(expectedThreadsCount, threadsCount, "Expected $expectedThreadsCount pool threads, but has $threadsCount") + } + + /** + * Asserts that any number of pool worker threads in [range] were created. + * Note that 'created' doesn't mean 'exists' because pool supports dynamic shrinking + */ + fun checkPoolThreadsCreated(range: IntRange, base: Int = CORES_COUNT) { + val maxSequenceNumber = maxSequenceNumber()!! + val r = (range.first)..(range.last + base) + assertTrue( + maxSequenceNumber in r, + "Expected pool threads to be in interval $r, but has $maxSequenceNumber" + ) + } + + private fun maxSequenceNumber(): Int? { + return Thread.getAllStackTraces().keys.asSequence().filter { it is CoroutineScheduler.Worker } + .map { sequenceNumber(it.name) }.maxOrNull() + } + + private fun sequenceNumber(threadName: String): Int { + val suffix = threadName.substring(threadName.lastIndexOf("-") + 1) + val separatorIndex = suffix.indexOf(' ') + if (separatorIndex == -1) { + return suffix.toInt() + } + + return suffix.substring(0, separatorIndex).toInt() + } + + suspend fun Iterable.joinAll() = forEach { it.join() } + } + + protected var corePoolSize = CORES_COUNT + protected var maxPoolSize = 1024 + protected var idleWorkerKeepAliveNs = IDLE_WORKER_KEEP_ALIVE_NS + + private var _dispatcher: SchedulerCoroutineDispatcher? = null + protected val dispatcher: CoroutineDispatcher + get() { + if (_dispatcher == null) { + _dispatcher = SchedulerCoroutineDispatcher( + corePoolSize, + maxPoolSize, + idleWorkerKeepAliveNs + ) + } + + return _dispatcher!! + } + + protected var blockingDispatcher = lazy { + blockingDispatcher(1000) + } + + protected fun blockingDispatcher(parallelism: Int): CoroutineDispatcher { + val intitialize = dispatcher + return _dispatcher!!.blocking(parallelism) + } + + protected fun view(parallelism: Int): CoroutineDispatcher { + val intitialize = dispatcher + return _dispatcher!!.limitedParallelism(parallelism) + } + + @After + fun after() { + runBlocking { + withTimeout(5_000) { + _dispatcher?.close() + } + } + } +} + +/** + * Implementation note: + * Our [Dispatcher.IO] is a [limitedParallelism][CoroutineDispatcher.limitedParallelism] dispatcher + * on top of unbounded scheduler. We want to test this scenario, but on top of non-singleton + * scheduler so we can control the number of threads, thus this method. + */ +internal fun SchedulerCoroutineDispatcher.blocking(parallelism: Int = 16): CoroutineDispatcher { + return object : CoroutineDispatcher() { + + @InternalCoroutinesApi + override fun dispatchYield(context: CoroutineContext, block: Runnable) { + this@blocking.dispatchWithContext(block, BlockingContext, true) + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + this@blocking.dispatchWithContext(block, BlockingContext, false) + } + }.limitedParallelism(parallelism) +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/SharingWorkerClassTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/SharingWorkerClassTest.kt new file mode 100644 index 0000000000..87103d76e4 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/SharingWorkerClassTest.kt @@ -0,0 +1,43 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class SharingWorkerClassTest : SchedulerTestBase() { + private val threadLocal = ThreadLocal() + + @Test + fun testSharedThread() = runTest { + val dispatcher = SchedulerCoroutineDispatcher(1, schedulerName = "first") + val dispatcher2 = SchedulerCoroutineDispatcher(1, schedulerName = "second") + + try { + withContext(dispatcher) { + assertNull(threadLocal.get()) + threadLocal.set(239) + withContext(dispatcher2) { + assertNull(threadLocal.get()) + threadLocal.set(42) + } + + assertEquals(239, threadLocal.get()) + } + } finally { + dispatcher.close() + dispatcher2.close() + } + } + + @Test(timeout = 5000L) + fun testProgress() = runTest { + // See #990 + val cores = Runtime.getRuntime().availableProcessors() + repeat(cores + 1) { + CoroutineScope(Dispatchers.Default).launch { + SchedulerCoroutineDispatcher(1).close() + }.join() + } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/TestTimeSource.kt b/kotlinx-coroutines-core/jvm/test/scheduling/TestTimeSource.kt new file mode 100644 index 0000000000..39df9a606a --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/TestTimeSource.kt @@ -0,0 +1,11 @@ +package kotlinx.coroutines.scheduling + + +internal class TestTimeSource(var time: Long) : SchedulerTimeSource() { + + override fun nanoTime() = time + + fun step(delta: Long = WORK_STEALING_TIME_RESOLUTION_NS) { + time += delta + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueStressTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueStressTest.kt new file mode 100644 index 0000000000..1bf2434f16 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueStressTest.kt @@ -0,0 +1,121 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.concurrent.* +import kotlin.jvm.internal.* +import kotlin.test.* + +class WorkQueueStressTest : TestBase() { + + private val threads = mutableListOf() + private val offerIterations = 100_000 * stressTestMultiplierSqrt // memory pressure, not CPU time + private val stealersCount = 6 + private val stolenTasks = Array(stealersCount) { GlobalQueue() } + private val globalQueue = GlobalQueue() // only producer will use it + private val producerQueue = WorkQueue() + + @Volatile + private var producerFinished = false + + @Before + fun setUp() { + schedulerTimeSource = TestTimeSource(Long.MAX_VALUE) // always steal + } + + @After + fun tearDown() { + schedulerTimeSource = NanoTimeSource + } + + @Test + fun testStealing() { + val startLatch = CountDownLatch(1) + + threads += thread(name = "producer") { + startLatch.await() + for (i in 1..offerIterations) { + while (producerQueue.size > BUFFER_CAPACITY / 2) { + Thread.yield() + } + + producerQueue.add(task(i.toLong()))?.let { globalQueue.addLast(it) } + } + + producerFinished = true + } + + for (i in 0 until stealersCount) { + threads += thread(name = "stealer $i") { + val ref = Ref.ObjectRef() + val myQueue = WorkQueue() + startLatch.await() + while (!producerFinished || producerQueue.size != 0) { + stolenTasks[i].addAll(myQueue.drain(ref).map { task(it) }) + producerQueue.trySteal(ref) + } + + // Drain last element which is not counted in buffer + stolenTasks[i].addAll(myQueue.drain(ref).map { task(it) }) + producerQueue.trySteal(ref) + stolenTasks[i].addAll(myQueue.drain(ref).map { task(it) }) + } + } + + startLatch.countDown() + threads.forEach { it.join() } + validate() + } + + @Test + fun testSingleProducerSingleStealer() { + val startLatch = CountDownLatch(1) + threads += thread(name = "producer") { + startLatch.await() + for (i in 1..offerIterations) { + while (producerQueue.size == BUFFER_CAPACITY - 1) { + Thread.yield() + } + + // No offloading to global queue here + producerQueue.add(task(i.toLong())) + } + } + + val stolen = GlobalQueue() + threads += thread(name = "stealer") { + val myQueue = WorkQueue() + val ref = Ref.ObjectRef() + startLatch.await() + while (stolen.size != offerIterations) { + if (producerQueue.trySteal(ref) != NOTHING_TO_STEAL) { + stolen.addAll(myQueue.drain(ref).map { task(it) }) + } + } + stolen.addAll(myQueue.drain(ref).map { task(it) }) + } + + startLatch.countDown() + threads.forEach { it.join() } + assertEquals((1L..offerIterations).toSet(), stolen.map { it.submissionTime }.toSet()) + } + + private fun validate() { + val result = mutableSetOf() + for (stolenTask in stolenTasks) { + assertEquals(stolenTask.size, stolenTask.map { it }.toSet().size) + result += stolenTask.map { it.submissionTime } + } + + result += globalQueue.map { it.submissionTime } + val expected = (1L..offerIterations).toSet() + assertEquals(expected, result, "Following elements are missing: ${(expected - result)}") + } + + private fun GlobalQueue.addAll(tasks: Collection) { + tasks.forEach { addLast(it) } + } +} diff --git a/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueTest.kt b/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueTest.kt new file mode 100644 index 0000000000..08ed5ca113 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/scheduling/WorkQueueTest.kt @@ -0,0 +1,108 @@ +package kotlinx.coroutines.scheduling + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import kotlin.jvm.internal.Ref.ObjectRef +import kotlin.test.* + +class WorkQueueTest : TestBase() { + + private val timeSource = TestTimeSource(0) + + @Before + fun setUp() { + schedulerTimeSource = timeSource + } + + @After + fun tearDown() { + schedulerTimeSource = NanoTimeSource + } + + @Test + fun testLastScheduledComesFirst() { + val queue = WorkQueue() + (1L..4L).forEach { queue.add(task(it)) } + assertEquals(listOf(4L, 1L, 2L, 3L), queue.drain(ObjectRef())) + } + + @Test + fun testAddWithOffload() { + val queue = WorkQueue() + val size = 130L + val offload = GlobalQueue() + (0 until size).forEach { queue.add(task(it))?.let { t -> offload.addLast(t) } } + + val expectedResult = listOf(129L) + (0L..126L).toList() + val actualResult = queue.drain(ObjectRef()) + assertEquals(expectedResult, actualResult) + assertEquals((0L until size).toSet().minus(expectedResult.toSet()), offload.drain().toSet()) + } + + @Test + fun testWorkOffloadPrecision() { + val queue = WorkQueue() + val globalQueue = GlobalQueue() + repeat(128) { assertNull(queue.add(task(it.toLong()))) } + assertTrue(globalQueue.isEmpty) + assertEquals(127L, queue.add(task(0))?.submissionTime) + } + + @Test + fun testStealingFromHead() { + val victim = WorkQueue() + victim.add(task(1L)) + victim.add(task(2L)) + timeSource.step() + timeSource.step(3) + + val stealer = WorkQueue() + val ref = ObjectRef() + assertEquals(TASK_STOLEN, victim.trySteal(ref)) + assertEquals(arrayListOf(1L), stealer.drain(ref)) + + assertEquals(TASK_STOLEN, victim.trySteal(ref)) + assertEquals(arrayListOf(2L), stealer.drain(ref)) + } + + @Test + fun testPollBlocking() { + val queue = WorkQueue() + assertNull(queue.pollBlocking()) + val blockingTask = blockingTask(1L) + queue.add(blockingTask) + queue.add(task(1L)) + assertSame(blockingTask, queue.pollBlocking()) + } +} + +internal fun task(n: Long) = Runnable {}.asTask(n, NonBlockingContext) +internal fun blockingTask(n: Long) = Runnable {}.asTask(n, BlockingContext) + +internal fun WorkQueue.drain(ref: ObjectRef): List { + var task: Task? = poll() + val result = arrayListOf() + while (task != null) { + result += task.submissionTime + task = poll() + } + if (ref.element != null) { + result += ref.element!!.submissionTime + ref.element = null + } + return result +} + +internal fun GlobalQueue.drain(): List { + var task: Task? = removeFirstOrNull() + val result = arrayListOf() + while (task != null) { + result += task.submissionTime + task = removeFirstOrNull() + } + return result +} + +internal fun WorkQueue.trySteal(stolenTaskRef: ObjectRef): Long = trySteal(STEAL_ANY, stolenTaskRef) diff --git a/kotlinx-coroutines-core/jvm/test/selects/SelectDeadlockStressTest.kt b/kotlinx-coroutines-core/jvm/test/selects/SelectDeadlockStressTest.kt new file mode 100644 index 0000000000..466aecf9f2 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/selects/SelectDeadlockStressTest.kt @@ -0,0 +1,59 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +/** + * A simple stress-test that does select sending/receiving into opposite channels to ensure that they + * don't deadlock. See https://github.com/Kotlin/kotlinx.coroutines/issues/504 + */ +class SelectDeadlockStressTest : TestBase() { + private val pool = newFixedThreadPoolContext(2, "SelectDeadlockStressTest") + private val nSeconds = 3 * stressTestMultiplier + + @After + fun tearDown() { + pool.close() + } + + @Test + fun testStress() = runTest { + val c1 = Channel() + val c2 = Channel() + val s1 = Stats() + val s2 = Stats() + launchSendReceive(c1, c2, s1) + launchSendReceive(c2, c1, s2) + for (i in 1..nSeconds) { + delay(1000) + println("$i: First: $s1; Second: $s2") + } + coroutineContext.cancelChildren() + } + + private class Stats { + var sendIndex = 0L + var receiveIndex = 0L + + override fun toString(): String = "send=$sendIndex, received=$receiveIndex" + } + + private fun CoroutineScope.launchSendReceive(c1: Channel, c2: Channel, s: Stats) = launch(pool) { + while (true) { + if (s.sendIndex % 1000 == 0L) yield() + select { + c1.onSend(s.sendIndex) { + s.sendIndex++ + } + c2.onReceive { i -> + assertEquals(s.receiveIndex, i) + s.receiveIndex++ + } + } + } + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/selects/SelectMemoryLeakStressTest.kt b/kotlinx-coroutines-core/jvm/test/selects/SelectMemoryLeakStressTest.kt new file mode 100644 index 0000000000..4b0ce46916 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/selects/SelectMemoryLeakStressTest.kt @@ -0,0 +1,56 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlin.test.* + +class SelectMemoryLeakStressTest : TestBase() { + private val nRepeat = 1_000_000 * stressTestMultiplier + + @Test + fun testLeakRegisterSend() = runTest { + expect(1) + val leak = Channel() + val data = Channel(1) + repeat(nRepeat) { value -> + data.send(value) + val bigValue = bigValue() // new instance + select { + leak.onSend("LEAK") { + println("Capture big value into this lambda: $bigValue") + expectUnreached() + } + data.onReceive { received -> + assertEquals(value, received) + expect(value + 2) + } + } + } + finish(nRepeat + 2) + } + + @Test + fun testLeakRegisterReceive() = runTest { + expect(1) + val leak = Channel() + val data = Channel(1) + repeat(nRepeat) { value -> + val bigValue = bigValue() // new instance + select { + leak.onReceive { + println("Capture big value into this lambda: $bigValue") + expectUnreached() + } + data.onSend(value) { + expect(value + 2) + } + } + assertEquals(value, data.receive()) + } + finish(nRepeat + 2) + } + + // capture big value for fast OOM in case of a bug + private fun bigValue(): ByteArray = ByteArray(4096) +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/jvm/test/selects/SelectPhilosophersStressTest.kt b/kotlinx-coroutines-core/jvm/test/selects/SelectPhilosophersStressTest.kt new file mode 100644 index 0000000000..423c1a8a55 --- /dev/null +++ b/kotlinx-coroutines-core/jvm/test/selects/SelectPhilosophersStressTest.kt @@ -0,0 +1,62 @@ +package kotlinx.coroutines.selects + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import org.junit.Test +import kotlin.test.* + +class SelectPhilosophersStressTest : TestBase() { + private val TEST_DURATION = 3000L * stressTestMultiplierSqrt + + val n = 10 // number of philosophers + private val forks = Array(n) { Mutex() } + + private suspend fun eat(id: Int, desc: String) { + val left = forks[id] + val right = forks[(id + 1) % n] + while (true) { + val pair = selectUnbiased> { + left.onLock(desc) { left to right } + right.onLock(desc) { right to left } + } + if (pair.second.tryLock(desc)) break + pair.first.unlock(desc) + pair.second.lock(desc) + if (pair.first.tryLock(desc)) break + pair.second.unlock(desc) + } + assertTrue(left.isLocked && right.isLocked) + // om, nom, nom --> eating!!! + right.unlock(desc) + left.unlock(desc) + } + + @Test + fun testPhilosophers() = runBlocking { + val timeLimit = System.currentTimeMillis() + TEST_DURATION + val philosophers = List>(n) { id -> + async { + val desc = "Philosopher $id" + var eatsCount = 0 + while (System.currentTimeMillis() < timeLimit) { + eat(id, desc) + eatsCount++ + yield() + } + println("Philosopher $id done, eats $eatsCount times") + eatsCount + } + } + val debugJob = launch { + delay(3 * TEST_DURATION) + println("Test is failing. Lock states are:") + forks.withIndex().forEach { (id, mutex) -> println("$id: $mutex") } + } + val eats = withTimeout(5 * TEST_DURATION) { philosophers.map { it.await() } } + debugJob.cancel() + eats.withIndex().forEach { (id, eats) -> + assertTrue(eats > 0, "$id shall not starve") + } + } +} diff --git a/kotlinx-coroutines-core/knit.properties b/kotlinx-coroutines-core/knit.properties new file mode 100644 index 0000000000..f27241a43c --- /dev/null +++ b/kotlinx-coroutines-core/knit.properties @@ -0,0 +1,6 @@ +knit.package=kotlinx.coroutines.examples +knit.dir=jvm/test/examples/ + +test.package=kotlinx.coroutines.examples.test +test.dir=jvm/test/examples/test/ + diff --git a/kotlinx-coroutines-core/native/src/Builders.kt b/kotlinx-coroutines-core/native/src/Builders.kt new file mode 100644 index 0000000000..4f94f19b53 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/Builders.kt @@ -0,0 +1,147 @@ +@file:OptIn(ExperimentalContracts::class, ObsoleteWorkersApi::class) +@file:Suppress("LEAKED_IN_PLACE_LAMBDA", "WRONG_INVOCATION_KIND") + +package kotlinx.coroutines + +import kotlinx.cinterop.* +import kotlin.contracts.* +import kotlin.coroutines.* +import kotlin.native.concurrent.* + +/** + * Runs a new coroutine and **blocks** the current thread _interruptibly_ until its completion. + * + * It is designed to bridge regular blocking code to libraries that are written in suspending style, to be used in + * `main` functions and in tests. + * + * Calling [runBlocking] from a suspend function is redundant. + * For example, the following code is incorrect: + * ``` + * suspend fun loadConfiguration() { + * // DO NOT DO THIS: + * val data = runBlocking { // <- redundant and blocks the thread, do not do that + * fetchConfigurationData() // suspending function + * } + * ``` + * + * Here, instead of releasing the thread on which `loadConfiguration` runs if `fetchConfigurationData` suspends, it will + * block, potentially leading to thread starvation issues. + * + * The default [CoroutineDispatcher] for this builder is an internal implementation of event loop that processes continuations + * in this blocked thread until the completion of this coroutine. + * See [CoroutineDispatcher] for the other implementations that are provided by `kotlinx.coroutines`. + * + * When [CoroutineDispatcher] is explicitly specified in the [context], then the new coroutine runs in the context of + * the specified dispatcher while the current thread is blocked. If the specified dispatcher is an event loop of another `runBlocking`, + * then this invocation uses the outer event loop. + * + * If this blocked thread is interrupted (see [Thread.interrupt]), then the coroutine job is cancelled and + * this `runBlocking` invocation throws [InterruptedException]. + * + * See [newCoroutineContext][CoroutineScope.newCoroutineContext] for a description of debugging facilities that are available + * for a newly created coroutine. + * + * @param context the context of the coroutine. The default value is an event loop on the current thread. + * @param block the coroutine code. + */ +public actual fun runBlocking(context: CoroutineContext, block: suspend CoroutineScope.() -> T): T { + contract { + callsInPlace(block, InvocationKind.EXACTLY_ONCE) + } + val contextInterceptor = context[ContinuationInterceptor] + val eventLoop: EventLoop? + val newContext: CoroutineContext + if (contextInterceptor == null) { + // create or use private event loop if no dispatcher is specified + eventLoop = ThreadLocalEventLoop.eventLoop + newContext = GlobalScope.newCoroutineContext(context + eventLoop) + } else { + // See if context's interceptor is an event loop that we shall use (to support TestContext) + // or take an existing thread-local event loop if present to avoid blocking it (but don't create one) + eventLoop = (contextInterceptor as? EventLoop)?.takeIf { it.shouldBeProcessedFromContext() } + ?: ThreadLocalEventLoop.currentOrNull() + newContext = GlobalScope.newCoroutineContext(context) + } + val coroutine = BlockingCoroutine(newContext, eventLoop) + var completed = false + ThreadLocalKeepAlive.addCheck { !completed } + try { + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) + return coroutine.joinBlocking() + } finally { + completed = true + } +} + +@ThreadLocal +private object ThreadLocalKeepAlive { + /** If any of these checks passes, this means this [Worker] is still used. */ + private var checks = mutableListOf<() -> Boolean>() + + /** Whether the worker currently tries to keep itself alive. */ + private var keepAliveLoopActive = false + + /** Adds another stopgap that must be passed before the [Worker] can be terminated. */ + fun addCheck(terminationForbidden: () -> Boolean) { + checks.add(terminationForbidden) + if (!keepAliveLoopActive) keepAlive() + } + + /** + * Send a ping to the worker to prevent it from terminating while this coroutine is running, + * ensuring that continuations don't get dropped and forgotten. + */ + private fun keepAlive() { + // only keep the checks that still forbid the termination + checks = checks.filter { it() }.toMutableList() + // if there are no checks left, we no longer keep the worker alive, it can be terminated + keepAliveLoopActive = checks.isNotEmpty() + if (keepAliveLoopActive) { + Worker.current.executeAfter(afterMicroseconds = 100_000) { + keepAlive() + } + } + } +} + +private class BlockingCoroutine( + parentContext: CoroutineContext, + private val eventLoop: EventLoop? +) : AbstractCoroutine(parentContext, true, true) { + private val joinWorker = Worker.current + + override val isScopedCoroutine: Boolean get() = true + + override fun afterCompletion(state: Any?) { + // wake up blocked thread + if (joinWorker != Worker.current) { + // Unpark waiting worker + joinWorker.executeAfter(0L, {}) // send an empty task to unpark the waiting event loop + } + } + + @Suppress("UNCHECKED_CAST") + fun joinBlocking(): T { + try { + eventLoop?.incrementUseCount() + while (true) { + var parkNanos: Long + // Workaround for bug in BE optimizer that cannot eliminate boxing here + if (eventLoop != null) { + parkNanos = eventLoop.processNextEvent() + } else { + parkNanos = Long.MAX_VALUE + } + // note: processNextEvent may lose unpark flag, so check if completed before parking + if (isCompleted) break + joinWorker.park(parkNanos / 1000L, true) + } + } finally { // paranoia + eventLoop?.decrementUseCount() + } + // now return result + val state = state.unboxState() + (state as? CompletedExceptionally)?.let { throw it.cause } + return state as T + } +} diff --git a/kotlinx-coroutines-core/native/src/CloseableCoroutineDispatcher.kt b/kotlinx-coroutines-core/native/src/CloseableCoroutineDispatcher.kt new file mode 100644 index 0000000000..3ea73ad7a1 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/CloseableCoroutineDispatcher.kt @@ -0,0 +1,5 @@ +package kotlinx.coroutines + +public actual abstract class CloseableCoroutineDispatcher actual constructor() : CoroutineDispatcher(), AutoCloseable { + public actual abstract override fun close() +} diff --git a/kotlinx-coroutines-core/native/src/CoroutineContext.kt b/kotlinx-coroutines-core/native/src/CoroutineContext.kt new file mode 100644 index 0000000000..3f4c8d9a01 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/CoroutineContext.kt @@ -0,0 +1,53 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +internal actual object DefaultExecutor : CoroutineDispatcher(), Delay { + + private val delegate = WorkerDispatcher(name = "DefaultExecutor") + + override fun dispatch(context: CoroutineContext, block: Runnable) { + delegate.dispatch(context, block) + } + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + delegate.scheduleResumeAfterDelay(timeMillis, continuation) + } + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { + return delegate.invokeOnTimeout(timeMillis, block, context) + } + + actual fun enqueue(task: Runnable): Unit { + delegate.dispatch(EmptyCoroutineContext, task) + } +} + +internal expect fun createDefaultDispatcher(): CoroutineDispatcher + +@PublishedApi +internal actual val DefaultDelay: Delay = DefaultExecutor + +public actual fun CoroutineScope.newCoroutineContext(context: CoroutineContext): CoroutineContext { + val combined = coroutineContext + context + return if (combined !== Dispatchers.Default && combined[ContinuationInterceptor] == null) + combined + Dispatchers.Default else combined +} + +public actual fun CoroutineContext.newCoroutineContext(addedContext: CoroutineContext): CoroutineContext { + return this + addedContext +} + +// No debugging facilities on native +internal actual inline fun withCoroutineContext(context: CoroutineContext, countOrElement: Any?, block: () -> T): T = block() +internal actual inline fun withContinuationContext(continuation: Continuation<*>, countOrElement: Any?, block: () -> T): T = block() +internal actual fun Continuation<*>.toDebugString(): String = toString() +internal actual val CoroutineContext.coroutineName: String? get() = null // not supported on native + +internal actual class UndispatchedCoroutine actual constructor( + context: CoroutineContext, + uCont: Continuation +) : ScopeCoroutine(context, uCont) { + override fun afterResume(state: Any?) = uCont.resumeWith(recoverResult(state, uCont)) +} diff --git a/kotlinx-coroutines-core/native/src/Debug.kt b/kotlinx-coroutines-core/native/src/Debug.kt new file mode 100644 index 0000000000..3cf24889d9 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/Debug.kt @@ -0,0 +1,12 @@ +package kotlinx.coroutines + +import kotlin.math.* +import kotlin.native.* + +internal actual val DEBUG: Boolean = false + +internal actual val Any.hexAddress: String get() = identityHashCode().toUInt().toString(16) + +internal actual val Any.classSimpleName: String get() = this::class.simpleName ?: "Unknown" + +internal actual inline fun assert(value: () -> Boolean) {} diff --git a/kotlinx-coroutines-core/native/src/Dispatchers.kt b/kotlinx-coroutines-core/native/src/Dispatchers.kt new file mode 100644 index 0000000000..e66c05f61d --- /dev/null +++ b/kotlinx-coroutines-core/native/src/Dispatchers.kt @@ -0,0 +1,51 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + + +public actual object Dispatchers { + public actual val Default: CoroutineDispatcher = createDefaultDispatcher() + public actual val Main: MainCoroutineDispatcher + get() = injectedMainDispatcher ?: mainDispatcher + public actual val Unconfined: CoroutineDispatcher get() = kotlinx.coroutines.Unconfined // Avoid freezing + + private val mainDispatcher = createMainDispatcher(Default) + + private var injectedMainDispatcher: MainCoroutineDispatcher? = null + + @PublishedApi + internal fun injectMain(dispatcher: MainCoroutineDispatcher) { + injectedMainDispatcher = dispatcher + } + + internal val IO: CoroutineDispatcher = DefaultIoScheduler +} + +internal object DefaultIoScheduler : CoroutineDispatcher() { + // 2048 is an arbitrary KMP-friendly constant + private val unlimitedPool = newFixedThreadPoolContext(2048, "Dispatchers.IO") + private val io = unlimitedPool.limitedParallelism(64) // Default JVM size + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + // See documentation to Dispatchers.IO for the rationale + return unlimitedPool.limitedParallelism(parallelism, name) + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + io.dispatch(context, block) + } + + @InternalCoroutinesApi + override fun dispatchYield(context: CoroutineContext, block: Runnable) { + io.dispatchYield(context, block) + } + + override fun toString(): String = "Dispatchers.IO" +} + + +@Suppress("EXTENSION_SHADOWED_BY_MEMBER") +public actual val Dispatchers.IO: CoroutineDispatcher get() = IO + +internal expect fun createMainDispatcher(default: CoroutineDispatcher): MainCoroutineDispatcher diff --git a/kotlinx-coroutines-core/native/src/EventLoop.kt b/kotlinx-coroutines-core/native/src/EventLoop.kt new file mode 100644 index 0000000000..58128d52fd --- /dev/null +++ b/kotlinx-coroutines-core/native/src/EventLoop.kt @@ -0,0 +1,32 @@ +@file:OptIn(ObsoleteWorkersApi::class) + +package kotlinx.coroutines + +import kotlin.coroutines.* +import kotlin.native.concurrent.* +import kotlin.time.* + +internal actual abstract class EventLoopImplPlatform : EventLoop() { + + private val current = Worker.current + + protected actual fun unpark() { + current.executeAfter(0L, {})// send an empty task to unpark the waiting event loop + } + + protected actual fun reschedule(now: Long, delayedTask: EventLoopImplBase.DelayedTask) { + val delayTimeMillis = delayNanosToMillis(delayedTask.nanoTime - now) + DefaultExecutor.invokeOnTimeout(delayTimeMillis, delayedTask, EmptyCoroutineContext) + } +} + +internal class EventLoopImpl: EventLoopImplBase() { + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + DefaultDelay.invokeOnTimeout(timeMillis, block, context) +} + +internal actual fun createEventLoop(): EventLoop = EventLoopImpl() + +private val startingPoint = TimeSource.Monotonic.markNow() + +internal actual fun nanoTime(): Long = (TimeSource.Monotonic.markNow() - startingPoint).inWholeNanoseconds diff --git a/kotlinx-coroutines-core/native/src/Exceptions.kt b/kotlinx-coroutines-core/native/src/Exceptions.kt new file mode 100644 index 0000000000..c6173afc26 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/Exceptions.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines + +/** + * Thrown by cancellable suspending functions if the [Job] of the coroutine is cancelled while it is suspending. + * It indicates _normal_ cancellation of a coroutine. + * **It is not printed to console/log by default uncaught exception handler**. + * (see [CoroutineExceptionHandler]). + */ +public actual typealias CancellationException = kotlin.coroutines.cancellation.CancellationException + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +@kotlin.internal.LowPriorityInOverloadResolution +public actual fun CancellationException(message: String?, cause: Throwable?): CancellationException = + CancellationException(message, cause) + +/** + * Thrown by cancellable suspending functions if the [Job] of the coroutine is cancelled or completed + * without cause, or with a cause or exception that is not [CancellationException] + * (see [Job.getCancellationException]). + */ +internal actual class JobCancellationException public actual constructor( + message: String, + cause: Throwable?, + internal actual val job: Job +) : CancellationException(message, cause) { + override fun toString(): String = "${super.toString()}; job=$job" + override fun equals(other: Any?): Boolean = + other === this || + other is JobCancellationException && other.message == message && other.job == job && other.cause == cause + override fun hashCode(): Int = + (message!!.hashCode() * 31 + job.hashCode()) * 31 + (cause?.hashCode() ?: 0) +} + +// For use in tests +internal actual val RECOVER_STACK_TRACES: Boolean = false diff --git a/kotlinx-coroutines-core/native/src/MultithreadedDispatchers.kt b/kotlinx-coroutines-core/native/src/MultithreadedDispatchers.kt new file mode 100644 index 0000000000..8b217285a5 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/MultithreadedDispatchers.kt @@ -0,0 +1,181 @@ +@file:OptIn(ObsoleteWorkersApi::class) + +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.concurrent.AtomicReference +import kotlin.native.concurrent.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds + +@DelicateCoroutinesApi +public actual fun newFixedThreadPoolContext(nThreads: Int, name: String): CloseableCoroutineDispatcher { + require(nThreads >= 1) { "Expected at least one thread, but got: $nThreads" } + return MultiWorkerDispatcher(name, nThreads) +} + +internal class WorkerDispatcher(name: String) : CloseableCoroutineDispatcher(), Delay { + private val worker = Worker.start(name = name) + + override fun dispatch(context: CoroutineContext, block: Runnable) { + worker.executeAfter(0L) { block.run() } + } + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val handle = schedule(timeMillis, Runnable { + with(continuation) { resumeUndispatched(Unit) } + }) + continuation.disposeOnCancellation(handle) + } + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + schedule(timeMillis, block) + + private fun schedule(timeMillis: Long, block: Runnable): DisposableHandle { + // Workers don't have an API to cancel sent "executeAfter" block, but we are trying + // to control the damage and reduce reachable objects by nulling out `block` + // that may retain a lot of references, and leaving only an empty shell after a timely disposal + // This is a class and not an object with `block` in a closure because that would defeat the purpose. + class DisposableBlock(block: Runnable) : DisposableHandle, Function0 { + private val disposableHolder = AtomicReference(block) + + override fun invoke() { + disposableHolder.value?.run() + } + + override fun dispose() { + disposableHolder.value = null + } + + fun isDisposed() = disposableHolder.value == null + } + + fun Worker.runAfterDelay(block: DisposableBlock, targetMoment: TimeMark) { + if (block.isDisposed()) return + val durationUntilTarget = -targetMoment.elapsedNow() + val quantum = 100.milliseconds + if (durationUntilTarget > quantum) { + executeAfter(quantum.inWholeMicroseconds) { runAfterDelay(block, targetMoment) } + } else { + executeAfter(maxOf(0, durationUntilTarget.inWholeMicroseconds), block) + } + } + + val disposableBlock = DisposableBlock(block) + val targetMoment = TimeSource.Monotonic.markNow() + timeMillis.milliseconds + worker.runAfterDelay(disposableBlock, targetMoment) + return disposableBlock + } + + override fun close() { + worker.requestTermination().result // Note: calling "result" blocks + } +} + +private class MultiWorkerDispatcher( + private val name: String, + private val workersCount: Int +) : CloseableCoroutineDispatcher() { + private val tasksQueue = Channel(Channel.UNLIMITED) + private val availableWorkers = Channel>(Channel.UNLIMITED) + private val workerPool = OnDemandAllocatingPool(workersCount) { + Worker.start(name = "$name-$it").apply { + executeAfter { workerRunLoop() } + } + } + + /** + * (number of tasks - number of workers) * 2 + (1 if closed) + */ + private val tasksAndWorkersCounter = atomic(0L) + + @Suppress("NOTHING_TO_INLINE") + private inline fun Long.isClosed() = this and 1L == 1L + @Suppress("NOTHING_TO_INLINE") + private inline fun Long.hasTasks() = this >= 2 + @Suppress("NOTHING_TO_INLINE") + private inline fun Long.hasWorkers() = this < 0 + + private fun workerRunLoop() = runBlocking { + while (true) { + val state = tasksAndWorkersCounter.getAndUpdate { + if (it.isClosed() && !it.hasTasks()) return@runBlocking + it - 2 + } + if (state.hasTasks()) { + // we promised to process a task, and there are some + tasksQueue.receive().run() + } else { + try { + suspendCancellableCoroutine { + val result = availableWorkers.trySend(it) + checkChannelResult(result) + }.run() + } catch (e: CancellationException) { + /** we are cancelled from [close] and thus will never get back to this branch of code, + but there may still be pending work, so we can't just exit here. */ + } + } + } + } + + // a worker that promised to be here and should actually arrive, so we wait for it in a blocking manner. + private fun obtainWorker(): CancellableContinuation = + availableWorkers.tryReceive().getOrNull() ?: runBlocking { availableWorkers.receive() } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + val state = tasksAndWorkersCounter.getAndUpdate { + if (it.isClosed()) + throw IllegalStateException("Dispatcher $name was closed, attempted to schedule: $block") + it + 2 + } + if (state.hasWorkers()) { + // there are workers that have nothing to do, let's grab one of them + obtainWorker().resume(block) + } else { + workerPool.allocate() + // no workers are available, we must queue the task + val result = tasksQueue.trySend(block) + checkChannelResult(result) + } + } + + override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher { + parallelism.checkParallelism() + if (parallelism >= workersCount) { + return namedOrThis(name) + } + return super.limitedParallelism(parallelism, name) + } + + override fun close() { + tasksAndWorkersCounter.getAndUpdate { if (it.isClosed()) it else it or 1L } + val workers = workerPool.close() // no new workers will be created + while (true) { + // check if there are workers that await tasks in their personal channels, we need to wake them up + val state = tasksAndWorkersCounter.getAndUpdate { + if (it.hasWorkers()) it + 2 else it + } + if (!state.hasWorkers()) + break + obtainWorker().cancel() + } + /* + * Here we cannot avoid waiting on `.result`, otherwise it will lead + * to a native memory leak, including a pthread handle. + */ + val requests = workers.map { it.requestTermination() } + requests.map { it.result } + } + + private fun checkChannelResult(result: ChannelResult<*>) { + if (!result.isSuccess) + throw IllegalStateException( + "Internal invariants of $this were violated, please file a bug to kotlinx.coroutines", + result.exceptionOrNull() + ) + } +} diff --git a/kotlinx-coroutines-core/native/src/Runnable.kt b/kotlinx-coroutines-core/native/src/Runnable.kt new file mode 100644 index 0000000000..d93e3f2073 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/Runnable.kt @@ -0,0 +1,24 @@ +package kotlinx.coroutines + +/** + * A runnable task for [CoroutineDispatcher.dispatch]. + * + * Equivalent to the type `() -> Unit`. + */ +public actual fun interface Runnable { + /** + * @suppress + */ + public actual fun run() +} + +@Deprecated( + "Preserved for binary compatibility, see https://github.com/Kotlin/kotlinx.coroutines/issues/4309", + level = DeprecationLevel.HIDDEN +) +public inline fun Runnable(crossinline block: () -> Unit): Runnable = + object : Runnable { + override fun run() { + block() + } + } diff --git a/kotlinx-coroutines-core/native/src/SchedulerTask.kt b/kotlinx-coroutines-core/native/src/SchedulerTask.kt new file mode 100644 index 0000000000..24b2311268 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/SchedulerTask.kt @@ -0,0 +1,3 @@ +package kotlinx.coroutines + +internal actual abstract class SchedulerTask : Runnable diff --git a/kotlinx-coroutines-core/native/src/flow/internal/FlowExceptions.kt b/kotlinx-coroutines-core/native/src/flow/internal/FlowExceptions.kt new file mode 100644 index 0000000000..1109b15fe4 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/flow/internal/FlowExceptions.kt @@ -0,0 +1,10 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* + +internal actual class AbortFlowException actual constructor( + actual val owner: Any +) : CancellationException("Flow was aborted, no more elements needed") +internal actual class ChildCancelledException : CancellationException("Child of the scoped flow was cancelled") + diff --git a/kotlinx-coroutines-core/native/src/flow/internal/SafeCollector.kt b/kotlinx-coroutines-core/native/src/flow/internal/SafeCollector.kt new file mode 100644 index 0000000000..ded11e22f5 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/flow/internal/SafeCollector.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines.flow.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.coroutines.* + +internal actual class SafeCollector actual constructor( + internal actual val collector: FlowCollector, + internal actual val collectContext: CoroutineContext +) : FlowCollector { + + // Note, it is non-capturing lambda, so no extra allocation during init of SafeCollector + internal actual val collectContextSize = collectContext.fold(0) { count, _ -> count + 1 } + private var lastEmissionContext: CoroutineContext? = null + + actual override suspend fun emit(value: T) { + val currentContext = currentCoroutineContext() + currentContext.ensureActive() + if (lastEmissionContext !== currentContext) { + checkContext(currentContext) + lastEmissionContext = currentContext + } + collector.emit(value) + } + + public actual fun releaseIntercepted() { + } +} diff --git a/kotlinx-coroutines-core/native/src/internal/Concurrent.kt b/kotlinx-coroutines-core/native/src/internal/Concurrent.kt new file mode 100644 index 0000000000..4999361bf5 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/Concurrent.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* +import kotlinx.cinterop.* +import kotlinx.atomicfu.locks.withLock as withLock2 + +internal actual typealias ReentrantLock = kotlinx.atomicfu.locks.SynchronizedObject + +internal actual inline fun ReentrantLock.withLock(action: () -> T): T = this.withLock2(action) + +internal actual fun identitySet(expectedSize: Int): MutableSet = HashSet() + +internal actual typealias BenignDataRace = kotlin.concurrent.Volatile + +internal actual class WorkaroundAtomicReference actual constructor(value: V) { + + private val nativeAtomic = kotlin.concurrent.AtomicReference(value) + + public actual fun get(): V= nativeAtomic.value + + public actual fun set(value: V) { + nativeAtomic.value = value + } + + public actual fun getAndSet(value: V): V = nativeAtomic.getAndSet(value) + + public actual fun compareAndSet(expected: V, value: V): Boolean = nativeAtomic.compareAndSet(expected, value) +} diff --git a/kotlinx-coroutines-core/native/src/internal/CopyOnWriteList.kt b/kotlinx-coroutines-core/native/src/internal/CopyOnWriteList.kt new file mode 100644 index 0000000000..32e1c4ac94 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/CopyOnWriteList.kt @@ -0,0 +1,75 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* + +@Suppress("UNCHECKED_CAST") +internal class CopyOnWriteList : AbstractMutableList() { + + private val _array = atomic(arrayOfNulls(0)) + private var array: Array + get() = _array.value + set(value) { _array.value = value } + + override val size: Int + get() = array.size + + override fun add(element: E): Boolean { + val n = size + val update = array.copyOf(n + 1) + update[n] = element + array = update + return true + } + + override fun add(index: Int, element: E) { + rangeCheck(index) + val n = size + val update = arrayOfNulls(n + 1) + array.copyInto(destination = update, endIndex = index) + update[index] = element + array.copyInto(destination = update, destinationOffset = index + 1, startIndex = index, endIndex = n + 1) + array = update + } + + override fun remove(element: E): Boolean { + val index = array.indexOf(element as Any) + if (index == -1) return false + removeAt(index) + return true + } + + override fun removeAt(index: Int): E { + rangeCheck(index) + val n = size + val element = array[index] + val update = arrayOfNulls(n - 1) + array.copyInto(destination = update, endIndex = index) + array.copyInto(destination = update, destinationOffset = index, startIndex = index + 1, endIndex = n) + array = update + return element as E + } + + override fun iterator(): MutableIterator = IteratorImpl(array as Array) + override fun listIterator(): MutableListIterator = throw UnsupportedOperationException("Operation is not supported") + override fun listIterator(index: Int): MutableListIterator = throw UnsupportedOperationException("Operation is not supported") + override fun isEmpty(): Boolean = size == 0 + override fun set(index: Int, element: E): E = throw UnsupportedOperationException("Operation is not supported") + override fun get(index: Int): E = array[rangeCheck(index)] as E + + private class IteratorImpl(private val array: Array) : MutableIterator { + private var current = 0 + + override fun hasNext(): Boolean = current != array.size + + override fun next(): E { + if (!hasNext()) throw NoSuchElementException() + return array[current++] + } + + override fun remove() = throw UnsupportedOperationException("Operation is not supported") + } + + private fun rangeCheck(index: Int) = index.apply { + if (index < 0 || index >= size) throw IndexOutOfBoundsException("index: $index, size: $size") + } +} diff --git a/kotlinx-coroutines-core/native/src/internal/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/native/src/internal/CoroutineExceptionHandlerImpl.kt new file mode 100644 index 0000000000..621b8a9c08 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/CoroutineExceptionHandlerImpl.kt @@ -0,0 +1,27 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.native.* + +private val lock = SynchronizedObject() + +internal actual val platformExceptionHandlers: Collection + get() = synchronized(lock) { platformExceptionHandlers_ } + +private val platformExceptionHandlers_ = mutableSetOf() + +internal actual fun ensurePlatformExceptionHandlerLoaded(callback: CoroutineExceptionHandler) { + synchronized(lock) { + platformExceptionHandlers_ += callback + } +} + +@OptIn(ExperimentalStdlibApi::class) +internal actual fun propagateExceptionFinalResort(exception: Throwable) { + // log exception + processUnhandledException(exception) +} + +internal actual class DiagnosticCoroutineContextException actual constructor(context: CoroutineContext) : + RuntimeException(context.toString()) diff --git a/kotlinx-coroutines-core/native/src/internal/LocalAtomics.kt b/kotlinx-coroutines-core/native/src/internal/LocalAtomics.kt new file mode 100644 index 0000000000..cbbeec1eed --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/LocalAtomics.kt @@ -0,0 +1,16 @@ +package kotlinx.coroutines.internal + +import kotlinx.atomicfu.* + +internal actual class LocalAtomicInt actual constructor(value: Int) { + + private val iRef = atomic(value) + + actual fun set(value: Int) { + iRef.value = value + } + + actual fun get(): Int = iRef.value + + actual fun decrementAndGet(): Int = iRef.decrementAndGet() +} diff --git a/kotlinx-coroutines-core/native/src/internal/ProbesSupport.kt b/kotlinx-coroutines-core/native/src/internal/ProbesSupport.kt new file mode 100644 index 0000000000..7afce8f8a4 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/ProbesSupport.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +@Suppress("NOTHING_TO_INLINE") +internal actual inline fun probeCoroutineCreated(completion: Continuation): Continuation = completion + +@Suppress("NOTHING_TO_INLINE") +internal actual inline fun probeCoroutineResumed(completion: Continuation) { } diff --git a/kotlinx-coroutines-core/native/src/internal/StackTraceRecovery.kt b/kotlinx-coroutines-core/native/src/internal/StackTraceRecovery.kt new file mode 100644 index 0000000000..20732e017f --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/StackTraceRecovery.kt @@ -0,0 +1,21 @@ +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +internal actual fun recoverStackTrace(exception: E, continuation: Continuation<*>): E = exception +internal actual fun recoverStackTrace(exception: E): E = exception + +@PublishedApi +internal actual fun unwrap(exception: E): E = exception +internal actual suspend inline fun recoverAndThrow(exception: Throwable): Nothing = throw exception + +@Suppress("UNUSED") +internal actual interface CoroutineStackFrame { + public actual val callerFrame: CoroutineStackFrame? + public actual fun getStackTraceElement(): StackTraceElement? +} + +internal actual typealias StackTraceElement = Any + +internal actual fun Throwable.initCause(cause: Throwable) { +} diff --git a/kotlinx-coroutines-core/native/src/internal/Synchronized.kt b/kotlinx-coroutines-core/native/src/internal/Synchronized.kt new file mode 100644 index 0000000000..43ff8bd9fc --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/Synchronized.kt @@ -0,0 +1,17 @@ +package kotlinx.coroutines.internal + +import kotlinx.cinterop.* +import kotlinx.coroutines.* +import kotlinx.atomicfu.locks.withLock as withLock2 + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public actual typealias SynchronizedObject = kotlinx.atomicfu.locks.SynchronizedObject + +/** + * @suppress **This an internal API and should not be used from general code.** + */ +@InternalCoroutinesApi +public actual inline fun synchronizedImpl(lock: SynchronizedObject, block: () -> T): T = lock.withLock2(block) diff --git a/kotlinx-coroutines-core/native/src/internal/SystemProps.kt b/kotlinx-coroutines-core/native/src/internal/SystemProps.kt new file mode 100644 index 0000000000..accc247d37 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/SystemProps.kt @@ -0,0 +1,3 @@ +package kotlinx.coroutines.internal + +internal actual fun systemProp(propertyName: String): String? = null diff --git a/kotlinx-coroutines-core/native/src/internal/ThreadContext.kt b/kotlinx-coroutines-core/native/src/internal/ThreadContext.kt new file mode 100644 index 0000000000..3f56f99d6c --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/ThreadContext.kt @@ -0,0 +1,5 @@ +package kotlinx.coroutines.internal + +import kotlin.coroutines.* + +internal actual fun threadContextElements(context: CoroutineContext): Any = 0 diff --git a/kotlinx-coroutines-core/native/src/internal/ThreadLocal.kt b/kotlinx-coroutines-core/native/src/internal/ThreadLocal.kt new file mode 100644 index 0000000000..0c803a7e36 --- /dev/null +++ b/kotlinx-coroutines-core/native/src/internal/ThreadLocal.kt @@ -0,0 +1,14 @@ +package kotlinx.coroutines.internal + +import kotlin.native.concurrent.ThreadLocal + +internal actual class CommonThreadLocal(private val name: Symbol) { + @Suppress("UNCHECKED_CAST") + actual fun get(): T = Storage[name] as T + actual fun set(value: T) { Storage[name] = value } +} + +internal actual fun commonThreadLocal(name: Symbol): CommonThreadLocal = CommonThreadLocal(name) + +@ThreadLocal +private object Storage: MutableMap by mutableMapOf() diff --git a/kotlinx-coroutines-core/native/test/ConcurrentTestUtilities.kt b/kotlinx-coroutines-core/native/test/ConcurrentTestUtilities.kt new file mode 100644 index 0000000000..ff1638fd9a --- /dev/null +++ b/kotlinx-coroutines-core/native/test/ConcurrentTestUtilities.kt @@ -0,0 +1,8 @@ +package kotlinx.coroutines.exceptions + +import platform.posix.* +import kotlin.native.concurrent.* + +actual inline fun yieldThread() { sched_yield() } + +actual fun currentThreadName(): String = Worker.current.name diff --git a/kotlinx-coroutines-core/native/test/DelayExceptionTest.kt b/kotlinx-coroutines-core/native/test/DelayExceptionTest.kt new file mode 100644 index 0000000000..8ffa0e2ed1 --- /dev/null +++ b/kotlinx-coroutines-core/native/test/DelayExceptionTest.kt @@ -0,0 +1,20 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +class DelayExceptionTest : TestBase() { + + @Test + fun testMaxDelay() = runBlocking { + expect(1) + val job = launch { + expect(2) + delay(Long.MAX_VALUE) + } + yield() + job.cancel() + finish(3) + } +} diff --git a/kotlinx-coroutines-core/native/test/MultithreadedDispatchersTest.kt b/kotlinx-coroutines-core/native/test/MultithreadedDispatchersTest.kt new file mode 100644 index 0000000000..beaace279c --- /dev/null +++ b/kotlinx-coroutines-core/native/test/MultithreadedDispatchersTest.kt @@ -0,0 +1,79 @@ +package kotlinx.coroutines + +import kotlinx.atomicfu.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.internal.* +import kotlin.native.concurrent.* +import kotlin.test.* +import kotlin.time.Duration.Companion.seconds + +private class BlockingBarrier(val n: Int) { + val counter = atomic(0) + val wakeUp = Channel(n - 1) + fun await() { + val count = counter.addAndGet(1) + if (count == n) { + repeat(n - 1) { + runBlocking { + wakeUp.send(Unit) + } + } + } else if (count < n) { + runBlocking { + wakeUp.receive() + } + } + } +} + +class MultithreadedDispatchersTest { + /** + * Test that [newFixedThreadPoolContext] does not allocate more dispatchers than it needs to. + * Incidentally also tests that it will allocate enough workers for its needs. Otherwise, the test will hang. + */ + @Test + fun testNotAllocatingExtraDispatchers() { + val barrier = BlockingBarrier(2) + val lock = SynchronizedObject() + suspend fun spin(set: MutableSet) { + repeat(100) { + synchronized(lock) { set.add(Worker.current) } + delay(1) + } + } + val dispatcher = newFixedThreadPoolContext(64, "test") + try { + runBlocking { + val encounteredWorkers = mutableSetOf() + val coroutine1 = launch(dispatcher) { + barrier.await() + spin(encounteredWorkers) + } + val coroutine2 = launch(dispatcher) { + barrier.await() + spin(encounteredWorkers) + } + listOf(coroutine1, coroutine2).joinAll() + assertEquals(2, encounteredWorkers.size) + } + } finally { + dispatcher.close() + } + } + + /** + * Test that [newSingleThreadContext] will not wait for the cancelled scheduled coroutines before closing. + */ + @Test + fun timeoutsNotPreventingClosing(): Unit = runBlocking { + val dispatcher = WorkerDispatcher("test") + withContext(dispatcher) { + withTimeout(5.seconds) { + } + } + withTimeout(1.seconds) { + dispatcher.close() // should not wait for the timeout + yield() + } + } +} diff --git a/kotlinx-coroutines-core/native/test/WorkerTest.kt b/kotlinx-coroutines-core/native/test/WorkerTest.kt new file mode 100644 index 0000000000..1eb9d92a7d --- /dev/null +++ b/kotlinx-coroutines-core/native/test/WorkerTest.kt @@ -0,0 +1,62 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.channels.* +import kotlin.coroutines.* +import kotlin.native.concurrent.* +import kotlin.test.* + +class WorkerTest : TestBase() { + + @Test + fun testLaunchInWorker() { + val worker = Worker.start() + worker.execute(TransferMode.SAFE, { }) { + runBlocking { + launch { }.join() + delay(1) + } + }.result + worker.requestTermination() + } + + @Test + fun testLaunchInWorkerThroughGlobalScope() { + val worker = Worker.start() + worker.execute(TransferMode.SAFE, { }) { + runBlocking { + CoroutineScope(EmptyCoroutineContext).launch { + delay(10) + }.join() + } + }.result + worker.requestTermination() + } + + /** + * Test that [runBlocking] does not crash after [Worker.requestTermination] is called on the worker that runs it. + */ + @Test + fun testRunBlockingInTerminatedWorker() { + val workerInRunBlocking = Channel() + val workerTerminated = Channel() + val checkResumption = Channel() + val finished = Channel() + val worker = Worker.start() + worker.executeAfter(0) { + runBlocking { + workerInRunBlocking.send(Unit) + workerTerminated.receive() + checkResumption.receive() + finished.send(Unit) + } + } + runBlocking { + workerInRunBlocking.receive() + worker.requestTermination() + workerTerminated.send(Unit) + checkResumption.send(Unit) + finished.receive() + } + } +} diff --git a/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt b/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt new file mode 100644 index 0000000000..786f0f215d --- /dev/null +++ b/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt @@ -0,0 +1,104 @@ +@file:OptIn(BetaInteropApi::class) + +package kotlinx.coroutines + +import kotlinx.cinterop.* +import platform.CoreFoundation.* +import platform.darwin.* +import kotlin.coroutines.* +import kotlin.concurrent.* +import kotlin.native.internal.NativePtr + +internal fun isMainThread(): Boolean = CFRunLoopGetCurrent() == CFRunLoopGetMain() + +internal actual fun createMainDispatcher(default: CoroutineDispatcher): MainCoroutineDispatcher = DarwinMainDispatcher(false) + +internal actual fun createDefaultDispatcher(): CoroutineDispatcher = DarwinGlobalQueueDispatcher + +private object DarwinGlobalQueueDispatcher : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + autoreleasepool { + dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT.convert(), 0u)) { + block.run() + } + } + } +} + +private class DarwinMainDispatcher( + private val invokeImmediately: Boolean +) : MainCoroutineDispatcher(), Delay { + + override val immediate: MainCoroutineDispatcher = + if (invokeImmediately) this else DarwinMainDispatcher(true) + + override fun isDispatchNeeded(context: CoroutineContext): Boolean = !(invokeImmediately && isMainThread()) + + override fun dispatch(context: CoroutineContext, block: Runnable) { + autoreleasepool { + dispatch_async(dispatch_get_main_queue()) { + block.run() + } + } + } + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val timer = Timer() + val timerBlock: TimerBlock = { + timer.dispose() + continuation.resume(Unit) + } + timer.start(timeMillis, timerBlock) + continuation.disposeOnCancellation(timer) + } + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { + val timer = Timer() + val timerBlock: TimerBlock = { + timer.dispose() + block.run() + } + timer.start(timeMillis, timerBlock) + return timer + } + + override fun toString(): String = + if (invokeImmediately) "Dispatchers.Main.immediate" else "Dispatchers.Main" +} + +private typealias TimerBlock = (CFRunLoopTimerRef?) -> Unit + +private val TIMER_NEW = NativePtr.NULL +private val TIMER_DISPOSED = NativePtr.NULL.plus(1) + +private class Timer : DisposableHandle { + private val ref = AtomicNativePtr(TIMER_NEW) + + fun start(timeMillis: Long, timerBlock: TimerBlock) { + val fireDate = CFAbsoluteTimeGetCurrent() + timeMillis / 1000.0 + val timer = CFRunLoopTimerCreateWithHandler(null, fireDate, 0.0, 0u, 0, timerBlock) + CFRunLoopAddTimer(CFRunLoopGetMain(), timer, kCFRunLoopCommonModes) + if (!ref.compareAndSet(TIMER_NEW, timer.rawValue)) { + // dispose was already called concurrently + release(timer) + } + } + + override fun dispose() { + while (true) { + val ptr = ref.value + if (ptr == TIMER_DISPOSED) return + if (ref.compareAndSet(ptr, TIMER_DISPOSED)) { + if (ptr != TIMER_NEW) release(interpretCPointer(ptr)) + return + } + } + } + + private fun release(timer: CFRunLoopTimerRef?) { + CFRunLoopRemoveTimer(CFRunLoopGetMain(), timer, kCFRunLoopCommonModes) + CFRelease(timer) + } +} + +internal actual inline fun platformAutoreleasePool(crossinline block: () -> Unit): Unit = autoreleasepool { block() } diff --git a/kotlinx-coroutines-core/nativeDarwin/test/Launcher.kt b/kotlinx-coroutines-core/nativeDarwin/test/Launcher.kt new file mode 100644 index 0000000000..4f63946852 --- /dev/null +++ b/kotlinx-coroutines-core/nativeDarwin/test/Launcher.kt @@ -0,0 +1,18 @@ +package kotlinx.coroutines + +import platform.CoreFoundation.* +import kotlin.native.concurrent.* +import kotlin.native.internal.test.* +import kotlin.system.* + +// This is a separate entry point for tests in background +fun mainBackground(args: Array) { + val worker = Worker.start(name = "main-background") + worker.execute(TransferMode.SAFE, { args }) { + val result = testLauncherEntryPoint(it) + exitProcess(result) + } + CFRunLoopRun() + error("CFRunLoopRun should never return") +} + diff --git a/kotlinx-coroutines-core/nativeDarwin/test/MainDispatcherTest.kt b/kotlinx-coroutines-core/nativeDarwin/test/MainDispatcherTest.kt new file mode 100644 index 0000000000..0094bcd7a1 --- /dev/null +++ b/kotlinx-coroutines-core/nativeDarwin/test/MainDispatcherTest.kt @@ -0,0 +1,25 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.cinterop.* +import kotlinx.coroutines.testing.* +import platform.CoreFoundation.* +import platform.darwin.* +import kotlin.coroutines.* +import kotlin.test.* + +class MainDispatcherTest : MainDispatcherTestBase.WithRealTimeDelay() { + + override fun isMainThread(): Boolean = CFRunLoopGetCurrent() == CFRunLoopGetMain() + + // skip if already on the main thread, run blocking doesn't really work well with that + override fun shouldSkipTesting(): Boolean = isMainThread() + + override fun scheduleOnMainQueue(block: () -> Unit) { + autoreleasepool { + dispatch_async(dispatch_get_main_queue()) { + block() + } + } + } +} diff --git a/kotlinx-coroutines-core/nativeOther/src/Dispatchers.kt b/kotlinx-coroutines-core/nativeOther/src/Dispatchers.kt new file mode 100644 index 0000000000..5d200d328a --- /dev/null +++ b/kotlinx-coroutines-core/nativeOther/src/Dispatchers.kt @@ -0,0 +1,31 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* +import kotlin.native.* + +internal actual fun createMainDispatcher(default: CoroutineDispatcher): MainCoroutineDispatcher = + MissingMainDispatcher + +internal actual fun createDefaultDispatcher(): CoroutineDispatcher = DefaultDispatcher + +private object DefaultDispatcher : CoroutineDispatcher() { + // Be consistent with JVM -- at least 2 threads to provide some liveness guarantees in case of improper uses + @OptIn(ExperimentalStdlibApi::class) + private val ctx = newFixedThreadPoolContext(Platform.getAvailableProcessors().coerceAtLeast(2), "Dispatchers.Default") + + override fun dispatch(context: CoroutineContext, block: Runnable) { + ctx.dispatch(context, block) + } +} + +private object MissingMainDispatcher : MainCoroutineDispatcher() { + override val immediate: MainCoroutineDispatcher + get() = notImplemented() + override fun dispatch(context: CoroutineContext, block: Runnable) = notImplemented() + override fun isDispatchNeeded(context: CoroutineContext): Boolean = notImplemented() + override fun dispatchYield(context: CoroutineContext, block: Runnable) = notImplemented() + + private fun notImplemented(): Nothing = TODO("Dispatchers.Main is missing on the current platform") +} + +internal actual inline fun platformAutoreleasePool(crossinline block: () -> Unit) = block() diff --git a/kotlinx-coroutines-core/nativeOther/test/Launcher.kt b/kotlinx-coroutines-core/nativeOther/test/Launcher.kt new file mode 100644 index 0000000000..a957a5112a --- /dev/null +++ b/kotlinx-coroutines-core/nativeOther/test/Launcher.kt @@ -0,0 +1,14 @@ +package kotlinx.coroutines + +import kotlin.native.concurrent.* +import kotlin.native.internal.test.* +import kotlin.system.* + +// This is a separate entry point for tests in background +fun mainBackground(args: Array) { + val worker = Worker.start(name = "main-background") + worker.execute(TransferMode.SAFE, { args }) { + val result = testLauncherEntryPoint(it) + exitProcess(result) + }.result // block main thread +} diff --git a/kotlinx-coroutines-core/pom.xml b/kotlinx-coroutines-core/pom.xml deleted file mode 100644 index 4f24e7f078..0000000000 --- a/kotlinx-coroutines-core/pom.xml +++ /dev/null @@ -1,61 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - - - kotlinx-coroutines-core - jar - - - ${env.JDK_16} - - - - src/main/kotlin - src/test/kotlin - - - - maven-surefire-plugin - - once - ${env.JDK_16}/bin/java - -ea -Xmx1g -Xms1g - - - - org.apache.maven.plugins - maven-jar-plugin - - - - test-jar - - - - - - - diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Builders.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Builders.kt deleted file mode 100644 index e312f278b4..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Builders.kt +++ /dev/null @@ -1,195 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import java.util.concurrent.locks.LockSupport -import kotlin.coroutines.experimental.* -import kotlin.coroutines.experimental.intrinsics.startCoroutineUninterceptedOrReturn -import kotlin.coroutines.experimental.intrinsics.suspendCoroutineOrReturn - -// --------------- basic coroutine builders --------------- - -/** - * Launches new coroutine without blocking current thread and returns a reference to the coroutine as a [Job]. - * The coroutine is cancelled when the resulting job is [cancelled][Job.cancel]. - * - * The [context] for the new coroutine must be explicitly specified. - * See [CoroutineDispatcher] for the standard [context] implementations that are provided by `kotlinx.coroutines`. - * The [context][CoroutineScope.context] of the parent coroutine from its [scope][CoroutineScope] may be used, - * in which case the [Job] of the resulting coroutine is a child of the job of the parent coroutine. - * - * By default, the coroutine is immediately started. - * An optional [start] parameter can be set to [CoroutineStart.LAZY] to start coroutine _lazily_. In this case, - * the coroutine [Job] is created in _new_ state. It can be explicitly started with [start][Job.start] function - * and will be started implicitly on the first invocation of [join][Job.join]. - * - * Uncaught exceptions in this coroutine cancel parent job in the context by default - * (unless [CoroutineExceptionHandler] is explicitly specified), which means that when `launch` is used with - * the context of another coroutine, then any uncaught exception leads to the cancellation of parent coroutine. - * - * See [newCoroutineContext] for a description of debugging facilities that are available for newly created coroutine. - - * @param context context of the coroutine - * @param start coroutine start option - * @param block the coroutine code - */ -public fun launch( - context: CoroutineContext, - start: CoroutineStart = CoroutineStart.DEFAULT, - block: suspend CoroutineScope.() -> Unit -): Job { - val newContext = newCoroutineContext(context) - val coroutine = if (start.isLazy) - LazyStandaloneCoroutine(newContext, block) else - StandaloneCoroutine(newContext, active = true) - coroutine.initParentJob(context[Job]) - start(block, coroutine, coroutine) - return coroutine -} - -/** - * @suppress **Deprecated**: Use `start = CoroutineStart.XXX` parameter - */ -@Deprecated(message = "Use `start = CoroutineStart.XXX` parameter", - replaceWith = ReplaceWith("launch(context, if (start) CoroutineStart.DEFAULT else CoroutineStart.LAZY, block)")) -public fun launch(context: CoroutineContext, start: Boolean, block: suspend CoroutineScope.() -> Unit): Job = - launch(context, if (start) CoroutineStart.DEFAULT else CoroutineStart.LAZY, block) - -/** - * Calls the specified suspending block with a given coroutine context, suspends until it completes, and returns - * the result. - * - * This function immediately applies dispatcher from the new context, shifting execution of the block into the - * different thread inside the block, and back when it completes. - * The specified [context] is added onto the current coroutine context for the execution of the block. - */ -public suspend fun run(context: CoroutineContext, block: suspend () -> T): T = - suspendCoroutineOrReturn sc@ { cont -> - val oldContext = cont.context - // fast path #1 if there is no change in the actual context: - if (context === oldContext || context is CoroutineContext.Element && oldContext[context.key] === context) - return@sc block.startCoroutineUninterceptedOrReturn(cont) - // compute new context - val newContext = oldContext + context - // fast path #2 if the result is actually the same - if (newContext === oldContext) - return@sc block.startCoroutineUninterceptedOrReturn(cont) - // fast path #3 if the new dispatcher is the same as the old one. - // `equals` is used by design (see equals implementation is wrapper context like ExecutorCoroutineDispatcher) - if (newContext[ContinuationInterceptor] == oldContext[ContinuationInterceptor]) { - val newContinuation = RunContinuationDirect(newContext, cont) - return@sc block.startCoroutineUninterceptedOrReturn(newContinuation) - } - // slowest path otherwise -- use new interceptor, sync to its result via a - // full-blown instance of CancellableContinuation - val newContinuation = RunContinuationCoroutine(newContext, cont) - newContinuation.initCancellability() - block.startCoroutine(newContinuation) - newContinuation.getResult() - } - -/** - * Runs new coroutine and **blocks** current thread _interruptibly_ until its completion. - * This function should not be used from coroutine. It is designed to bridge regular blocking code - * to libraries that are written in suspending style, to be used in `main` functions and in tests. - * - * The default [CoroutineDispatcher] for this builder in an implementation of [EventLoop] that processes continuations - * in this blocked thread until the completion of this coroutine. - * See [CoroutineDispatcher] for the other implementations that are provided by `kotlinx.coroutines`. - * - * If this blocked thread is interrupted (see [Thread.interrupt]), then the coroutine job is cancelled and - * this `runBlocking` invocation throws [InterruptedException]. - * - * See [newCoroutineContext] for a description of debugging facilities that are available for newly created coroutine. - */ -@Throws(InterruptedException::class) -public fun runBlocking(context: CoroutineContext = EmptyCoroutineContext, block: suspend CoroutineScope.() -> T): T { - val currentThread = Thread.currentThread() - val eventLoop = if (context[ContinuationInterceptor] == null) EventLoopImpl(currentThread) else null - val newContext = newCoroutineContext(context + (eventLoop ?: EmptyCoroutineContext)) - val coroutine = BlockingCoroutine(newContext, currentThread, privateEventLoop = eventLoop != null) - coroutine.initParentJob(context[Job]) - eventLoop?.initParentJob(coroutine) - block.startCoroutine(coroutine, coroutine) - return coroutine.joinBlocking() -} - -// --------------- implementation --------------- - -private open class StandaloneCoroutine( - override val parentContext: CoroutineContext, - active: Boolean -) : AbstractCoroutine(active) { - override fun afterCompletion(state: Any?, mode: Int) { - // note the use of the parent's job context below! - if (state is CompletedExceptionally) handleCoroutineException(parentContext, state.exception) - } -} - -private class LazyStandaloneCoroutine( - parentContext: CoroutineContext, - private val block: suspend CoroutineScope.() -> Unit -) : StandaloneCoroutine(parentContext, active = false) { - override fun onStart() { - block.startCoroutine(this, this) - } -} - -private class RunContinuationDirect( - override val context: CoroutineContext, - continuation: Continuation -) : Continuation by continuation - -private class RunContinuationCoroutine( - override val parentContext: CoroutineContext, - continuation: Continuation -) : CancellableContinuationImpl(continuation, active = true) - -private class BlockingCoroutine( - override val parentContext: CoroutineContext, - private val blockedThread: Thread, - private val privateEventLoop: Boolean -) : AbstractCoroutine(active = true) { - val eventLoop: EventLoop? = parentContext[ContinuationInterceptor] as? EventLoop - - init { - if (privateEventLoop) require(eventLoop is EventLoopImpl) - } - - override fun afterCompletion(state: Any?, mode: Int) { - if (Thread.currentThread() != blockedThread) - LockSupport.unpark(blockedThread) - } - - @Suppress("UNCHECKED_CAST") - fun joinBlocking(): T { - while (true) { - if (Thread.interrupted()) throw InterruptedException().also { cancel(it) } - val parkNanos = eventLoop?.processNextEvent() ?: Long.MAX_VALUE - // note: process next even may look unpark flag, so check !isActive before parking - if (!isActive) break - LockSupport.parkNanos(this, parkNanos) - } - // process queued events (that could have been added after last processNextEvent and before cancel - if (privateEventLoop) (eventLoop as EventLoopImpl).shutdown() - // now return result - val state = this.state - (state as? CompletedExceptionally)?.let { throw it.exception } - return state as T - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CancellableContinuation.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CancellableContinuation.kt deleted file mode 100644 index cfabad120d..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CancellableContinuation.kt +++ /dev/null @@ -1,278 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlinx.coroutines.experimental.internal.LockFreeLinkedListNode -import java.util.concurrent.atomic.AtomicIntegerFieldUpdater -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.intrinsics.COROUTINE_SUSPENDED -import kotlin.coroutines.experimental.intrinsics.suspendCoroutineOrReturn -import kotlin.coroutines.experimental.suspendCoroutine - -// --------------- cancellable continuations --------------- - -/** - * Cancellable continuation. Its job is _completed_ when it is resumed or cancelled. - * When [cancel] function is explicitly invoked, this continuation resumes with [CancellationException] or - * with the specified cancel cause. - * - * Cancellable continuation has three states: - * - * | **State** | [isActive] | [isCompleted] | [isCancelled] | - * | ----------------------------------- | ---------- | ------------- | ------------- | - * | _Active_ (initial state) | `true` | `false` | `false` | - * | _Resumed_ (final _completed_ state) | `false` | `true` | `false` | - * | _Canceled_ (final _completed_ state)| `false` | `true` | `true` | - * - * Invocation of [cancel] transitions this continuation from _active_ to _cancelled_ state, while - * invocation of [resume] or [resumeWithException] transitions it from _active_ to _resumed_ state. - * - * Invocation of [resume] or [resumeWithException] in _resumed_ state produces [IllegalStateException] - * but is ignored in _cancelled_ state. - */ -public interface CancellableContinuation : Continuation, Job { - /** - * Returns `true` if this continuation was [cancelled][cancel]. - * - * It implies that [isActive] is `false` and [isCompleted] is `true`. - */ - public val isCancelled: Boolean - - /** - * Tries to resume this continuation with a given value and returns non-null object token if it was successful, - * or `null` otherwise (it was already resumed or cancelled). When non-null object was returned, - * [completeResume] must be invoked with it. - * - * When [idempotent] is not `null`, this function performs _idempotent_ operation, so that - * further invocations with the same non-null reference produce the same result. - * - * @suppress **This is unstable API and it is subject to change.** - */ - public fun tryResume(value: T, idempotent: Any? = null): Any? - - /** - * Tries to resume this continuation with a given exception and returns non-null object token if it was successful, - * or `null` otherwise (it was already resumed or cancelled). When non-null object was returned, - * [completeResume] must be invoked with it. - * - * @suppress **This is unstable API and it is subject to change.** - */ - public fun tryResumeWithException(exception: Throwable): Any? - - /** - * Completes the execution of [tryResume] or [tryResumeWithException] on its non-null result. - * - * @suppress **This is unstable API and it is subject to change.** - */ - public fun completeResume(token: Any) - - /** - * Makes this continuation cancellable. Use it with `holdCancellability` optional parameter to - * [suspendCancellableCoroutine] function. It throws [IllegalStateException] if invoked more than once. - */ - public fun initCancellability() - - /** - * Resumes this continuation with a given [value] in the invoker thread without going though - * [dispatch][CoroutineDispatcher.dispatch] function of the [CoroutineDispatcher] in the [context]. - * This function is designed to be used only by the [CoroutineDispatcher] implementations themselves. - * **It should not be used in general code**. - * - * The receiver [CoroutineDispatcher] of this function be equal to the context dispatcher or - * [IllegalArgumentException] if thrown. - */ - public fun CoroutineDispatcher.resumeUndispatched(value: T) - - /** - * Resumes this continuation with a given [exception] in the invoker thread without going though - * [dispatch][CoroutineDispatcher.dispatch] function of the [CoroutineDispatcher] in the [context]. - * This function is designed to be used only by the [CoroutineDispatcher] implementations themselves. - * **It should not be used in general code**. - * - * The receiver [CoroutineDispatcher] of this function be equal to the context dispatcher or - * [IllegalArgumentException] if thrown. - */ - public fun CoroutineDispatcher.resumeUndispatchedWithException(exception: Throwable) -} - -/** - * Suspend coroutine similar to [suspendCoroutine], but provide an implementation of [CancellableContinuation] to - * the [block]. This function throws [CancellationException] if the coroutine is cancelled while suspended. - * - * If [holdCancellability] optional parameter is `true`, then the coroutine is suspended, but it is not - * cancellable until [CancellableContinuation.initCancellability] is invoked. - */ -public inline suspend fun suspendCancellableCoroutine( - holdCancellability: Boolean = false, - crossinline block: (CancellableContinuation) -> Unit -): T = - suspendCoroutineOrReturn { cont -> - val cancellable = CancellableContinuationImpl(cont, active = true) - if (!holdCancellability) cancellable.initCancellability() - block(cancellable) - cancellable.getResult() - } - - -/** - * Removes a given node on cancellation. - * @suppress **This is unstable API and it is subject to change.** - */ -public fun CancellableContinuation<*>.removeOnCancel(node: LockFreeLinkedListNode): DisposableHandle = - invokeOnCompletion(RemoveOnCancel(this, node)) - -// --------------- implementation details --------------- - -private class RemoveOnCancel( - cont: CancellableContinuation<*>, - val node: LockFreeLinkedListNode -) : JobNode>(cont) { - override fun invoke(reason: Throwable?) { - if (job.isCancelled) - node.remove() - } - override fun toString() = "RemoveOnCancel[$node]" -} - -internal const val MODE_DISPATCHED = 0 -internal const val MODE_UNDISPATCHED = 1 -internal const val MODE_DIRECT = 2 - -@PublishedApi -internal open class CancellableContinuationImpl( - @JvmField - protected val delegate: Continuation, - active: Boolean -) : AbstractCoroutine(active), CancellableContinuation { - @Volatile - private var decision = UNDECIDED - - override val parentContext: CoroutineContext - get() = delegate.context - - protected companion object { - @JvmField - val DECISION: AtomicIntegerFieldUpdater> = - AtomicIntegerFieldUpdater.newUpdater(CancellableContinuationImpl::class.java, "decision") - - const val UNDECIDED = 0 - const val SUSPENDED = 1 - const val RESUMED = 2 - - @Suppress("UNCHECKED_CAST") - fun getSuccessfulResult(state: Any?): T = if (state is CompletedIdempotentResult) state.result as T else state as T - } - - override fun initCancellability() { - initParentJob(parentContext[Job]) - } - - @PublishedApi - internal fun getResult(): Any? { - val decision = this.decision // volatile read - if (decision == UNDECIDED && DECISION.compareAndSet(this, UNDECIDED, SUSPENDED)) return COROUTINE_SUSPENDED - // otherwise, afterCompletion was already invoked, and the result is in the state - val state = this.state - if (state is CompletedExceptionally) throw state.exception - return getSuccessfulResult(state) - } - - override val isCancelled: Boolean get() = state is Cancelled - - override fun tryResume(value: T, idempotent: Any?): Any? { - while (true) { // lock-free loop on state - val state = this.state // atomic read - when (state) { - is Incomplete -> { - val idempotentStart = state.idempotentStart - val update: Any? = if (idempotent == null && idempotentStart == null) value else - CompletedIdempotentResult(idempotentStart, idempotent, value, state) - if (tryUpdateState(state, update)) return state - } - is CompletedIdempotentResult -> { - if (state.idempotentResume === idempotent) { - check(state.result === value) { "Non-idempotent resume" } - return state.token - } else - return null - } - else -> return null // cannot resume -- not active anymore - } - } - } - - override fun tryResumeWithException(exception: Throwable): Any? { - while (true) { // lock-free loop on state - val state = this.state // atomic read - when (state) { - is Incomplete -> { - if (tryUpdateState(state, CompletedExceptionally(state.idempotentStart, exception))) return state - } - else -> return null // cannot resume -- not active anymore - } - } - } - - override fun completeResume(token: Any) { - completeUpdateState(token, state, defaultResumeMode) - } - - override fun afterCompletion(state: Any?, mode: Int) { - val decision = this.decision // volatile read - if (decision == UNDECIDED && DECISION.compareAndSet(this, UNDECIDED, RESUMED)) return // will get result in getResult - // otherwise, getResult has already commenced, i.e. it was resumed later or in other thread - if (state is CompletedExceptionally) { - val exception = state.exception - when (mode) { - MODE_DISPATCHED -> delegate.resumeWithException(exception) - MODE_UNDISPATCHED -> (delegate as DispatchedContinuation).resumeUndispatchedWithException(exception) - MODE_DIRECT -> delegate.resumeDirectWithException(exception) - else -> error("Invalid mode $mode") - } - } else { - val value = getSuccessfulResult(state) - when (mode) { - MODE_DISPATCHED -> delegate.resume(value) - MODE_UNDISPATCHED -> (delegate as DispatchedContinuation).resumeUndispatched(value) - MODE_DIRECT -> delegate.resumeDirect(value) - else -> error("Invalid mode $mode") - } - } - } - - override fun CoroutineDispatcher.resumeUndispatched(value: T) { - val dc = delegate as? DispatchedContinuation ?: throw IllegalArgumentException("Must be used with DispatchedContinuation") - check(dc.dispatcher === this) { "Must be invoked from the context CoroutineDispatcher"} - resume(value, MODE_UNDISPATCHED) - } - - override fun CoroutineDispatcher.resumeUndispatchedWithException(exception: Throwable) { - val dc = delegate as? DispatchedContinuation ?: throw IllegalArgumentException("Must be used with DispatchedContinuation") - check(dc.dispatcher === this) { "Must be invoked from the context CoroutineDispatcher"} - resumeWithException(exception, MODE_UNDISPATCHED) - } - - private class CompletedIdempotentResult( - idempotentStart: Any?, - @JvmField val idempotentResume: Any?, - @JvmField val result: Any?, - @JvmField val token: Incomplete - ) : CompletedIdempotentStart(idempotentStart) { - override fun toString(): String = "CompletedIdempotentResult[$result]" - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CommonPool.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CommonPool.kt deleted file mode 100644 index 6198492ecf..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CommonPool.kt +++ /dev/null @@ -1,87 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import java.util.concurrent.ExecutorService -import java.util.concurrent.Executors -import java.util.concurrent.TimeUnit -import java.util.concurrent.atomic.AtomicInteger -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Represents common pool of shared threads as coroutine dispatcher for compute-intensive tasks. - * It uses [java.util.concurrent.ForkJoinPool] when available, which implements efficient work-stealing algorithm for its queues, so every - * coroutine resumption is dispatched as a separate task even when it already executes inside the pool. - * When available, it wraps `ForkJoinPool.commonPool` and provides a similar shared pool where not. - */ -object CommonPool : CoroutineDispatcher() { - private var usePrivatePool = false - - @Volatile - private var _pool: ExecutorService? = null - - private inline fun Try(block: () -> T) = try { block() } catch (e: Throwable) { null } - - private fun createPool(): ExecutorService { - val fjpClass = Try { Class.forName("java.util.concurrent.ForkJoinPool") } - ?: return createPlainPool() - if (!usePrivatePool) { - Try { fjpClass.getMethod("commonPool")?.invoke(null) as? ExecutorService } - ?.let { return it } - } - Try { fjpClass.getConstructor(Int::class.java).newInstance(defaultParallelism()) as? ExecutorService } - ?. let { return it } - return createPlainPool() - } - - private fun createPlainPool(): ExecutorService { - val threadId = AtomicInteger() - return Executors.newFixedThreadPool(defaultParallelism()) { - Thread(it, "CommonPool-worker-${threadId.incrementAndGet()}").apply { isDaemon = true } - } - } - - private fun defaultParallelism() = (Runtime.getRuntime().availableProcessors() - 1).coerceAtLeast(1) - - @Synchronized - private fun getOrCreatePoolSync(): ExecutorService = - _pool ?: createPool().also { _pool = it } - - override fun dispatch(context: CoroutineContext, block: Runnable) = - (_pool ?: getOrCreatePoolSync()).execute(block) - - // used for tests - @Synchronized - internal fun usePrivatePool() { - shutdownAndRelease(0) - usePrivatePool = true - } - - // used for tests - @Synchronized - internal fun shutdownAndRelease(timeout: Long) { - _pool?.apply { - shutdown() - if (timeout > 0) - awaitTermination(timeout, TimeUnit.MILLISECONDS) - _pool = null - } - usePrivatePool = false - } - - override fun toString(): String = "CommonPool" -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineContext.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineContext.kt deleted file mode 100644 index b056fbfd58..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineContext.kt +++ /dev/null @@ -1,124 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import java.util.concurrent.atomic.AtomicLong -import kotlin.coroutines.experimental.AbstractCoroutineContextElement -import kotlin.coroutines.experimental.CoroutineContext - -private const val DEBUG_PROPERTY_NAME = "kotlinx.coroutines.debug" - -private val DEBUG = run { - val value = System.getProperty(DEBUG_PROPERTY_NAME) - when (value) { - "auto", null -> CoroutineId::class.java.desiredAssertionStatus() - "on", "" -> true - "off" -> false - else -> error("System property '$DEBUG_PROPERTY_NAME' has unrecognized value '$value'") - } -} - -private val COROUTINE_ID = AtomicLong() - -// for tests only -internal fun resetCoroutineId() { - COROUTINE_ID.set(0) -} - -/** - * A coroutine dispatcher that is not confined to any specific thread. - * It executes initial continuation of the coroutine _right here_ in the current call-frame - * and let the coroutine resume in whatever thread that is used by the corresponding suspending function, without - * mandating any specific threading policy. - * - * Note, that if you need your coroutine to be confined to a particular thread or a thread-pool after resumption, - * but still want to execute it in the current call-frame until its first suspension, then you can use - * an optional [CoroutineStart] parameter in coroutine builders like [launch] and [async] setting it to the - * the value of [CoroutineStart.UNDISPATCHED]. - */ -public object Unconfined : CoroutineDispatcher() { - override fun isDispatchNeeded(context: CoroutineContext): Boolean = false - override fun dispatch(context: CoroutineContext, block: Runnable) { throw UnsupportedOperationException() } - override fun toString(): String = "Unconfined" -} - -/** - * @suppress **Deprecated**: `Here` was renamed to `Unconfined`. - */ -@Deprecated(message = "`Here` was renamed to `Unconfined`", - replaceWith = ReplaceWith(expression = "Unconfined")) -public typealias Here = Unconfined - -/** - * Creates context for the new coroutine with optional support for debugging facilities (when turned on). - * - * **Debugging facilities:** In debug mode every coroutine is assigned a unique consecutive identifier. - * Every thread that executes a coroutine has its name modified to include the name and identifier of the - * currently currently running coroutine. - * When one coroutine is suspended and resumes another coroutine that is dispatched in the same thread, - * then the thread name displays - * the whole stack of coroutine descriptions that are being executed on this thread. - * - * Enable debugging facilities with "`kotlinx.coroutines.debug`" system property, use the following values: - * * "`auto`" (default mode) -- enabled when assertions are enabled with "`-ea`" JVM option. - * * "`on`" or empty string -- enabled. - * * "`off`" -- disabled. - * - * Coroutine name can be explicitly assigned using [CoroutineName] context element. - * The string "coroutine" is used as a default name. - */ -public fun newCoroutineContext(context: CoroutineContext): CoroutineContext = - if (DEBUG) context + CoroutineId(COROUTINE_ID.incrementAndGet()) else context - -/** - * Executes a block using a given coroutine context. - */ -internal inline fun withCoroutineContext(context: CoroutineContext, block: () -> T): T { - val oldName = updateContext(context) - try { - return block() - } finally { - restoreContext(oldName) - } -} - -@PublishedApi -internal fun updateContext(context: CoroutineContext): String? { - if (!DEBUG) return null - val newId = context[CoroutineId] ?: return null - val currentThread = Thread.currentThread() - val oldName = currentThread.name - val coroutineName = context[CoroutineName]?.name ?: "coroutine" - currentThread.name = buildString(oldName.length + coroutineName.length + 10) { - append(oldName) - append(" @") - append(coroutineName) - append('#') - append(newId.id) - } - return oldName -} - -@PublishedApi -internal fun restoreContext(oldName: String?) { - if (oldName != null) Thread.currentThread().name = oldName -} - -private class CoroutineId(val id: Long) : AbstractCoroutineContextElement(CoroutineId) { - companion object Key : CoroutineContext.Key - override fun toString(): String = "CoroutineId($id)" -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineDispatcher.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineDispatcher.kt deleted file mode 100644 index e81d4ca33b..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineDispatcher.kt +++ /dev/null @@ -1,165 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlin.coroutines.experimental.AbstractCoroutineContextElement -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.ContinuationInterceptor -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Base class that shall be extended by all coroutine dispatcher implementations. - * - * The following standard implementations are provided by `kotlinx.coroutines`: - * * [Unconfined] -- starts coroutine execution in the current call-frame until the first suspension. - * On first suspension the coroutine builder function returns. - * The coroutine will resume in whatever thread that is used by the - * corresponding suspending function, without confining it to any specific thread or pool. - * This in an appropriate choice for IO-intensive coroutines that do not consume CPU resources. - * * [CommonPool] -- immediately returns from the coroutine builder and schedules coroutine execution to - * a common pool of shared background threads. - * This is an appropriate choice for compute-intensive coroutines that consume a lot of CPU resources. - * * Private thread pools can be created with [newSingleThreadContext] and [newFixedThreadPoolContext]. - * * An arbitrary [Executor][java.util.concurrent.Executor] can be converted to dispatcher with [asCoroutineDispatcher] extension function. - * - * This class ensures that debugging facilities in [newCoroutineContext] function work properly. - */ -public abstract class CoroutineDispatcher : - AbstractCoroutineContextElement(ContinuationInterceptor), ContinuationInterceptor { - /** - * Returns `true` if execution shall be dispatched onto another thread. - * The default behaviour for most dispatchers is to return `true`. - * - * UI dispatchers _should not_ override `isDispatchNeeded`, but leave a default implementation that - * returns `true`. To understand the rationale beyond this recommendation, consider the following code: - * - * ```kotlin - * fun asyncUpdateUI() = async(MainThread) { - * // do something here that updates something in UI - * } - * ``` - * - * When you invoke `asyncUpdateUI` in some background thread, it immediately continues to the next - * line, while UI update happens asynchronously in the UI thread. However, if you invoke - * it in the UI thread itself, it updates UI _synchronously_ if your `isDispatchNeeded` is - * overridden with a thread check. Checking if we are already in the UI thread seems more - * efficient (and it might indeed save a few CPU cycles), but this subtle and context-sensitive - * difference in behavior makes the resulting async code harder to debug. - * - * Basically, the choice here is between "JS-style" asynchronous approach (async actions - * are always postponed to be executed later in the even dispatch thread) and "C#-style" approach - * (async actions are executed in the invoker thread until the first suspension point). - * While, C# approach seems to be more efficient, it ends up with recommendations like - * "use `yield` if you need to ....". This is error-prone. JS-style approach is more consistent - * and does not require programmers to think about whether they need to yield or not. - * - * However, coroutine builders like [launch] and [async] accept an optional [CoroutineStart] - * parameter that allows one to optionally choose C#-style [CoroutineStart.UNDISPATCHED] behaviour - * whenever it is needed for efficiency. - */ - public open fun isDispatchNeeded(context: CoroutineContext): Boolean = true - - /** - * Dispatches execution of a runnable [block] onto another thread in the given [context]. - */ - public abstract fun dispatch(context: CoroutineContext, block: Runnable) - - /** - * Returns continuation that wraps the original [continuation], thus intercepting all resumptions. - */ - public override fun interceptContinuation(continuation: Continuation): Continuation = - DispatchedContinuation(this, continuation) - - /** - * @suppress **Error**: Operator '+' on two CoroutineDispatcher objects is meaningless. - * CoroutineDispatcher is a coroutine context element and `+` is a set-sum operator for coroutine contexts. - * The dispatcher to the right of `+` just replaces the dispatcher the left of `+`. - */ - @Suppress("DeprecatedCallableAddReplaceWith") - @Deprecated(message = "Operator '+' on two CoroutineDispatcher objects is meaningless. " + - "CoroutineDispatcher is a coroutine context element and `+` is a set-sum operator for coroutine contexts. " + - "The dispatcher to the right of `+` just replaces the dispatcher the left of `+`.", - level = DeprecationLevel.ERROR) - public operator fun plus(other: CoroutineDispatcher) = other - - // for nicer debugging - override fun toString(): String = - "${this::class.java.simpleName}@${Integer.toHexString(System.identityHashCode(this))}" - -} - -internal class DispatchedContinuation( - @JvmField val dispatcher: CoroutineDispatcher, - @JvmField val continuation: Continuation -): Continuation by continuation { - override fun resume(value: T) { - val context = continuation.context - if (dispatcher.isDispatchNeeded(context)) - dispatcher.dispatch(context, Runnable { - resumeUndispatched(value) - }) - else - resumeUndispatched(value) - } - - @Suppress("NOTHING_TO_INLINE") // we need it inline to save us an entry on the stack - inline fun resumeUndispatched(value: T) { - withCoroutineContext(context) { - continuation.resume(value) - } - } - - override fun resumeWithException(exception: Throwable) { - val context = continuation.context - if (dispatcher.isDispatchNeeded(context)) - dispatcher.dispatch(context, Runnable { - resumeUndispatchedWithException(exception) - }) - else - resumeUndispatchedWithException(exception) - } - - @Suppress("NOTHING_TO_INLINE") // we need it inline to save us an entry on the stack - inline fun resumeUndispatchedWithException(exception: Throwable) { - withCoroutineContext(context) { - continuation.resumeWithException(exception) - } - } - - // used by "yield" implementation - internal fun dispatchYield(job: Job?, value: T) { - val context = continuation.context - dispatcher.dispatch(context, Runnable { - withCoroutineContext(context) { - if (job != null && job.isCompleted) - continuation.resumeWithException(job.getCompletionException()) - else - continuation.resume(value) - } - }) - } -} - -internal fun Continuation.resumeDirect(value: T) = when (this) { - is DispatchedContinuation -> continuation.resume(value) - else -> resume(value) -} - -internal fun Continuation.resumeDirectWithException(exception: Throwable) = when (this) { - is DispatchedContinuation -> continuation.resumeWithException(exception) - else -> resumeWithException(exception) -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineExceptionHandler.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineExceptionHandler.kt deleted file mode 100644 index 30add0a07c..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineExceptionHandler.kt +++ /dev/null @@ -1,61 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlin.coroutines.experimental.CoroutineContext - - -/** - * Helper function for coroutine builder implementations to handle uncaught exception in coroutines. - * It tries to handle uncaught exception in the following way: - * * If there is [CoroutineExceptionHandler] in the context, then it is used. - * * Otherwise, if exception is [CancellationException] then it is ignored - * (because that is the supposed mechanism to cancel the running coroutine) - * * Otherwise, if there is a [Job] in the context, then [Job.cancel] is invoked and if it - * returns `true` (it was still active), then the exception is considered to be handled. - * * Otherwise, current thread's [Thread.uncaughtExceptionHandler] is used. - */ -fun handleCoroutineException(context: CoroutineContext, exception: Throwable) { - context[CoroutineExceptionHandler]?.let { - it.handleException(context, exception) - return - } - // ignore CancellationException (they are normal means to terminate a coroutine) - if (exception is CancellationException) return - // quit if successfully pushed exception as cancellation reason - if (context[Job]?.cancel(exception) ?: false) return - // otherwise just use thread's handler - val currentThread = Thread.currentThread() - currentThread.uncaughtExceptionHandler.uncaughtException(currentThread, exception) -} - -/** - * An optional element on the coroutine context to handle uncaught exceptions. - * See [handleCoroutineException]. - */ -public interface CoroutineExceptionHandler : CoroutineContext.Element { - /** - * Key for [CoroutineExceptionHandler] instance in the coroutine context. - */ - companion object Key : CoroutineContext.Key - - /** - * Handles uncaught [exception] in the given [context]. It is invoked - * if coroutine has an uncaught exception. See [handleCoroutineException]. - */ - public fun handleException(context: CoroutineContext, exception: Throwable) -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineName.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineName.kt deleted file mode 100644 index 7a53d8984a..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineName.kt +++ /dev/null @@ -1,41 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlin.coroutines.experimental.AbstractCoroutineContextElement -import kotlin.coroutines.experimental.CoroutineContext - -/** - * User-specified name of coroutine. This name is used in debugging mode. - * See [newCoroutineContext] for the description of coroutine debugging facilities. - */ -public data class CoroutineName( - /** - * User-defined coroutine name. - */ - val name: String -) : AbstractCoroutineContextElement(CoroutineName) { - /** - * Key for [CoroutineName] instance in the coroutine context. - */ - public companion object Key : CoroutineContext.Key - - /** - * Returns a string representation of the object. - */ - override fun toString(): String = "CoroutineName($name)" -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineScope.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineScope.kt deleted file mode 100644 index e976f7e72b..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineScope.kt +++ /dev/null @@ -1,127 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Receiver interface for generic coroutine builders, so that the code inside coroutine has a convenient access - * to its [context] and its cancellation status via [isActive]. - */ -public interface CoroutineScope { - /** - * Returns `true` when this coroutine is still active (has not completed yet). - * - * Check this property in long-running computation loops to support cancellation: - * ``` - * while (isActive) { - * // do some computation - * } - * ``` - * - * This property is a shortcut for `context[Job]!!.isActive`. See [context] and [Job]. - */ - public val isActive: Boolean - - /** - * Returns the context of this coroutine. - */ - public val context: CoroutineContext -} - -/** - * Abstract class to simplify writing of coroutine completion objects that - * implement completion [Continuation], [Job], and [CoroutineScope] interfaces. - * It stores the result of continuation in the state of the job. - * - * @param active when `true` coroutine is created in _active_ state, when `false` in _new_ state. See [Job] for details. - * @suppress **This is unstable API and it is subject to change.** - */ -public abstract class AbstractCoroutine( - active: Boolean -) : JobSupport(active), Continuation, CoroutineScope { - // context must be Ok for unsafe publishing (it is persistent), - // so we don't mark this _context variable as volatile, but leave potential benign race here - private var _context: CoroutineContext? = null // created on first need - - @Suppress("LeakingThis") - public final override val context: CoroutineContext - get() = _context ?: createContext().also { _context = it } - - protected abstract val parentContext: CoroutineContext - - protected open fun createContext() = parentContext + this - - protected open val defaultResumeMode: Int get() = MODE_DISPATCHED - - protected open val ignoreRepeatedResume: Boolean get() = false - - final override fun resume(value: T) = resume(value, defaultResumeMode) - - protected fun resume(value: T, mode: Int) { - while (true) { // lock-free loop on state - val state = this.state // atomic read - when (state) { - is Incomplete -> if (updateState(state, value, mode)) return - is Cancelled -> return // ignore resumes on cancelled continuation - else -> { - if (ignoreRepeatedResume) { - return - } else - throw IllegalStateException("Already resumed, but got value $value") - } - } - } - } - - final override fun resumeWithException(exception: Throwable) = resumeWithException(exception, defaultResumeMode) - - protected fun resumeWithException(exception: Throwable, mode: Int) { - while (true) { // lock-free loop on state - val state = this.state // atomic read - when (state) { - is Incomplete -> { - if (updateState(state, CompletedExceptionally(state.idempotentStart, exception), mode)) return - } - is Cancelled -> { - // ignore resumes on cancelled continuation, but handle exception if a different one is here - if (exception != state.exception) handleCoroutineException(context, exception) - return - } - else -> { - if (ignoreRepeatedResume) { - handleCoroutineException(context, exception) - return - } else - throw IllegalStateException("Already resumed, but got exception $exception", exception) - } - } - } - } - - final override fun handleCompletionException(closeException: Throwable) { - handleCoroutineException(context, closeException) - } - - // for nicer debugging - override fun toString(): String { - val state = this.state - val result = if (state is Incomplete) "" else "[$state]" - return "${this::class.java.simpleName}{${stateToString(state)}}$result@${Integer.toHexString(System.identityHashCode(this))}" - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineStart.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineStart.kt deleted file mode 100644 index 6389d53929..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/CoroutineStart.kt +++ /dev/null @@ -1,74 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.startCoroutine - -/** - * Defines start option for coroutines builders. - * It is used in `start` parameter of [launch], [async], and [actor][kotlinx.coroutines.experimental.channels.actor] - * coroutine builder functions. - */ -public enum class CoroutineStart { - /** - * Default -- schedules coroutine for execution according to its context. - * - * If the [CoroutineDispatcher] of the coroutine context returns `true` from [CoroutineDispatcher.isDispatchNeeded] - * function as most dispatchers do, then the coroutine code is dispatched for execution later, while the code that - * invoked the coroutine builder continues execution. - * - * Note, that [Unconfined] dispatcher always returns `false` from its [CoroutineDispatcher.isDispatchNeeded] - * function, so starting coroutine with [Unconfined] dispatcher by [DEFAULT] is the same as using [UNDISPATCHED]. - */ - DEFAULT, - - /** - * Starts coroutine lazily, only when it is needed. - * - * See the documentation for the corresponding coroutine builders for details: - * [launch], [async], and [actor][kotlinx.coroutines.experimental.channels.actor]. - */ - LAZY, - - /** - * Immediately executes coroutine until its first suspension point _in the current thread_ as if it the - * coroutine was started using [Unconfined] dispatcher. However, when coroutine is resumed from suspension - * it is dispatched according to the [CoroutineDispatcher] in its context. - */ - UNDISPATCHED; - - /** - * Starts the corresponding block as a coroutine with this coroutine start strategy. - * - * * [DEFAULT] uses [startCoroutine]. - * * [UNDISPATCHED] uses [startCoroutineUndispatched]. - * * [LAZY] does nothing. - */ - public operator fun invoke(block: suspend R.() -> T, receiver: R, completion: Continuation) = - when (this) { - CoroutineStart.DEFAULT -> block.startCoroutine(receiver, completion) - CoroutineStart.UNDISPATCHED -> block.startCoroutineUndispatched(receiver, completion) - CoroutineStart.LAZY -> Unit // will start lazily - } - - /** - * Returns `true` when [LAZY]. - */ - public val isLazy: Boolean get() = this === LAZY -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Deferred.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Deferred.kt deleted file mode 100644 index e36a74bbe8..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Deferred.kt +++ /dev/null @@ -1,241 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import kotlinx.coroutines.experimental.selects.SelectBuilder -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlinx.coroutines.experimental.selects.select -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Deferred value is a non-blocking cancellable future. - * It is created with [async] coroutine builder. - * It is in [active][isActive] state while the value is being computed. - * - * Deferred value has four or five possible states. - * - * | **State** | [isActive] | [isCompleted] | [isCompletedExceptionally] | [isCancelled] | - * | -------------------------------- | ---------- | ------------- | -------------------------- | ------------- | - * | _New_ (optional initial state) | `false` | `false` | `false` | `false` | - * | _Active_ (default initial state) | `true` | `false` | `false` | `false` | - * | _Resolved_ (final state) | `false` | `true` | `false` | `false` | - * | _Failed_ (final state) | `false` | `true` | `true` | `false` | - * | _Cancelled_ (final state) | `false` | `true` | `true` | `true` | - * - * Usually, a deferred value is created in _active_ state (it is created and started), so its only visible - * states are _active_ and _completed_ (_resolved_, _failed_, or _cancelled_) state. - * However, [async] coroutine builder has an optional `start` parameter that creates a deferred value in _new_ state - * when this parameter is set to [CoroutineStart.LAZY]. - * Such a deferred can be be made _active_ by invoking [start], [join], or [await]. - */ -public interface Deferred : Job { - /** - * Returns `true` if computation of this deferred value has _completed exceptionally_ -- it had - * either _failed_ with exception during computation or was [cancelled][cancel]. - * - * It implies that [isActive] is `false` and [isCompleted] is `true`. - */ - val isCompletedExceptionally: Boolean - - /** - * Returns `true` if computation of this deferred value was [cancelled][cancel]. - * - * It implies that [isActive] is `false`, [isCompleted] is `true`, and [isCompletedExceptionally] is `true`. - */ - val isCancelled: Boolean - - /** - * Awaits for completion of this value without blocking a thread and resumes when deferred computation is complete, - * returning the resulting value or throwing the corresponding exception if the deferred had completed exceptionally. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * This function can be used in [select] invocation with [onAwait][SelectBuilder.onAwait] clause. - * Use [isCompleted] to check for completion of this deferred value without waiting. - */ - public suspend fun await(): T - - /** - * Registers [onAwait][SelectBuilder.onAwait] select clause. - * @suppress **This is unstable API and it is subject to change.** - */ - public fun registerSelectAwait(select: SelectInstance, block: suspend (T) -> R) - - /** - * Returns *completed* result or throws [IllegalStateException] if this deferred value has not - * [completed][isCompleted] yet. It throws the corresponding exception if this deferred has - * [completed exceptionally][isCompletedExceptionally]. - * - * This function is designed to be used from [invokeOnCompletion] handlers, when there is an absolute certainty that - * the value is already complete. - */ - public fun getCompleted(): T - - /** - * @suppress **Deprecated**: Use `isActive`. - */ - @Deprecated(message = "Use `isActive`", replaceWith = ReplaceWith("isActive")) - public val isComputing: Boolean get() = isActive -} - -/** - * Creates new coroutine and returns its future result as an implementation of [Deferred]. - * - * The running coroutine is cancelled when the resulting object is [cancelled][Job.cancel]. - * The [context] for the new coroutine must be explicitly specified. - * See [CoroutineDispatcher] for the standard [context] implementations that are provided by `kotlinx.coroutines`. - * The [context][CoroutineScope.context] of the parent coroutine from its [scope][CoroutineScope] may be used, - * in which case the [Job] of the resulting coroutine is a child of the job of the parent coroutine. - * - * By default, the coroutine is immediately started. - * An optional [start] parameter can be set to [CoroutineStart.LAZY] to start coroutine _lazily_. In this case,, - * the resulting [Deferred] is created in _new_ state. It can be explicitly started with [start][Job.start] - * function and will be started implicitly on the first invocation of [join][Job.join] or [await][Deferred.await]. - * - * @param context context of the coroutine - * @param start coroutine start option - * @param block the coroutine code - */ -public fun async( - context: CoroutineContext, - start: CoroutineStart = CoroutineStart.DEFAULT, - block: suspend CoroutineScope.() -> T -): Deferred { - val newContext = newCoroutineContext(context) - val coroutine = if (start.isLazy) - LazyDeferredCoroutine(newContext, block) else - DeferredCoroutine(newContext, active = true) - coroutine.initParentJob(context[Job]) - start(block, coroutine, coroutine) - return coroutine -} - -/** - * @suppress **Deprecated**: Use `start = CoroutineStart.XXX` parameter - */ -@Deprecated(message = "Use `start = CoroutineStart.XXX` parameter", - replaceWith = ReplaceWith("async(context, if (start) CoroutineStart.DEFAULT else CoroutineStart.LAZY, block)")) -public fun async(context: CoroutineContext, start: Boolean, block: suspend CoroutineScope.() -> T): Deferred = - async(context, if (start) CoroutineStart.DEFAULT else CoroutineStart.LAZY, block) - -/** - * @suppress **Deprecated**: `defer` was renamed to `async`. - */ -@Deprecated(message = "`defer` was renamed to `async`", level = DeprecationLevel.WARNING, - replaceWith = ReplaceWith("async(context, block = block)")) -public fun defer(context: CoroutineContext, block: suspend CoroutineScope.() -> T): Deferred = - async(context, block = block) - -private open class DeferredCoroutine( - override val parentContext: CoroutineContext, - active: Boolean -) : AbstractCoroutine(active), Deferred { - override val isCompletedExceptionally: Boolean get() = state is CompletedExceptionally - override val isCancelled: Boolean get() = state is Cancelled - - @Suppress("UNCHECKED_CAST") - suspend override fun await(): T { - // fast-path -- check state (avoid extra object creation) - while(true) { // lock-free loop on state - val state = this.state - if (state !is Incomplete) { - // already complete -- just return result - if (state is CompletedExceptionally) throw state.exception - return state as T - - } - if (startInternal(state) >= 0) break // break unless needs to retry - } - return awaitSuspend() // slow-path - } - - @Suppress("UNCHECKED_CAST") - private suspend fun awaitSuspend(): T = suspendCancellableCoroutine { cont -> - cont.disposeOnCompletion(invokeOnCompletion { - val state = this.state - check(state !is Incomplete) - if (state is CompletedExceptionally) - cont.resumeWithException(state.exception) - else - cont.resume(state as T) - }) - } - - @Suppress("UNCHECKED_CAST") - override fun registerSelectAwait(select: SelectInstance, block: suspend (T) -> R) { - // fast-path -- check state and select/return if needed - while (true) { - if (select.isSelected) return - val state = this.state - if (state !is Incomplete) { - // already complete -- select result - if (select.trySelect(idempotent = null)) { - if (state is CompletedExceptionally) - select.resumeSelectWithException(state.exception, MODE_DIRECT) - else - block.startCoroutineUndispatched(state as T, select.completion) - } - return - } - if (startInternal(state) == 0) { - // slow-path -- register waiter for completion - select.disposeOnSelect(invokeOnCompletion(SelectAwaitOnCompletion(this, select, block))) - return - } - } - } - - @Suppress("UNCHECKED_CAST") - internal fun selectAwaitCompletion(select: SelectInstance, block: suspend (T) -> R, state: Any? = this.state) { - if (select.trySelect(idempotent = null)) { - if (state is CompletedExceptionally) - select.resumeSelectWithException(state.exception, MODE_DISPATCHED) - else - block.startCoroutine(state as T, select.completion) - } - } - - @Suppress("UNCHECKED_CAST") - override fun getCompleted(): T { - val state = this.state - check(state !is Incomplete) { "This deferred value has not completed yet" } - if (state is CompletedExceptionally) throw state.exception - return state as T - } -} - -private class SelectAwaitOnCompletion( - deferred: DeferredCoroutine, - private val select: SelectInstance, - private val block: suspend (T) -> R -) : JobNode>(deferred) { - override fun invoke(reason: Throwable?) = job.selectAwaitCompletion(select, block) - override fun toString(): String = "SelectAwaitOnCompletion[$select]" -} - -private class LazyDeferredCoroutine( - parentContext: CoroutineContext, - private val block: suspend CoroutineScope.() -> T -) : DeferredCoroutine(parentContext, active = false) { - override fun onStart() { - block.startCoroutine(this, this) - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Delay.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Delay.kt deleted file mode 100644 index 2c984cf0a7..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Delay.kt +++ /dev/null @@ -1,103 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlinx.coroutines.experimental.selects.SelectBuilder -import kotlinx.coroutines.experimental.selects.select -import java.util.concurrent.Future -import java.util.concurrent.TimeUnit -import kotlin.coroutines.experimental.ContinuationInterceptor - -/** - * This dispatcher _feature_ is implemented by [CoroutineDispatcher] implementations that natively support - * scheduled execution of tasks. - * - * Implementation of this interface affects operation of - * [delay][kotlinx.coroutines.experimental.delay] and [withTimeout] functions. - */ -public interface Delay { - /** - * Delays coroutine for a given time without blocking a thread and resumes it after a specified time. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * immediately resumes with [CancellationException]. - */ - suspend fun delay(time: Long, unit: TimeUnit = TimeUnit.MILLISECONDS) { - require(time >= 0) { "Delay time $time cannot be negative" } - if (time <= 0) return // don't delay - return suspendCancellableCoroutine { scheduleResumeAfterDelay(time, unit, it) } - } - - /** - * Schedules resume of a specified [continuation] after a specified delay [time]. - * - * Continuation **must be scheduled** to resume even if it is already cancelled, because a cancellation is just - * an exception that the coroutine that used `delay` might wanted to catch and process. It might - * need to close some resources in its `finally` blocks, for example. - * - * This implementation is supposed to use dispatcher's native ability for scheduled execution in its thread(s). - * In order to avoid an extra delay of execution, the following code shall be used to resume this - * [continuation] when the code is already executing in the appropriate thread: - * - * ```kotlin - * with(continuation) { resumeUndispatched(Unit) } - * ``` - */ - fun scheduleResumeAfterDelay(time: Long, unit: TimeUnit, continuation: CancellableContinuation) - - /** - * Schedules invocation of a specified [block] after a specified delay [time]. - * The resulting [DisposableHandle] can be used to [dispose][DisposableHandle.dispose] of this invocation - * request if it is not needed anymore. - * - * This implementation uses a built-in single-threaded scheduled executor service. - */ - fun invokeOnTimeout(time: Long, unit: TimeUnit, block: Runnable): DisposableHandle = - DisposableFutureHandle(scheduledExecutor.schedule(block, time, unit)) -} - -/** - * Delays coroutine for a given time without blocking a thread and resumes it after a specified time. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * immediately resumes with [CancellationException]. - * - * Note, that delay can be used in [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. - * - * This function delegates to [Delay.scheduleResumeAfterDelay] if the context [CoroutineDispatcher] - * implements [Delay] interface, otherwise it resumes using a built-in single-threaded scheduled executor service. - */ -suspend fun delay(time: Long, unit: TimeUnit = TimeUnit.MILLISECONDS) { - require(time >= 0) { "Delay time $time cannot be negative" } - if (time <= 0) return // don't delay - return suspendCancellableCoroutine sc@ { cont: CancellableContinuation -> - val delay = cont.context[ContinuationInterceptor] as? Delay - if (delay != null) - delay.scheduleResumeAfterDelay(time, unit, cont) else - cont.cancelFutureOnCompletion(scheduledExecutor.schedule(ResumeRunnable(cont), time, unit)) - } -} - -/** - * An implementation of [DisposableHandle] that cancels the specified future on dispose. - */ -public class DisposableFutureHandle(private val future: Future<*>) : DisposableHandle { - override fun dispose() { - future.cancel(false) - } - override fun toString(): String = "DisposableFutureHandle[$future]" -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/EventLoop.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/EventLoop.kt deleted file mode 100644 index 4e8ada9a10..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/EventLoop.kt +++ /dev/null @@ -1,201 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlinx.coroutines.experimental.internal.LockFreeLinkedListHead -import kotlinx.coroutines.experimental.internal.LockFreeLinkedListNode -import java.util.concurrent.ConcurrentSkipListMap -import java.util.concurrent.TimeUnit -import java.util.concurrent.atomic.AtomicLong -import java.util.concurrent.locks.LockSupport -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Implemented by [CoroutineDispatcher] implementations that have event loop inside and can - * be asked to process next event from their event queue. - * - * It may optionally implement [Delay] interface and support time-scheduled tasks. It is used by [runBlocking] to - * continue processing events when invoked from the event dispatch thread. - */ -public interface EventLoop { - /** - * Processes next event in this event loop. - * - * The result of this function is to be interpreted like this: - * * `<= 0` -- there are potentially more events for immediate processing; - * * `> 0` -- a number of nanoseconds to wait for next scheduled event; - * * [Long.MAX_VALUE] -- no more events, or was invoked from the wrong thread. - */ - public fun processNextEvent(): Long - - public companion object Factory { - /** - * Creates a new event loop that is bound the specified [thread] (current thread by default) and - * stops accepting new events when [parentJob] completes. Every continuation that is scheduled - * onto this event loop unparks the specified thread via [LockSupport.unpark]. - * - * The main event-processing loop using the resulting `eventLoop` object should look like this: - * ``` - * while (needsToBeRunning) { - * if (Thread.interrupted()) break // or handle somehow - * LockSupport.parkNanos(eventLoop.processNextEvent()) // event loop will unpark - * } - * ``` - */ - public operator fun invoke(thread: Thread = Thread.currentThread(), parentJob: Job? = null): CoroutineDispatcher = - EventLoopImpl(thread).apply { - if (parentJob != null) initParentJob(parentJob) - } - } -} - -internal class EventLoopImpl( - private val thread: Thread -) : CoroutineDispatcher(), EventLoop, Delay { - private val queue = LockFreeLinkedListHead() - private val delayed = ConcurrentSkipListMap() - private val nextSequence = AtomicLong() - private var parentJob: Job? = null - - fun initParentJob(coroutine: Job) { - require(this.parentJob == null) - this.parentJob = coroutine - } - - override fun dispatch(context: CoroutineContext, block: Runnable) { - if (scheduleQueued(QueuedRunnableTask(block))) { - unpark() - } else { - block.run() - } - } - - override fun scheduleResumeAfterDelay(time: Long, unit: TimeUnit, continuation: CancellableContinuation) { - if (scheduleDelayed(DelayedResumeTask(time, unit, continuation))) { - // todo: we should unpark only when this delayed task became first in the queue - unpark() - } else { - scheduledExecutor.schedule(ResumeRunnable(continuation), time, unit) - } - } - - override fun invokeOnTimeout(time: Long, unit: TimeUnit, block: Runnable): DisposableHandle = - DelayedRunnableTask(time, unit, block).also { scheduleDelayed(it) } - - override fun processNextEvent(): Long { - if (Thread.currentThread() !== thread) return Long.MAX_VALUE - // queue all delayed tasks that are due to be executed - while (true) { - val delayedTask = delayed.firstEntry()?.key ?: break - val now = System.nanoTime() - if (delayedTask.nanoTime - now > 0) break - if (!scheduleQueued(delayedTask)) break - delayed.remove(delayedTask) - } - // then process one event from queue - (queue.removeFirstOrNull() as? QueuedTask)?.let { queuedTask -> - queuedTask() - } - if (!queue.isEmpty) return 0 - val nextDelayedTask = delayed.firstEntry()?.key ?: return Long.MAX_VALUE - return nextDelayedTask.nanoTime - System.nanoTime() - } - - fun shutdown() { - // complete processing of all queued tasks - while (true) { - val queuedTask = (queue.removeFirstOrNull() ?: break) as QueuedTask - queuedTask() - } - // cancel all delayed tasks - while (true) { - val delayedTask = delayed.pollFirstEntry()?.key ?: break - delayedTask.cancel() - } - } - - private fun scheduleQueued(queuedTask: QueuedTask): Boolean { - if (parentJob == null) { - queue.addLast(queuedTask) - return true - } - return queue.addLastIf(queuedTask, { !parentJob!!.isCompleted }) - } - - private fun scheduleDelayed(delayedTask: DelayedTask): Boolean { - delayed.put(delayedTask, delayedTask) - if (parentJob?.isActive != false) return true - delayedTask.dispose() - return false - } - - private fun unpark() { - if (Thread.currentThread() !== thread) - LockSupport.unpark(thread) - } - - private abstract class QueuedTask : LockFreeLinkedListNode(), () -> Unit - - private class QueuedRunnableTask( - private val block: Runnable - ) : QueuedTask() { - override fun invoke() { block.run() } - } - - private abstract inner class DelayedTask( - time: Long, timeUnit: TimeUnit - ) : QueuedTask(), Comparable, DisposableHandle { - @JvmField val nanoTime: Long = System.nanoTime() + timeUnit.toNanos(time) - @JvmField val sequence: Long = nextSequence.getAndIncrement() - - override fun compareTo(other: DelayedTask): Int { - val dTime = nanoTime - other.nanoTime - if (dTime > 0) return 1 - if (dTime < 0) return -1 - val dSequence = sequence - other.sequence - return if (dSequence > 0) 1 else if (dSequence < 0) -1 else 0 - } - - override final fun dispose() { - delayed.remove(this) - cancel() - } - - open fun cancel() {} - } - - private inner class DelayedResumeTask( - time: Long, timeUnit: TimeUnit, - private val cont: CancellableContinuation - ) : DelayedTask(time, timeUnit) { - override fun invoke() { - with(cont) { resumeUndispatched(Unit) } - } - override fun cancel() { - if (!cont.isActive) return - val remaining = nanoTime - System.nanoTime() - scheduledExecutor.schedule(ResumeRunnable(cont), remaining, TimeUnit.NANOSECONDS) - } - } - - private inner class DelayedRunnableTask( - time: Long, timeUnit: TimeUnit, - private val block: Runnable - ) : DelayedTask(time, timeUnit) { - override fun invoke() { block.run() } - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Executors.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Executors.kt deleted file mode 100644 index caafc08b30..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Executors.kt +++ /dev/null @@ -1,81 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import java.util.concurrent.Executor -import java.util.concurrent.ScheduledExecutorService -import java.util.concurrent.TimeUnit -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Converts an instance of [Executor] to an implementation of [CoroutineDispatcher]. - * @suppress **Deprecated**: Renamed to [asCoroutineDispatcher]. - */ -@Deprecated("Renamed to `asCoroutineDispatcher`", - replaceWith = ReplaceWith("asCoroutineDispatcher()")) -public fun Executor.toCoroutineDispatcher(): CoroutineDispatcher = - ExecutorCoroutineDispatcher(this) - -/** - * Converts an instance of [Executor] to an implementation of [CoroutineDispatcher]. - */ -public fun Executor.asCoroutineDispatcher(): CoroutineDispatcher = - ExecutorCoroutineDispatcher(this) - -private class ExecutorCoroutineDispatcher(override val executor: Executor) : ExecutorCoroutineDispatcherBase() - -internal abstract class ExecutorCoroutineDispatcherBase : CoroutineDispatcher(), Delay { - abstract val executor: Executor - - override fun dispatch(context: CoroutineContext, block: Runnable) = executor.execute(block) - - override fun scheduleResumeAfterDelay(time: Long, unit: TimeUnit, continuation: CancellableContinuation) { - val timeout = (executor as? ScheduledExecutorService) - ?.schedule(ResumeUndispatchedRunnable(this, continuation), time, unit) - ?: scheduledExecutor.schedule(ResumeRunnable(continuation), time, unit) - continuation.cancelFutureOnCompletion(timeout) - } - - override fun invokeOnTimeout(time: Long, unit: TimeUnit, block: Runnable): DisposableHandle { - val timeout = (executor as? ScheduledExecutorService) - ?.schedule(block, time, unit) - ?: scheduledExecutor.schedule(block, time, unit) - return DisposableFutureHandle(timeout) - } - - override fun toString(): String = executor.toString() - override fun equals(other: Any?): Boolean = other is ExecutorCoroutineDispatcherBase && other.executor === executor - override fun hashCode(): Int = System.identityHashCode(executor) -} - -internal class ResumeRunnable( - private val continuation: Continuation -) : Runnable { - override fun run() { - continuation.resume(Unit) - } -} - -private class ResumeUndispatchedRunnable( - private val dispatcher: CoroutineDispatcher, - private val continuation: CancellableContinuation -) : Runnable { - override fun run() { - with(continuation) { dispatcher.resumeUndispatched(Unit) } - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Job.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Job.kt deleted file mode 100644 index c6bf0bc33b..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Job.kt +++ /dev/null @@ -1,850 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlinx.coroutines.experimental.internal.* -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import kotlinx.coroutines.experimental.selects.SelectBuilder -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlinx.coroutines.experimental.selects.select -import java.util.concurrent.Future -import java.util.concurrent.atomic.AtomicReferenceFieldUpdater -import kotlin.coroutines.experimental.AbstractCoroutineContextElement -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -// --------------- core job interfaces --------------- - -/** - * A background job. It is created with [launch] coroutine builder or with a - * [`Job()`][Job.Key.invoke] factory function. - * A job can be _cancelled_ at any time with [cancel] function that forces it to become _completed_ immediately. - * - * A job has two or three states: - * - * | **State** | [isActive] | [isCompleted] | - * | -------------------------------- | ---------- | ------------- | - * | _New_ (optional initial state) | `false` | `false` | - * | _Active_ (default initial state) | `true` | `false` | - * | _Completed_ (final state) | `false` | `true` | - * - * Usually, a job is created in _active_ state (it is created and started), so its only visible - * states are _active_ and _completed_. However, coroutine builders that provide an optional `start` parameter - * create a coroutine in _new_ state when this parameter is set to [CoroutineStart.LAZY]. Such a job can - * be made _active_ by invoking [start] or [join]. - * - * A job in the coroutine [context][CoroutineScope.context] represents the coroutine itself. - * A job is active while the coroutine is working and job's cancellation aborts the coroutine when - * the coroutine is suspended on a _cancellable_ suspension point by throwing [CancellationException] - * or the cancellation cause inside the coroutine. - * - * A job can have a _parent_. A job with a parent is cancelled when its parent completes. - * - * All functions on this interface and on all interfaces derived from it are **thread-safe** and can - * be safely invoked from concurrent coroutines without external synchronization. - */ -public interface Job : CoroutineContext.Element { - /** - * Key for [Job] instance in the coroutine context. - */ - public companion object Key : CoroutineContext.Key { - /** - * Creates a new job object in _active_ state. - * It is optionally a child of a [parent] job. - */ - public operator fun invoke(parent: Job? = null): Job = JobImpl(parent) - } - - /** - * Returns `true` when this job is active. - */ - public val isActive: Boolean - - /** - * Returns `true` when this job has completed for any reason. - */ - public val isCompleted: Boolean - - /** - * Starts coroutine related to this job (if any) if it was not started yet. - * The result `true` if this invocation actually started coroutine or `false` - * if it was already started or completed. - */ - public fun start(): Boolean - - /** - * Returns the exception that signals the completion of this job -- it returns the original - * [cancel] cause or an instance of [CancellationException] if this job had completed - * normally or was cancelled without a cause. This function throws - * [IllegalStateException] when invoked for an job that has not [completed][isCompleted] yet. - * - * The [cancellable][suspendCancellableCoroutine] suspending functions throw this exception - * when trying to suspend in the context of this job. - */ - fun getCompletionException(): Throwable - - /** - * Registers handler that is **synchronously** invoked on completion of this job. - * When job is already complete, then the handler is immediately invoked - * with a cancellation cause or `null`. Otherwise, handler will be invoked once when this - * job is complete. Note, that [cancellation][cancel] is also a form of completion. - * - * The resulting [DisposableHandle] can be used to [dispose][DisposableHandle.dispose] the - * registration of this handler and release its memory if its invocation is no longer needed. - * There is no need to dispose the handler after completion of this job. The reference to - * all the handlers are released when this job completes. - */ - public fun invokeOnCompletion(handler: CompletionHandler): DisposableHandle - - /** - * Suspends coroutine until this job is complete. This invocation resumes normally (without exception) - * when the job is complete for any reason. This function also [starts][Job.start] the corresponding coroutine - * if the [Job] was still in _new_ state. - * - * This suspending function is cancellable. If the [Job] of the invoking coroutine is completed while this - * suspending function is suspended, this function immediately resumes with [CancellationException]. - * - * This function can be used in [select] invocation with [onJoin][SelectBuilder.onJoin] clause. - * Use [isCompleted] to check for completion of this job without waiting. - */ - public suspend fun join() - - /** - * Registers [onJoin][SelectBuilder.onJoin] select clause. - * @suppress **This is unstable API and it is subject to change.** - */ - public fun registerSelectJoin(select: SelectInstance, block: suspend () -> R) - - /** - * Cancel this activity with an optional cancellation [cause]. The result is `true` if this job was - * cancelled as a result of this invocation and `false` otherwise - * (if it was already _completed_ or if it is [NonCancellable]). - * Repeated invocations of this function have no effect and always produce `false`. - * - * When cancellation has a clear reason in the code, an instance of [CancellationException] should be created - * at the corresponding original cancellation site and passed into this method to aid in debugging by providing - * both the context of cancellation and text description of the reason. - */ - public fun cancel(cause: Throwable? = null): Boolean - - /** - * @suppress **Error**: Operator '+' on two Job objects is meaningless. - * Job is a coroutine context element and `+` is a set-sum operator for coroutine contexts. - * The job to the right of `+` just replaces the job the left of `+`. - */ - @Suppress("DeprecatedCallableAddReplaceWith") - @Deprecated(message = "Operator '+' on two Job objects is meaningless. " + - "Job is a coroutine context element and `+` is a set-sum operator for coroutine contexts. " + - "The job to the right of `+` just replaces the job the left of `+`.", - level = DeprecationLevel.ERROR) - public operator fun plus(other: Job) = other - - /** - * Registration object for [invokeOnCompletion]. It can be used to [unregister] if needed. - * There is no need to unregister after completion. - * @suppress **Deprecated**: Replace with `DisposableHandle` - */ - @Deprecated(message = "Replace with `DisposableHandle`", - replaceWith = ReplaceWith("DisposableHandle")) - public interface Registration { - /** - * Unregisters completion handler. - * @suppress **Deprecated**: Replace with `dispose` - */ - @Deprecated(message = "Replace with `dispose`", - replaceWith = ReplaceWith("dispose()")) - public fun unregister() - } -} - -/** - * A handle to an allocated object that can be disposed to make it eligible for garbage collection. - */ -@Suppress("DEPRECATION") // todo: remove when Job.Registration is removed -public interface DisposableHandle : Job.Registration { - /** - * Disposes the corresponding object, making it eligible for garbage collection. - * Repeated invocation of this function has no effect. - */ - public fun dispose() - - /** - * Unregisters completion handler. - * @suppress **Deprecated**: Replace with `dispose` - */ - @Deprecated(message = "Replace with `dispose`", - replaceWith = ReplaceWith("dispose()")) - public override fun unregister() = dispose() -} - -/** - * Handler for [Job.invokeOnCompletion]. - */ -public typealias CompletionHandler = (Throwable?) -> Unit - -/** - * Thrown by cancellable suspending functions if the [Job] of the coroutine is cancelled while it is suspending. - */ -public typealias CancellationException = java.util.concurrent.CancellationException - -/** - * Unregisters a specified [registration] when this job is complete. - * - * This is a shortcut for the following code with slightly more efficient implementation (one fewer object created). - * ``` - * invokeOnCompletion { registration.unregister() } - * ``` - * @suppress: **Deprecated**: Renamed to `disposeOnCompletion`. - */ -@Deprecated(message = "Renamed to `disposeOnCompletion`", - replaceWith = ReplaceWith("disposeOnCompletion(registration)")) -public fun Job.unregisterOnCompletion(registration: DisposableHandle): DisposableHandle = - invokeOnCompletion(DisposeOnCompletion(this, registration)) - -/** - * Disposes a specified [handle] when this job is complete. - * - * This is a shortcut for the following code with slightly more efficient implementation (one fewer object created). - * ``` - * invokeOnCompletion { handle.dispose() } - * ``` - */ -public fun Job.disposeOnCompletion(handle: DisposableHandle): DisposableHandle = - invokeOnCompletion(DisposeOnCompletion(this, handle)) - -/** - * Cancels a specified [future] when this job is complete. - * - * This is a shortcut for the following code with slightly more efficient implementation (one fewer object created). - * ``` - * invokeOnCompletion { future.cancel(false) } - * ``` - */ -public fun Job.cancelFutureOnCompletion(future: Future<*>): DisposableHandle = - invokeOnCompletion(CancelFutureOnCompletion(this, future)) - -/** - * @suppress **Deprecated**: `join` is now a member function of `Job`. - */ -@Suppress("EXTENSION_SHADOWED_BY_MEMBER", "DeprecatedCallableAddReplaceWith") -@Deprecated(message = "`join` is now a member function of `Job`") -public suspend fun Job.join() = this.join() - -/** - * No-op implementation of [Job.Registration]. - */ -@Deprecated(message = "Replace with `NonDisposableHandle`", - replaceWith = ReplaceWith("NonDisposableHandle")) -typealias EmptyRegistration = NonDisposableHandle - -/** - * No-op implementation of [DisposableHandle]. - */ -public object NonDisposableHandle : DisposableHandle { - /** Does not do anything. */ - override fun dispose() {} - - /** Returns "NonDisposableHandle" string. */ - override fun toString(): String = "NonDisposableHandle" -} - -// --------------- utility classes to simplify job implementation - -/** - * A concrete implementation of [Job]. It is optionally a child to a parent job. - * This job is cancelled when the parent is complete, but not vise-versa. - * - * This is an open class designed for extension by more specific classes that might augment the - * state and mare store addition state information for completed jobs, like their result values. - * - * @param active when `true` the job is created in _active_ state, when `false` in _new_ state. See [Job] for details. - * @suppress **This is unstable API and it is subject to change.** - */ -public open class JobSupport(active: Boolean) : AbstractCoroutineContextElement(Job), Job { - /* - === Internal states === - - name state class public state description - ------ ------------ ------------ ----------- - EMPTY_N EmptyNew : New no completion listeners - EMPTY_A EmptyActive : Active no completion listeners - SINGLE JobNode : Active a single completion listener - SINGLE+ JobNode : Active a single completion listener + NodeList added as its next - LIST_N NodeList : New a list of listeners (promoted once, does not got back to EmptyNew) - LIST_A NodeList : Active a list of listeners (promoted once, does not got back to JobNode/EmptyActive) - FINAL_C Cancelled : Completed cancelled (final state) - FINAL_F Failed : Completed failed for other reason (final state) - FINAL_R : Completed produced some result - - === Transitions === - - New states Active states Inactive states - +---------+ +---------+ +----------+ - | EMPTY_N | --+-> | EMPTY_A | --+-> | FINAL_* | - +---------+ | +---------+ | +----------+ - | | | ^ | - | | V | | - | | +---------+ | - | | | SINGLE | --+ - | | +---------+ | - | | | | - | | V | - | | +---------+ | - | +-- | SINGLE+ | --+ - | +---------+ | - | | | - V V | - +---------+ +---------+ | - | LIST_N | ----> | LIST_A | --+ - +---------+ +---------+ - - This state machine and its transition matrix are optimized for the common case when job is created in active - state (EMPTY_A) and at most one completion listener is added to it during its life-time. - - Note, that the actual `_state` variable can also be a reference to atomic operation descriptor `OpDescriptor` - */ - - @Volatile - private var _state: Any? = if (active) EmptyActive else EmptyNew // shared objects while we have no listeners - - @Volatile - private var parentHandle: DisposableHandle? = null - - protected companion object { - private val STATE: AtomicReferenceFieldUpdater = - AtomicReferenceFieldUpdater.newUpdater(JobSupport::class.java, Any::class.java, "_state") - - fun stateToString(state: Any?): String = - if (state is Incomplete) - if (state.isActive) "Active" else "New" - else "Completed" - } - - /** - * Initializes parent job. - * It shall be invoked at most once after construction after all other initialization. - */ - public fun initParentJob(parent: Job?) { - check(parentHandle == null) - if (parent == null) { - parentHandle = NonDisposableHandle - return - } - // directly pass HandlerNode to parent scope to optimize one closure object (see makeNode) - val newRegistration = parent.invokeOnCompletion(ParentOnCompletion(parent, this)) - parentHandle = newRegistration - // now check our state _after_ registering (see updateState order of actions) - if (isCompleted) newRegistration.dispose() - } - - internal open fun onParentCompletion(cause: Throwable?) { - // if parent was completed with CancellationException then use it as the cause of our cancellation, too. - // however, we shall not use application specific exceptions here. So if parent crashes due to IOException, - // we cannot and should not cancel the child with IOException - cancel(cause as? CancellationException) - } - - /** - * Returns current state of this job. - */ - protected val state: Any? get() { - while (true) { // lock-free helping loop - val state = _state - if (state !is OpDescriptor) return state - state.perform(this) - } - } - - /** - * Updates current [state] of this job. - */ - protected fun updateState(expect: Any, update: Any?, mode: Int): Boolean { - if (!tryUpdateState(expect, update)) return false - completeUpdateState(expect, update, mode) - return true - } - - /** - * Tries to initiate update of the current [state] of this job. - */ - protected fun tryUpdateState(expect: Any, update: Any?): Boolean { - require(expect is Incomplete && update !is Incomplete) // only incomplete -> completed transition is allowed - if (!STATE.compareAndSet(this, expect, update)) return false - // Unregister from parent job - parentHandle?.dispose() // volatile read parentHandle _after_ state was updated - return true // continues in completeUpdateState - } - - /** - * Completes update of the current [state] of this job. - */ - protected fun completeUpdateState(expect: Any, update: Any?, mode: Int) { - // Invoke completion handlers - val cause = (update as? CompletedExceptionally)?.cause - var completionException: Throwable? = null - when (expect) { - // SINGLE/SINGLE+ state -- one completion handler (common case) - is JobNode<*> -> try { - expect.invoke(cause) - } catch (ex: Throwable) { - completionException = ex - } - // LIST state -- a list of completion handlers - is NodeList -> expect.forEach> { node -> - try { - node.invoke(cause) - } catch (ex: Throwable) { - completionException?.apply { addSuppressed(ex) } ?: run { completionException = ex } - } - - } - // otherwise -- do nothing (it was Empty*) - else -> check(expect is Empty) - } - // handle invokeOnCompletion exceptions - completionException?.let { handleCompletionException(it) } - // Do other (overridable) processing after completion handlers - afterCompletion(update, mode) - } - - public final override val isActive: Boolean get() { - val state = this.state - return state is Incomplete && state.isActive - } - - public final override val isCompleted: Boolean get() = state !is Incomplete - - // this is for `select` operator. `isSelected` state means "not new" (== was started or already completed) - public val isSelected: Boolean get() { - val state = this.state - return state !is Incomplete || state.isActive - } - - public final override fun start(): Boolean { - while (true) { // lock-free loop on state - when (startInternal(state)) { - 0 -> return false - 1 -> return true - } - } - } - - // return: 0 -> false (not new), 1 -> true (started), -1 -> retry - internal fun startInternal(state: Any?): Int { - when { - state === EmptyNew -> { // EMPTY_NEW state -- no completion handlers, new - if (!STATE.compareAndSet(this, state, EmptyActive)) return -1 - onStart() - return 1 - } - state is NodeList -> { // LIST -- a list of completion handlers (either new or active) - if (state.isActive) return 0 - if (!NodeList.ACTIVE.compareAndSet(state, null, NodeList.ACTIVE_STATE)) return -1 - onStart() - return 1 - } - else -> return 0 // not a new state - } - } - - // it is just like start(), but support idempotent start - public fun trySelect(idempotent: Any?): Boolean { - if (idempotent == null) return start() // non idempotent -- use plain start - check(idempotent !is OpDescriptor) { "cannot use OpDescriptor as idempotent marker"} - while (true) { // lock-free loop on state - val state = this.state - when { - state === EmptyNew -> { // EMPTY_NEW state -- no completion handlers, new - // try to promote it to list in new state - STATE.compareAndSet(this, state, NodeList(active = false)) - } - state is NodeList -> { // LIST -- a list of completion handlers (either new or active) - val active = state.active - if (active === idempotent) return true // was activated with the same marker --> true - if (active != null) return false - if (NodeList.ACTIVE.compareAndSet(state, null, idempotent)) { - onStart() - return true - } - } - state is CompletedIdempotentStart -> { // remembers idempotent start token - return state.idempotentStart === idempotent - } - else -> return false - } - } - } - - public fun performAtomicTrySelect(desc: AtomicDesc): Any? = AtomicSelectOp(desc, true).perform(null) - public fun performAtomicIfNotSelected(desc: AtomicDesc): Any? = AtomicSelectOp(desc, false).perform(null) - - private inner class AtomicSelectOp( - @JvmField val desc: AtomicDesc, - @JvmField val activate: Boolean - ) : AtomicOp () { - override fun prepare(): Any? = prepareIfNotSelected() ?: desc.prepare(this) - - override fun complete(affected: Any?, failure: Any?) { - completeSelect(failure) - desc.complete(this, failure) - } - - fun prepareIfNotSelected(): Any? { - while (true) { // lock-free loop on state - val state = _state - when { - state === this@AtomicSelectOp -> return null // already in progress - state is OpDescriptor -> state.perform(this@JobSupport) // help - state === EmptyNew -> { // EMPTY_NEW state -- no completion handlers, new - if (STATE.compareAndSet(this@JobSupport, state, this@AtomicSelectOp)) return null // success - } - state is NodeList -> { // LIST -- a list of completion handlers (either new or active) - val active = state._active - when { - active == null -> { - if (NodeList.ACTIVE.compareAndSet(state, null, this@AtomicSelectOp)) return null // success - } - active === this@AtomicSelectOp -> return null // already in progress - active is OpDescriptor -> active.perform(state) // help - else -> return ALREADY_SELECTED // active state - } - } - else -> return ALREADY_SELECTED // not a new state - } - } - } - - private fun completeSelect(failure: Any?) { - val success = failure == null - val state = _state - when { - state === this -> { - val update = if (success && activate) EmptyActive else EmptyNew - if (STATE.compareAndSet(this@JobSupport, this, update)) { - if (success) onStart() - } - } - state is NodeList -> { // LIST -- a list of completion handlers (either new or active) - if (state._active === this) { - val update = if (success && activate) NodeList.ACTIVE_STATE else null - if (NodeList.ACTIVE.compareAndSet(state, this, update)) { - if (success) onStart() - } - } - } - } - - } - } - - /** - * Override to provide the actual [start] action. - */ - protected open fun onStart() {} - - final override fun getCompletionException(): Throwable { - val state = this.state - return when (state) { - is Incomplete -> throw IllegalStateException("Job has not completed yet") - is CompletedExceptionally -> state.exception - else -> CancellationException("Job has completed normally") - } - } - - final override fun invokeOnCompletion(handler: CompletionHandler): DisposableHandle { - var nodeCache: JobNode<*>? = null - while (true) { // lock-free loop on state - val state = this.state - when { - state === EmptyActive -> { // EMPTY_ACTIVE state -- no completion handlers, active - // try move to SINGLE state - val node = nodeCache ?: makeNode(handler).also { nodeCache = it } - if (STATE.compareAndSet(this, state, node)) return node - } - state === EmptyNew -> { // EMPTY_NEW state -- no completion handlers, new - // try to promote it to list in new state - STATE.compareAndSet(this, state, NodeList(active = false)) - } - state is JobNode<*> -> { // SINGLE/SINGLE+ state -- one completion handler - // try to promote it to list (SINGLE+ state) - state.addOneIfEmpty(NodeList(active = true)) - // it must be in SINGLE+ state or state has changed (node could have need removed from state) - val list = state.next // either NodeList or somebody else won the race, updated state - // just attempt converting it to list if state is still the same, then continue lock-free loop - STATE.compareAndSet(this, state, list) - } - state is NodeList -> { // LIST -- a list of completion handlers (either new or active) - val node = nodeCache ?: makeNode(handler).also { nodeCache = it } - if (state.addLastIf(node) { this.state === state }) return node - } - else -> { // is inactive - handler((state as? CompletedExceptionally)?.exception) - return NonDisposableHandle - } - } - } - } - - final override suspend fun join() { - while (true) { // lock-free loop on state - val state = this.state as? Incomplete ?: return // fast-path - no need to wait - if (startInternal(state) >= 0) break // break unless needs to retry - } - return joinSuspend() // slow-path - } - - private suspend fun joinSuspend() = suspendCancellableCoroutine { cont -> - cont.disposeOnCompletion(invokeOnCompletion(ResumeOnCompletion(this, cont))) - } - - override fun registerSelectJoin(select: SelectInstance, block: suspend () -> R) { - // fast-path -- check state and select/return if needed - while (true) { - if (select.isSelected) return - val state = this.state - if (state !is Incomplete) { - // already complete -- select result - if (select.trySelect(idempotent = null)) - block.startCoroutineUndispatched(select.completion) - return - } - if (startInternal(state) == 0) { - // slow-path -- register waiter for completion - select.disposeOnSelect(invokeOnCompletion(SelectJoinOnCompletion(this, select, block))) - return - } - } - } - - internal fun removeNode(node: JobNode<*>) { - // remove logic depends on the state of the job - while (true) { // lock-free loop on job state - val state = this.state - when (state) { - // SINGE/SINGLE+ state -- one completion handler - is JobNode<*> -> { - if (state !== this) return // a different job node --> we were already removed - // try remove and revert back to empty state - if (STATE.compareAndSet(this, state, EmptyActive)) return - } - // LIST -- a list of completion handlers - is NodeList -> { - // remove node from the list - node.remove() - return - } - // it is inactive or Empty* (does not have any completion handlers) - else -> return - } - } - } - - final override fun cancel(cause: Throwable?): Boolean { - while (true) { // lock-free loop on state - val state = this.state as? Incomplete ?: return false // quit if already complete - if (updateState(state, Cancelled(state.idempotentStart, cause), mode = 0)) return true - } - } - - /** - * Override to process any exceptions that were encountered while invoking [invokeOnCompletion] handlers. - */ - protected open fun handleCompletionException(closeException: Throwable) { - throw closeException - } - - /** - * Override for post-completion actions that need to do something with the state. - */ - protected open fun afterCompletion(state: Any?, mode: Int) {} - - private fun makeNode(handler: CompletionHandler): JobNode<*> = - (handler as? JobNode<*>)?.also { require(it.job === this) } - ?: InvokeOnCompletion(this, handler) - - // for nicer debugging - override fun toString(): String = "${this::class.java.simpleName}{${stateToString(state)}}@${Integer.toHexString(System.identityHashCode(this))}" - - /** - * Interface for incomplete [state] of a job. - */ - public interface Incomplete { - val isActive: Boolean - val idempotentStart: Any? // != null if this state is a descendant of trySelect(idempotent) - } - - private class NodeList( - active: Boolean - ) : LockFreeLinkedListHead(), Incomplete { - @Volatile - @JvmField - var _active: Any? = if (active) ACTIVE_STATE else null - - val active: Any? get() { - while (true) { // helper loop for atomic ops - val active = this._active - if (active !is OpDescriptor) return active - active.perform(this) - } - } - - override val isActive: Boolean get() = active != null - - override val idempotentStart: Any? get() { - val active = this.active - return if (active === ACTIVE_STATE) null else active - } - - companion object { - @JvmField - val ACTIVE: AtomicReferenceFieldUpdater = - AtomicReferenceFieldUpdater.newUpdater(NodeList::class.java, Any::class.java, "_active") - - @JvmField - val ACTIVE_STATE = Symbol("ACTIVE_STATE") - } - - override fun toString(): String = buildString { - append("List") - append(if (isActive) "{Active}" else "{New}") - append("[") - var first = true - this@NodeList.forEach> { node -> - if (first) first = false else append(", ") - append(node) - } - append("]") - } - } - - public open class CompletedIdempotentStart( - @JvmField val idempotentStart: Any? - ) - - /** - * Class for a [state] of a job that had completed exceptionally, including cancellation. - * - * @param cause the exceptional completion cause. If `cause` is null, then a [CancellationException] - * if created on first get from [exception] property. - */ - public open class CompletedExceptionally( - idempotentStart: Any?, - @JvmField val cause: Throwable? - ) : CompletedIdempotentStart(idempotentStart) { - @Volatile - private var _exception: Throwable? = cause // materialize CancellationException on first need - - /** - * Returns completion exception. - */ - public val exception: Throwable get() = - _exception ?: // atomic read volatile var or else create new - CancellationException("Job was cancelled").also { _exception = it } - - override fun toString(): String = "${this::class.java.simpleName}[$exception]" - } - - /** - * A specific subclass of [CompletedExceptionally] for cancelled jobs. - */ - public class Cancelled( - idempotentStart: Any?, - cause: Throwable? - ) : CompletedExceptionally(idempotentStart, cause) -} - -internal val ALREADY_SELECTED: Any = Symbol("ALREADY_SELECTED") - -private val EmptyNew = Empty(false) -private val EmptyActive = Empty(true) - -private class Empty(override val isActive: Boolean) : JobSupport.Incomplete { - override val idempotentStart: Any? get() = null - override fun toString(): String = "Empty{${if (isActive) "Active" else "New" }}" -} - -internal abstract class JobNode( - @JvmField val job: J -) : LockFreeLinkedListNode(), DisposableHandle, CompletionHandler, JobSupport.Incomplete { - final override val isActive: Boolean get() = true - final override val idempotentStart: Any? get() = null - // if unregister is called on this instance, then Job was an instance of JobSupport that added this node it itself - // directly without wrapping - final override fun dispose() = (job as JobSupport).removeNode(this) - override abstract fun invoke(reason: Throwable?) -} - -private class InvokeOnCompletion( - job: Job, - @JvmField val handler: CompletionHandler -) : JobNode(job) { - override fun invoke(reason: Throwable?) = handler.invoke(reason) - override fun toString() = "InvokeOnCompletion[${handler::class.java.name}@${Integer.toHexString(System.identityHashCode(handler))}]" -} - -private class ResumeOnCompletion( - job: Job, - @JvmField val continuation: Continuation -) : JobNode(job) { - override fun invoke(reason: Throwable?) = continuation.resume(Unit) - override fun toString() = "ResumeOnCompletion[$continuation]" -} - -internal class DisposeOnCompletion( - job: Job, - @JvmField val handle: DisposableHandle -) : JobNode(job) { - override fun invoke(reason: Throwable?) = handle.dispose() - override fun toString(): String = "DisposeOnCompletion[$handle]" -} - -private class ParentOnCompletion( - parentJob: Job, - @JvmField val subordinateJob: JobSupport -) : JobNode(parentJob) { - override fun invoke(reason: Throwable?) { subordinateJob.onParentCompletion(reason) } - override fun toString(): String = "ParentOnCompletion[$subordinateJob]" -} - -private class CancelFutureOnCompletion( - job: Job, - @JvmField val future: Future<*> -) : JobNode(job) { - override fun invoke(reason: Throwable?) { - // Don't interrupt when cancelling future on completion, because no one is going to reset this - // interruption flag and it will cause spurious failures elsewhere - future.cancel(false) - } - override fun toString() = "CancelFutureOnCompletion[$future]" -} - -private class SelectJoinOnCompletion( - job: JobSupport, - private val select: SelectInstance, - private val block: suspend () -> R -) : JobNode(job) { - override fun invoke(reason: Throwable?) { - if (select.trySelect(idempotent = null)) - block.startCoroutine(select.completion) - } - override fun toString(): String = "SelectJoinOnCompletion[$select]" -} - -private class JobImpl(parent: Job? = null) : JobSupport(true) { - init { initParentJob(parent) } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/LazyDeferred.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/LazyDeferred.kt deleted file mode 100644 index 4efa86be98..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/LazyDeferred.kt +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlin.coroutines.experimental.CoroutineContext - -/** - * @suppress **Deprecated**: `Deferred` incorporates functionality of `LazyDeferred`. See [Deferred]. - */ -@Deprecated(message = "`Deferred` incorporates functionality of `LazyDeferred`", level = DeprecationLevel.WARNING, - replaceWith = ReplaceWith("Deferred")) -typealias LazyDeferred = Deferred - -/** - * @suppress **Deprecated**: Replace with `async(context, start = false) { ... }`. See [async]. - */ -@Deprecated(message = "This functionality is incorporated into `async", level = DeprecationLevel.WARNING, - replaceWith = ReplaceWith("async(context, start = false, block = block)")) -public fun lazyDefer(context: CoroutineContext, block: suspend CoroutineScope.() -> T) : Deferred = - async(context, start = false, block = block) diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/NonCancellable.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/NonCancellable.kt deleted file mode 100644 index 801d7412bf..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/NonCancellable.kt +++ /dev/null @@ -1,65 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlinx.coroutines.experimental.NonCancellable.isActive -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlin.coroutines.experimental.AbstractCoroutineContextElement - -/** - * A non-cancelable job that is always [active][isActive]. It is designed to be used with [run] builder - * to prevent cancellation of code blocks that need to run without cancellation. - * - * Use it like this: - * ``` - * run(NonCancellable) { - * // this code will not be cancelled - * } - * ``` - */ -object NonCancellable : AbstractCoroutineContextElement(Job), Job { - /** Always returns `true`. */ - override val isActive: Boolean get() = true - - /** Always returns `false`. */ - override val isCompleted: Boolean get() = false - - /** Always returns `false`. */ - override fun start(): Boolean = false - - /** Always throws [UnsupportedOperationException]. */ - suspend override fun join() { - throw UnsupportedOperationException("This job is always active") - } - - /** - * Always throws [UnsupportedOperationException]. - * @suppress **This is unstable API and it is subject to change.** - */ - override fun registerSelectJoin(select: SelectInstance, block: suspend () -> R) { - throw UnsupportedOperationException("This job is always active") - } - - /** Always throws [IllegalStateException]. */ - override fun getCompletionException(): CancellationException = throw IllegalStateException("This job is always active") - - /** Always returns [NonDisposableHandle]. */ - override fun invokeOnCompletion(handler: CompletionHandler): DisposableHandle = NonDisposableHandle - - /** Always returns `false`. */ - override fun cancel(cause: Throwable?): Boolean = false -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Scheduled.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Scheduled.kt deleted file mode 100644 index c58c233039..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Scheduled.kt +++ /dev/null @@ -1,171 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlinx.coroutines.experimental.selects.SelectBuilder -import kotlinx.coroutines.experimental.selects.select -import java.util.concurrent.ScheduledExecutorService -import java.util.concurrent.ScheduledThreadPoolExecutor -import java.util.concurrent.TimeUnit -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.ContinuationInterceptor -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.intrinsics.startCoroutineUninterceptedOrReturn -import kotlin.coroutines.experimental.intrinsics.suspendCoroutineOrReturn - -private val KEEP_ALIVE = java.lang.Long.getLong("kotlinx.coroutines.ScheduledExecutor.keepAlive", 1000L) - -@Volatile -private var _scheduledExecutor: ScheduledExecutorService? = null - -internal val scheduledExecutor: ScheduledExecutorService get() = - _scheduledExecutor ?: getOrCreateScheduledExecutorSync() - -@Synchronized -private fun getOrCreateScheduledExecutorSync(): ScheduledExecutorService = - _scheduledExecutor ?: ScheduledThreadPoolExecutor(1) { r -> - Thread(r, "kotlinx.coroutines.ScheduledExecutor").apply { isDaemon = true } - }.apply { - setKeepAliveTime(KEEP_ALIVE, TimeUnit.MILLISECONDS) - allowCoreThreadTimeOut(true) - executeExistingDelayedTasksAfterShutdownPolicy = false - // "setRemoveOnCancelPolicy" is available only since JDK7, so try it via reflection - try { - val m = this::class.java.getMethod("setRemoveOnCancelPolicy", Boolean::class.javaPrimitiveType) - m.invoke(this, true) - } catch (ex: Throwable) { /* ignore */ } - _scheduledExecutor = this - } - -// used for tests -@Synchronized -internal fun scheduledExecutorShutdownNow() { - _scheduledExecutor?.shutdownNow() -} - -@Synchronized -internal fun scheduledExecutorShutdownNowAndRelease() { - _scheduledExecutor?.apply { - shutdownNow() - _scheduledExecutor = null - } -} - -/** - * Runs a given suspending [block] of code inside a coroutine with a specified timeout and throws - * [CancellationException] if timeout was exceeded. - * - * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of - * cancellable suspending function inside the block throws [CancellationException], so normally that exception, - * if uncaught, also gets thrown by `withTimeout` as a result. - * However, the code in the block can suppresses [CancellationException]. - * - * The sibling function that does not throw exception on timeout is [withTimeoutOrNull]. - * Note, that timeout action can be specified for [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. - * - * This function delegates to [Delay.invokeOnTimeout] if the context [CoroutineDispatcher] - * implements [Delay] interface, otherwise it tracks time using a built-in single-threaded scheduled executor service. - * - * @param time timeout time - * @param unit timeout unit (milliseconds by default) - */ -public suspend fun withTimeout(time: Long, unit: TimeUnit = TimeUnit.MILLISECONDS, block: suspend () -> T): T { - require(time >= 0) { "Timeout time $time cannot be negative" } - if (time <= 0L) throw CancellationException("Timed out immediately") - return suspendCoroutineOrReturn { cont: Continuation -> - val context = cont.context - val coroutine = TimeoutExceptionCoroutine(time, unit, cont) - val delay = context[ContinuationInterceptor] as? Delay - // schedule cancellation of this coroutine on time - if (delay != null) - coroutine.disposeOnCompletion(delay.invokeOnTimeout(time, unit, coroutine)) else - coroutine.cancelFutureOnCompletion(scheduledExecutor.schedule(coroutine, time, unit)) - coroutine.initParentJob(context[Job]) - // restart block using new coroutine with new job, - // however start it as undispatched coroutine, because we are already in the proper context - block.startCoroutineUninterceptedOrReturn(coroutine) - } -} - -private class TimeoutExceptionCoroutine( - private val time: Long, - private val unit: TimeUnit, - private val cont: Continuation -) : JobSupport(active = true), Runnable, Continuation { - override val context: CoroutineContext = cont.context + this // mix in this Job into the context - override fun run() { cancel(TimeoutException(time, unit)) } - override fun resume(value: T) { cont.resumeDirect(value) } - override fun resumeWithException(exception: Throwable) { cont.resumeDirectWithException(exception) } -} - -/** - * Runs a given suspending block of code inside a coroutine with a specified timeout and returns - * `null` if timeout was exceeded. - * - * The code that is executing inside the [block] is cancelled on timeout and the active or next invocation of - * cancellable suspending function inside the block throws [CancellationException]. Normally that exception, - * if uncaught by the block, gets converted into the `null` result of `withTimeoutOrNull`. - * However, the code in the block can suppresses [CancellationException]. - * - * The sibling function that throws exception on timeout is [withTimeout]. - * Note, that timeout action can be specified for [select] invocation with [onTimeout][SelectBuilder.onTimeout] clause. - * - * This function delegates to [Delay.invokeOnTimeout] if the context [CoroutineDispatcher] - * implements [Delay] interface, otherwise it tracks time using a built-in single-threaded scheduled executor service. - * - * @param time timeout time - * @param unit timeout unit (milliseconds by default) - */ -public suspend fun withTimeoutOrNull(time: Long, unit: TimeUnit = TimeUnit.MILLISECONDS, block: suspend () -> T): T? { - require(time >= 0) { "Timeout time $time cannot be negative" } - if (time <= 0L) return null - return suspendCoroutineOrReturn { cont: Continuation -> - val context = cont.context - val coroutine = TimeoutNullCoroutine(time, unit, cont) - val delay = context[ContinuationInterceptor] as? Delay - // schedule cancellation of this coroutine on time - if (delay != null) - coroutine.disposeOnCompletion(delay.invokeOnTimeout(time, unit, coroutine)) else - coroutine.cancelFutureOnCompletion(scheduledExecutor.schedule(coroutine, time, unit)) - coroutine.initParentJob(context[Job]) - // restart block using new coroutine with new job, - // however start it as undispatched coroutine, because we are already in the proper context - try { - block.startCoroutineUninterceptedOrReturn(coroutine) - } catch (e: TimeoutException) { - null // replace inner timeout exception with null result - } - } -} - -private class TimeoutNullCoroutine( - private val time: Long, - private val unit: TimeUnit, - private val cont: Continuation -) : JobSupport(active = true), Runnable, Continuation { - override val context: CoroutineContext = cont.context + this // mix in this Job into the context - override fun run() { cancel(TimeoutException(time, unit)) } - override fun resume(value: T) { cont.resumeDirect(value) } - override fun resumeWithException(exception: Throwable) { - // suppress inner timeout exception and replace it with null - if (exception is TimeoutException) - cont.resumeDirect(null) else - cont.resumeDirectWithException(exception) - } -} - -private class TimeoutException(time: Long, unit: TimeUnit) : CancellationException("Timed out waiting for $time $unit") \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/ThreadPoolDispatcher.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/ThreadPoolDispatcher.kt deleted file mode 100644 index bcf941e80a..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/ThreadPoolDispatcher.kt +++ /dev/null @@ -1,71 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import java.util.concurrent.Executors -import java.util.concurrent.ScheduledExecutorService -import java.util.concurrent.TimeUnit -import java.util.concurrent.atomic.AtomicInteger -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Creates new coroutine execution context with the a single thread and built-in [yield] and [delay] support. - * All continuations are dispatched immediately when invoked inside the thread of this context. - * Resources of this pool (its thread) are reclaimed when job of this context is cancelled. - * The specified [name] defines the name of the new thread. - * An optional [parent] job may be specified upon creation. - */ -fun newSingleThreadContext(name: String, parent: Job? = null): CoroutineContext = - newFixedThreadPoolContext(1, name, parent) - -/** - * Creates new coroutine execution context with the fixed-size thread-pool and built-in [yield] and [delay] support. - * All continuations are dispatched immediately when invoked inside the threads of this context. - * Resources of this pool (its threads) are reclaimed when job of this context is cancelled. - * The specified [name] defines the names of the threads. - * An optional [parent] job may be specified upon creation. - */ -fun newFixedThreadPoolContext(nThreads: Int, name: String, parent: Job? = null): CoroutineContext { - require(nThreads >= 1) { "Expected at least one thread, but $nThreads specified" } - val job = Job(parent) - return job + ThreadPoolDispatcher(nThreads, name, job) -} - -internal class PoolThread( - @JvmField val dispatcher: ThreadPoolDispatcher, // for debugging & tests - target: Runnable, name: String -) : Thread(target, name) { - init { isDaemon = true } -} - -internal class ThreadPoolDispatcher( - private val nThreads: Int, - private val name: String, - job: Job -) : ExecutorCoroutineDispatcherBase() { - private val threadNo = AtomicInteger() - - override val executor: ScheduledExecutorService = Executors.newScheduledThreadPool(nThreads) { target -> - PoolThread(this, target, if (nThreads == 1) name else name + "-" + threadNo.incrementAndGet()) - } - - init { - job.invokeOnCompletion { executor.shutdown() } - } - - override fun toString(): String = "ThreadPoolDispatcher[$nThreads, $name]" -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Yield.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Yield.kt deleted file mode 100644 index 014f555731..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/Yield.kt +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import kotlin.coroutines.experimental.intrinsics.COROUTINE_SUSPENDED -import kotlin.coroutines.experimental.intrinsics.suspendCoroutineOrReturn - -/** - * Yields a thread (or thread pool) of the current coroutine dispatcher to other coroutines to run. - * If the coroutine dispatcher does not have its own thread pool (like [Unconfined] dispatcher) then this - * function does nothing, but checks if the coroutine [Job] was completed. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed when this suspending function is invoked or while - * this function is waiting for dispatching, it resumes with [CancellationException]. - */ -suspend fun yield(): Unit = suspendCoroutineOrReturn sc@ { cont -> - val context = cont.context - val job = context[Job] - if (job != null && job.isCompleted) throw job.getCompletionException() - if (cont !is DispatchedContinuation) return@sc Unit - if (!cont.dispatcher.isDispatchNeeded(context)) return@sc Unit - cont.dispatchYield(job, Unit) - COROUTINE_SUSPENDED -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/AbstractChannel.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/AbstractChannel.kt deleted file mode 100644 index fb0396e2e2..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/AbstractChannel.kt +++ /dev/null @@ -1,879 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.internal.* -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlin.coroutines.experimental.startCoroutine - -/** - * Abstract send channel. It is a base class for all send channel implementations. - */ -public abstract class AbstractSendChannel : SendChannel { - /** @suppress **This is unstable API and it is subject to change.** */ - protected val queue = LockFreeLinkedListHead() - - // ------ extension points for buffered channels ------ - - /** - * Returns `true` if [isBufferFull] is always `true`. - * @suppress **This is unstable API and it is subject to change.** - */ - protected abstract val isBufferAlwaysFull: Boolean - - /** - * Returns `true` if this channel's buffer is full. - * @suppress **This is unstable API and it is subject to change.** - */ - protected abstract val isBufferFull: Boolean - - // ------ internal functions for override by buffered channels ------ - - /** - * Tries to add element to buffer or to queued receiver. - * Return type is `OFFER_SUCCESS | OFFER_FAILED | Closed`. - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun offerInternal(element: E): Any { - while (true) { - val receive = takeFirstReceiveOrPeekClosed() ?: return OFFER_FAILED - val token = receive.tryResumeReceive(element, idempotent = null) - if (token != null) { - receive.completeResumeReceive(token) - return receive.offerResult - } - } - } - - /** - * Tries to add element to buffer or to queued receiver if select statement clause was not selected yet. - * Return type is `ALREADY_SELECTED | OFFER_SUCCESS | OFFER_FAILED | Closed`. - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - // offer atomically with select - val offerOp = describeTryOffer(element) - val failure = select.performAtomicTrySelect(offerOp) - if (failure != null) return failure - val receive = offerOp.result - receive.completeResumeReceive(offerOp.resumeToken!!) - return receive.offerResult - } - - // ------ state functions & helpers for concrete implementations ------ - - /** - * Returns non-null closed token if it is last in the queue. - * @suppress **This is unstable API and it is subject to change.** - */ - protected val closedForSend: Closed<*>? get() = queue.prev as? Closed<*> - - /** - * Returns non-null closed token if it is first in the queue. - * @suppress **This is unstable API and it is subject to change.** - */ - protected val closedForReceive: Closed<*>? get() = queue.next as? Closed<*> - - /** - * Retrieves first sending waiter from the queue or returns closed token. - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun takeFirstSendOrPeekClosed(): Send? = - queue.removeFirstIfIsInstanceOfOrPeekIf { it is Closed<*> } - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun sendBuffered(element: E): Boolean = - queue.addLastIfPrev(SendBuffered(element), { it !is ReceiveOrClosed<*> }) - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun sendConflated(element: E): Boolean { - val node = SendBuffered(element) - if (!queue.addLastIfPrev(node, { it !is ReceiveOrClosed<*> })) return false - // remove previous SendBuffered - val prev = node.prev - if (prev is SendBuffered<*>) - prev.remove() - return true - } - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun describeSendBuffered(element: E): AddLastDesc<*> = SendBufferedDesc(queue, element) - - private open class SendBufferedDesc( - queue: LockFreeLinkedListHead, - element: E - ) : AddLastDesc>(queue, SendBuffered(element)) { - override fun failure(affected: LockFreeLinkedListNode, next: Any): Any? { - if (affected is ReceiveOrClosed<*>) return OFFER_FAILED - return null - } - } - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun describeSendConflated(element: E): AddLastDesc<*> = SendConflatedDesc(queue, element) - - private class SendConflatedDesc( - queue: LockFreeLinkedListHead, - element: E - ) : SendBufferedDesc(queue, element) { - override fun finishOnSuccess(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode) { - super.finishOnSuccess(affected, next) - // remove previous SendBuffered - if (affected is SendBuffered<*>) - affected.remove() - } - } - - // ------ SendChannel ------ - - public final override val isClosedForSend: Boolean get() = closedForSend != null - public final override val isFull: Boolean get() = queue.next !is ReceiveOrClosed<*> && isBufferFull - - public final override suspend fun send(element: E) { - // fast path -- try offer non-blocking - if (offer(element)) return - // slow-path does suspend - return sendSuspend(element) - } - - public final override fun offer(element: E): Boolean { - val result = offerInternal(element) - return when { - result === OFFER_SUCCESS -> true - result === OFFER_FAILED -> false - result is Closed<*> -> throw result.sendException - else -> error("offerInternal returned $result") - } - } - - private suspend fun sendSuspend(element: E): Unit = suspendCancellableCoroutine(true) sc@ { cont -> - val send = SendElement(element, cont) - loop@ while (true) { - if (enqueueSend(send)) { - cont.initCancellability() // make it properly cancellable - cont.removeOnCancel(send) - return@sc - } - // hm... something is not right. try to offer - val result = offerInternal(element) - when { - result === OFFER_SUCCESS -> { - cont.resume(Unit) - return@sc - } - result === OFFER_FAILED -> continue@loop - result is Closed<*> -> { - cont.resumeWithException(result.sendException) - return@sc - } - else -> error("offerInternal returned $result") - } - } - } - - private fun enqueueSend(send: SendElement) = - if (isBufferAlwaysFull) - queue.addLastIfPrev(send, { it !is ReceiveOrClosed<*> }) else - queue.addLastIfPrevAndIf(send, { it !is ReceiveOrClosed<*> }, { isBufferFull }) - - public override fun close(cause: Throwable?): Boolean { - val closed = Closed(cause) - while (true) { - val receive = takeFirstReceiveOrPeekClosed() - if (receive == null) { - // queue empty or has only senders -- try add last "Closed" item to the queue - if (queue.addLastIfPrev(closed, { it !is ReceiveOrClosed<*> })) { - afterClose(cause) - return true - } - continue // retry on failure - } - if (receive is Closed<*>) return false // already marked as closed -- nothing to do - receive as Receive // type assertion - receive.resumeReceiveClosed(closed) - } - } - - /** - * Invoked after successful [close]. - */ - protected open fun afterClose(cause: Throwable?) {} - - /** - * Retrieves first receiving waiter from the queue or returns closed token. - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun takeFirstReceiveOrPeekClosed(): ReceiveOrClosed? = - queue.removeFirstIfIsInstanceOfOrPeekIf>({ it is Closed<*> }) - - // ------ registerSelectSend ------ - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun describeTryOffer(element: E): TryOfferDesc = TryOfferDesc(element, queue) - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected class TryOfferDesc( - @JvmField val element: E, - queue: LockFreeLinkedListHead - ) : RemoveFirstDesc>(queue) { - @JvmField var resumeToken: Any? = null - - override fun failure(affected: LockFreeLinkedListNode, next: Any): Any? { - if (affected !is ReceiveOrClosed<*>) return OFFER_FAILED - if (affected is Closed<*>) return affected - return null - } - - override fun validatePrepared(node: ReceiveOrClosed): Boolean { - val token = node.tryResumeReceive(element, idempotent = this) ?: return false - resumeToken = token - return true - } - } - - private inner class TryEnqueueSendDesc( - element: E, - select: SelectInstance, - block: suspend () -> R - ) : AddLastDesc>(queue, SendSelect(element, select, block)) { - override fun failure(affected: LockFreeLinkedListNode, next: Any): Any? { - if (affected is ReceiveOrClosed<*>) { - return affected as? Closed<*> ?: ENQUEUE_FAILED - } - return null - } - - override fun onPrepare(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode): Any? { - if (!isBufferFull) return ENQUEUE_FAILED - return super.onPrepare(affected, next) - } - - override fun finishOnSuccess(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode) { - super.finishOnSuccess(affected, next) - // we can actually remove on select start, but this is also Ok (it'll get removed if discovered there) - node.disposeOnSelect() - } - } - - override fun registerSelectSend(select: SelectInstance, element: E, block: suspend () -> R) { - while (true) { - if (select.isSelected) return - if (isFull) { - val enqueueOp = TryEnqueueSendDesc(element, select, block) - val enqueueResult = select.performAtomicIfNotSelected(enqueueOp) ?: return - when { - enqueueResult === ALREADY_SELECTED -> return - enqueueResult === ENQUEUE_FAILED -> {} // retry - enqueueResult is Closed<*> -> throw enqueueResult.sendException - else -> error("performAtomicIfNotSelected(TryEnqueueSendDesc) returned $enqueueResult") - } - } else { - val offerResult = offerSelectInternal(element, select) - when { - offerResult === ALREADY_SELECTED -> return - offerResult === OFFER_FAILED -> {} // retry - offerResult === OFFER_SUCCESS -> { - block.startCoroutineUndispatched(select.completion) - return - } - offerResult is Closed<*> -> throw offerResult.sendException - else -> error("offerSelectInternal returned $offerResult") - } - } - } - } - - // ------ private ------ - - private class SendSelect( - override val pollResult: Any?, - @JvmField val select: SelectInstance, - @JvmField val block: suspend () -> R - ) : LockFreeLinkedListNode(), Send, DisposableHandle { - override fun tryResumeSend(idempotent: Any?): Any? = - if (select.trySelect(idempotent)) SELECT_STARTED else null - - override fun completeResumeSend(token: Any) { - check(token === SELECT_STARTED) - block.startCoroutine(select.completion) - } - - fun disposeOnSelect() { - select.disposeOnSelect(this) - } - - override fun dispose() { - remove() - } - - override fun toString(): String = "SendSelect($pollResult)[$select]" - } - - private class SendBuffered( - @JvmField val element: E - ) : LockFreeLinkedListNode(), Send { - override val pollResult: Any? get() = element - override fun tryResumeSend(idempotent: Any?): Any? = SEND_RESUMED - override fun completeResumeSend(token: Any) { check(token === SEND_RESUMED) } - } -} - -/** - * Abstract send/receive channel. It is a base class for all channel implementations. - */ -public abstract class AbstractChannel : AbstractSendChannel(), Channel { - // ------ extension points for buffered channels ------ - - /** - * Returns `true` if [isBufferEmpty] is always `true`. - * @suppress **This is unstable API and it is subject to change.** - */ - protected abstract val isBufferAlwaysEmpty: Boolean - - /** - * Returns `true` if this channel's buffer is empty. - * @suppress **This is unstable API and it is subject to change.** - */ - protected abstract val isBufferEmpty: Boolean - - // ------ internal functions for override by buffered channels ------ - - /** - * Tries to remove element from buffer or from queued sender. - * Return type is `E | POLL_FAILED | Closed` - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun pollInternal(): Any? { - while (true) { - val send = takeFirstSendOrPeekClosed() ?: return POLL_FAILED - val token = send.tryResumeSend(idempotent = null) - if (token != null) { - send.completeResumeSend(token) - return send.pollResult - } - } - } - - /** - * Tries to remove element from buffer or from queued sender if select statement clause was not selected yet. - * Return type is `ALREADY_SELECTED | E | POLL_FAILED | Closed` - * @suppress **This is unstable API and it is subject to change.** - */ - protected open fun pollSelectInternal(select: SelectInstance<*>): Any? { - // poll atomically with select - val pollOp = describeTryPoll() - val failure = select.performAtomicTrySelect(pollOp) - if (failure != null) return failure - val send = pollOp.result - send.completeResumeSend(pollOp.resumeToken!!) - return pollOp.pollResult - } - - // ------ state functions & helpers for concrete implementations ------ - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected val hasReceiveOrClosed: Boolean get() = queue.next is ReceiveOrClosed<*> - - // ------ ReceiveChannel ------ - - public final override val isClosedForReceive: Boolean get() = closedForReceive != null && isBufferEmpty - public final override val isEmpty: Boolean get() = queue.next !is Send && isBufferEmpty - - @Suppress("UNCHECKED_CAST") - public final override suspend fun receive(): E { - // fast path -- try poll non-blocking - val result = pollInternal() - if (result !== POLL_FAILED) return receiveResult(result) - // slow-path does suspend - return receiveSuspend() - } - - @Suppress("UNCHECKED_CAST") - private fun receiveResult(result: Any?): E { - if (result is Closed<*>) throw result.receiveException - return result as E - } - - @Suppress("UNCHECKED_CAST") - private suspend fun receiveSuspend(): E = suspendCancellableCoroutine(true) sc@ { cont -> - val receive = ReceiveElement(cont as CancellableContinuation, nullOnClose = false) - while (true) { - if (enqueueReceive(receive)) { - cont.initCancellability() // make it properly cancellable - removeReceiveOnCancel(cont, receive) - return@sc - } - // hm... something is not right. try to poll - val result = pollInternal() - if (result is Closed<*>) { - cont.resumeWithException(result.receiveException) - return@sc - } - if (result !== POLL_FAILED) { - cont.resume(result as E) - return@sc - } - } - } - - private fun enqueueReceive(receive: Receive): Boolean { - val result = if (isBufferAlwaysEmpty) - queue.addLastIfPrev(receive, { it !is Send }) else - queue.addLastIfPrevAndIf(receive, { it !is Send }, { isBufferEmpty }) - if (result) onEnqueuedReceive() - return result - } - - @Suppress("UNCHECKED_CAST") - public final override suspend fun receiveOrNull(): E? { - // fast path -- try poll non-blocking - val result = pollInternal() - if (result !== POLL_FAILED) return receiveOrNullResult(result) - // slow-path does suspend - return receiveOrNullSuspend() - } - - @Suppress("UNCHECKED_CAST") - private fun receiveOrNullResult(result: Any?): E? { - if (result is Closed<*>) { - if (result.closeCause != null) throw result.closeCause - return null - } - return result as E - } - - @Suppress("UNCHECKED_CAST") - private suspend fun receiveOrNullSuspend(): E? = suspendCancellableCoroutine(true) sc@ { cont -> - val receive = ReceiveElement(cont, nullOnClose = true) - while (true) { - if (enqueueReceive(receive)) { - cont.initCancellability() // make it properly cancellable - removeReceiveOnCancel(cont, receive) - return@sc - } - // hm... something is not right. try to poll - val result = pollInternal() - if (result is Closed<*>) { - if (result.closeCause == null) - cont.resume(null) - else - cont.resumeWithException(result.closeCause) - return@sc - } - if (result !== POLL_FAILED) { - cont.resume(result as E) - return@sc - } - } - } - - @Suppress("UNCHECKED_CAST") - public final override fun poll(): E? { - val result = pollInternal() - return if (result === POLL_FAILED) null else receiveOrNullResult(result) - } - - public final override fun iterator(): ChannelIterator = Itr(this) - - // ------ registerSelectReceive ------ - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected fun describeTryPoll(): TryPollDesc = TryPollDesc(queue) - - /** - * @suppress **This is unstable API and it is subject to change.** - */ - protected class TryPollDesc(queue: LockFreeLinkedListHead) : RemoveFirstDesc(queue) { - @JvmField var resumeToken: Any? = null - @JvmField var pollResult: E? = null - - override fun failure(affected: LockFreeLinkedListNode, next: Any): Any? { - if (affected is Closed<*>) return affected - if (affected !is Send) return POLL_FAILED - return null - } - - @Suppress("UNCHECKED_CAST") - override fun validatePrepared(node: Send): Boolean { - val token = node.tryResumeSend(this) ?: return false - resumeToken = token - pollResult = node.pollResult as E - return true - } - } - - private inner class TryEnqueueReceiveDesc( - select: SelectInstance, - block: suspend (E?) -> R, - nullOnClose: Boolean - ) : AddLastDesc>(queue, ReceiveSelect(select, block, nullOnClose)) { - override fun failure(affected: LockFreeLinkedListNode, next: Any): Any? { - if (affected is Send) return ENQUEUE_FAILED - return null - } - - override fun onPrepare(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode): Any? { - if (!isBufferEmpty) return ENQUEUE_FAILED - return super.onPrepare(affected, next) - } - - override fun finishOnSuccess(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode) { - super.finishOnSuccess(affected, next) - // notify the there is one more receiver - onEnqueuedReceive() - // we can actually remove on select start, but this is also Ok (it'll get removed if discovered there) - node.removeOnSelectCompletion() - } - } - - @Suppress("UNCHECKED_CAST") - override fun registerSelectReceive(select: SelectInstance, block: suspend (E) -> R) { - while (true) { - if (select.isSelected) return - if (isEmpty) { - val enqueueOp = TryEnqueueReceiveDesc(select, block as (suspend (E?) -> R), nullOnClose = false) - val enqueueResult = select.performAtomicIfNotSelected(enqueueOp) ?: return - when { - enqueueResult === ALREADY_SELECTED -> return - enqueueResult === ENQUEUE_FAILED -> {} // retry - else -> error("performAtomicIfNotSelected(TryEnqueueReceiveDesc) returned $enqueueResult") - } - } else { - val pollResult = pollSelectInternal(select) - when { - pollResult === ALREADY_SELECTED -> return - pollResult === POLL_FAILED -> {} // retry - pollResult is Closed<*> -> throw pollResult.receiveException - else -> { - block.startCoroutineUndispatched(pollResult as E, select.completion) - return - } - } - } - } - } - - @Suppress("UNCHECKED_CAST") - override fun registerSelectReceiveOrNull(select: SelectInstance, block: suspend (E?) -> R) { - while (true) { - if (select.isSelected) return - if (isEmpty) { - val enqueueOp = TryEnqueueReceiveDesc(select, block, nullOnClose = true) - val enqueueResult = select.performAtomicIfNotSelected(enqueueOp) ?: return - when { - enqueueResult === ALREADY_SELECTED -> return - enqueueResult === ENQUEUE_FAILED -> {} // retry - else -> error("performAtomicIfNotSelected(TryEnqueueReceiveDesc) returned $enqueueResult") - } - } else { - val pollResult = pollSelectInternal(select) - when { - pollResult === ALREADY_SELECTED -> return - pollResult === POLL_FAILED -> {} // retry - pollResult is Closed<*> -> { - if (pollResult.closeCause == null) { - if (select.trySelect(idempotent = null)) - block.startCoroutineUndispatched(null, select.completion) - return - } else - throw pollResult.closeCause - } - else -> { - // selected successfully - block.startCoroutineUndispatched(pollResult as E, select.completion) - return - } - } - } - } - } - - // ------ protected ------ - - /** - * Invoked when receiver is successfully enqueued to the queue of waiting receivers. - */ - protected open fun onEnqueuedReceive() {} - - /** - * Invoked when enqueued receiver was successfully cancelled. - */ - protected open fun onCancelledReceive() {} - - // ------ private ------ - - private fun removeReceiveOnCancel(cont: CancellableContinuation<*>, receive: Receive<*>) { - cont.invokeOnCompletion { - if (cont.isCancelled && receive.remove()) - onCancelledReceive() - } - } - - private class Itr(val channel: AbstractChannel) : ChannelIterator { - var result: Any? = POLL_FAILED // E | POLL_FAILED | Closed - - suspend override fun hasNext(): Boolean { - // check for repeated hasNext - if (result !== POLL_FAILED) return hasNextResult(result) - // fast path -- try poll non-blocking - result = channel.pollInternal() - if (result !== POLL_FAILED) return hasNextResult(result) - // slow-path does suspend - return hasNextSuspend() - } - - private fun hasNextResult(result: Any?): Boolean { - if (result is Closed<*>) { - if (result.closeCause != null) throw result.receiveException - return false - } - return true - } - - private suspend fun hasNextSuspend(): Boolean = suspendCancellableCoroutine(true) sc@ { cont -> - val receive = ReceiveHasNext(this, cont) - while (true) { - if (channel.enqueueReceive(receive)) { - cont.initCancellability() // make it properly cancellable - channel.removeReceiveOnCancel(cont, receive) - return@sc - } - // hm... something is not right. try to poll - val result = channel.pollInternal() - this.result = result - if (result is Closed<*>) { - if (result.closeCause == null) - cont.resume(false) - else - cont.resumeWithException(result.receiveException) - return@sc - } - if (result !== POLL_FAILED) { - cont.resume(true) - return@sc - } - } - } - - @Suppress("UNCHECKED_CAST") - suspend override fun next(): E { - val result = this.result - if (result is Closed<*>) throw result.receiveException - if (result !== POLL_FAILED) { - this.result = POLL_FAILED - return result as E - } - // rare case when hasNext was not invoked yet -- just delegate to receive (leave state as is) - return channel.receive() - } - } - - private class ReceiveElement( - @JvmField val cont: CancellableContinuation, - @JvmField val nullOnClose: Boolean - ) : Receive() { - override fun tryResumeReceive(value: E, idempotent: Any?): Any? = cont.tryResume(value, idempotent) - override fun completeResumeReceive(token: Any) = cont.completeResume(token) - override fun resumeReceiveClosed(closed: Closed<*>) { - if (closed.closeCause == null && nullOnClose) - cont.resume(null) - else - cont.resumeWithException(closed.receiveException) - } - override fun toString(): String = "ReceiveElement[$cont,nullOnClose=$nullOnClose]" - } - - private class ReceiveHasNext( - @JvmField val iterator: Itr, - @JvmField val cont: CancellableContinuation - ) : Receive() { - override fun tryResumeReceive(value: E, idempotent: Any?): Any? { - val token = cont.tryResume(true, idempotent) - if (token != null) { - /* - When idempotent != null this invocation can be stale and we cannot directly update iterator.result - Instead, we save both token & result into a temporary IdempotentTokenValue object and - set iterator result only in completeResumeReceive that is going to be invoked just once - */ - if (idempotent != null) return IdempotentTokenValue(token, value) - iterator.result = value - } - return token - } - - override fun completeResumeReceive(token: Any) { - if (token is IdempotentTokenValue<*>) { - iterator.result = token.value - cont.completeResume(token.token) - } else - cont.completeResume(token) - } - - override fun resumeReceiveClosed(closed: Closed<*>) { - val token = if (closed.closeCause == null) - cont.tryResume(false) - else - cont.tryResumeWithException(closed.receiveException) - if (token != null) { - iterator.result = closed - cont.completeResume(token) - } - } - override fun toString(): String = "ReceiveHasNext[$cont]" - } - - private inner class ReceiveSelect( - @JvmField val select: SelectInstance, - @JvmField val block: suspend (E?) -> R, - @JvmField val nullOnClose: Boolean - ) : Receive(), DisposableHandle { - override fun tryResumeReceive(value: E, idempotent: Any?): Any? = - if (select.trySelect(idempotent)) (value ?: NULL_VALUE) else null - - @Suppress("UNCHECKED_CAST") - override fun completeResumeReceive(token: Any) { - val value: E = (if (token === NULL_VALUE) null else token) as E - block.startCoroutine(value, select.completion) - } - - override fun resumeReceiveClosed(closed: Closed<*>) { - if (select.trySelect(idempotent = null)) { - if (closed.closeCause == null && nullOnClose) { - block.startCoroutine(null, select.completion) - } else - select.resumeSelectWithException(closed.receiveException, MODE_DISPATCHED) - } - } - - fun removeOnSelectCompletion() { - select.disposeOnSelect(this) - } - - override fun dispose() { // invoked on select completion - if (remove()) - onCancelledReceive() // notify cancellation of receive - } - - override fun toString(): String = "ReceiveSelect[$select,nullOnClose=$nullOnClose]" - } - - private class IdempotentTokenValue( - @JvmField val token: Any, - @JvmField val value: E - ) -} - -/** @suppress **This is unstable API and it is subject to change.** */ -@JvmField val OFFER_SUCCESS: Any = Symbol("OFFER_SUCCESS") - -/** @suppress **This is unstable API and it is subject to change.** */ -@JvmField val OFFER_FAILED: Any = Symbol("OFFER_FAILED") - -/** @suppress **This is unstable API and it is subject to change.** */ -@JvmField val POLL_FAILED: Any = Symbol("POLL_FAILED") - -/** @suppress **This is unstable API and it is subject to change.** */ -@JvmField val ENQUEUE_FAILED: Any = Symbol("ENQUEUE_FAILED") - -/** @suppress **This is unstable API and it is subject to change.** */ -@JvmField val SELECT_STARTED: Any = Symbol("SELECT_STARTED") - -/** @suppress **This is unstable API and it is subject to change.** */ -@JvmField val NULL_VALUE: Any = Symbol("NULL_VALUE") - -/** @suppress **This is unstable API and it is subject to change.** */ -@JvmField val CLOSE_RESUMED: Any = Symbol("CLOSE_RESUMED") - -/** @suppress **This is unstable API and it is subject to change.** */ -@JvmField val SEND_RESUMED = Symbol("SEND_RESUMED") - -/** - * Represents sending waiter in the queue. - * @suppress **This is unstable API and it is subject to change.** - */ -public interface Send { - val pollResult: Any? // E | Closed - fun tryResumeSend(idempotent: Any?): Any? - fun completeResumeSend(token: Any) -} - -/** - * Represents receiver waiter in the queue or closed token. - * @suppress **This is unstable API and it is subject to change.** - */ -public interface ReceiveOrClosed { - val offerResult: Any // OFFER_SUCCESS | Closed - fun tryResumeReceive(value: E, idempotent: Any?): Any? - fun completeResumeReceive(token: Any) -} - -/** - * Represents closed channel. - * @suppress **This is unstable API and it is subject to change.** - */ -@Suppress("UNCHECKED_CAST") -public class SendElement( - override val pollResult: Any?, - @JvmField val cont: CancellableContinuation -) : LockFreeLinkedListNode(), Send { - override fun tryResumeSend(idempotent: Any?): Any? = cont.tryResume(Unit, idempotent) - override fun completeResumeSend(token: Any) = cont.completeResume(token) - override fun toString(): String = "SendElement($pollResult)[$cont]" -} - -/** - * Represents closed channel. - * @suppress **This is unstable API and it is subject to change.** - */ -public class Closed( - @JvmField val closeCause: Throwable? -) : LockFreeLinkedListNode(), Send, ReceiveOrClosed { - val sendException: Throwable get() = closeCause ?: ClosedSendChannelException(DEFAULT_CLOSE_MESSAGE) - val receiveException: Throwable get() = closeCause ?: ClosedReceiveChannelException(DEFAULT_CLOSE_MESSAGE) - - override val offerResult get() = this - override val pollResult get() = this - override fun tryResumeSend(idempotent: Any?): Any? = CLOSE_RESUMED - override fun completeResumeSend(token: Any) { check(token === CLOSE_RESUMED) } - override fun tryResumeReceive(value: E, idempotent: Any?): Any? = throw sendException - override fun completeResumeReceive(token: Any) = throw sendException - override fun toString(): String = "Closed[$closeCause]" -} - -private abstract class Receive : LockFreeLinkedListNode(), ReceiveOrClosed { - override val offerResult get() = OFFER_SUCCESS - abstract fun resumeReceiveClosed(closed: Closed<*>) -} - diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Actor.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Actor.kt deleted file mode 100644 index 1b957168bf..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Actor.kt +++ /dev/null @@ -1,130 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Scope for [actor] coroutine builder. - */ -public interface ActorScope : CoroutineScope, ReceiveChannel { - /** - * A reference to the mailbox channel that this coroutine [receives][receive] messages from. - * It is provided for convenience, so that the code in the coroutine can refer - * to the channel as `channel` as apposed to `this`. - * All the [ReceiveChannel] functions on this interface delegate to - * the channel instance returned by this function. - */ - val channel: Channel -} - -/** - * Return type for [actor] coroutine builder. - */ -public interface ActorJob : Job, SendChannel { - /** - * A reference to the mailbox channel that this coroutine is receiving messages from. - * All the [SendChannel] functions on this interface delegate to - * the channel instance returned by this function. - */ - val channel: SendChannel -} - -/** - * Launches new coroutine that is receiving messages from its mailbox channel - * and returns a reference to the coroutine as an [ActorJob]. The resulting - * object can be used to [send][SendChannel.send] messages to this coroutine. - * - * The scope of the coroutine contains [ActorScope] interface, which implements - * both [CoroutineScope] and [ReceiveChannel], so that coroutine can invoke - * [receive][ReceiveChannel.receive] directly. The channel is [closed][SendChannel.close] - * when the coroutine completes. - * The running coroutine is cancelled when the its job is [cancelled][Job.cancel]. - * - * The [context] for the new coroutine must be explicitly specified. - * See [CoroutineDispatcher] for the standard [context] implementations that are provided by `kotlinx.coroutines`. - * The [context][CoroutineScope.context] of the parent coroutine from its [scope][CoroutineScope] may be used, - * in which case the [Job] of the resulting coroutine is a child of the job of the parent coroutine. - * - * By default, the coroutine is immediately started. - * An optional [start] parameter can be set to [CoroutineStart.LAZY] to start coroutine _lazily_. In this case, - * the coroutine [Job] is created in _new_ state. It can be explicitly started with [start][Job.start] function - * and will be started implicitly on the first invocation of [join][Job.join] or on a first message - * [sent][SendChannel.send] to this coroutine's mailbox channel. - * - * Uncaught exceptions in this coroutine close the channel with this exception as a cause and - * the resulting channel becomes _failed_, so that any attempt to send to such a channel throws exception. - * - * See [newCoroutineContext] for a description of debugging facilities that are available for newly created coroutine. - * - * @param context context of the coroutine - * @param capacity capacity of the channel's buffer (no buffer by default) - * @param start coroutine start option - * @param block the coroutine code - */ -public fun actor( - context: CoroutineContext, - capacity: Int = 0, - start: CoroutineStart = CoroutineStart.DEFAULT, - block: suspend ActorScope.() -> Unit -): ActorJob { - val newContext = newCoroutineContext(context) - val channel = Channel(capacity) - val coroutine = if (start.isLazy) - LazyActorCoroutine(newContext, channel, block) else - ActorCoroutine(newContext, channel, active = true) - coroutine.initParentJob(context[Job]) - start(block, coroutine, coroutine) - return coroutine -} - -private open class ActorCoroutine( - parentContext: CoroutineContext, - channel: Channel, - active: Boolean -) : ChannelCoroutine(parentContext, channel, active), ActorScope, ActorJob - -private class LazyActorCoroutine( - parentContext: CoroutineContext, - channel: Channel, - private val block: suspend ActorScope.() -> Unit -) : ActorCoroutine(parentContext, channel, active = false) { - override val channel: Channel get() = this - - override fun onStart() { - block.startCoroutine(this, this) - } - - suspend override fun send(element: E) { - start() - return super.send(element) - } - - override fun offer(element: E): Boolean { - start() - return super.offer(element) - } - - override fun registerSelectSend(select: SelectInstance, element: E, block: suspend () -> R) { - start() - return super.registerSelectSend(select, element, block) - } -} - diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ArrayBroadcastChannel.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ArrayBroadcastChannel.kt deleted file mode 100644 index 128e500fe5..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ArrayBroadcastChannel.kt +++ /dev/null @@ -1,346 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.ALREADY_SELECTED -import kotlinx.coroutines.experimental.selects.SelectInstance -import java.util.concurrent.CopyOnWriteArrayList -import java.util.concurrent.locks.ReentrantLock -import kotlin.concurrent.withLock - -/** - * Broadcast channel with array buffer of a fixed [capacity]. - * Sender suspends only when buffer is fully due to one of the receives not being late and - * receiver suspends only when buffer is empty. - * - * Note, that elements that are sent to the broadcast channel while there are no [open] subscribers are immediately - * lost. - * - * This implementation uses lock to protect the buffer, which is held only during very short buffer-update operations. - * The lock at each subscription is also used to manage concurrent attempts to receive from the same subscriber. - * The lists of suspended senders or receivers are lock-free. - */ -class ArrayBroadcastChannel( - /** - * Buffer capacity. - */ - val capacity: Int -) : AbstractSendChannel(), BroadcastChannel { - init { - check(capacity >= 1) { "ArrayBroadcastChannel capacity must be at least 1, but $capacity was specified" } - } - - private val bufferLock = ReentrantLock() - private val buffer: Array = arrayOfNulls(capacity) // guarded by lock - - // head & tail are Long (64 bits) and we assume that they never wrap around - // head, tail, and size are guarded by bufferLock - @Volatile - private var head: Long = 0 // do modulo on use of head - @Volatile - private var tail: Long = 0 // do modulo on use of tail - @Volatile - private var size: Int = 0 - - private val subs = CopyOnWriteArrayList>() - - override val isBufferAlwaysFull: Boolean get() = false - override val isBufferFull: Boolean get() = size >= capacity - - override fun open(): SubscriptionReceiveChannel { - val sub = Subscriber(this, head) - subs.add(sub) - // between creating and adding of subscription into the list the buffer head could have been bumped past it, - // so here we check if it did happen and update the head in subscription in this case - // we did not leak newly created subscription yet, so its subHead cannot update - val head = this.head // volatile read after sub was added to subs - if (head != sub.subHead) { - // needs update - sub.subHead = head - updateHead() // and also must recompute head of the buffer - } - return sub - } - - override fun close(cause: Throwable?): Boolean { - if (!super.close(cause)) return false - checkSubOffers() - return true - } - - // result is `OFFER_SUCCESS | OFFER_FAILED | Closed` - override fun offerInternal(element: E): Any { - bufferLock.withLock { - // check if closed for send (under lock, so size cannot change) - closedForSend?.let { return it } - val size = this.size - if (size >= capacity) return OFFER_FAILED - val tail = this.tail - buffer[(tail % capacity).toInt()] = element - this.size = size + 1 - this.tail = tail + 1 - } - // if offered successfully, then check subs outside of lock - checkSubOffers() - return OFFER_SUCCESS - } - - // result is `ALREADY_SELECTED | OFFER_SUCCESS | OFFER_FAILED | Closed` - override fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - bufferLock.withLock { - // check if closed for send (under lock, so size cannot change) - closedForSend?.let { return it } - val size = this.size - if (size >= capacity) return OFFER_FAILED - // let's try to select sending this element to buffer - if (!select.trySelect(null)) { // :todo: move trySelect completion outside of lock - return ALREADY_SELECTED - } - val tail = this.tail - buffer[(tail % capacity).toInt()] = element - this.size = size + 1 - this.tail = tail + 1 - } - // if offered successfully, then check subs outside of lock - checkSubOffers() - return OFFER_SUCCESS - } - - private fun closeSubscriber(sub: Subscriber) { - subs.remove(sub) - if (head == sub.subHead) - updateHead() - } - - private fun checkSubOffers() { - var updated = false - @Suppress("LoopToCallChain") // must invoke `checkOffer` on every sub - for (sub in subs) { - if (sub.checkOffer()) updated = true - } - if (updated) - updateHead() - } - - private fun updateHead() { - // compute minHead w/o lock (it will be eventually consistent) - val minHead = computeMinHead() - // update head in a loop - while (true) { - var send: Send? = null - var token: Any? = null - bufferLock.withLock { - val tail = this.tail - var head = this.head - val targetHead = minHead.coerceAtMost(tail) - if (targetHead <= head) return // nothing to do -- head was already moved - var size = this.size - // clean up removed (on not need if we don't have any subscribers anymore) - while (head < targetHead) { - buffer[(head % capacity).toInt()] = null - val wasFull = size >= capacity - // update the size before checking queue (no more senders can queue up) - this.head = ++head - this.size = --size - if (wasFull) { - while (true) { - send = takeFirstSendOrPeekClosed() ?: break // when when no sender - if (send is Closed<*>) break // break when closed for send - token = send!!.tryResumeSend(idempotent = null) - if (token != null) { - // put sent element to the buffer - buffer[(tail % capacity).toInt()] = (send as Send).pollResult - this.size = size + 1 - this.tail = tail + 1 - return@withLock // go out of lock to wakeup this sender - } - } - } - } - return // done updating here -> return - } - // we only get out of the lock normally when there is a sender to resume - send!!.completeResumeSend(token!!) - // since we've just sent an element, we might need to resume some receivers - checkSubOffers() - } - } - - private fun computeMinHead(): Long { - var minHead = Long.MAX_VALUE - for (sub in subs) - minHead = minHead.coerceAtMost(sub.subHead) // volatile (atomic) reads of subHead - return minHead - } - - @Suppress("UNCHECKED_CAST") - private fun elementAt(index: Long): E = buffer[(index % capacity).toInt()] as E - - private class Subscriber( - private val broadcastChannel: ArrayBroadcastChannel, - @Volatile @JvmField var subHead: Long // guarded by lock - ) : AbstractChannel(), SubscriptionReceiveChannel { - private val lock = ReentrantLock() - - override val isBufferAlwaysEmpty: Boolean get() = false - override val isBufferEmpty: Boolean get() = subHead >= broadcastChannel.tail - override val isBufferAlwaysFull: Boolean get() = error("Should not be used") - override val isBufferFull: Boolean get() = error("Should not be used") - - override fun close() { - if (close(cause = null)) - broadcastChannel.closeSubscriber(this) - } - - // returns true if subHead was updated and broadcast channel's head must be checked - // this method is lock-free (it never waits on lock) - @Suppress("UNCHECKED_CAST") - fun checkOffer(): Boolean { - var updated = false - var closed: Closed<*>? = null - loop@ - while (needsToCheckOfferWithoutLock()) { - // just use `tryLock` here and break when some other thread is checking under lock - // it means that `checkOffer` must be retried after every `unlock` - if (!lock.tryLock()) break - val receive: ReceiveOrClosed? - val token: Any? - try { - val result = peekUnderLock() - when { - result === POLL_FAILED -> continue@loop // must retest `needsToCheckOfferWithoutLock` outside of the lock - result is Closed<*> -> { - closed = result - break@loop // was closed - } - } - // find a receiver for an element - receive = takeFirstReceiveOrPeekClosed() ?: break // break when no one's receiving - if (receive is Closed<*>) break // noting more to do if this sub already closed - token = receive.tryResumeReceive(result as E, idempotent = null) - if (token == null) continue // bail out here to next iteration (see for next receiver) - val subHead = this.subHead - this.subHead = subHead + 1 // retrieved element for this subscriber - updated = true - } finally { - lock.unlock() - } - receive!!.completeResumeReceive(token!!) - } - // do close outside of lock if needed - closed?.also { close(cause = it.closeCause) } - return updated - } - - // result is `E | POLL_FAILED | Closed` - override fun pollInternal(): Any? { - var updated = false - val result: Any? - lock.lock() - try { - result = peekUnderLock() - when { - result is Closed<*> -> { /* just bail out of lock */ } - result === POLL_FAILED -> { /* just bail out of lock */ } - else -> { - // update subHead after retrieiving element from buffer - val subHead = this.subHead - this.subHead = subHead + 1 - updated = true - } - } - } finally { - lock.unlock() - } - // do close outside of lock - (result as? Closed<*>)?.also { close(cause = it.closeCause) } - // there could have been checkOffer attempt while we were holding lock - // now outside the lock recheck if anything else to offer - if (checkOffer()) - updated = true - // and finally update broadcast's channel head if needed - if (updated) - broadcastChannel.updateHead() - return result - } - - // result is `ALREADY_SELECTED | E | POLL_FAILED | Closed` - override fun pollSelectInternal(select: SelectInstance<*>): Any? { - var updated = false - var result: Any? - lock.lock() - try { - result = peekUnderLock() - when { - result is Closed<*> -> { /* just bail out of lock */ } - result === POLL_FAILED -> { /* just bail out of lock */ } - else -> { - // let's try to select receiving this element from buffer - if (!select.trySelect(null)) { // :todo: move trySelect completion outside of lock - result = ALREADY_SELECTED - } else { - // update subHead after retrieiving element from buffer - val subHead = this.subHead - this.subHead = subHead + 1 - updated = true - } - } - } - } finally { - lock.unlock() - } - // do close outside of lock - (result as? Closed<*>)?.also { close(cause = it.closeCause) } - // there could have been checkOffer attempt while we were holding lock - // now outside the lock recheck if anything else to offer - if (checkOffer()) - updated = true - // and finally update broadcast's channel head if needed - if (updated) - broadcastChannel.updateHead() - return result - } - - // Must invoke this check this after lock, because offer's invocation of `checkOffer` might have failed - // to `tryLock` just before the lock was about to unlocked, thus loosing notification to this - // subscription about an element that was just offered - private fun needsToCheckOfferWithoutLock(): Boolean { - if (closedForReceive != null) - return false // already closed -> nothing to do - if (isBufferEmpty && broadcastChannel.closedForReceive == null) - return false // no data for us && broadcast channel was not closed yet -> nothing to do - return true // check otherwise - } - - // guarded by lock, returns: - // E - the element from the buffer at subHead - // Closed<*> when closed; - // POLL_FAILED when there seems to be no data, but must retest `needsToCheckOfferWithoutLock` outside of lock - private fun peekUnderLock(): Any? { - val subHead = this.subHead // guarded read (can be non-volatile read) - // note: from the broadcastChannel we must read closed token first, then read its tail - // because it is Ok if tail moves in between the reads (we make decision based on tail first) - val closed = broadcastChannel.closedForReceive // unguarded volatile read - val tail = broadcastChannel.tail // unguarded volatile read - if (subHead >= tail) { - // no elements to poll from the queue -- check if closed - return closed ?: POLL_FAILED // must retest `needsToCheckOfferWithoutLock` outside of the lock - } - return broadcastChannel.elementAt(subHead) - } - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ArrayChannel.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ArrayChannel.kt deleted file mode 100644 index 6a7afc7cf5..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ArrayChannel.kt +++ /dev/null @@ -1,232 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.ALREADY_SELECTED -import kotlinx.coroutines.experimental.selects.SelectInstance -import java.util.concurrent.locks.ReentrantLock -import kotlin.concurrent.withLock - -/** - * Channel with array buffer of a fixed [capacity]. - * Sender suspends only when buffer is fully and receiver suspends only when buffer is empty. - * - * This implementation uses lock to protect the buffer, which is held only during very short buffer-update operations. - * The lists of suspended senders or receivers are lock-free. - */ -public open class ArrayChannel( - /** - * Buffer capacity. - */ - val capacity: Int -) : AbstractChannel() { - init { - check(capacity >= 1) { "ArrayChannel capacity must be at least 1, but $capacity was specified" } - } - - private val lock = ReentrantLock() - private val buffer: Array = arrayOfNulls(capacity) - private var head: Int = 0 - @Volatile - private var size: Int = 0 - - protected final override val isBufferAlwaysEmpty: Boolean get() = false - protected final override val isBufferEmpty: Boolean get() = size == 0 - protected final override val isBufferAlwaysFull: Boolean get() = false - protected final override val isBufferFull: Boolean get() = size == capacity - - // result is `OFFER_SUCCESS | OFFER_FAILED | Closed` - protected override fun offerInternal(element: E): Any { - var receive: ReceiveOrClosed? = null - var token: Any? = null - lock.withLock { - val size = this.size - closedForSend?.let { return it } - if (size < capacity) { - // tentatively put element to buffer - this.size = size + 1 // update size before checking queue (!!!) - // check for receivers that were waiting on empty queue - if (size == 0) { - loop@ while (true) { - receive = takeFirstReceiveOrPeekClosed() ?: break@loop // break when no receivers queued - if (receive is Closed) { - this.size = size // restore size - return receive!! - } - token = receive!!.tryResumeReceive(element, idempotent = null) - if (token != null) { - this.size = size // restore size - return@withLock - } - } - } - buffer[(head + size) % capacity] = element // actually queue element - return OFFER_SUCCESS - } - // size == capacity: full - return OFFER_FAILED - } - // breaks here if offer meets receiver - receive!!.completeResumeReceive(token!!) - return receive!!.offerResult - } - - // result is `ALREADY_SELECTED | OFFER_SUCCESS | OFFER_FAILED | Closed` - protected override fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - var receive: ReceiveOrClosed? = null - var token: Any? = null - lock.withLock { - val size = this.size - closedForSend?.let { return it } - if (size < capacity) { - // tentatively put element to buffer - this.size = size + 1 // update size before checking queue (!!!) - // check for receivers that were waiting on empty queue - if (size == 0) { - loop@ while (true) { - val offerOp = describeTryOffer(element) - val failure = select.performAtomicTrySelect(offerOp) - when { - failure == null -> { // offered successfully - this.size = size // restore size - receive = offerOp.result - token = offerOp.resumeToken - check(token != null) - return@withLock - } - failure === OFFER_FAILED -> break@loop // cannot offer -> Ok to queue to buffer - failure === ALREADY_SELECTED || failure is Closed<*> -> { - this.size = size // restore size - return failure - } - else -> error("performAtomicTrySelect(describeTryOffer) returned $failure") - } - } - } - // let's try to select sending this element to buffer - if (!select.trySelect(null)) { // :todo: move trySelect completion outside of lock - this.size = size // restore size - return ALREADY_SELECTED - } - buffer[(head + size) % capacity] = element // actually queue element - return OFFER_SUCCESS - } - // size == capacity: full - return OFFER_FAILED - } - // breaks here if offer meets receiver - receive!!.completeResumeReceive(token!!) - return receive!!.offerResult - } - - // result is `E | POLL_FAILED | Closed` - protected override fun pollInternal(): Any? { - var send: Send? = null - var token: Any? = null - var result: Any? = null - lock.withLock { - val size = this.size - if (size == 0) return closedForSend ?: POLL_FAILED // when nothing can be read from buffer - // size > 0: not empty -- retrieve element - result = buffer[head] - buffer[head] = null - this.size = size - 1 // update size before checking queue (!!!) - // check for senders that were waiting on full queue - var replacement: Any? = POLL_FAILED - if (size == capacity) { - loop@ while (true) { - send = takeFirstSendOrPeekClosed() ?: break - token = send!!.tryResumeSend(idempotent = null) - if (token != null) { - replacement = send!!.pollResult - break@loop - } - } - } - if (replacement !== POLL_FAILED && replacement !is Closed<*>) { - this.size = size // restore size - buffer[(head + size) % capacity] = replacement - } - head = (head + 1) % capacity - } - // complete send the we're taken replacement from - if (token != null) - send!!.completeResumeSend(token!!) - return result - } - - // result is `ALREADY_SELECTED | E | POLL_FAILED | Closed` - protected override fun pollSelectInternal(select: SelectInstance<*>): Any? { - var send: Send? = null - var token: Any? = null - var result: Any? = null - lock.withLock { - val size = this.size - if (size == 0) return closedForSend ?: POLL_FAILED - // size > 0: not empty -- retrieve element - result = buffer[head] - buffer[head] = null - this.size = size - 1 // update size before checking queue (!!!) - // check for senders that were waiting on full queue - var replacement: Any? = POLL_FAILED - if (size == capacity) { - loop@ while (true) { - val pollOp = describeTryPoll() - val failure = select.performAtomicTrySelect(pollOp) - when { - failure == null -> { // polled successfully - send = pollOp.result - token = pollOp.resumeToken - check(token != null) - replacement = send!!.pollResult - break@loop - } - failure === POLL_FAILED -> break@loop // cannot poll -> Ok to take from buffer - failure === ALREADY_SELECTED -> { - this.size = size // restore size - buffer[head] = result // restore head - return failure - } - failure is Closed<*> -> { - send = failure - token = failure.tryResumeSend(idempotent = null) - replacement = failure - break@loop - } - else -> error("performAtomicTrySelect(describeTryOffer) returned $failure") - } - } - } - if (replacement !== POLL_FAILED && replacement !is Closed<*>) { - this.size = size // restore size - buffer[(head + size) % capacity] = replacement - } else { - // failed to poll or is already closed --> let's try to select receiving this element from buffer - if (!select.trySelect(null)) { // :todo: move trySelect completion outside of lock - this.size = size // restore size - buffer[head] = result // restore head - return ALREADY_SELECTED - } - } - head = (head + 1) % capacity - } - // complete send the we're taken replacement from - if (token != null) - send!!.completeResumeSend(token!!) - return result - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/BroadcastChannel.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/BroadcastChannel.kt deleted file mode 100644 index 0c268dcfd8..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/BroadcastChannel.kt +++ /dev/null @@ -1,44 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import java.io.Closeable - -/** - * Broadcast channel is a non-blocking primitive for communication between the sender and multiple receivers - * that subscribe for the elements using [open] function and unsubscribe using [SubscriptionReceiveChannel.close] - * function. - */ -public interface BroadcastChannel : SendChannel { - /** - * Subscribes to this [BroadcastChannel] and returns a channel to receive elements from it. - * The resulting channel shall be [closed][SubscriptionReceiveChannel.close] to unsubscribe from this - * broadcast channel. - */ - public fun open(): SubscriptionReceiveChannel -} - -/** - * Return type for [BroadcastChannel.open] that can be used to [receive] elements from the - * open subscription and to [close] it to unsubscribe. - */ -public interface SubscriptionReceiveChannel : ReceiveChannel, Closeable { - /** - * Closes this subscription. - */ - public override fun close() -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Channel.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Channel.kt deleted file mode 100644 index 1ee18401a3..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Channel.kt +++ /dev/null @@ -1,277 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.CancellationException -import kotlinx.coroutines.experimental.CoroutineScope -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.selects.SelectBuilder -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlinx.coroutines.experimental.selects.select -import kotlinx.coroutines.experimental.yield - -/** - * Sender's interface to [Channel]. - */ -public interface SendChannel { - /** - * Returns `true` if this channel was closed by invocation of [close] and thus - * the [send] attempt throws [ClosedSendChannelException]. If the channel was closed because of the exception, it - * is considered closed, too, but it is called a _failed_ channel. All suspending attempts to send - * an element to a failed channel throw the original [close] cause exception. - */ - public val isClosedForSend: Boolean - - /** - * Returns `true` if the channel is full (out of capacity) and the [send] attempt will suspend. - * This function returns `false` for [isClosedForSend] channel. - */ - public val isFull: Boolean - - /** - * Adds [element] into to this channel, suspending the caller while this channel [isFull], - * or throws [ClosedSendChannelException] if the channel [isClosedForSend] _normally_. - * It throws the original [close] cause exception if the channel has _failed_. - * - * Note, that closing a channel _after_ this function had suspended does not cause this suspended send invocation - * to abort, because closing a channel is conceptually like sending a special "close token" over this channel. - * All elements that are sent over the channel are delivered in first-in first-out order. The element that - * is being sent will get delivered to receivers before a close token. - * - * This suspending function is cancellable. If the [Job] of the current coroutine is completed while this - * function is suspended, this function immediately resumes with [CancellationException]. - * Cancellation of suspended send is *atomic* -- when this function - * throws [CancellationException] it means that the [element] was not sent to this channel. - * - * Note, that this function does not check for cancellation when it is not suspended. - * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. - * - * This function can be used in [select] invocation with [onSend][SelectBuilder.onSend] clause. - * Use [offer] to try sending to this channel without waiting. - */ - public suspend fun send(element: E) - - /** - * Adds [element] into this queue if it is possible to do so immediately without violating capacity restrictions - * and returns `true`. Otherwise, it returns `false` immediately - * or throws [ClosedSendChannelException] if the channel [isClosedForSend] _normally_. - * It throws the original [close] cause exception if the channel has _failed_. - */ - public fun offer(element: E): Boolean - - /** - * Registers [onSend][SelectBuilder.onSend] select clause. - * @suppress **This is unstable API and it is subject to change.** - */ - public fun registerSelectSend(select: SelectInstance, element: E, block: suspend () -> R) - - /** - * Closes this channel with an optional exceptional [cause]. - * This is an idempotent operation -- repeated invocations of this function have no effect and return `false`. - * Conceptually, its sends a special "close token" over this channel. Immediately after invocation of this function - * [isClosedForSend] starts returning `true`. However, [isClosedForReceive][ReceiveChannel.isClosedForReceive] - * on the side of [ReceiveChannel] starts returning `true` only after all previously sent elements - * are received. - * - * A channel that was closed without a [cause], is considered to be _closed normally_. - * A channel that was closed with non-null [cause] is called a _failed channel_. Attempts to send or - * receive on a failed channel throw this cause exception. - */ - public fun close(cause: Throwable? = null): Boolean -} - -/** - * Receiver's interface to [Channel]. - */ -public interface ReceiveChannel { - /** - * Returns `true` if this channel was closed by invocation of [close][SendChannel.close] on the [SendChannel] - * side and all previously sent items were already received, so that the [receive] attempt - * throws [ClosedReceiveChannelException]. If the channel was closed because of the exception, it - * is considered closed, too, but it is called a _failed_ channel. All suspending attempts to receive - * an element from a failed channel throw the original [close][SendChannel.close] cause exception. - */ - public val isClosedForReceive: Boolean - - /** - * Returns `true` if the channel is empty (contains no elements) and the [receive] attempt will suspend. - * This function returns `false` for [isClosedForReceive] channel. - */ - public val isEmpty: Boolean - - /** - * Retrieves and removes the element from this channel suspending the caller while this channel [isEmpty] - * or throws [ClosedReceiveChannelException] if the channel [isClosedForReceive]. - * If the channel was closed because of the exception, it is called a _failed_ channel and this function - * throws the original [close][SendChannel.close] cause exception. - * - * This suspending function is cancellable. If the [Job] of the current coroutine is completed while this - * function is suspended, this function immediately resumes with [CancellationException]. - * Cancellation of suspended receive is *atomic* -- when this function - * throws [CancellationException] it means that the element was not retrieved from this channel. - * - * Note, that this function does not check for cancellation when it is not suspended. - * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. - * - * This function can be used in [select] invocation with [onReceive][SelectBuilder.onReceive] clause. - * Use [poll] to try receiving from this channel without waiting. - */ - public suspend fun receive(): E - - /** - * Retrieves and removes the element from this channel suspending the caller while this channel [isEmpty] - * or returns `null` if the channel is [closed][isClosedForReceive] _normally_, - * or throws the original [close][SendChannel.close] cause exception if the channel has _failed_. - * - * This suspending function is cancellable. If the [Job] of the current coroutine is completed while this - * function is suspended, this function immediately resumes with [CancellationException]. - * Cancellation of suspended receive is *atomic* -- when this function - * throws [CancellationException] it means that the element was not retrieved from this channel. - * - * Note, that this function does not check for cancellation when it is not suspended. - * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. - * - * This function can be used in [select] invocation with [onReceiveOrNull][SelectBuilder.onReceiveOrNull] clause. - * Use [poll] to try receiving from this channel without waiting. - */ - public suspend fun receiveOrNull(): E? - - /** - * Retrieves and removes the head of this queue, or returns `null` if this queue [isEmpty] - * or is [closed][isClosedForReceive] _normally_, - * or throws the original [close][SendChannel.close] cause exception if the channel has _failed_. - */ - public fun poll(): E? - - /** - * Returns new iterator to receive elements from this channels using `for` loop. - * Iteration completes normally when the channel is [closed][isClosedForReceive] _normally_ and - * throws the original [close][SendChannel.close] cause exception if the channel has _failed_. - */ - public operator fun iterator(): ChannelIterator - - /** - * Registers [onReceive][SelectBuilder.onReceive] select clause. - * @suppress **This is unstable API and it is subject to change.** - */ - public fun registerSelectReceive(select: SelectInstance, block: suspend (E) -> R) - - /** - * Registers [onReceiveOrNull][SelectBuilder.onReceiveOrNull] select clause. - * @suppress **This is unstable API and it is subject to change.** - */ - public fun registerSelectReceiveOrNull(select: SelectInstance, block: suspend (E?) -> R) -} - -/** - * Iterator for [ReceiveChannel]. Instances of this interface are *not thread-safe* and shall not be used - * from concurrent coroutines. - */ -public interface ChannelIterator { - /** - * Returns `true` if the channel has more elements suspending the caller while this channel - * [isEmpty][ReceiveChannel.isEmpty] or returns `false` if the channel - * [isClosedForReceive][ReceiveChannel.isClosedForReceive] _normally_. - * It throws the original [close][SendChannel.close] cause exception if the channel has _failed_. - * - * This function retrieves and removes the element from this channel for the subsequent invocation - * of [next]. - * - * This suspending function is cancellable. If the [Job] of the current coroutine is completed while this - * function is suspended, this function immediately resumes with [CancellationException]. - * Cancellation of suspended receive is *atomic* -- when this function - * throws [CancellationException] it means that the element was not retrieved from this channel. - * - * Note, that this function does not check for cancellation when it is not suspended. - * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. - */ - public suspend operator fun hasNext(): Boolean - - /** - * Retrieves and removes the element from this channel suspending the caller while this channel - * [isEmpty][ReceiveChannel.isEmpty] or throws [ClosedReceiveChannelException] if the channel - * [isClosedForReceive][ReceiveChannel.isClosedForReceive]. - * It throws the original [close][SendChannel.close] cause exception if the channel has _failed_. - * - * This suspending function is cancellable. If the [Job] of the current coroutine is completed while this - * function is suspended, this function immediately resumes with [CancellationException]. - * Cancellation of suspended receive is *atomic* -- when this function - * throws [CancellationException] it means that the element was not retrieved from this channel. - * - * Note, that this function does not check for cancellation when it is not suspended. - * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. - */ - public suspend operator fun next(): E -} - -/** - * Channel is a non-blocking primitive for communication between sender using [SendChannel] and receiver using [ReceiveChannel]. - * Conceptually, a channel is similar to [BlockingQueue][java.util.concurrent.BlockingQueue], - * but it has suspending operations instead of blocking ones and it can be closed. - * - * See [Channel()][Channel.invoke] factory function for the description of available channel implementations. - */ -public interface Channel : SendChannel, ReceiveChannel { - /** - * Factory for channels. - */ - public companion object Factory { - /** - * Requests channel with unlimited capacity buffer in [Channel()][invoke] factory function -- - * the [LinkedListChannel] gets created. - */ - public const val UNLIMITED = Int.MAX_VALUE - - /** - * Requests conflated channel in [Channel()][invoke] factory function -- - * the [ConflatedChannel] gets created. - */ - public const val CONFLATED = -1 - - /** - * Creates a channel with specified buffer capacity (or without a buffer by default). - * - * The resulting channel type depends on the specified [capacity] parameter: - * * when `capacity` is 0 -- creates [RendezvousChannel] without a buffer; - * * when `capacity` is [UNLIMITED] -- creates [LinkedListChannel] with buffer of unlimited size; - * * when `capacity` is [CONFLATED] -- creates [ConflatedChannel] that conflates back-to-back sends; - * * otherwise -- creates [ArrayChannel] with a buffer of the specified `capacity`. - */ - public operator fun invoke(capacity: Int = 0): Channel { - return when (capacity) { - 0 -> RendezvousChannel() - UNLIMITED -> LinkedListChannel() - CONFLATED -> ConflatedChannel() - else -> ArrayChannel(capacity) - } - } - } -} - -/** - * Indicates attempt to [send][SendChannel.send] on [isClosedForSend][SendChannel.isClosedForSend] channel - * that was closed _normally_. A _failed_ channel rethrows the original [close][SendChannel.close] cause - * exception on send attempts. - */ -public class ClosedSendChannelException(message: String?) : CancellationException(message) - -/** - * Indicates attempt to [receive][ReceiveChannel.receive] on [isClosedForReceive][ReceiveChannel.isClosedForReceive] - * channel that was closed _normally_. A _failed_ channel rethrows the original [close][SendChannel.close] cause - * exception on receive attempts. - */ -public class ClosedReceiveChannelException(message: String?) : NoSuchElementException(message) diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ChannelCoroutine.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ChannelCoroutine.kt deleted file mode 100644 index e1b64b2be9..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ChannelCoroutine.kt +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.JobSupport -import kotlinx.coroutines.experimental.handleCoroutineException -import kotlin.coroutines.experimental.CoroutineContext - -internal open class ChannelCoroutine( - override val parentContext: CoroutineContext, - open val channel: Channel, - active: Boolean -) : AbstractCoroutine(active), Channel by channel { - override fun afterCompletion(state: Any?, mode: Int) { - val cause = (state as? JobSupport.CompletedExceptionally)?.cause - if (!channel.close(cause) && cause != null) - handleCoroutineException(context, cause) - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Channels.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Channels.kt deleted file mode 100644 index ec9ec7d072..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Channels.kt +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -internal const val DEFAULT_CLOSE_MESSAGE = "Channel was closed" - -/** - * Performs the given [action] for each received element. - */ -// :todo: make it inline when this bug is fixed: https://youtrack.jetbrains.com/issue/KT-16448 -public suspend fun ReceiveChannel.consumeEach(action: suspend (E) -> Unit) { - for (element in this) action(element) -} - -/** - * Subscribes to this [BroadcastChannel] and performs the specified action for each received element. - */ -// :todo: make it inline when this bug is fixed: https://youtrack.jetbrains.com/issue/KT-16448 -public suspend fun BroadcastChannel.consumeEach(action: suspend (E) -> Unit) { - open().use { channel -> - for (x in channel) action(x) - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ConflatedBroadcastChannel.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ConflatedBroadcastChannel.kt deleted file mode 100644 index e1642ab71d..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ConflatedBroadcastChannel.kt +++ /dev/null @@ -1,270 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.MODE_DIRECT -import kotlinx.coroutines.experimental.internal.Symbol -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import kotlinx.coroutines.experimental.selects.SelectInstance -import java.util.concurrent.atomic.AtomicIntegerFieldUpdater -import java.util.concurrent.atomic.AtomicReferenceFieldUpdater - -/** - * Broadcasts the most recently sent element (aka [value]) to all [open] subscribers. - * - * Back-to-send sent elements are _conflated_ -- only the the most recently sent value is received, - * while previously sent elements **are lost**. - * Every subscriber immediately receives the most recently sent element. - * Sender to this broadcast channel never suspends and [offer] always returns `true`. - * - * A secondary constructor can be used to create an instance of this class that already holds a value. - * - * This implementation is fully lock-free. In this implementation - * [opening][open] and [closing][SubscriptionReceiveChannel.close] subscription takes O(N) time, where N is the - * number of subscribers. - */ -public class ConflatedBroadcastChannel() : BroadcastChannel { - /** - * Creates an instance of this class that already holds a value. - * - * It is as a shortcut to creating an instance with a default constructor and - * immediately sending an element: `ConflatedBroadcastChannel().apply { offer(value) }`. - */ - constructor(value: E) : this() { - state = State(value, null) - } - - @Suppress("UNCHECKED_CAST") - @Volatile - private var state: Any = INITIAL_STATE // State | Closed - - @Volatile - private var updating = 0 - - private companion object { - @JvmField - val STATE: AtomicReferenceFieldUpdater, Any> = AtomicReferenceFieldUpdater. - newUpdater(ConflatedBroadcastChannel::class.java, Any::class.java, "state") - - @JvmField - val UPDATING: AtomicIntegerFieldUpdater> = AtomicIntegerFieldUpdater. - newUpdater(ConflatedBroadcastChannel::class.java, "updating") - - @JvmField - val CLOSED = Closed(null) - - @JvmField - val UNDEFINED = Symbol("UNDEFINED") - - @JvmField - val INITIAL_STATE = State(UNDEFINED, null) - } - - private class State( - @JvmField val value: Any?, // UNDEFINED | E - @JvmField val subscribers: Array>? - ) - - private class Closed(@JvmField val closeCause: Throwable?) { - val sendException: Throwable get() = closeCause ?: ClosedSendChannelException(DEFAULT_CLOSE_MESSAGE) - val valueException: Throwable get() = closeCause ?: IllegalStateException(DEFAULT_CLOSE_MESSAGE) - } - - /** - * The most recently sent element to this channel. - * - * Access to this property throws [IllegalStateException] when this class is constructed without - * initial value and no value was sent yet or if it was [closed][close] _normally_ and - * throws the original [close][SendChannel.close] cause exception if the channel has _failed_. - */ - @Suppress("UNCHECKED_CAST") - public val value: E get() { - val state = this.state - when (state) { - is Closed -> throw state.valueException - is State<*> -> { - if (state.value === UNDEFINED) throw IllegalStateException("No value") - return state.value as E - } - else -> error("Invalid state $state") - } - } - - /** - * The most recently sent element to this channel or `null` when this class is constructed without - * initial value and no value was sent yet or if it was [closed][close]. - */ - @Suppress("UNCHECKED_CAST") - public val valueOrNull: E? get() { - val state = this.state - when (state) { - is Closed -> return null - is State<*> -> { - if (state.value === UNDEFINED) return null - return state.value as E - } - else -> error("Invalid state $state") - } - } - - override val isClosedForSend: Boolean get() = state is Closed - override val isFull: Boolean get() = false - - @Suppress("UNCHECKED_CAST") - override fun open(): SubscriptionReceiveChannel { - val subscriber = Subscriber(this) - while (true) { // lock-free loop on state - val state = this.state - when (state) { - is Closed -> { - subscriber.close(state.closeCause) - return subscriber - } - is State<*> -> { - if (state.value !== UNDEFINED) - subscriber.offerInternal(state.value as E) - val update = State(state.value, addSubscriber((state as State).subscribers, subscriber)) - if (STATE.compareAndSet(this, state, update)) - return subscriber - } - else -> error("Invalid state $state") - } - } - } - - @Suppress("UNCHECKED_CAST") - private fun closeSubscriber(subscriber: Subscriber) { - while (true) { // lock-free loop on state - val state = this.state - when (state) { - is Closed -> return - is State<*> -> { - val update = State(state.value, removeSubscriber((state as State).subscribers!!, subscriber)) - if (STATE.compareAndSet(this, state, update)) - return - } - else -> error("Invalid state $state") - } - } - } - - private fun addSubscriber(list: Array>?, subscriber: Subscriber): Array> { - if (list == null) return Array>(1) { subscriber } - return list + subscriber - } - - @Suppress("UNCHECKED_CAST") - private fun removeSubscriber(list: Array>, subscriber: Subscriber): Array>? { - val n = list.size - val i = list.indexOf(subscriber) - check(i >= 0) - if (n == 1) return null - val update = arrayOfNulls>(n - 1) - System.arraycopy(list, 0, update, 0, i) - System.arraycopy(list, i + 1, update, i, n - i - 1) - return update as Array> - } - - @Suppress("UNCHECKED_CAST") - override fun close(cause: Throwable?): Boolean { - while (true) { // lock-free loop on state - val state = this.state - when (state) { - is Closed -> return false - is State<*> -> { - val update = if (cause == null) CLOSED else Closed(cause) - if (STATE.compareAndSet(this, state, update)) { - (state as State).subscribers?.forEach { it.close(cause) } - return true - } - } - else -> error("Invalid state $state") - } - } - } - - /** - * Sends the value to all subscribed receives and stores this value as the most recent state for - * future subscribers. This implementation never suspends. - * - * It throws [ClosedSendChannelException] if the channel [isClosedForSend] _normally_. - * It throws the original [close] cause exception if the channel has _failed_. - */ - suspend override fun send(element: E) { - offerInternal(element)?.let { throw it.sendException } - } - - /** - * Sends the value to all subscribed receives and stores this value as the most recent state for - * future subscribers. This implementation always returns `true`. - * - * It throws [ClosedSendChannelException] if the channel [isClosedForSend] _normally_. - * It throws the original [close] cause exception if the channel has _failed_. - */ - override fun offer(element: E): Boolean { - offerInternal(element)?.let { throw it.sendException } - return true - } - - @Suppress("UNCHECKED_CAST") - private fun offerInternal(element: E): Closed? { - // If some other thread is updating the state in its offer operation we assume that our offer had linearized - // before that offer (we lost) and that offer overwrote us and conflated our offer. - if (!UPDATING.compareAndSet(this, 0, 1)) return null - try { - while (true) { // lock-free loop on state - val state = this.state - when (state) { - is Closed -> return state - is State<*> -> { - val update = State(element, (state as State).subscribers) - if (STATE.compareAndSet(this, state, update)) { - // Note: Using offerInternal here to ignore the case when this subscriber was - // already concurrently closed (assume the close had conflated our offer for this - // particular subscriber). - state.subscribers?.forEach { it.offerInternal(element) } - return null - } - } - else -> error("Invalid state $state") - } - } - } finally { - updating = 0 // reset the updating flag to zero even when something goes wrong - } - } - - override fun registerSelectSend(select: SelectInstance, element: E, block: suspend () -> R) { - if (!select.trySelect(idempotent = null)) return - offerInternal(element)?.let { - select.resumeSelectWithException(it.sendException, MODE_DIRECT) - return - } - block.startCoroutineUndispatched(select.completion) - } - - private class Subscriber( - private val broadcastChannel: ConflatedBroadcastChannel - ) : ConflatedChannel(), SubscriptionReceiveChannel { - override fun close() { - if (close(cause = null)) - broadcastChannel.closeSubscriber(this) - } - - public override fun offerInternal(element: E): Any = super.offerInternal(element) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ConflatedChannel.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ConflatedChannel.kt deleted file mode 100644 index 4d12000721..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/ConflatedChannel.kt +++ /dev/null @@ -1,69 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.ALREADY_SELECTED -import kotlinx.coroutines.experimental.selects.SelectInstance - -/** - * Channel that buffers at most one element and conflates all subsequent `send` and `offer` invocations, - * so that the receiver always gets the most recently sent element. - * Back-to-send sent elements are _conflated_ -- only the the most recently sent element is received, - * while previously sent elements **are lost**. - * Sender to this channel never suspends and [offer] always returns `true`. - * - * This implementation is fully lock-free. - */ -public open class ConflatedChannel : AbstractChannel() { - protected final override val isBufferAlwaysEmpty: Boolean get() = true - protected final override val isBufferEmpty: Boolean get() = true - protected final override val isBufferAlwaysFull: Boolean get() = false - protected final override val isBufferFull: Boolean get() = false - - // result is always `OFFER_SUCCESS | Closed` - protected override fun offerInternal(element: E): Any { - while (true) { - val result = super.offerInternal(element) - when { - result === OFFER_SUCCESS -> return OFFER_SUCCESS - result === OFFER_FAILED -> { // try to buffer - if (sendConflated(element)) - return OFFER_SUCCESS - } - result is Closed<*> -> return result - else -> error("Invalid offerInternal result $result") - } - } - } - - // result is always `ALREADY_SELECTED | OFFER_SUCCESS | Closed`. - protected override fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - while (true) { - val result = if (hasReceiveOrClosed) - super.offerSelectInternal(element, select) else - (select.performAtomicTrySelect(describeSendConflated(element)) ?: OFFER_SUCCESS) - when { - result === ALREADY_SELECTED -> return ALREADY_SELECTED - result === OFFER_SUCCESS -> return OFFER_SUCCESS - result === OFFER_FAILED -> {} // retry - result is Closed<*> -> return result - else -> error("Invalid result $result") - } - } - } -} - diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/LinkedListChannel.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/LinkedListChannel.kt deleted file mode 100644 index 83e67bb957..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/LinkedListChannel.kt +++ /dev/null @@ -1,66 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.ALREADY_SELECTED -import kotlinx.coroutines.experimental.selects.SelectInstance - -/** - * Channel with linked-list buffer of a unlimited capacity (limited only by available memory). - * Sender to this channel never suspends and [offer] always returns `true`. - * - * This implementation is fully lock-free. - */ -public open class LinkedListChannel : AbstractChannel() { - protected final override val isBufferAlwaysEmpty: Boolean get() = true - protected final override val isBufferEmpty: Boolean get() = true - protected final override val isBufferAlwaysFull: Boolean get() = false - protected final override val isBufferFull: Boolean get() = false - - // result is always `OFFER_SUCCESS | Closed` - protected override fun offerInternal(element: E): Any { - while (true) { - val result = super.offerInternal(element) - when { - result === OFFER_SUCCESS -> return OFFER_SUCCESS - result === OFFER_FAILED -> { // try to buffer - if (sendBuffered(element)) - return OFFER_SUCCESS - } - result is Closed<*> -> return result - else -> error("Invalid offerInternal result $result") - } - } - } - - // result is always `ALREADY_SELECTED | OFFER_SUCCESS | Closed`. - protected override fun offerSelectInternal(element: E, select: SelectInstance<*>): Any { - while (true) { - val result = if (hasReceiveOrClosed) - super.offerSelectInternal(element, select) else - (select.performAtomicTrySelect(describeSendBuffered(element)) ?: OFFER_SUCCESS) - when { - result === ALREADY_SELECTED -> return ALREADY_SELECTED - result === OFFER_SUCCESS -> return OFFER_SUCCESS - result === OFFER_FAILED -> {} // retry - result is Closed<*> -> return result - else -> error("Invalid result $result") - } - } - } -} - diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Produce.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Produce.kt deleted file mode 100644 index 0590ebe6aa..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/Produce.kt +++ /dev/null @@ -1,113 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.CoroutineDispatcher -import kotlinx.coroutines.experimental.CoroutineScope -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.newCoroutineContext -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Scope for [produce] coroutine builder. - */ -public interface ProducerScope : CoroutineScope, SendChannel { - /** - * A reference to the channel that this coroutine [sends][send] elements to. - * It is provided for convenience, so that the code in the coroutine can refer - * to the channel as `channel` as apposed to `this`. - * All the [SendChannel] functions on this interface delegate to - * the channel instance returned by this function. - */ - val channel: SendChannel -} - -/** - * @suppress **Deprecated**: Renamed to `ProducerScope`. - */ -@Deprecated(message = "Renamed to `ProducerScope`", replaceWith = ReplaceWith("ProducerScope")) -typealias ChannelBuilder = ProducerScope - -/** - * Return type for [produce] coroutine builder. - */ -public interface ProducerJob : Job, ReceiveChannel { - /** - * A reference to the channel that this coroutine is producing. - * All the [ReceiveChannel] functions on this interface delegate to - * the channel instance returned by this function. - */ - val channel: ReceiveChannel -} - -/** - * @suppress **Deprecated**: Renamed to `ProducerJob`. - */ -@Deprecated(message = "Renamed to `ProducerJob`", replaceWith = ReplaceWith("ProducerJob")) -typealias ChannelJob = ProducerJob - -/** - * Launches new coroutine to produce a stream of values by sending them to a channel - * and returns a reference to the coroutine as a [ProducerJob]. This resulting - * object can be used to [receive][ReceiveChannel.receive] elements produced by this coroutine. - * - * The scope of the coroutine contains [ProducerScope] interface, which implements - * both [CoroutineScope] and [SendChannel], so that coroutine can invoke - * [send][SendChannel.send] directly. The channel is [closed][SendChannel.close] - * when the coroutine completes. - * The running coroutine is cancelled when the its job is [cancelled][Job.cancel]. - * - * The [context] for the new coroutine must be explicitly specified. - * See [CoroutineDispatcher] for the standard [context] implementations that are provided by `kotlinx.coroutines`. - * The [context][CoroutineScope.context] of the parent coroutine from its [scope][CoroutineScope] may be used, - * in which case the [Job] of the resulting coroutine is a child of the job of the parent coroutine. - * - * Uncaught exceptions in this coroutine close the channel with this exception as a cause and - * the resulting channel becomes _failed_, so that any attempt to receive from such a channel throws exception. - * - * See [newCoroutineContext] for a description of debugging facilities that are available for newly created coroutine. - * - * @param context context of the coroutine - * @param capacity capacity of the channel's buffer (no buffer by default) - * @param block the coroutine code - */ -public fun produce( - context: CoroutineContext, - capacity: Int = 0, - block: suspend ProducerScope.() -> Unit -): ProducerJob { - val channel = Channel(capacity) - return ProducerCoroutine(newCoroutineContext(context), channel).apply { - initParentJob(context[Job]) - block.startCoroutine(this, this) - } -} - -/** - * @suppress **Deprecated**: Renamed to `produce`. - */ -@Deprecated(message = "Renamed to `produce`", replaceWith = ReplaceWith("produce(context, capacity, block)")) -public fun buildChannel( - context: CoroutineContext, - capacity: Int = 0, - block: suspend ProducerScope.() -> Unit -): ProducerJob = - produce(context, capacity, block) - -private class ProducerCoroutine(parentContext: CoroutineContext, channel: Channel) : - ChannelCoroutine(parentContext, channel, active = true), ProducerScope, ProducerJob diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/RendezvousChannel.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/RendezvousChannel.kt deleted file mode 100644 index 6b7891ca4d..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/channels/RendezvousChannel.kt +++ /dev/null @@ -1,31 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -/** - * Rendezvous channel. This channel does not have any buffer at all. An element is transferred from sender - * to receiver only when [send] and [receive] invocations meet in time (rendezvous), so [send] suspends - * until another coroutine invokes [receive] and [receive] suspends until another coroutine invokes [send]. - * - * This implementation is fully lock-free. - */ -public open class RendezvousChannel : AbstractChannel() { - protected final override val isBufferAlwaysEmpty: Boolean get() = true - protected final override val isBufferEmpty: Boolean get() = true - protected final override val isBufferAlwaysFull: Boolean get() = true - protected final override val isBufferFull: Boolean get() = true -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/internal/Atomic.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/internal/Atomic.kt deleted file mode 100644 index dd42680e78..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/internal/Atomic.kt +++ /dev/null @@ -1,89 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.internal - -import java.util.concurrent.atomic.AtomicReferenceFieldUpdater - -/** - * The most abstract operation that can be in process. Other threads observing an instance of this - * class in the fields of their object shall invoke [perform] to help. - * - * @suppress **This is unstable API and it is subject to change.** - */ -public abstract class OpDescriptor { - /** - * Returns `null` is operation was performed successfully or some other - * object that indicates the failure reason. - */ - abstract fun perform(affected: Any?): Any? -} - -/** - * Descriptor for multi-word atomic operation. - * Based on paper - * ["A Practical Multi-Word Compare-and-Swap Operation"](http://www.cl.cam.ac.uk/research/srg/netos/papers/2002-casn.pdf) - * by Timothy L. Harris, Keir Fraser and Ian A. Pratt. - * - * Note: parts of atomic operation must be globally ordered. Otherwise, this implementation will produce - * [StackOverflowError]. - * - * @suppress **This is unstable API and it is subject to change.** - */ -public abstract class AtomicOp : OpDescriptor() { - @Volatile - private var _consensus: Any? = UNDECIDED - - companion object { - private val CONSENSUS: AtomicReferenceFieldUpdater = - AtomicReferenceFieldUpdater.newUpdater(AtomicOp::class.java, Any::class.java, "_consensus") - - private val UNDECIDED: Any = Symbol("UNDECIDED") - } - - val isDecided: Boolean get() = _consensus !== UNDECIDED - - fun tryDecide(decision: Any?): Boolean { - check(decision !== UNDECIDED) - return CONSENSUS.compareAndSet(this, UNDECIDED, decision) - } - - private fun decide(decision: Any?): Any? = if (tryDecide(decision)) decision else _consensus - - abstract fun prepare(): Any? // `null` if Ok, or failure reason - - abstract fun complete(affected: Any?, failure: Any?) // failure != null if failed to prepare op - - // returns `null` on success - final override fun perform(affected: Any?): Any? { - // make decision on status - var decision = this._consensus - if (decision === UNDECIDED) - decision = decide(prepare()) - complete(affected, decision) - return decision - } -} - -/** - * A part of multi-step atomic operation [AtomicOp]. - * - * @suppress **This is unstable API and it is subject to change.** - */ -public abstract class AtomicDesc { - abstract fun prepare(op: AtomicOp): Any? // returns `null` if prepared successfully - abstract fun complete(op: AtomicOp, failure: Any?) // decision == null if success -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedList.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedList.kt deleted file mode 100644 index 9d8b12321f..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedList.kt +++ /dev/null @@ -1,617 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.internal - -import java.util.concurrent.atomic.AtomicReferenceFieldUpdater - -private typealias Node = LockFreeLinkedListNode - -@PublishedApi -internal const val UNDECIDED = 0 - -@PublishedApi -internal const val SUCCESS = 1 - -@PublishedApi -internal const val FAILURE = 2 - -@PublishedApi -internal val CONDITION_FALSE: Any = Symbol("CONDITION_FALSE") - -@PublishedApi -internal val ALREADY_REMOVED: Any = Symbol("ALREADY_REMOVED") - -@PublishedApi -internal val LIST_EMPTY: Any = Symbol("LIST_EMPTY") - -private val REMOVE_PREPARED: Any = Symbol("REMOVE_PREPARED") - -/** - * @suppress **This is unstable API and it is subject to change.** - */ -public typealias RemoveFirstDesc = LockFreeLinkedListNode.RemoveFirstDesc - -/** - * @suppress **This is unstable API and it is subject to change.** - */ -public typealias AddLastDesc = LockFreeLinkedListNode.AddLastDesc - -/** - * Doubly-linked concurrent list node with remove support. - * Based on paper - * ["Lock-Free and Practical Doubly Linked List-Based Deques Using Single-Word Compare-and-Swap"](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.140.4693&rep=rep1&type=pdf) - * by Sundell and Tsigas. - * - * Important notes: - * * The instance of this class serves both as list head/tail sentinel and as the list item. - * Sentinel node should be never removed. - * * There are no operations to add items to left side of the list, only to the end (right side), because we cannot - * efficiently linearize them with atomic multi-step head-removal operations. In short, - * support for [describeRemoveFirst] operation precludes ability to add items at the beginning. - * - * @suppress **This is unstable API and it is subject to change.** - */ -@Suppress("LeakingThis") -public open class LockFreeLinkedListNode { - @Volatile - private var _next: Any = this // Node | Removed | OpDescriptor - @Volatile - private var _prev: Any = this // Node | Removed - @Volatile - private var _removedRef: Removed? = null // lazily cached removed ref to this - - private companion object { - @JvmField - val NEXT: AtomicReferenceFieldUpdater = - AtomicReferenceFieldUpdater.newUpdater(Node::class.java, Any::class.java, "_next") - @JvmField - val PREV: AtomicReferenceFieldUpdater = - AtomicReferenceFieldUpdater.newUpdater(Node::class.java, Any::class.java, "_prev") - @JvmField - val REMOVED_REF: AtomicReferenceFieldUpdater = - AtomicReferenceFieldUpdater.newUpdater(Node::class.java, Removed::class.java, "_removedRef") - } - - private fun removed(): Removed = - _removedRef ?: Removed(this).also { REMOVED_REF.lazySet(this, it) } - - @PublishedApi - internal abstract class CondAddOp( - @JvmField val newNode: Node - ) : AtomicOp() { - @JvmField var oldNext: Node? = null - - override fun complete(affected: Any?, failure: Any?) { - affected as Node // type assertion - val success = failure == null - val update = if (success) newNode else oldNext - if (NEXT.compareAndSet(affected, this, update)) { - // only the thread the makes this update actually finishes add operation - if (success) newNode.finishAdd(oldNext!!) - } - } - } - - @PublishedApi - internal inline fun makeCondAddOp(node: Node, crossinline condition: () -> Boolean): CondAddOp = - object : CondAddOp(node) { - override fun prepare(): Any? = if (condition()) null else CONDITION_FALSE - } - - public val isRemoved: Boolean get() = next is Removed - - // LINEARIZABLE. Returns Node | Removed - public val next: Any get() { - while (true) { // operation helper loop on _next - val next = this._next - if (next !is OpDescriptor) return next - next.perform(this) - } - } - - // LINEARIZABLE. Returns Node | Removed - public val prev: Any get() { - while (true) { // insert helper loop on _prev - val prev = this._prev - if (prev is Removed) return prev - prev as Node // otherwise, it can be only node otherwise - if (prev.next === this) return prev - helpInsert(prev, null) - } - } - - // ------ addOneIfEmpty ------ - - public fun addOneIfEmpty(node: Node): Boolean { - PREV.lazySet(node, this) - NEXT.lazySet(node, this) - while (true) { - val next = next - if (next !== this) return false // this is not an empty list! - if (NEXT.compareAndSet(this, this, node)) { - // added successfully (linearized add) -- fixup the list - node.finishAdd(this) - return true - } - } - } - - // ------ addLastXXX ------ - - /** - * Adds last item to this list. - */ - public fun addLast(node: Node) { - while (true) { // lock-free loop on prev.next - val prev = prev as Node // sentinel node is never removed, so prev is always defined - if (prev.addNext(node, this)) return - } - } - - public fun describeAddLast(node: T): AddLastDesc = AddLastDesc(this, node) - - /** - * Adds last item to this list atomically if the [condition] is true. - */ - public inline fun addLastIf(node: Node, crossinline condition: () -> Boolean): Boolean { - val condAdd = makeCondAddOp(node, condition) - while (true) { // lock-free loop on prev.next - val prev = prev as Node // sentinel node is never removed, so prev is always defined - when (prev.tryCondAddNext(node, this, condAdd)) { - SUCCESS -> return true - FAILURE -> return false - } - } - } - - public inline fun addLastIfPrev(node: Node, predicate: (Node) -> Boolean): Boolean { - while (true) { // lock-free loop on prev.next - val prev = prev as Node // sentinel node is never removed, so prev is always defined - if (!predicate(prev)) return false - if (prev.addNext(node, this)) return true - } - } - - public inline fun addLastIfPrevAndIf( - node: Node, - predicate: (Node) -> Boolean, // prev node predicate - crossinline condition: () -> Boolean // atomically checked condition - ): Boolean { - val condAdd = makeCondAddOp(node, condition) - while (true) { // lock-free loop on prev.next - val prev = prev as Node // sentinel node is never removed, so prev is always defined - if (!predicate(prev)) return false - when (prev.tryCondAddNext(node, this, condAdd)) { - SUCCESS -> return true - FAILURE -> return false - } - } - } - - // ------ addXXX util ------ - - @PublishedApi - internal fun addNext(node: Node, next: Node): Boolean { - PREV.lazySet(node, this) - NEXT.lazySet(node, next) - if (!NEXT.compareAndSet(this, next, node)) return false - // added successfully (linearized add) -- fixup the list - node.finishAdd(next) - return true - } - - // returns UNDECIDED, SUCCESS or FAILURE - @PublishedApi - internal fun tryCondAddNext(node: Node, next: Node, condAdd: CondAddOp): Int { - PREV.lazySet(node, this) - NEXT.lazySet(node, next) - condAdd.oldNext = next - if (!NEXT.compareAndSet(this, next, condAdd)) return UNDECIDED - // added operation successfully (linearized) -- complete it & fixup the list - return if (condAdd.perform(this) == null) SUCCESS else FAILURE - } - - // ------ removeXXX ------ - - /** - * Removes this node from the list. Returns `true` when removed successfully. - */ - public open fun remove(): Boolean { - while (true) { // lock-free loop on next - val next = this.next - if (next is Removed) return false // was already removed -- don't try to help (original thread will take care) - check(next !== this) // sanity check -- can be true for sentinel nodes only, but they are never removed - val removed = (next as Node).removed() - if (NEXT.compareAndSet(this, next, removed)) { - // was removed successfully (linearized remove) -- fixup the list - finishRemove(next) - return true - } - } - } - - public open fun describeRemove() : AtomicDesc? { - if (isRemoved) return null // fast path if was already removed - return object : AbstractAtomicDesc() { - override val affectedNode: Node? get() = this@LockFreeLinkedListNode - override var originalNext: Node? = null - override fun failure(affected: Node, next: Any): Any? = - if (next is Removed) ALREADY_REMOVED else null - override fun onPrepare(affected: Node, next: Node): Any? { - originalNext = next - return null // always success - } - override fun updatedNext(affected: Node, next: Node) = next.removed() - override fun finishOnSuccess(affected: Node, next: Node) = finishRemove(next) - } - } - - public fun removeFirstOrNull(): Node? { - while (true) { // try to linearize - val first = next as Node - if (first === this) return null - if (first.remove()) return first - first.helpDelete() // must help delete, or loose lock-freedom - } - } - - public fun describeRemoveFirst(): RemoveFirstDesc = RemoveFirstDesc(this) - - public inline fun removeFirstIfIsInstanceOf(): T? { - while (true) { // try to linearize - val first = next as Node - if (first === this) return null - if (first !is T) return null - if (first.remove()) return first - first.helpDelete() // must help delete, or loose lock-freedom - } - } - - // just peek at item when predicate is true - public inline fun removeFirstIfIsInstanceOfOrPeekIf(predicate: (T) -> Boolean): T? { - while (true) { // try to linearize - val first = next as Node - if (first === this) return null - if (first !is T) return null - if (predicate(first)) return first // just peek when predicate is true - if (first.remove()) return first - first.helpDelete() // must help delete, or loose lock-freedom - } - } - - // ------ multi-word atomic operations helpers ------ - - public open class AddLastDesc( - @JvmField val queue: Node, - @JvmField val node: T - ) : AbstractAtomicDesc() { - init { - // require freshly allocated node here - check(node._next === node && node._prev === node) - } - - final override fun takeAffectedNode(op: OpDescriptor): Node { - while (true) { - val prev = queue._prev as Node // this sentinel node is never removed - val next = prev._next - if (next === queue) return prev // all is good -> linked properly - if (next === op) return prev // all is good -> our operation descriptor is already there - if (next is OpDescriptor) { // some other operation descriptor -> help & retry - next.perform(prev) - continue - } - // linked improperly -- help insert - queue.helpInsert(prev, op) - } - } - - final override var affectedNode: Node? = null - final override val originalNext: Node? get() = queue - - override fun retry(affected: Node, next: Any): Boolean = next !== queue - - override fun onPrepare(affected: Node, next: Node): Any? { - affectedNode = affected - return null // always success - } - - override fun updatedNext(affected: Node, next: Node): Any { - // it is invoked only on successfully completion of operation, but this invocation can be stale, - // so we must use CAS to set both prev & next pointers - PREV.compareAndSet(node, node, affected) - NEXT.compareAndSet(node, node, queue) - return node - } - - override fun finishOnSuccess(affected: Node, next: Node) { - node.finishAdd(queue) - } - } - - public open class RemoveFirstDesc( - @JvmField val queue: Node - ) : AbstractAtomicDesc() { - @Suppress("UNCHECKED_CAST") - public val result: T get() = affectedNode!! as T - - final override fun takeAffectedNode(op: OpDescriptor): Node = queue.next as Node - final override var affectedNode: Node? = null - final override var originalNext: Node? = null - - // check node predicates here, must signal failure if affect is not of type T - protected override fun failure(affected: Node, next: Any): Any? = - if (affected === queue) LIST_EMPTY else null - - // validate the resulting node (return false if it should be deleted) - protected open fun validatePrepared(node: T): Boolean = true // false means remove node & retry - - final override fun retry(affected: Node, next: Any): Boolean { - if (next !is Removed) return false - affected.helpDelete() // must help delete, or loose lock-freedom - return true - } - - @Suppress("UNCHECKED_CAST") - final override fun onPrepare(affected: Node, next: Node): Any? { - check(affected !is LockFreeLinkedListHead) - if (!validatePrepared(affected as T)) return REMOVE_PREPARED - affectedNode = affected - originalNext = next - return null // ok - } - - final override fun updatedNext(affected: Node, next: Node): Any = next.removed() - final override fun finishOnSuccess(affected: Node, next: Node) = affected.finishRemove(next) - } - - public abstract class AbstractAtomicDesc : AtomicDesc() { - protected abstract val affectedNode: Node? - protected abstract val originalNext: Node? - protected open fun takeAffectedNode(op: OpDescriptor): Node = affectedNode!! - protected open fun failure(affected: Node, next: Any): Any? = null // next: Node | Removed - protected open fun retry(affected: Node, next: Any): Boolean = false // next: Node | Removed - protected abstract fun onPrepare(affected: Node, next: Node): Any? // non-null on failure - protected abstract fun updatedNext(affected: Node, next: Node): Any - protected abstract fun finishOnSuccess(affected: Node, next: Node) - - // This is Harris's RDCSS (Restricted Double-Compare Single Swap) operation - // It inserts "op" descriptor of when "op" status is still undecided (rolls back otherwise) - private class PrepareOp( - @JvmField val next: Node, - @JvmField val op: AtomicOp, - @JvmField val desc: AbstractAtomicDesc - ) : OpDescriptor() { - override fun perform(affected: Any?): Any? { - affected as Node // type assertion - val decision = desc.onPrepare(affected, next) - if (decision != null) { - if (decision === REMOVE_PREPARED) { - // remove element on failure - val removed = next.removed() - if (NEXT.compareAndSet(affected, this, removed)) { - affected.helpDelete() - } - } else { - // some other failure -- mark as decided - op.tryDecide(decision) - // undo preparations - NEXT.compareAndSet(affected, this, next) - } - return decision - } - check(desc.affectedNode === affected) - check(desc.originalNext === next) - val update: Any = if (op.isDecided) next else op // restore if decision was already reached - NEXT.compareAndSet(affected, this, update) - return null // ok - } - } - - final override fun prepare(op: AtomicOp): Any? { - while (true) { // lock free loop on next - val affected = takeAffectedNode(op) - // read its original next pointer first - val next = affected._next - // then see if already reached consensus on overall operation - if (op.isDecided) return null // already decided -- go to next desc - if (next === op) return null // already in process of operation -- all is good - if (next is OpDescriptor) { - // some other operation is in process -- help it - next.perform(affected) - continue // and retry - } - // next: Node | Removed - val failure = failure(affected, next) - if (failure != null) return failure // signal failure - if (retry(affected, next)) continue // retry operation - val prepareOp = PrepareOp(next as Node, op, this) - if (NEXT.compareAndSet(affected, next, prepareOp)) { - // prepared -- complete preparations - val prepFail = prepareOp.perform(affected) - if (prepFail === REMOVE_PREPARED) continue // retry - return prepFail - } - } - } - - final override fun complete(op: AtomicOp, failure: Any?) { - val success = failure == null - val affectedNode = affectedNode ?: run { check(!success); return } - val originalNext = this.originalNext ?: run { check(!success); return } - val update = if (success) updatedNext(affectedNode, originalNext) else originalNext - if (NEXT.compareAndSet(affectedNode, op, update)) { - if (success) finishOnSuccess(affectedNode, originalNext) - } - } - } - - // ------ other helpers ------ - - private fun finishAdd(next: Node) { - while (true) { - val nextPrev = next._prev - if (nextPrev is Removed || this.next !== next) return // next was removed, remover fixes up links - if (PREV.compareAndSet(next, nextPrev, this)) { - if (this.next is Removed) { - // already removed - next.helpInsert(nextPrev as Node, null) - } - return - } - } - } - - private fun finishRemove(next: Node) { - helpDelete() - next.helpInsert(_prev.unwrap(), null) - } - - private fun markPrev(): Node { - while (true) { // lock-free loop on prev - val prev = this._prev - if (prev is Removed) return prev.ref - if (PREV.compareAndSet(this, prev, (prev as Node).removed())) return prev - } - } - - // fixes next links to the left of this node - @PublishedApi - internal fun helpDelete() { - var last: Node? = null // will set to the node left of prev when found - var prev: Node = markPrev() - var next: Node = (this._next as Removed).ref - while (true) { - // move to the right until first non-removed node - val nextNext = next.next - if (nextNext is Removed) { - next.markPrev() - next = nextNext.ref - continue - } - // move the the left until first non-removed node - val prevNext = prev.next - if (prevNext is Removed) { - if (last != null) { - prev.markPrev() - NEXT.compareAndSet(last, prev, prevNext.ref) - prev = last - last = null - } else { - prev = prev._prev.unwrap() - } - continue - } - if (prevNext !== this) { - // skipped over some removed nodes to the left -- setup to fixup the next links - last = prev - prev = prevNext as Node - if (prev === next) return // already done!!! - continue - } - // Now prev & next are Ok - if (NEXT.compareAndSet(prev, this, next)) return // success! - } - } - - // fixes prev links from this node - private fun helpInsert(_prev: Node, op: OpDescriptor?) { - var prev: Node = _prev - var last: Node? = null // will be set so that last.next === prev - while (true) { - // move the the left until first non-removed node - val prevNext = prev._next - if (prevNext === op) return // part of the same op -- don't recurse - if (prevNext is OpDescriptor) { // help & retry - prevNext.perform(prev) - continue - } - if (prevNext is Removed) { - if (last !== null) { - prev.markPrev() - NEXT.compareAndSet(last, prev, prevNext.ref) - prev = last - last = null - } else { - prev = prev._prev.unwrap() - } - continue - } - val oldPrev = this._prev - if (oldPrev is Removed) return // this node was removed, too -- its remover will take care - if (prevNext !== this) { - // need to fixup next - last = prev - prev = prevNext as Node - continue - } - if (oldPrev === prev) return // it is already linked as needed - if (PREV.compareAndSet(this, oldPrev, prev)) { - if (prev._prev !is Removed) return // finish only if prev was not concurrently removed - } - } - } - - internal fun validateNode(prev: Node, next: Node) { - check(prev === this._prev) - check(next === this._next) - } - - override fun toString(): String = "${this::class.java.simpleName}@${Integer.toHexString(System.identityHashCode(this))}" -} - -private class Removed(@JvmField val ref: Node) { - override fun toString(): String = "Removed[$ref]" -} - -@PublishedApi -internal fun Any.unwrap(): Node = if (this is Removed) ref else this as Node - -/** - * Head (sentinel) item of the linked list that is never removed. - * - * @suppress **This is unstable API and it is subject to change.** - */ -public open class LockFreeLinkedListHead : LockFreeLinkedListNode() { - public val isEmpty: Boolean get() = next === this - - /** - * Iterates over all elements in this list of a specified type. - */ - public inline fun forEach(block: (T) -> Unit) { - var cur: Node = next as Node - while (cur != this) { - if (cur is T) block(cur) - cur = cur.next.unwrap() - } - } - - // just a defensive programming -- makes sure that list head sentinel is never removed - public final override fun remove() = throw UnsupportedOperationException() - public final override fun describeRemove(): AtomicDesc? = throw UnsupportedOperationException() - - internal fun validate() { - var prev: Node = this - var cur: Node = next as Node - while (cur != this) { - val next = cur.next.unwrap() - cur.validateNode(prev, next) - prev = cur - cur = next - } - validateNode(prev, next as Node) - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/internal/Symbol.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/internal/Symbol.kt deleted file mode 100644 index 177715b10c..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/internal/Symbol.kt +++ /dev/null @@ -1,22 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.internal - -/** @suppress **This is unstable API and it is subject to change.** */ -public class Symbol(val symbol: String) { - override fun toString(): String = symbol -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/intrinsics/Undispatched.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/intrinsics/Undispatched.kt deleted file mode 100644 index e5b18c0b77..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/intrinsics/Undispatched.kt +++ /dev/null @@ -1,55 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.intrinsics - -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.intrinsics.* -import kotlin.coroutines.experimental.suspendCoroutine - -/** - * Use this function to restart coroutine directly from inside of [suspendCoroutine]. - * - * @suppress **This is unstable API and it is subject to change.** - */ -@Suppress("PLATFORM_CLASS_MAPPED_TO_KOTLIN", "UNCHECKED_CAST") -internal fun (suspend () -> R).startCoroutineUndispatched(completion: Continuation) { - val value = try { - startCoroutineUninterceptedOrReturn(completion) - } catch (e: Throwable) { - completion.resumeWithException(e) - return - } - if (value !== COROUTINE_SUSPENDED) - completion.resume(value as R) -} - -/** - * Use this function to restart coroutine directly from inside of [suspendCoroutine]. - * - * @suppress **This is unstable API and it is subject to change.** - */ -@Suppress("PLATFORM_CLASS_MAPPED_TO_KOTLIN", "UNCHECKED_CAST") -internal fun (suspend (E) -> R).startCoroutineUndispatched(element: E, completion: Continuation) { - val value = try { - startCoroutineUninterceptedOrReturn(element, completion) - } catch (e: Throwable) { - completion.resumeWithException(e) - return - } - if (value !== COROUTINE_SUSPENDED) - completion.resume(value as R) -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/selects/Select.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/selects/Select.kt deleted file mode 100644 index fc75e1742c..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/selects/Select.kt +++ /dev/null @@ -1,277 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.ClosedReceiveChannelException -import kotlinx.coroutines.experimental.channels.ClosedSendChannelException -import kotlinx.coroutines.experimental.channels.ReceiveChannel -import kotlinx.coroutines.experimental.channels.SendChannel -import kotlinx.coroutines.experimental.internal.AtomicDesc -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import kotlinx.coroutines.experimental.sync.Mutex -import java.util.concurrent.TimeUnit -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.ContinuationInterceptor -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.intrinsics.suspendCoroutineOrReturn -import kotlin.coroutines.experimental.startCoroutine - -/** - * Scope for [select] invocation. - */ -public interface SelectBuilder { - /** - * Clause for [Job.join] suspending function that selects the given [block] when the job is complete. - * This clause never fails, even if the job completes exceptionally. - */ - public fun Job.onJoin(block: suspend () -> R) - - /** - * Clause for [Deferred.await] suspending function that selects the given [block] with the deferred value is - * resolved. The [select] invocation fails if the deferred value completes exceptionally (either fails or - * it cancelled). - */ - public fun Deferred.onAwait(block: suspend (T) -> R) - - /** - * Clause for [SendChannel.send] suspending function that selects the given [block] when the [element] is sent to - * the channel. The [select] invocation fails with [ClosedSendChannelException] if the channel - * [isClosedForSend][SendChannel.isClosedForSend] _normally_ or with the original - * [close][SendChannel.close] cause exception if the channel has _failed_. - */ - public fun SendChannel.onSend(element: E, block: suspend () -> R) - - /** - * Clause for [ReceiveChannel.receive] suspending function that selects the given [block] with the element that - * is received from the channel. The [select] invocation fails with [ClosedReceiveChannelException] if the channel - * [isClosedForReceive][ReceiveChannel.isClosedForReceive] _normally_ or with the original - * [close][SendChannel.close] cause exception if the channel has _failed_. - */ - public fun ReceiveChannel.onReceive(block: suspend (E) -> R) - - /** - * Clause for [ReceiveChannel.receiveOrNull] suspending function that selects the given [block] with the element that - * is received from the channel or selects the given [block] with `null` if if the channel - * [isClosedForReceive][ReceiveChannel.isClosedForReceive] _normally_. The [select] invocation fails with - * the original [close][SendChannel.close] cause exception if the channel has _failed_. - */ - public fun ReceiveChannel.onReceiveOrNull(block: suspend (E?) -> R) - - /** - * Clause for [Mutex.lock] suspending function that selects the given [block] when the mutex is locked. - * - * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex - * is already locked with the same token (same identity), this clause throws [IllegalStateException]. - */ - public fun Mutex.onLock(owner: Any? = null, block: suspend () -> R) - - /** - * Clause that selects the given [block] after a specified timeout passes. - * - * @param time timeout time - * @param unit timeout unit (milliseconds by default) - */ - public fun onTimeout(time: Long, unit: TimeUnit = TimeUnit.MILLISECONDS, block: suspend () -> R) -} - -/** - * Internal representation of select instance. This instance is called _selected_ when - * the clause to execute is already picked. - * - * @suppress **This is unstable API and it is subject to change.** - */ -public interface SelectInstance { - /** - * Returns `true` when this [select] statement had already picked a clause to execute. - */ - public val isSelected: Boolean - - /** - * Tries to select this instance. - */ - public fun trySelect(idempotent: Any?): Boolean - - /** - * Performs action atomically with [trySelect]. - */ - public fun performAtomicTrySelect(desc: AtomicDesc): Any? - - /** - * Performs action atomically when [isSelected] is `false`. - */ - public fun performAtomicIfNotSelected(desc: AtomicDesc): Any? - - /** - * Returns completion continuation of this select instance. - * This select instance must be _selected_ first. - * All resumption through this instance happen _directly_ (as if `mode` is [MODE_DIRECT]). - */ - public val completion: Continuation - - public fun resumeSelectWithException(exception: Throwable, mode: Int) - - // This function is actually implemented to dispose the handle only when the whole - // select expression complete. It is later than it could be, but if resource will get released anyway - // :todo: Invoke this function on select really - public fun disposeOnSelect(handle: DisposableHandle) -} - -/** - * Waits for the result of multiple suspending functions simultaneously, which are specified using _clauses_ - * in the [builder] scope of this select invocation. The caller is suspended until one of the clauses - * is either _selected_ or _fails_. - * - * At most one clause is *atomically* selected and its block is executed. The result of the selected clause - * becomes the result of the select. If any clause _fails_, then the select invocation produces the - * corresponding exception. No clause is selected in this case. - * - * This select function is _biased_ to the first clause. When multiple clauses can be selected at the same time, - * the first one of them gets priority. Use [selectUnbiased] for an unbiased (randomized) selection among - * the clauses. - - * There is no `default` clause for select expression. Instead, each selectable suspending function has the - * corresponding non-suspending version that can be used with a regular `when` expression to select one - * of the alternatives or to perform default (`else`) action if none of them can be immediately selected. - * - * | **Receiver** | **Suspending function** | **Select clause** | **Non-suspending version** - * | ---------------- | --------------------------------------------- | ------------------------------------------------ | -------------------------- - * | [Job] | [join][Job.join] | [onJoin][SelectBuilder.onJoin] | [isCompleted][Job.isCompleted] - * | [Deferred] | [await][Deferred.await] | [onAwait][SelectBuilder.onAwait] | [isCompleted][Job.isCompleted] - * | [SendChannel] | [send][SendChannel.send] | [onSend][SelectBuilder.onSend] | [offer][SendChannel.offer] - * | [ReceiveChannel] | [receive][ReceiveChannel.receive] | [onReceive][SelectBuilder.onReceive] | [poll][ReceiveChannel.poll] - * | [ReceiveChannel] | [receiveOrNull][ReceiveChannel.receiveOrNull] | [onReceiveOrNull][SelectBuilder.onReceiveOrNull] | [poll][ReceiveChannel.poll] - * | [Mutex] | [lock][Mutex.lock] | [onLock][SelectBuilder.onLock] | [tryLock][Mutex.tryLock] - * | none | [delay] | [onTimeout][SelectBuilder.onTimeout] | none - * - * This suspending function is cancellable. If the [Job] of the current coroutine is completed while this - * function is suspended, this function immediately resumes with [CancellationException]. - * Cancellation of suspended select is *atomic* -- when this function - * throws [CancellationException] it means that no clause was selected. - * - * Note, that this function does not check for cancellation when it is not suspended. - * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. - */ -public inline suspend fun select(crossinline builder: SelectBuilder.() -> Unit): R = - suspendCoroutineOrReturn { cont -> - val scope = SelectBuilderImpl(cont) - try { - builder(scope) - } catch (e: Throwable) { - scope.handleBuilderException(e) - } - scope.initSelectResult() - } - -/* - :todo: It is best to rewrite this class without the use of CancellableContinuationImpl and JobSupport infrastructure - This way JobSupport will not have to provide trySelect(idempotent) operation can can save some checks and bytes - to carry on that idempotent start token. - */ -@PublishedApi -internal class SelectBuilderImpl( - delegate: Continuation -) : CancellableContinuationImpl(delegate, active = false), SelectBuilder, SelectInstance { - @PublishedApi - internal fun handleBuilderException(e: Throwable) { - if (trySelect(idempotent = null)) { - val token = tryResumeWithException(e) - if (token != null) - completeResume(token) - else - handleCoroutineException(context, e) - } - } - - @PublishedApi - internal fun initSelectResult(): Any? { - if (!isSelected) initCancellability() - return getResult() - } - - // coroutines that are started inside this select are directly subordinate to the parent job - override fun createContext(): CoroutineContext = delegate.context - - override fun onParentCompletion(cause: Throwable?) { - /* - Select is cancelled only when no clause was selected yet. If a clause was selected, then - it is the concern of the coroutine that was started by that clause to cancel on its suspension - points. - */ - if (trySelect(null)) - cancel(cause) - } - - override val defaultResumeMode get() = MODE_DIRECT // all resumes through completion are dispatched directly - - override val completion: Continuation get() { - check(isSelected) { "Must be selected first" } - return this - } - - override fun resumeSelectWithException(exception: Throwable, mode: Int) { - check(isSelected) { "Must be selected first" } - resumeWithException(exception, mode) - } - - override fun Job.onJoin(block: suspend () -> R) { - registerSelectJoin(this@SelectBuilderImpl, block) - } - - override fun Deferred.onAwait(block: suspend (T) -> R) { - registerSelectAwait(this@SelectBuilderImpl, block) - } - - override fun SendChannel.onSend(element: E, block: suspend () -> R) { - registerSelectSend(this@SelectBuilderImpl, element, block) - } - - override fun ReceiveChannel.onReceive(block: suspend (E) -> R) { - registerSelectReceive(this@SelectBuilderImpl, block) - } - - override fun ReceiveChannel.onReceiveOrNull(block: suspend (E?) -> R) { - registerSelectReceiveOrNull(this@SelectBuilderImpl, block) - } - - override fun Mutex.onLock(owner: Any?, block: suspend () -> R) { - registerSelectLock(this@SelectBuilderImpl, owner, block) - } - - override fun onTimeout(time: Long, unit: TimeUnit, block: suspend () -> R) { - require(time >= 0) { "Timeout time $time cannot be negative" } - if (time == 0L) { - if (trySelect(idempotent = null)) - block.startCoroutineUndispatched(completion) - return - } - val action = Runnable { - // todo: we could have replaced startCoroutine with startCoroutineUndispatched - // But we need a way to know that Delay.invokeOnTimeout had used the right thread - if (trySelect(idempotent = null)) - block.startCoroutine(completion) - } - val delay = context[ContinuationInterceptor] as? Delay - if (delay != null) - disposeOnSelect(delay.invokeOnTimeout(time, unit, action)) else - cancelFutureOnCompletion(scheduledExecutor.schedule(action, time, unit)) - } - - override fun disposeOnSelect(handle: DisposableHandle) { - invokeOnCompletion(DisposeOnCompletion(this, handle)) - } -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/selects/SelectUnbiased.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/selects/SelectUnbiased.kt deleted file mode 100644 index a08f976112..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/selects/SelectUnbiased.kt +++ /dev/null @@ -1,99 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.Deferred -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.channels.ReceiveChannel -import kotlinx.coroutines.experimental.channels.SendChannel -import kotlinx.coroutines.experimental.sync.Mutex -import java.util.* -import java.util.concurrent.TimeUnit -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.intrinsics.suspendCoroutineOrReturn - -/** - * Waits for the result of multiple suspending functions simultaneously like [select], but in an _unbiased_ - * way when multiple clauses are selectable at the same time. - * - * This unbiased implementation of `select` expression randomly shuffles the clauses before checking - * if they are selectable, thus ensuring that there is no statistical bias to the selection of the first - * clauses. - * - * See [select] function description for all the other details. - */ -public inline suspend fun selectUnbiased(crossinline builder: SelectBuilder.() -> Unit): R = - suspendCoroutineOrReturn { cont -> - val scope = UnbiasedSelectBuilderImpl(cont) - try { - builder(scope) - } catch (e: Throwable) { - scope.handleBuilderException(e) - } - scope.initSelectResult() - } - - -@PublishedApi -internal class UnbiasedSelectBuilderImpl(cont: Continuation) : SelectBuilder { - val instance = SelectBuilderImpl(cont) - val clauses = arrayListOf<() -> Unit>() - - @PublishedApi - internal fun handleBuilderException(e: Throwable) = instance.handleBuilderException(e) - - @PublishedApi - internal fun initSelectResult(): Any? { - if (!instance.isSelected) { - try { - Collections.shuffle(clauses) - clauses.forEach { it.invoke() } - } catch (e: Throwable) { - instance.handleBuilderException(e) - } - } - return instance.initSelectResult() - } - - override fun Job.onJoin(block: suspend () -> R) { - clauses += { registerSelectJoin(instance, block) } - } - - override fun Deferred.onAwait(block: suspend (T) -> R) { - clauses += { registerSelectAwait(instance, block) } - } - - override fun SendChannel.onSend(element: E, block: suspend () -> R) { - clauses += { registerSelectSend(instance, element, block) } - } - - override fun ReceiveChannel.onReceive(block: suspend (E) -> R) { - clauses += { registerSelectReceive(instance, block) } - } - - override fun ReceiveChannel.onReceiveOrNull(block: suspend (E?) -> R) { - clauses += { registerSelectReceiveOrNull(instance, block) } - } - - override fun Mutex.onLock(owner: Any?, block: suspend () -> R) { - clauses += { registerSelectLock(instance, owner, block) } - } - - override fun onTimeout(time: Long, unit: TimeUnit, block: suspend () -> R) { - clauses += { instance.onTimeout(time, unit, block) } - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/selects/WhileSelect.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/selects/WhileSelect.kt deleted file mode 100644 index 05486853ac..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/selects/WhileSelect.kt +++ /dev/null @@ -1,39 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -/** - * Loops while [select] expression returns `true`. - * - * The statement of the form: - * - * ``` - * whileSelect { - * /*body*/ - * } - * ``` - * - * is a shortcut for: - * - * ``` - * while(select { - * /*body*/ - * }) {} - */ -suspend fun whileSelect(builder: SelectBuilder.() -> Unit) { - while(select(builder)) {} -} diff --git a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/sync/Mutex.kt b/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/sync/Mutex.kt deleted file mode 100644 index 1a935f0743..0000000000 --- a/kotlinx-coroutines-core/src/main/kotlin/kotlinx/coroutines/experimental/sync/Mutex.kt +++ /dev/null @@ -1,426 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.sync - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.internal.* -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import kotlinx.coroutines.experimental.selects.SelectBuilder -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlinx.coroutines.experimental.selects.select -import java.util.concurrent.atomic.AtomicReferenceFieldUpdater -import kotlin.coroutines.experimental.startCoroutine - -/** - * Mutual exclusion for coroutines. - * - * Mutex has two states: _locked_ and _unlocked_. - * It is **non-reentrant**, that is invoking [lock] even from the same thread/coroutine that currently holds - * the lock still suspends the invoker. - */ -public interface Mutex { - /** - * Factory for [Mutex] instances. - */ - public companion object Factory { - /** - * Creates new [Mutex] instance. - * @param locked initial state of the mutex. - */ - public operator fun invoke(locked: Boolean = false) : Mutex = MutexImpl(locked) - } - - /** - * Returns `true` when this mutex is locked. - */ - public val isLocked: Boolean - - /** - * Tries to lock this mutex, returning `false` if this mutex is already locked. - * - * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex - * is already locked with the same token (same identity), this function throws [IllegalStateException]. - */ - public fun tryLock(owner: Any? = null): Boolean - - /** - * Locks this mutex, suspending caller while the mutex is locked. - * - * This suspending function is cancellable. If the [Job] of the current coroutine is completed while this - * function is suspended, this function immediately resumes with [CancellationException]. - * Cancellation of suspended lock invocation is *atomic* -- when this function - * throws [CancellationException] it means that the mutex was not locked. - * - * Note, that this function does not check for cancellation when it is not suspended. - * Use [yield] or [CoroutineScope.isActive] to periodically check for cancellation in tight loops if needed. - * - * This function can be used in [select] invocation with [onLock][SelectBuilder.onLock] clause. - * Use [tryLock] to try acquire lock without waiting. - * - * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex - * is already locked with the same token (same identity), this function throws [IllegalStateException]. - */ - public suspend fun lock(owner: Any? = null) - - /** - * Registers [onLock][SelectBuilder.onLock] select clause. - * @suppress **This is unstable API and it is subject to change.** - */ - public fun registerSelectLock(select: SelectInstance, owner: Any?, block: suspend () -> R) - - /** - * Unlocks this mutex. Throws [IllegalStateException] if invoked on a mutex that is not locked. - * - * @param owner Optional owner token for debugging. When `owner` is specified (non-null value) and this mutex - * was locked with the different token (by identity), this function throws [IllegalStateException]. - */ - public fun unlock(owner: Any? = null) -} - -/** - * Executes the given [action] under this mutex's lock. - * @return the return value of the action. - */ -// :todo: this function needs to be make inline as soon as this bug is fixed: https://youtrack.jetbrains.com/issue/KT-16448 -public suspend fun Mutex.withLock(action: suspend () -> T): T { - lock() - try { - return action() - } finally { - unlock() - } -} - -/** - * @suppress: **Deprecated**: Use [withLock] - */ -@Deprecated("Use `withLock`", replaceWith = ReplaceWith("withLock(action)")) -public suspend fun Mutex.withMutex(action: suspend () -> T): T { - lock() - try { - return action() - } finally { - unlock() - } -} - -internal class MutexImpl(locked: Boolean) : Mutex { - // State is: Empty | LockedQueue | OpDescriptor - @Volatile - private var _state: Any? = if (locked) EmptyLocked else EmptyUnlocked // shared objects while we have no waiters - - private companion object { - @JvmField - val STATE: AtomicReferenceFieldUpdater = - AtomicReferenceFieldUpdater.newUpdater(MutexImpl::class.java, Any::class.java, "_state") - - @JvmField - val LOCK_FAIL = Symbol("LOCK_FAIL") - - @JvmField - val ENQUEUE_FAIL = Symbol("ENQUEUE_FAIL") - - @JvmField - val UNLOCK_FAIL = Symbol("UNLOCK_FAIL") - - @JvmField - val SELECT_SUCCESS = Symbol("SELECT_SUCCESS") - - @JvmField - val LOCKED = Symbol("LOCKED") - - @JvmField - val UNLOCKED = Symbol("UNLOCKED") - - @JvmField - val EmptyLocked = Empty(LOCKED) - - @JvmField - val EmptyUnlocked = Empty(UNLOCKED) - - } - - public override val isLocked: Boolean get() { - while (true) { // lock-free loop on state - val state = this._state - when (state) { - is Empty -> return state.locked !== UNLOCKED - is LockedQueue -> return true - is OpDescriptor -> state.perform(this) // help - else -> error("Illegal state $state") - } - } - } - - // for tests - internal val isLockedEmptyQueueState: Boolean get() { - val state = this._state - return state is LockedQueue && state.isEmpty - } - - public override fun tryLock(owner: Any?): Boolean { - while (true) { // lock-free loop on state - val state = this._state - when (state) { - is Empty -> { - if (state.locked !== UNLOCKED) return false - val update = if (owner == null) EmptyLocked else Empty(owner) - if (STATE.compareAndSet(this, state, update)) return true - } - is LockedQueue -> { - check(state.owner !== owner) { "Already locked by $owner" } - return false - } - is OpDescriptor -> state.perform(this) // help - else -> error("Illegal state $state") - } - } - } - - public override suspend fun lock(owner: Any?) { - // fast-path -- try lock - if (tryLock(owner)) return - // slow-path -- suspend - return lockSuspend(owner) - } - - private suspend fun lockSuspend(owner: Any?) = suspendCancellableCoroutine(holdCancellability = true) sc@ { cont -> - val waiter = LockCont(owner, cont) - while (true) { // lock-free loop on state - val state = this._state - when (state) { - is Empty -> { - if (state.locked !== UNLOCKED) { // try upgrade to queue & retry - STATE.compareAndSet(this, state, LockedQueue(state.locked)) - } else { - // try lock - val update = if (owner == null) EmptyLocked else Empty(owner) - if (STATE.compareAndSet(this, state, update)) { // locked - cont.resume(Unit) - return@sc - } - } - } - is LockedQueue -> { - val curOwner = state.owner - check(curOwner !== owner) { "Already locked by $owner" } - if (state.addLastIf(waiter, { this._state === state })) { - // added to waiter list! - cont.initCancellability() - cont.removeOnCancel(waiter) - return@sc - } - } - is OpDescriptor -> state.perform(this) // help - else -> error("Illegal state $state") - } - } - } - - override fun registerSelectLock(select: SelectInstance, owner: Any?, block: suspend () -> R) { - while (true) { // lock-free loop on state - if (select.isSelected) return - val state = this._state - when (state) { - is Empty -> { - if (state.locked !== UNLOCKED) { // try upgrade to queue & retry - STATE.compareAndSet(this, state, LockedQueue(state.locked)) - } else { - // try lock - val failure = select.performAtomicTrySelect(TryLockDesc(this, owner)) - when { - failure == null -> { // success - block.startCoroutineUndispatched(select.completion) - return - } - failure === ALREADY_SELECTED -> return // already selected -- bail out - failure === LOCK_FAIL -> {} // retry - else -> error("performAtomicTrySelect(TryLockDesc) returned $failure") - } - } - } - is LockedQueue -> { - check(state.owner !== owner) { "Already locked by $owner" } - val enqueueOp = TryEnqueueLockDesc(this, owner, state, select, block) - val failure = select.performAtomicIfNotSelected(enqueueOp) - when { - failure == null -> { // successfully enqueued - select.disposeOnSelect(enqueueOp.node) - return - } - failure === ALREADY_SELECTED -> return // already selected -- bail out - failure === ENQUEUE_FAIL -> {} // retry - else -> error("performAtomicIfNotSelected(TryEnqueueLockDesc) returned $failure") - } - } - is OpDescriptor -> state.perform(this) // help - else -> error("Illegal state $state") - } - } - } - - private class TryLockDesc( - @JvmField val mutex: MutexImpl, - @JvmField val owner: Any? - ) : AtomicDesc() { - // This is Harris's RDCSS (Restricted Double-Compare Single Swap) operation - private inner class PrepareOp(private val op: AtomicOp) : OpDescriptor() { - override fun perform(affected: Any?): Any? { - val update: Any = if (op.isDecided) EmptyUnlocked else op // restore if was already decided - STATE.compareAndSet(affected as MutexImpl, this, update) - return null // ok - } - } - - override fun prepare(op: AtomicOp): Any? { - val prepare = PrepareOp(op) - if (!STATE.compareAndSet(mutex, EmptyUnlocked, prepare)) return LOCK_FAIL - return prepare.perform(mutex) - } - - override fun complete(op: AtomicOp, failure: Any?) { - val update = if (failure != null) EmptyUnlocked else { - if (owner == null) EmptyLocked else Empty(owner) - } - STATE.compareAndSet(mutex, op, update) - } - } - - private class TryEnqueueLockDesc( - @JvmField val mutex: MutexImpl, - owner: Any?, - queue: LockedQueue, - select: SelectInstance, - block: suspend () -> R - ) : AddLastDesc>(queue, LockSelect(owner, select, block)) { - override fun onPrepare(affected: LockFreeLinkedListNode, next: LockFreeLinkedListNode): Any? { - if (mutex._state !== queue) return ENQUEUE_FAIL - return super.onPrepare(affected, next) - } - } - - public override fun unlock(owner: Any?) { - while (true) { // lock-free loop on state - val state = this._state - when (state) { - is Empty -> { - if (owner == null) - check(state.locked !== UNLOCKED) { "Mutex is not locked" } - else - check(state.locked === owner) { "Mutex is locked by ${state.locked} but expected $owner" } - if (STATE.compareAndSet(this, state, EmptyUnlocked)) return - } - is OpDescriptor -> state.perform(this) - is LockedQueue -> { - if (owner != null) - check(state.owner === owner) { "Mutex is locked by ${state.owner} but expected $owner" } - val waiter = state.removeFirstOrNull() - if (waiter == null) { - val op = UnlockOp(state) - if (STATE.compareAndSet(this, state, op) && op.perform(this) == null) return - } else { - val token = (waiter as LockWaiter).tryResumeLockWaiter() - if (token != null) { // successfully resumed waiter that now is holding the lock - state.owner = waiter.owner ?: LOCKED - waiter.completeResumeLockWaiter(token) - return - } - } - } - else -> error("Illegal state $state") - } - } - } - - override fun toString(): String { - while (true) { - val state = this._state - when (state) { - is Empty -> return "Mutex[${state.locked}]" - is OpDescriptor -> state.perform(this) - is LockedQueue -> return "Mutex[${state.owner}]" - else -> error("Illegal state $state") - } - } - } - - private class Empty( - @JvmField val locked: Any - ) { - override fun toString(): String = "Empty[$locked]" - } - - private class LockedQueue( - @JvmField var owner: Any - ) : LockFreeLinkedListHead() { - override fun toString(): String = "LockedQueue[$owner]" - } - - private abstract class LockWaiter( - @JvmField val owner: Any? - ) : LockFreeLinkedListNode(), DisposableHandle { - final override fun dispose() { remove() } - abstract fun tryResumeLockWaiter(): Any? - abstract fun completeResumeLockWaiter(token: Any) - } - - private class LockCont( - owner: Any?, - @JvmField val cont: CancellableContinuation - ) : LockWaiter(owner) { - override fun tryResumeLockWaiter() = cont.tryResume(Unit) - override fun completeResumeLockWaiter(token: Any) = cont.completeResume(token) - override fun toString(): String = "LockCont[$owner, $cont]" - } - - private class LockSelect( - owner: Any?, - @JvmField val select: SelectInstance, - @JvmField val block: suspend () -> R - ) : LockWaiter(owner) { - override fun tryResumeLockWaiter(): Any? = if (select.trySelect(null)) SELECT_SUCCESS else null - override fun completeResumeLockWaiter(token: Any) { - check(token === SELECT_SUCCESS) - block.startCoroutine(select.completion) - } - override fun toString(): String = "LockSelect[$owner, $select]" - } - - // atomic unlock operation that checks that waiters queue is empty - private class UnlockOp( - @JvmField val queue: LockedQueue - ) : OpDescriptor() { - override fun perform(affected: Any?): Any? { - /* - Note: queue cannot change while this UnlockOp is in progress, so all concurrent attempts to - make a decision will reach it consistently. It does not matter what is a proposed - decision when this UnlockOp is no longer active, because in this case the following CAS - will fail anyway. - */ - val success = queue.isEmpty - val update: Any = if (success) EmptyUnlocked else queue - STATE.compareAndSet(affected as MutexImpl, this@UnlockOp, update) - /* - `perform` invocation from the original `unlock` invocation may be coming too late, when - some other thread had already helped to complete it (either successfully or not). - That operation was unsuccessful if `state` was restored to this `queue` reference and - that is what is being checked below. - */ - return if (affected._state === queue) UNLOCK_FAIL else null - } - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-01.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-01.kt deleted file mode 100644 index b920fe85c7..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-01.kt +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.basic.example01 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) { - launch(CommonPool) { // create new coroutine in common thread pool - delay(1000L) // non-blocking delay for 1 second (default time unit is ms) - println("World!") // print after delay - } - println("Hello,") // main function continues while coroutine is delayed - Thread.sleep(2000L) // block main thread for 2 seconds to keep JVM alive -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-02.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-02.kt deleted file mode 100644 index 6e3a9ee668..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-02.kt +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.basic.example02 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { // start main coroutine - launch(CommonPool) { // create new coroutine in common thread pool - delay(1000L) - println("World!") - } - println("Hello,") // main coroutine continues while child is delayed - delay(2000L) // non-blocking delay for 2 seconds to keep JVM alive -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-03.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-03.kt deleted file mode 100644 index cc1b748bc4..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-03.kt +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.basic.example03 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { // create new coroutine and keep a reference to its Job - delay(1000L) - println("World!") - } - println("Hello,") - job.join() // wait until child coroutine completes -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-04.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-04.kt deleted file mode 100644 index 231305e761..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-04.kt +++ /dev/null @@ -1,32 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.basic.example04 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { doWorld() } - println("Hello,") - job.join() -} - -// this is your first suspending function -suspend fun doWorld() { - delay(1000L) - println("World!") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-05.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-05.kt deleted file mode 100644 index ec76812864..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-05.kt +++ /dev/null @@ -1,30 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.basic.example05 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val jobs = List(100_000) { // create a lot of coroutines and list their jobs - launch(CommonPool) { - delay(1000L) - print(".") - } - } - jobs.forEach { it.join() } // wait for all jobs to complete -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-06.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-06.kt deleted file mode 100644 index 0cf0422cdc..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-basic-06.kt +++ /dev/null @@ -1,30 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.basic.example06 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - launch(CommonPool) { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } - delay(1300L) // just quit after delay -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-01.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-01.kt deleted file mode 100644 index 4dd92a6437..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-01.kt +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.cancel.example01 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to ensure it was cancelled indeed - println("main: Now I can quit.") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-02.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-02.kt deleted file mode 100644 index 1c658cb618..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-02.kt +++ /dev/null @@ -1,39 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.cancel.example02 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - var nextPrintTime = 0L - var i = 0 - while (i < 10) { // computation loop - val currentTime = System.currentTimeMillis() - if (currentTime >= nextPrintTime) { - println("I'm sleeping ${i++} ...") - nextPrintTime = currentTime + 500L - } - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to see if it was cancelled.... - println("main: Now I can quit.") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-03.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-03.kt deleted file mode 100644 index 619b6b7de9..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-03.kt +++ /dev/null @@ -1,39 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.cancel.example03 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - var nextPrintTime = 0L - var i = 0 - while (isActive) { // cancellable computation loop - val currentTime = System.currentTimeMillis() - if (currentTime >= nextPrintTime) { - println("I'm sleeping ${i++} ...") - nextPrintTime = currentTime + 500L - } - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to see if it was cancelled.... - println("main: Now I can quit.") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-04.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-04.kt deleted file mode 100644 index 28df0844aa..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-04.kt +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.cancel.example04 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - try { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } finally { - println("I'm running finally") - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to ensure it was cancelled indeed - println("main: Now I can quit.") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-05.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-05.kt deleted file mode 100644 index d379c08654..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-05.kt +++ /dev/null @@ -1,42 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.cancel.example05 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val job = launch(CommonPool) { - try { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } finally { - run(NonCancellable) { - println("I'm running finally") - delay(1000L) - println("And I've just delayed for 1 sec because I'm non-cancellable") - } - } - } - delay(1300L) // delay a bit - println("main: I'm tired of waiting!") - job.cancel() // cancels the job - delay(1300L) // delay a bit to ensure it was cancelled indeed - println("main: Now I can quit.") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-06.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-06.kt deleted file mode 100644 index 3320cef211..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-cancel-06.kt +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.cancel.example06 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - withTimeout(1300L) { - repeat(1000) { i -> - println("I'm sleeping $i ...") - delay(500L) - } - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-01.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-01.kt deleted file mode 100644 index 4f560dca82..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-01.kt +++ /dev/null @@ -1,32 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.channel.example01 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* - -fun main(args: Array) = runBlocking { - val channel = Channel() - launch(CommonPool) { - // this might be heavy CPU-consuming computation or async logic, we'll just send five squares - for (x in 1..5) channel.send(x * x) - } - // here we print five received integers: - repeat(5) { println(channel.receive()) } - println("Done!") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-02.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-02.kt deleted file mode 100644 index 3d4160867a..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-02.kt +++ /dev/null @@ -1,32 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.channel.example02 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* - -fun main(args: Array) = runBlocking { - val channel = Channel() - launch(CommonPool) { - for (x in 1..5) channel.send(x * x) - channel.close() // we're done sending - } - // here we print received values using `for` loop (until the channel is closed) - for (y in channel) println(y) - println("Done!") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-03.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-03.kt deleted file mode 100644 index e9132b1643..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-03.kt +++ /dev/null @@ -1,31 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.channel.example03 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* - -fun produceSquares() = produce(CommonPool) { - for (x in 1..5) send(x * x) -} - -fun main(args: Array) = runBlocking { - val squares = produceSquares() - squares.consumeEach { println(it) } - println("Done!") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-04.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-04.kt deleted file mode 100644 index 4d095c4735..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-04.kt +++ /dev/null @@ -1,39 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.channel.example04 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* - -fun produceNumbers() = produce(CommonPool) { - var x = 1 - while (true) send(x++) // infinite stream of integers starting from 1 -} - -fun square(numbers: ReceiveChannel) = produce(CommonPool) { - for (x in numbers) send(x * x) -} - -fun main(args: Array) = runBlocking { - val numbers = produceNumbers() // produces integers from 1 and on - val squares = square(numbers) // squares integers - for (i in 1..5) println(squares.receive()) // print first five - println("Done!") // we are done - squares.cancel() // need to cancel these coroutines in a larger app - numbers.cancel() -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-05.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-05.kt deleted file mode 100644 index d7976e22ab..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-05.kt +++ /dev/null @@ -1,40 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.channel.example05 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* -import kotlin.coroutines.experimental.CoroutineContext - -fun numbersFrom(context: CoroutineContext, start: Int) = produce(context) { - var x = start - while (true) send(x++) // infinite stream of integers from start -} - -fun filter(context: CoroutineContext, numbers: ReceiveChannel, prime: Int) = produce(context) { - for (x in numbers) if (x % prime != 0) send(x) -} - -fun main(args: Array) = runBlocking { - var cur = numbersFrom(context, 2) - for (i in 1..10) { - val prime = cur.receive() - println(prime) - cur = filter(context, cur, prime) - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-06.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-06.kt deleted file mode 100644 index 9c8d66c7b6..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-06.kt +++ /dev/null @@ -1,42 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.channel.example06 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* - -fun produceNumbers() = produce(CommonPool) { - var x = 1 // start from 1 - while (true) { - send(x++) // produce next - delay(100) // wait 0.1s - } -} - -fun launchProcessor(id: Int, channel: ReceiveChannel) = launch(CommonPool) { - channel.consumeEach { - println("Processor #$id received $it") - } -} - -fun main(args: Array) = runBlocking { - val producer = produceNumbers() - repeat(5) { launchProcessor(it, producer) } - delay(1000) - producer.cancel() // cancel producer coroutine and thus kill them all -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-07.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-07.kt deleted file mode 100644 index a9c1f42b47..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-07.kt +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.channel.example07 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* - -suspend fun sendString(channel: SendChannel, s: String, time: Long) { - while (true) { - delay(time) - channel.send(s) - } -} - -fun main(args: Array) = runBlocking { - val channel = Channel() - launch(context) { sendString(channel, "foo", 200L) } - launch(context) { sendString(channel, "BAR!", 500L) } - repeat(6) { // receive first six - println(channel.receive()) - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-08.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-08.kt deleted file mode 100644 index 96a425d15d..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-08.kt +++ /dev/null @@ -1,33 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.channel.example08 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* - -fun main(args: Array) = runBlocking { - val channel = Channel(4) // create buffered channel - launch(context) { // launch sender coroutine - repeat(10) { - println("Sending $it") // print before sending each element - channel.send(it) // will suspend when buffer is full - } - } - // don't receive anything... just wait.... - delay(1000) -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-09.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-09.kt deleted file mode 100644 index 2fdad2d81c..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-channel-09.kt +++ /dev/null @@ -1,41 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.channel.example09 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* - -data class Ball(var hits: Int) - -fun main(args: Array) = runBlocking { - val table = Channel() // a shared table - launch(context) { player("ping", table) } - launch(context) { player("pong", table) } - table.send(Ball(0)) // serve the ball - delay(1000) // delay 1 second - table.receive() // game over, grab the ball -} - -suspend fun player(name: String, table: Channel) { - for (ball in table) { // receive the ball in a loop - ball.hits++ - println("$name $ball") - delay(300) // wait a bit - table.send(ball) // send the ball back - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-01.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-01.kt deleted file mode 100644 index af083f30f2..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-01.kt +++ /dev/null @@ -1,40 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.compose.example01 - -import kotlinx.coroutines.experimental.* -import kotlin.system.measureTimeMillis - -suspend fun doSomethingUsefulOne(): Int { - delay(1000L) // pretend we are doing something useful here - return 13 -} - -suspend fun doSomethingUsefulTwo(): Int { - delay(1000L) // pretend we are doing something useful here, too - return 29 -} - -fun main(args: Array) = runBlocking { - val time = measureTimeMillis { - val one = doSomethingUsefulOne() - val two = doSomethingUsefulTwo() - println("The answer is ${one + two}") - } - println("Completed in $time ms") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-02.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-02.kt deleted file mode 100644 index 63965e93fd..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-02.kt +++ /dev/null @@ -1,40 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.compose.example02 - -import kotlinx.coroutines.experimental.* -import kotlin.system.measureTimeMillis - -suspend fun doSomethingUsefulOne(): Int { - delay(1000L) // pretend we are doing something useful here - return 13 -} - -suspend fun doSomethingUsefulTwo(): Int { - delay(1000L) // pretend we are doing something useful here, too - return 29 -} - -fun main(args: Array) = runBlocking { - val time = measureTimeMillis { - val one = async(CommonPool) { doSomethingUsefulOne() } - val two = async(CommonPool) { doSomethingUsefulTwo() } - println("The answer is ${one.await() + two.await()}") - } - println("Completed in $time ms") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-03.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-03.kt deleted file mode 100644 index cd56a18afa..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-03.kt +++ /dev/null @@ -1,40 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.compose.example03 - -import kotlinx.coroutines.experimental.* -import kotlin.system.measureTimeMillis - -suspend fun doSomethingUsefulOne(): Int { - delay(1000L) // pretend we are doing something useful here - return 13 -} - -suspend fun doSomethingUsefulTwo(): Int { - delay(1000L) // pretend we are doing something useful here, too - return 29 -} - -fun main(args: Array) = runBlocking { - val time = measureTimeMillis { - val one = async(CommonPool, CoroutineStart.LAZY) { doSomethingUsefulOne() } - val two = async(CommonPool, CoroutineStart.LAZY) { doSomethingUsefulTwo() } - println("The answer is ${one.await() + two.await()}") - } - println("Completed in $time ms") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-04.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-04.kt deleted file mode 100644 index aaedcb7a79..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-compose-04.kt +++ /dev/null @@ -1,56 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.compose.example04 - -import kotlinx.coroutines.experimental.* -import kotlin.system.measureTimeMillis - -suspend fun doSomethingUsefulOne(): Int { - delay(1000L) // pretend we are doing something useful here - return 13 -} - -suspend fun doSomethingUsefulTwo(): Int { - delay(1000L) // pretend we are doing something useful here, too - return 29 -} - -// The result type of asyncSomethingUsefulOne is Deferred -fun asyncSomethingUsefulOne() = async(CommonPool) { - doSomethingUsefulOne() -} - -// The result type of asyncSomethingUsefulTwo is Deferred -fun asyncSomethingUsefulTwo() = async(CommonPool) { - doSomethingUsefulTwo() -} - -// note, that we don't have `runBlocking` to the right of `main` in this example -fun main(args: Array) { - val time = measureTimeMillis { - // we can initiate async actions outside of a coroutine - val one = asyncSomethingUsefulOne() - val two = asyncSomethingUsefulTwo() - // but waiting for a result must involve either suspending or blocking. - // here we use `runBlocking { ... }` to block the main thread while waiting for the result - runBlocking { - println("The answer is ${one.await() + two.await()}") - } - } - println("Completed in $time ms") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-01.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-01.kt deleted file mode 100644 index 2411b7f8e4..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-01.kt +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.context.example01 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val jobs = arrayListOf() - jobs += launch(Unconfined) { // not confined -- will work with main thread - println(" 'Unconfined': I'm working in thread ${Thread.currentThread().name}") - } - jobs += launch(context) { // context of the parent, runBlocking coroutine - println(" 'context': I'm working in thread ${Thread.currentThread().name}") - } - jobs += launch(CommonPool) { // will get dispatched to ForkJoinPool.commonPool (or equivalent) - println(" 'CommonPool': I'm working in thread ${Thread.currentThread().name}") - } - jobs += launch(newSingleThreadContext("MyOwnThread")) { // will get its own new thread - println(" 'newSTC': I'm working in thread ${Thread.currentThread().name}") - } - jobs.forEach { it.join() } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-02.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-02.kt deleted file mode 100644 index 6a40e9bb8a..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-02.kt +++ /dev/null @@ -1,35 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.context.example02 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val jobs = arrayListOf() - jobs += launch(Unconfined) { // not confined -- will work with main thread - println(" 'Unconfined': I'm working in thread ${Thread.currentThread().name}") - delay(500) - println(" 'Unconfined': After delay in thread ${Thread.currentThread().name}") - } - jobs += launch(context) { // context of the parent, runBlocking coroutine - println(" 'context': I'm working in thread ${Thread.currentThread().name}") - delay(1000) - println(" 'context': After delay in thread ${Thread.currentThread().name}") - } - jobs.forEach { it.join() } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-03.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-03.kt deleted file mode 100644 index 89f30054eb..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-03.kt +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.context.example03 - -import kotlinx.coroutines.experimental.* - -fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") - -fun main(args: Array) = runBlocking { - val a = async(context) { - log("I'm computing a piece of the answer") - 6 - } - val b = async(context) { - log("I'm computing another piece of the answer") - 7 - } - log("The answer is ${a.await() * b.await()}") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-04.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-04.kt deleted file mode 100644 index 228b4fc80d..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-04.kt +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.context.example04 - -import kotlinx.coroutines.experimental.* - -fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") - -fun main(args: Array) { - val ctx1 = newSingleThreadContext("Ctx1") - val ctx2 = newSingleThreadContext("Ctx2") - runBlocking(ctx1) { - log("Started in ctx1") - run(ctx2) { - log("Working in ctx2") - } - log("Back to ctx1") - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-05.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-05.kt deleted file mode 100644 index 37c2636c6c..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-05.kt +++ /dev/null @@ -1,24 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.context.example05 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - println("My job is ${context[Job]}") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-06.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-06.kt deleted file mode 100644 index 80127df9a6..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-06.kt +++ /dev/null @@ -1,45 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.context.example06 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - // start a coroutine to process some kind of incoming request - val request = launch(CommonPool) { - // it spawns two other jobs, one with its separate context - val job1 = launch(CommonPool) { - println("job1: I have my own context and execute independently!") - delay(1000) - println("job1: I am not affected by cancellation of the request") - } - // and the other inherits the parent context - val job2 = launch(context) { - println("job2: I am a child of the request coroutine") - delay(1000) - println("job2: I will not execute this line if my parent request is cancelled") - } - // request completes when both its sub-jobs complete: - job1.join() - job2.join() - } - delay(500) - request.cancel() // cancel processing of the request - delay(1000) // delay a second to see what happens - println("main: Who has survived request cancellation?") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-07.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-07.kt deleted file mode 100644 index 7afff60e24..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-07.kt +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.context.example07 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - // start a coroutine to process some kind of incoming request - val request = launch(context) { // use the context of `runBlocking` - // spawns CPU-intensive child job in CommonPool !!! - val job = launch(context + CommonPool) { - println("job: I am a child of the request coroutine, but with a different dispatcher") - delay(1000) - println("job: I will not execute this line if my parent request is cancelled") - } - job.join() // request completes when its sub-job completes - } - delay(500) - request.cancel() // cancel processing of the request - delay(1000) // delay a second to see what happens - println("main: Who has survived request cancellation?") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-08.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-08.kt deleted file mode 100644 index 7cbfb21d9a..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-08.kt +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.context.example08 - -import kotlinx.coroutines.experimental.* - -fun log(msg: String) = println("[${Thread.currentThread().name}] $msg") - -fun main(args: Array) = runBlocking(CoroutineName("main")) { - log("Started main coroutine") - // run two background value computations - val v1 = async(CommonPool + CoroutineName("v1coroutine")) { - log("Computing v1") - delay(500) - 252 - } - val v2 = async(CommonPool + CoroutineName("v2coroutine")) { - log("Computing v2") - delay(1000) - 6 - } - log("The answer for v1 / v2 = ${v1.await() / v2.await()}") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-09.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-09.kt deleted file mode 100644 index 5074b4b7db..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-context-09.kt +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.context.example09 - -import kotlinx.coroutines.experimental.* - -fun main(args: Array) = runBlocking { - val job = Job() // create a job object to manage our lifecycle - // now launch ten coroutines for a demo, each working for a different time - val coroutines = List(10) { i -> - // they are all children of our job object - launch(context + job) { // we use the context of main runBlocking thread, but with our own job object - delay(i * 200L) // variable delay 0ms, 200ms, 400ms, ... etc - println("Coroutine $i is done") - } - } - println("Launched ${coroutines.size} coroutines") - delay(500L) // delay for half a second - println("Cancelling job!") - job.cancel() // cancel our job.. !!! - delay(1000L) // delay for more to see if our coroutines are still working -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-01.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-01.kt deleted file mode 100644 index 4e044b855f..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-01.kt +++ /dev/null @@ -1,56 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.select.example01 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* -import kotlinx.coroutines.experimental.selects.* -import kotlin.coroutines.experimental.CoroutineContext - -fun fizz(context: CoroutineContext) = produce(context) { - while (true) { // sends "Fizz" every 300 ms - delay(300) - send("Fizz") - } -} - -fun buzz(context: CoroutineContext) = produce(context) { - while (true) { // sends "Buzz!" every 500 ms - delay(500) - send("Buzz!") - } -} - -suspend fun selectFizzBuzz(fizz: ReceiveChannel, buzz: ReceiveChannel) { - select { // means that this select expression does not produce any result - fizz.onReceive { value -> // this is the first select clause - println("fizz -> '$value'") - } - buzz.onReceive { value -> // this is the second select clause - println("buzz -> '$value'") - } - } -} - -fun main(args: Array) = runBlocking { - val fizz = fizz(context) - val buzz = buzz(context) - repeat(7) { - selectFizzBuzz(fizz, buzz) - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-02.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-02.kt deleted file mode 100644 index 1c764acfdc..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-02.kt +++ /dev/null @@ -1,51 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.select.example02 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* -import kotlinx.coroutines.experimental.selects.* - -suspend fun selectAorB(a: ReceiveChannel, b: ReceiveChannel): String = - select { - a.onReceiveOrNull { value -> - if (value == null) - "Channel 'a' is closed" - else - "a -> '$value'" - } - b.onReceiveOrNull { value -> - if (value == null) - "Channel 'b' is closed" - else - "b -> '$value'" - } - } - -fun main(args: Array) = runBlocking { - // we are using the context of the main thread in this example for predictability ... - val a = produce(context) { - repeat(4) { send("Hello $it") } - } - val b = produce(context) { - repeat(4) { send("World $it") } - } - repeat(8) { // print first eight results - println(selectAorB(a, b)) - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-03.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-03.kt deleted file mode 100644 index 0009de0d4c..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-03.kt +++ /dev/null @@ -1,44 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.select.example03 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* -import kotlinx.coroutines.experimental.selects.* - -fun produceNumbers(side: SendChannel) = produce(CommonPool) { - for (num in 1..10) { // produce 10 numbers from 1 to 10 - delay(100) // every 100 ms - select { - onSend(num) {} // Send to the primary channel - side.onSend(num) {} // or to the side channel - } - } -} - -fun main(args: Array) = runBlocking { - val side = Channel() // allocate side channel - launch(context) { // this is a very fast consumer for the side channel - side.consumeEach { println("Side channel has $it") } - } - produceNumbers(side).consumeEach { - println("Consuming $it") - delay(250) // let us digest the consumed number properly, do not hurry - } - println("Done consuming") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-04.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-04.kt deleted file mode 100644 index 17488348f5..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-04.kt +++ /dev/null @@ -1,47 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.select.example04 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* -import kotlinx.coroutines.experimental.selects.* -import java.util.* - -fun asyncString(time: Int) = async(CommonPool) { - delay(time.toLong()) - "Waited for $time ms" -} - -fun asyncStringsList(): List> { - val random = Random(3) - return List(12) { asyncString(random.nextInt(1000)) } -} - -fun main(args: Array) = runBlocking { - val list = asyncStringsList() - val result = select { - list.withIndex().forEach { (index, deferred) -> - deferred.onAwait { answer -> - "Deferred $index produced answer '$answer'" - } - } - } - println(result) - val countActive = list.count { it.isActive } - println("$countActive coroutines are still active") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-05.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-05.kt deleted file mode 100644 index 0f27b6d738..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-select-05.kt +++ /dev/null @@ -1,66 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.select.example05 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* -import kotlinx.coroutines.experimental.selects.* - -fun switchMapDeferreds(input: ReceiveChannel>) = produce(CommonPool) { - var current = input.receive() // start with first received deferred value - while (isActive) { // loop while not cancelled/closed - val next = select?> { // return next deferred value from this select or null - input.onReceiveOrNull { update -> - update // replaces next value to wait - } - current.onAwait { value -> - send(value) // send value that current deferred has produced - input.receiveOrNull() // and use the next deferred from the input channel - } - } - if (next == null) { - println("Channel was closed") - break // out of loop - } else { - current = next - } - } -} - -fun asyncString(str: String, time: Long) = async(CommonPool) { - delay(time) - str -} - -fun main(args: Array) = runBlocking { - val chan = Channel>() // the channel for test - launch(context) { // launch printing coroutine - for (s in switchMapDeferreds(chan)) - println(s) // print each received string - } - chan.send(asyncString("BEGIN", 100)) - delay(200) // enough time for "BEGIN" to be produced - chan.send(asyncString("Slow", 500)) - delay(100) // not enough time to produce slow - chan.send(asyncString("Replace", 100)) - delay(500) // give it time before the last one - chan.send(asyncString("END", 500)) - delay(1000) // give it time to process - chan.close() // close the channel ... - delay(500) // and wait some time to let it finish -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-01.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-01.kt deleted file mode 100644 index 6d2de04b03..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-01.kt +++ /dev/null @@ -1,45 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.sync.example01 - -import kotlinx.coroutines.experimental.* -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.system.measureTimeMillis - -suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) { - val n = 1000 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - val jobs = List(n) { - launch(context) { - repeat(k) { action() } - } - } - jobs.forEach { it.join() } - } - println("Completed ${n * k} actions in $time ms") -} - -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { - counter++ - } - println("Counter = $counter") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-01b.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-01b.kt deleted file mode 100644 index 68b320d430..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-01b.kt +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.sync.example01b - -import kotlinx.coroutines.experimental.* -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.system.measureTimeMillis - -suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) { - val n = 1000 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - val jobs = List(n) { - launch(context) { - repeat(k) { action() } - } - } - jobs.forEach { it.join() } - } - println("Completed ${n * k} actions in $time ms") -} - -val mtContext = newFixedThreadPoolContext(2, "mtPool") // explicitly define context with two threads -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(mtContext) { // use it instead of CommonPool in this sample and below - counter++ - } - println("Counter = $counter") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-02.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-02.kt deleted file mode 100644 index ecba342d19..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-02.kt +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.sync.example02 - -import kotlinx.coroutines.experimental.* -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.system.measureTimeMillis - -suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) { - val n = 1000 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - val jobs = List(n) { - launch(context) { - repeat(k) { action() } - } - } - jobs.forEach { it.join() } - } - println("Completed ${n * k} actions in $time ms") -} - -@Volatile // in Kotlin `volatile` is an annotation -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { - counter++ - } - println("Counter = $counter") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-03.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-03.kt deleted file mode 100644 index bc1dfaf5e2..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-03.kt +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.sync.example03 - -import kotlinx.coroutines.experimental.* -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.system.measureTimeMillis -import java.util.concurrent.atomic.AtomicInteger - -suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) { - val n = 1000 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - val jobs = List(n) { - launch(context) { - repeat(k) { action() } - } - } - jobs.forEach { it.join() } - } - println("Completed ${n * k} actions in $time ms") -} - -var counter = AtomicInteger() - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { - counter.incrementAndGet() - } - println("Counter = ${counter.get()}") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-04.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-04.kt deleted file mode 100644 index b789617020..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-04.kt +++ /dev/null @@ -1,48 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.sync.example04 - -import kotlinx.coroutines.experimental.* -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.system.measureTimeMillis - -suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) { - val n = 1000 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - val jobs = List(n) { - launch(context) { - repeat(k) { action() } - } - } - jobs.forEach { it.join() } - } - println("Completed ${n * k} actions in $time ms") -} - -val counterContext = newSingleThreadContext("CounterContext") -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { // run each coroutine in CommonPool - run(counterContext) { // but confine each increment to the single-threaded context - counter++ - } - } - println("Counter = $counter") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-05.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-05.kt deleted file mode 100644 index adf8612aeb..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-05.kt +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.sync.example05 - -import kotlinx.coroutines.experimental.* -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.system.measureTimeMillis - -suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) { - val n = 1000 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - val jobs = List(n) { - launch(context) { - repeat(k) { action() } - } - } - jobs.forEach { it.join() } - } - println("Completed ${n * k} actions in $time ms") -} - -val counterContext = newSingleThreadContext("CounterContext") -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(counterContext) { // run each coroutine in the single-threaded context - counter++ - } - println("Counter = $counter") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-06.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-06.kt deleted file mode 100644 index 279b51d5d5..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-06.kt +++ /dev/null @@ -1,49 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.sync.example06 - -import kotlinx.coroutines.experimental.* -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.system.measureTimeMillis -import kotlinx.coroutines.experimental.sync.Mutex - -suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) { - val n = 1000 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - val jobs = List(n) { - launch(context) { - repeat(k) { action() } - } - } - jobs.forEach { it.join() } - } - println("Completed ${n * k} actions in $time ms") -} - -val mutex = Mutex() -var counter = 0 - -fun main(args: Array) = runBlocking { - massiveRun(CommonPool) { - mutex.lock() - try { counter++ } - finally { mutex.unlock() } - } - println("Counter = $counter") -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-07.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-07.kt deleted file mode 100644 index 1f7129ce20..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/example-sync-07.kt +++ /dev/null @@ -1,64 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.sync.example07 - -import kotlinx.coroutines.experimental.* -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.system.measureTimeMillis -import kotlinx.coroutines.experimental.channels.* - -suspend fun massiveRun(context: CoroutineContext, action: suspend () -> Unit) { - val n = 1000 // number of coroutines to launch - val k = 1000 // times an action is repeated by each coroutine - val time = measureTimeMillis { - val jobs = List(n) { - launch(context) { - repeat(k) { action() } - } - } - jobs.forEach { it.join() } - } - println("Completed ${n * k} actions in $time ms") -} - -// Message types for counterActor -sealed class CounterMsg -object IncCounter : CounterMsg() // one-way message to increment counter -class GetCounter(val response: SendChannel) : CounterMsg() // a request with reply - -// This function launches a new counter actor -fun counterActor() = actor(CommonPool) { - var counter = 0 // actor state - for (msg in channel) { // iterate over incoming messages - when (msg) { - is IncCounter -> counter++ - is GetCounter -> msg.response.send(counter) - } - } -} - -fun main(args: Array) = runBlocking { - val counter = counterActor() // create the actor - massiveRun(CommonPool) { - counter.send(IncCounter) - } - val response = Channel() - counter.send(GetCounter(response)) - println("Counter = ${response.receive()}") - counter.close() // shutdown the actor -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/test/GuideTest.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/test/GuideTest.kt deleted file mode 100644 index 9655454643..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/test/GuideTest.kt +++ /dev/null @@ -1,473 +0,0 @@ -// This file was automatically generated from coroutines-guide.md by Knit tool. Do not edit. -package guide.test - -import org.junit.Test - -class GuideTest { - - @Test - fun testGuideBasicExample01() { - test { guide.basic.example01.main(emptyArray()) }.verifyLines( - "Hello,", - "World!" - ) - } - - @Test - fun testGuideBasicExample02() { - test { guide.basic.example02.main(emptyArray()) }.verifyLines( - "Hello,", - "World!" - ) - } - - @Test - fun testGuideBasicExample03() { - test { guide.basic.example03.main(emptyArray()) }.verifyLines( - "Hello,", - "World!" - ) - } - - @Test - fun testGuideBasicExample04() { - test { guide.basic.example04.main(emptyArray()) }.verifyLines( - "Hello,", - "World!" - ) - } - - @Test - fun testGuideBasicExample05() { - test { guide.basic.example05.main(emptyArray()) }.also { lines -> - check(lines.size == 1 && lines[0] == ".".repeat(100_000)) - } - } - - @Test - fun testGuideBasicExample06() { - test { guide.basic.example06.main(emptyArray()) }.verifyLines( - "I'm sleeping 0 ...", - "I'm sleeping 1 ...", - "I'm sleeping 2 ..." - ) - } - - @Test - fun testGuideCancelExample01() { - test { guide.cancel.example01.main(emptyArray()) }.verifyLines( - "I'm sleeping 0 ...", - "I'm sleeping 1 ...", - "I'm sleeping 2 ...", - "main: I'm tired of waiting!", - "main: Now I can quit." - ) - } - - @Test - fun testGuideCancelExample02() { - test { guide.cancel.example02.main(emptyArray()) }.verifyLines( - "I'm sleeping 0 ...", - "I'm sleeping 1 ...", - "I'm sleeping 2 ...", - "main: I'm tired of waiting!", - "I'm sleeping 3 ...", - "I'm sleeping 4 ...", - "I'm sleeping 5 ...", - "main: Now I can quit." - ) - } - - @Test - fun testGuideCancelExample03() { - test { guide.cancel.example03.main(emptyArray()) }.verifyLines( - "I'm sleeping 0 ...", - "I'm sleeping 1 ...", - "I'm sleeping 2 ...", - "main: I'm tired of waiting!", - "main: Now I can quit." - ) - } - - @Test - fun testGuideCancelExample04() { - test { guide.cancel.example04.main(emptyArray()) }.verifyLines( - "I'm sleeping 0 ...", - "I'm sleeping 1 ...", - "I'm sleeping 2 ...", - "main: I'm tired of waiting!", - "I'm running finally", - "main: Now I can quit." - ) - } - - @Test - fun testGuideCancelExample05() { - test { guide.cancel.example05.main(emptyArray()) }.verifyLines( - "I'm sleeping 0 ...", - "I'm sleeping 1 ...", - "I'm sleeping 2 ...", - "main: I'm tired of waiting!", - "I'm running finally", - "And I've just delayed for 1 sec because I'm non-cancellable", - "main: Now I can quit." - ) - } - - @Test - fun testGuideCancelExample06() { - test { guide.cancel.example06.main(emptyArray()) }.verifyLinesStartWith( - "I'm sleeping 0 ...", - "I'm sleeping 1 ...", - "I'm sleeping 2 ...", - "Exception in thread \"main\" kotlinx.coroutines.experimental.TimeoutException: Timed out waiting for 1300 MILLISECONDS" - ) - } - - @Test - fun testGuideComposeExample01() { - test { guide.compose.example01.main(emptyArray()) }.verifyLinesFlexibleTime( - "The answer is 42", - "Completed in 2017 ms" - ) - } - - @Test - fun testGuideComposeExample02() { - test { guide.compose.example02.main(emptyArray()) }.verifyLinesFlexibleTime( - "The answer is 42", - "Completed in 1017 ms" - ) - } - - @Test - fun testGuideComposeExample03() { - test { guide.compose.example03.main(emptyArray()) }.verifyLinesFlexibleTime( - "The answer is 42", - "Completed in 2017 ms" - ) - } - - @Test - fun testGuideComposeExample04() { - test { guide.compose.example04.main(emptyArray()) }.verifyLinesFlexibleTime( - "The answer is 42", - "Completed in 1085 ms" - ) - } - - @Test - fun testGuideContextExample01() { - test { guide.context.example01.main(emptyArray()) }.verifyLinesStartUnordered( - " 'Unconfined': I'm working in thread main", - " 'CommonPool': I'm working in thread ForkJoinPool.commonPool-worker-1", - " 'newSTC': I'm working in thread MyOwnThread", - " 'context': I'm working in thread main" - ) - } - - @Test - fun testGuideContextExample02() { - test { guide.context.example02.main(emptyArray()) }.verifyLinesStart( - " 'Unconfined': I'm working in thread main", - " 'context': I'm working in thread main", - " 'Unconfined': After delay in thread kotlinx.coroutines.ScheduledExecutor", - " 'context': After delay in thread main" - ) - } - - @Test - fun testGuideContextExample03() { - test { guide.context.example03.main(emptyArray()) }.verifyLines( - "[main @coroutine#2] I'm computing a piece of the answer", - "[main @coroutine#3] I'm computing another piece of the answer", - "[main @coroutine#1] The answer is 42" - ) - } - - @Test - fun testGuideContextExample04() { - test { guide.context.example04.main(emptyArray()) }.verifyLines( - "[Ctx1 @coroutine#1] Started in ctx1", - "[Ctx2 @coroutine#1] Working in ctx2", - "[Ctx1 @coroutine#1] Back to ctx1" - ) - } - - @Test - fun testGuideContextExample05() { - test { guide.context.example05.main(emptyArray()) }.also { lines -> - check(lines.size == 1 && lines[0].startsWith("My job is BlockingCoroutine{Active}@")) - } - } - - @Test - fun testGuideContextExample06() { - test { guide.context.example06.main(emptyArray()) }.verifyLines( - "job1: I have my own context and execute independently!", - "job2: I am a child of the request coroutine", - "job1: I am not affected by cancellation of the request", - "main: Who has survived request cancellation?" - ) - } - - @Test - fun testGuideContextExample07() { - test { guide.context.example07.main(emptyArray()) }.verifyLines( - "job: I am a child of the request coroutine, but with a different dispatcher", - "main: Who has survived request cancellation?" - ) - } - - @Test - fun testGuideContextExample08() { - test { guide.context.example08.main(emptyArray()) }.verifyLinesFlexibleThread( - "[main @main#1] Started main coroutine", - "[ForkJoinPool.commonPool-worker-1 @v1coroutine#2] Computing v1", - "[ForkJoinPool.commonPool-worker-2 @v2coroutine#3] Computing v2", - "[main @main#1] The answer for v1 / v2 = 42" - ) - } - - @Test - fun testGuideContextExample09() { - test { guide.context.example09.main(emptyArray()) }.verifyLines( - "Launched 10 coroutines", - "Coroutine 0 is done", - "Coroutine 1 is done", - "Coroutine 2 is done", - "Cancelling job!" - ) - } - - @Test - fun testGuideChannelExample01() { - test { guide.channel.example01.main(emptyArray()) }.verifyLines( - "1", - "4", - "9", - "16", - "25", - "Done!" - ) - } - - @Test - fun testGuideChannelExample02() { - test { guide.channel.example02.main(emptyArray()) }.verifyLines( - "1", - "4", - "9", - "16", - "25", - "Done!" - ) - } - - @Test - fun testGuideChannelExample03() { - test { guide.channel.example03.main(emptyArray()) }.verifyLines( - "1", - "4", - "9", - "16", - "25", - "Done!" - ) - } - - @Test - fun testGuideChannelExample04() { - test { guide.channel.example04.main(emptyArray()) }.verifyLines( - "1", - "4", - "9", - "16", - "25", - "Done!" - ) - } - - @Test - fun testGuideChannelExample05() { - test { guide.channel.example05.main(emptyArray()) }.verifyLines( - "2", - "3", - "5", - "7", - "11", - "13", - "17", - "19", - "23", - "29" - ) - } - - @Test - fun testGuideChannelExample06() { - test { guide.channel.example06.main(emptyArray()) }.also { lines -> - check(lines.size == 10 && lines.withIndex().all { (i, line) -> line.startsWith("Processor #") && line.endsWith(" received ${i + 1}") }) - } - } - - @Test - fun testGuideChannelExample07() { - test { guide.channel.example07.main(emptyArray()) }.verifyLines( - "foo", - "foo", - "BAR!", - "foo", - "foo", - "BAR!" - ) - } - - @Test - fun testGuideChannelExample08() { - test { guide.channel.example08.main(emptyArray()) }.verifyLines( - "Sending 0", - "Sending 1", - "Sending 2", - "Sending 3", - "Sending 4" - ) - } - - @Test - fun testGuideChannelExample09() { - test { guide.channel.example09.main(emptyArray()) }.verifyLines( - "ping Ball(hits=1)", - "pong Ball(hits=2)", - "ping Ball(hits=3)", - "pong Ball(hits=4)", - "ping Ball(hits=5)" - ) - } - - @Test - fun testGuideSyncExample01() { - test { guide.sync.example01.main(emptyArray()) }.verifyLinesStart( - "Completed 1000000 actions in", - "Counter =" - ) - } - - @Test - fun testGuideSyncExample01b() { - test { guide.sync.example01b.main(emptyArray()) }.verifyLinesStart( - "Completed 1000000 actions in", - "Counter =" - ) - } - - @Test - fun testGuideSyncExample02() { - test { guide.sync.example02.main(emptyArray()) }.verifyLinesStart( - "Completed 1000000 actions in", - "Counter =" - ) - } - - @Test - fun testGuideSyncExample03() { - test { guide.sync.example03.main(emptyArray()) }.verifyLinesArbitraryTime( - "Completed 1000000 actions in xxx ms", - "Counter = 1000000" - ) - } - - @Test - fun testGuideSyncExample04() { - test { guide.sync.example04.main(emptyArray()) }.verifyLinesArbitraryTime( - "Completed 1000000 actions in xxx ms", - "Counter = 1000000" - ) - } - - @Test - fun testGuideSyncExample05() { - test { guide.sync.example05.main(emptyArray()) }.verifyLinesArbitraryTime( - "Completed 1000000 actions in xxx ms", - "Counter = 1000000" - ) - } - - @Test - fun testGuideSyncExample06() { - test { guide.sync.example06.main(emptyArray()) }.verifyLinesArbitraryTime( - "Completed 1000000 actions in xxx ms", - "Counter = 1000000" - ) - } - - @Test - fun testGuideSyncExample07() { - test { guide.sync.example07.main(emptyArray()) }.verifyLinesArbitraryTime( - "Completed 1000000 actions in xxx ms", - "Counter = 1000000" - ) - } - - @Test - fun testGuideSelectExample01() { - test { guide.select.example01.main(emptyArray()) }.verifyLines( - "fizz -> 'Fizz'", - "buzz -> 'Buzz!'", - "fizz -> 'Fizz'", - "fizz -> 'Fizz'", - "buzz -> 'Buzz!'", - "fizz -> 'Fizz'", - "buzz -> 'Buzz!'" - ) - } - - @Test - fun testGuideSelectExample02() { - test { guide.select.example02.main(emptyArray()) }.verifyLines( - "a -> 'Hello 0'", - "a -> 'Hello 1'", - "b -> 'World 0'", - "a -> 'Hello 2'", - "a -> 'Hello 3'", - "b -> 'World 1'", - "Channel 'a' is closed", - "Channel 'a' is closed" - ) - } - - @Test - fun testGuideSelectExample03() { - test { guide.select.example03.main(emptyArray()) }.verifyLines( - "Consuming 1", - "Side channel has 2", - "Side channel has 3", - "Consuming 4", - "Side channel has 5", - "Side channel has 6", - "Consuming 7", - "Side channel has 8", - "Side channel has 9", - "Consuming 10", - "Done consuming" - ) - } - - @Test - fun testGuideSelectExample04() { - test { guide.select.example04.main(emptyArray()) }.verifyLines( - "Deferred 4 produced answer 'Waited for 128 ms'", - "11 coroutines are still active" - ) - } - - @Test - fun testGuideSelectExample05() { - test { guide.select.example05.main(emptyArray()) }.verifyLines( - "BEGIN", - "Replace", - "END", - "Channel was closed" - ) - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/guide/test/TestUtil.kt b/kotlinx-coroutines-core/src/test/kotlin/guide/test/TestUtil.kt deleted file mode 100644 index 4cead9edc9..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/guide/test/TestUtil.kt +++ /dev/null @@ -1,147 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package guide.test - -import kotlinx.coroutines.experimental.* -import org.junit.Assert.assertEquals -import org.junit.Assert.assertTrue -import java.io.ByteArrayInputStream -import java.io.ByteArrayOutputStream -import java.io.PrintStream - -fun test(block: () -> Unit): List { - val oldOut = System.out - val oldErr = System.err - val bytesOut = ByteArrayOutputStream() - val ps = PrintStream(bytesOut) - System.setErr(ps) - System.setOut(ps) - CommonPool.usePrivatePool() - resetCoroutineId() - var bytes: ByteArray - try { - block() - } catch (e: Throwable) { - System.err.print("Exception in thread \"main\" ") - e.printStackTrace() - } finally { - // capture output - bytes = bytesOut.toByteArray() - // the shutdown - scheduledExecutorShutdownNow() - shutdownDispatcherPools() - CommonPool.shutdownAndRelease(10000L) // wait at most 10 sec - scheduledExecutorShutdownNowAndRelease() - System.setOut(oldOut) - System.setErr(oldErr) - - } - return ByteArrayInputStream(bytes).bufferedReader().readLines() -} - -private fun shutdownDispatcherPools() { - val threads = arrayOfNulls(Thread.activeCount()) - val n = Thread.enumerate(threads) - for (i in 0 until n) { - val thread = threads[i] - if (thread is PoolThread) - thread.dispatcher.executor.shutdownNow() - } -} - -enum class SanitizeMode { - NONE, - ARBITRARY_TIME, - FLEXIBLE_TIME, - FLEXIBLE_THREAD -} - -private fun sanitize(s: String, mode: SanitizeMode): String { - var res = s - when (mode) { - SanitizeMode.ARBITRARY_TIME -> { - res = res.replace(Regex(" [0-9]+ ms"), " xxx ms") - } - SanitizeMode.FLEXIBLE_TIME -> { - res = res.replace(Regex("[0-9][0-9][0-9] ms"), "xxx ms") - } - SanitizeMode.FLEXIBLE_THREAD -> { - res = res.replace(Regex("ForkJoinPool\\.commonPool-worker-[0-9]+"), "CommonPool") - res = res.replace(Regex("ForkJoinPool-[0-9]+-worker-[0-9]+"), "CommonPool") - res = res.replace(Regex("CommonPool-worker-[0-9]+"), "CommonPool") - res = res.replace(Regex("RxComputationThreadPool-[0-9]+"), "RxComputationThreadPool") - } - SanitizeMode.NONE -> {} - } - return res -} - -private fun List.verifyCommonLines(expected: Array, mode: SanitizeMode = SanitizeMode.NONE) { - val n = minOf(size, expected.size) - for (i in 0 until n) { - val exp = sanitize(expected[i], mode) - val act = sanitize(get(i), mode) - assertEquals("Line ${i + 1}", exp, act) - } -} - -private fun List.checkEqualNumberOfLines(expected: Array) { - if (size > expected.size) - error("Expected ${expected.size} lines, but found $size. Unexpected line '${get(expected.size)}'") - else if (size < expected.size) - error("Expected ${expected.size} lines, but found $size") -} - -fun List.verifyLines(vararg expected: String) { - verifyCommonLines(expected) - checkEqualNumberOfLines(expected) -} - -fun List.verifyLinesStartWith(vararg expected: String) { - verifyCommonLines(expected) - assertTrue("Number of lines", expected.size <= size) -} - -fun List.verifyLinesArbitraryTime(vararg expected: String) { - verifyCommonLines(expected, SanitizeMode.ARBITRARY_TIME) - checkEqualNumberOfLines(expected) -} - -fun List.verifyLinesFlexibleTime(vararg expected: String) { - verifyCommonLines(expected, SanitizeMode.FLEXIBLE_TIME) - checkEqualNumberOfLines(expected) -} - -fun List.verifyLinesFlexibleThread(vararg expected: String) { - verifyCommonLines(expected, SanitizeMode.FLEXIBLE_THREAD) - checkEqualNumberOfLines(expected) -} - -fun List.verifyLinesStartUnordered(vararg expected: String) { - val expectedSorted = expected.sorted().toTypedArray() - sorted().verifyLinesStart(*expectedSorted) -} - -fun List.verifyLinesStart(vararg expected: String) { - val n = minOf(size, expected.size) - for (i in 0 until n) { - val exp = sanitize(expected[i], SanitizeMode.FLEXIBLE_THREAD) - val act = sanitize(get(i), SanitizeMode.FLEXIBLE_THREAD) - assertEquals("Line ${i + 1}", exp, act.substring(0, minOf(act.length, exp.length))) - } - checkEqualNumberOfLines(expected) -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/AsyncLazyTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/AsyncLazyTest.kt deleted file mode 100644 index 415252b01f..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/AsyncLazyTest.kt +++ /dev/null @@ -1,190 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.junit.Test -import java.io.IOException - -class AsyncLazyTest : TestBase() { - @Test - fun testSimple(): Unit = runBlocking { - expect(1) - val d = async(context, CoroutineStart.LAZY) { - expect(3) - 42 - } - expect(2) - check(!d.isActive && !d.isCompleted) - check(d.await() == 42) - check(!d.isActive && d.isCompleted && !d.isCompletedExceptionally) - expect(4) - check(d.await() == 42) // second await -- same result - finish(5) - } - - @Test - fun testLazyDeferAndYield(): Unit = runBlocking { - expect(1) - val d = async(context, CoroutineStart.LAZY) { - expect(3) - yield() // this has not effect, because parent coroutine is waiting - expect(4) - 42 - } - expect(2) - check(!d.isActive && !d.isCompleted) - check(d.await() == 42) - check(!d.isActive && d.isCompleted && !d.isCompletedExceptionally) - expect(5) - check(d.await() == 42) // second await -- same result - finish(6) - } - - @Test - fun testLazyDeferAndYield2(): Unit = runBlocking { - expect(1) - val d = async(context, CoroutineStart.LAZY) { - expect(7) - 42 - } - expect(2) - check(!d.isActive && !d.isCompleted) - launch(context) { // see how it looks from another coroutine - expect(4) - check(!d.isActive && !d.isCompleted) - yield() // yield back to main - expect(6) - check(d.isActive && !d.isCompleted) // implicitly started by main's await - yield() // yield to d - } - expect(3) - check(!d.isActive && !d.isCompleted) - yield() // yield to second child (lazy async is not computing yet) - expect(5) - check(!d.isActive && !d.isCompleted) - check(d.await() == 42) // starts computing - check(!d.isActive && d.isCompleted && !d.isCompletedExceptionally) - finish(8) - } - - @Test(expected = IOException::class) - fun testSimpleException(): Unit = runBlocking { - expect(1) - val d = async(context, CoroutineStart.LAZY) { - finish(3) - throw IOException() - } - expect(2) - check(!d.isActive && !d.isCompleted) - d.await() // will throw IOException - } - - @Test(expected = IOException::class) - fun testLazyDeferAndYieldException(): Unit = runBlocking { - expect(1) - val d = async(context, CoroutineStart.LAZY) { - expect(3) - yield() // this has not effect, because parent coroutine is waiting - finish(4) - throw IOException() - } - expect(2) - check(!d.isActive && !d.isCompleted) - d.await() // will throw IOException - } - - @Test - fun testCatchException(): Unit = runBlocking { - expect(1) - val d = async(context, CoroutineStart.LAZY) { - expect(3) - throw IOException() - } - expect(2) - check(!d.isActive && !d.isCompleted) - try { - d.await() // will throw IOException - } catch (e: IOException) { - check(!d.isActive && d.isCompleted && d.isCompletedExceptionally && !d.isCancelled) - expect(4) - } - finish(5) - } - - @Test - fun testStart(): Unit = runBlocking { - expect(1) - val d = async(context, CoroutineStart.LAZY) { - expect(4) - 42 - } - expect(2) - check(!d.isActive && !d.isCompleted) - check(d.start()) - check(d.isActive && !d.isCompleted) - expect(3) - check(!d.start()) - yield() // yield to started coroutine - check(!d.isActive && d.isCompleted && !d.isCompletedExceptionally) // and it finishes - expect(5) - check(d.await() == 42) // await sees result - finish(6) - } - - @Test(expected = CancellationException::class) - fun testCancelBeforeStart(): Unit = runBlocking { - expect(1) - val d = async(context, CoroutineStart.LAZY) { - expectUnreached() - 42 - } - expect(2) - check(!d.isActive && !d.isCompleted) - check(d.cancel()) - check(!d.isActive && d.isCompleted && d.isCompletedExceptionally && d.isCancelled) - check(!d.cancel()) - check(!d.start()) - finish(3) - check(d.await() == 42) // await shall throw CancellationException - expectUnreached() - } - - @Test(expected = CancellationException::class) - fun testCancelWhileComputing(): Unit = runBlocking { - expect(1) - val d = async(context, CoroutineStart.LAZY) { - expect(4) - yield() // yield to main, that is going to cancel us - expectUnreached() - 42 - } - expect(2) - check(!d.isActive && !d.isCompleted) - check(d.start()) - check(d.isActive && !d.isCompleted) - expect(3) - yield() // yield to d - expect(5) - check(d.isActive && !d.isCompleted) - check(d.cancel()) - check(!d.isActive && d.isCancelled && d.isCompletedExceptionally && d.isCancelled) - check(!d.cancel()) - finish(6) - check(d.await() == 42) // await shall throw CancellationException - expectUnreached() - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/AsyncTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/AsyncTest.kt deleted file mode 100644 index a9684c44b5..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/AsyncTest.kt +++ /dev/null @@ -1,123 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.junit.Test -import java.io.IOException -import org.junit.Assert.* - -class AsyncTest : TestBase() { - @Test - fun testSimple(): Unit = runBlocking { - expect(1) - val d = async(context) { - expect(3) - 42 - } - expect(2) - check(d.isActive) - check(d.await() == 42) - check(!d.isActive) - expect(4) - check(d.await() == 42) // second await -- same result - finish(5) - } - - @Test - fun testUndispatched(): Unit = runBlocking { - expect(1) - val d = async(context, start = CoroutineStart.UNDISPATCHED) { - expect(2) - 42 - } - expect(3) - check(!d.isActive) - check(d.await() == 42) - finish(4) - } - - @Test(expected = IOException::class) - fun testSimpleException(): Unit = runBlocking { - expect(1) - val d = async(context) { - finish(3) - throw IOException() - } - expect(2) - d.await() // will throw IOException - } - - @Test(expected = IOException::class) - fun testDeferAndYieldException(): Unit = runBlocking { - expect(1) - val d = async(context) { - expect(3) - yield() // no effect, parent waiting - finish(4) - throw IOException() - } - expect(2) - d.await() // will throw IOException - } - - @Test - fun testDeferWithTwoWaiters() = runBlocking { - expect(1) - val d = async(context) { - expect(5) - yield() - expect(9) - 42 - } - expect(2) - launch(context) { - expect(6) - check(d.await() == 42) - expect(11) - } - expect(3) - launch(context) { - expect(7) - check(d.await() == 42) - expect(12) - } - expect(4) - yield() // this actually yields control to async, which produces results and resumes both waiters (in order) - expect(8) - yield() // yield again to "d", which completes - expect(10) - yield() // yield to both waiters - finish(13) - } - - class BadClass { - override fun equals(other: Any?): Boolean = error("equals") - override fun hashCode(): Int = error("hashCode") - override fun toString(): String = error("toString") - } - - @Test - fun testDeferBadClass() = runBlocking { - val bad = BadClass() - val d = async(context) { - expect(1) - bad - } - assertTrue(d.await() === bad) - finish(2) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/CancellableContinuationImplTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/CancellableContinuationImplTest.kt deleted file mode 100644 index 5139c9bd1e..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/CancellableContinuationImplTest.kt +++ /dev/null @@ -1,100 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.junit.Test -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.EmptyCoroutineContext -import kotlin.coroutines.experimental.intrinsics.COROUTINE_SUSPENDED - -class CancellableContinuationImplTest { - @Test - fun testIdempotentSelectResume() { - var resumed = false - val delegate = object : Continuation { - override val context: CoroutineContext get() = EmptyCoroutineContext - override fun resume(value: String) { - check(value === "OK") - resumed = true - } - override fun resumeWithException(exception: Throwable) { error("Should not happen") } - } - val c = CancellableContinuationImpl(delegate, false) - check(!c.isSelected) - check(!c.isActive) - check(c.trySelect("SELECT")) - check(c.isSelected) - check(c.isActive) - check(!c.start()) - check(!c.trySelect("OTHER")) - check(c.trySelect("SELECT")) - val token = c.tryResume("OK", "RESUME") ?: error("Failed") - check(c.isSelected) - check(!c.isActive) - check(null == c.tryResume("FAIL")) - check(!c.start()) - check(!c.trySelect("OTHER")) - check(c.trySelect("SELECT")) - check(token === c.tryResume("OK", "RESUME")) - check(c.getResult() === COROUTINE_SUSPENDED) - check(!resumed) - c.completeResume(token) - check(resumed) - check(c.isSelected) - check(!c.isActive) - check(null == c.tryResume("FAIL")) - check(!c.start()) - check(!c.trySelect("OTHER")) - check(c.trySelect("SELECT")) - check(token === c.tryResume("OK", "RESUME")) - check(c.getResult() === "OK") - } - - @Test - fun testIdempotentSelectCancel() { - var resumed = false - val delegate = object : Continuation { - override val context: CoroutineContext get() = EmptyCoroutineContext - override fun resume(value: String) { error("Should not happen") } - override fun resumeWithException(exception: Throwable) { - check(exception is CancellationException) - resumed = true - } - } - val c = CancellableContinuationImpl(delegate, false) - check(!c.isSelected) - check(!c.isActive) - check(c.trySelect("SELECT")) - check(c.isSelected) - check(c.isActive) - check(!c.start()) - check(!c.trySelect("OTHER")) - check(c.trySelect("SELECT")) - check(c.getResult() === COROUTINE_SUSPENDED) - check(!resumed) - c.cancel() - check(resumed) - check(c.isSelected) - check(!c.isActive) - check(null == c.tryResume("FAIL")) - check(!c.start()) - check(!c.trySelect("OTHER")) - check(c.trySelect("SELECT")) - check(null === c.tryResume("OK", "RESUME")) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/CoroutinesTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/CoroutinesTest.kt deleted file mode 100644 index d9af2a8a24..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/CoroutinesTest.kt +++ /dev/null @@ -1,190 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.junit.Test -import java.io.IOException - -class CoroutinesTest : TestBase() { - @Test - fun testSimple() = runBlocking { - expect(1) - finish(2) - } - - @Test - fun testYield() = runBlocking { - expect(1) - yield() // effectively does nothing, as we don't have other coroutines - finish(2) - } - - @Test - fun testLaunchAndYieldJoin() = runBlocking { - expect(1) - val job = launch(context) { - expect(3) - yield() - expect(4) - } - expect(2) - check(job.isActive && !job.isCompleted) - job.join() - check(!job.isActive && job.isCompleted) - finish(5) - } - - @Test - fun testLaunchUndispatched() = runBlocking { - expect(1) - val job = launch(context, start = CoroutineStart.UNDISPATCHED) { - expect(2) - yield() - expect(4) - } - expect(3) - check(job.isActive && !job.isCompleted) - job.join() - check(!job.isActive && job.isCompleted) - finish(5) - } - - @Test - fun testNested() = runBlocking { - expect(1) - val j1 = launch(context) { - expect(3) - val j2 = launch(context) { - expect(5) - } - expect(4) - j2.join() - expect(6) - } - expect(2) - j1.join() - finish(7) - } - - @Test - fun testCancelChildImplicit() = runBlocking { - expect(1) - launch(context) { - expect(3) - yield() // parent finishes earlier, does not wait for us - expectUnreached() - } - expect(2) - yield() - finish(4) - } - - @Test - fun testCancelChildExplicit() = runBlocking { - expect(1) - val job = launch(context) { - expect(3) - yield() - expectUnreached() - } - expect(2) - yield() - expect(4) - job.cancel() - finish(5) - } - - @Test - fun testCancelChildWithFinally() = runBlocking { - expect(1) - val job = launch(context) { - expect(3) - try { - yield() - } finally { - finish(6) // cancelled child will still execute finally - } - expectUnreached() - } - expect(2) - yield() - expect(4) - job.cancel() - expect(5) - } - - @Test - fun testCancelNestedImplicit() = runBlocking { - expect(1) - launch(context) { - expect(3) - launch(context) { - expect(6) - yield() // parent finishes earlier, does not wait for us - expectUnreached() - } - expect(4) - yield() - expect(7) - yield() // does not go further, because already cancelled - expectUnreached() - } - expect(2) - yield() - expect(5) - yield() - finish(8) - } - - @Test(expected = IOException::class) - fun testExceptionPropagation(): Unit = runBlocking { - finish(1) - throw IOException() - } - - @Test(expected = IOException::class) - fun testCancelParentOnChildException(): Unit = runBlocking { - expect(1) - launch(context) { - finish(3) - throw IOException() // does not propagate exception to launch, but cancels parent (!) - } - expect(2) - yield() - expectUnreached() // because of exception in child - } - - @Test(expected = IOException::class) - fun testCancelParentOnNestedException(): Unit = runBlocking { - expect(1) - launch(context) { - expect(3) - launch(context) { - finish(6) - throw IOException() // unhandled exception kills all parents - } - expect(4) - yield() - expectUnreached() // because of exception in child - } - expect(2) - yield() - expect(5) - yield() - expectUnreached() // because of exception in child - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/ExecutorsTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/ExecutorsTest.kt deleted file mode 100644 index e802c4e361..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/ExecutorsTest.kt +++ /dev/null @@ -1,101 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.junit.After -import org.junit.Before -import org.junit.Test -import java.util.concurrent.Executors - -class ExecutorsTest { - fun threadNames(): Set { - val arrayOfThreads = Array(Thread.activeCount()) { null } - val n = Thread.enumerate(arrayOfThreads) - val names = hashSetOf() - for (i in 0 until n) - names.add(arrayOfThreads[i]!!.name) - return names - } - - lateinit var threadNamesBefore: Set - - @Before - fun before() { - threadNamesBefore = threadNames() - } - - @After - fun after() { - // give threads some time to shutdown - val waitTill = System.currentTimeMillis() + 1000L - var diff: Set - do { - val threadNamesAfter = threadNames() - diff = threadNamesAfter - threadNamesBefore - if (diff.isEmpty()) break - } while (System.currentTimeMillis() <= waitTill) - diff.forEach { println("Lost thread '$it'") } - check(diff.isEmpty()) { "Lost ${diff.size} threads"} - } - - fun checkThreadName(prefix: String) { - val name = Thread.currentThread().name - check(name.startsWith(prefix)) { "Expected thread name to start with '$prefix', found: '$name'" } - } - - @Test - fun testSingleThread() { - val context = newSingleThreadContext("TestThread") - runBlocking(context) { - checkThreadName("TestThread") - } - context[Job]!!.cancel() - } - - @Test - fun testFixedThreadPool() { - val context = newFixedThreadPoolContext(2, "TestPool") - runBlocking(context) { - checkThreadName("TestPool") - } - context[Job]!!.cancel() - } - - @Test - fun testToExecutor() { - val executor = Executors.newSingleThreadExecutor { r -> Thread(r, "TestExecutor") } - runBlocking(executor.asCoroutineDispatcher()) { - checkThreadName("TestExecutor") - } - executor.shutdown() - } - - @Test - fun testTwoThreads() { - val ctx1 = newSingleThreadContext("Ctx1") - val ctx2 = newSingleThreadContext("Ctx2") - runBlocking(ctx1) { - checkThreadName("Ctx1") - run(ctx2) { - checkThreadName("Ctx2") - } - checkThreadName("Ctx1") - } - ctx1[Job]!!.cancel() - ctx2[Job]!!.cancel() - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/JobTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/JobTest.kt deleted file mode 100644 index 4afd94c9c4..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/JobTest.kt +++ /dev/null @@ -1,139 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.junit.Assert.assertEquals -import org.junit.Test - -class JobTest : TestBase() { - @Test - fun testState() { - val job = Job() - check(job.isActive) - job.cancel() - check(!job.isActive) - } - - @Test - fun testHandler() { - val job = Job() - var fireCount = 0 - job.invokeOnCompletion { fireCount++ } - check(job.isActive) - assertEquals(0, fireCount) - // cancel once - job.cancel() - check(!job.isActive) - assertEquals(1, fireCount) - // cancel again - job.cancel() - check(!job.isActive) - assertEquals(1, fireCount) - } - - @Test - fun testManyHandlers() { - val job = Job() - val n = 100 * stressTestMultiplier - val fireCount = IntArray(n) - for (i in 0 until n) job.invokeOnCompletion { fireCount[i]++ } - check(job.isActive) - for (i in 0 until n) assertEquals(0, fireCount[i]) - // cancel once - job.cancel() - check(!job.isActive) - for (i in 0 until n) assertEquals(1, fireCount[i]) - // cancel again - job.cancel() - check(!job.isActive) - for (i in 0 until n) assertEquals(1, fireCount[i]) - } - - @Test - fun testUnregisterInHandler() { - val job = Job() - val n = 100 * stressTestMultiplier - val fireCount = IntArray(n) - for (i in 0 until n) { - var registration: DisposableHandle? = null - registration = job.invokeOnCompletion { - fireCount[i]++ - registration!!.dispose() - } - } - check(job.isActive) - for (i in 0 until n) assertEquals(0, fireCount[i]) - // cancel once - job.cancel() - check(!job.isActive) - for (i in 0 until n) assertEquals(1, fireCount[i]) - // cancel again - job.cancel() - check(!job.isActive) - for (i in 0 until n) assertEquals(1, fireCount[i]) - } - - @Test - fun testManyHandlersWithUnregister() { - val job = Job() - val n = 100 * stressTestMultiplier - val fireCount = IntArray(n) - val registrations = Array(n) { i -> job.invokeOnCompletion { fireCount[i]++ } } - check(job.isActive) - fun unreg(i: Int) = i % 4 <= 1 - for (i in 0 until n) if (unreg(i)) registrations[i].dispose() - for (i in 0 until n) assertEquals(0, fireCount[i]) - job.cancel() - check(!job.isActive) - for (i in 0 until n) assertEquals(if (unreg(i)) 0 else 1, fireCount[i]) - } - - @Test - fun testExceptionsInHandler() { - val job = Job() - val n = 100 * stressTestMultiplier - val fireCount = IntArray(n) - class TestException : Throwable() - for (i in 0 until n) job.invokeOnCompletion { - fireCount[i]++ - throw TestException() - } - check(job.isActive) - for (i in 0 until n) assertEquals(0, fireCount[i]) - val tryCancel = Try { job.cancel() } - check(!job.isActive) - for (i in 0 until n) assertEquals(1, fireCount[i]) - check(tryCancel.exception is TestException) - } - - @Test - fun testMemoryRelease() { - val job = Job() - val n = 10_000_000 * stressTestMultiplier - var fireCount = 0 - for (i in 0 until n) job.invokeOnCompletion { fireCount++ }.dispose() - } - - @Test - fun testCancelledParent() { - val parent = Job() - parent.cancel() - check(!parent.isActive) - val child = Job(parent) - check(!child.isActive) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/LaunchLazyTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/LaunchLazyTest.kt deleted file mode 100644 index 28d7e822d9..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/LaunchLazyTest.kt +++ /dev/null @@ -1,65 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.junit.Test - -class LaunchLazyTest : TestBase() { - @Test - fun testLaunchAndYieldJoin() = runBlocking { - expect(1) - val job = launch(context, CoroutineStart.LAZY) { - expect(4) - yield() // does nothing -- main waits - expect(5) - } - expect(2) - yield() // does nothing, was not started yet - expect(3) - check(!job.isActive && !job.isCompleted) - job.join() - check(!job.isActive && job.isCompleted) - finish(6) - } - - @Test - fun testStart() = runBlocking { - expect(1) - val job = launch(context, CoroutineStart.LAZY) { - expect(5) - yield() // yields back to main - expect(7) - } - expect(2) - yield() // does nothing, was not started yet - expect(3) - check(!job.isActive && !job.isCompleted) - check(job.start()) - check(job.isActive && !job.isCompleted) - check(!job.start()) // start again -- does nothing - check(job.isActive && !job.isCompleted) - expect(4) - yield() // now yield to started coroutine - expect(6) - check(job.isActive && !job.isCompleted) - yield() // yield again - check(!job.isActive && job.isCompleted) // it completes this time - expect(8) - job.join() // immediately returns - finish(9) - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/RunTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/RunTest.kt deleted file mode 100644 index 5b13391c4d..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/RunTest.kt +++ /dev/null @@ -1,127 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.hamcrest.MatcherAssert.assertThat -import org.hamcrest.core.IsEqual -import org.junit.Test - -class RunTest : TestBase() { - @Test - fun testSameContextNoSuspend() = runBlocking { - expect(1) - launch(context) { // make sure there is not early dispatch here - finish(5) - } - expect(2) - val result = run(context) { // same context! - expect(3) // still here - "OK" - } - assertThat(result, IsEqual("OK")) - expect(4) - } - - @Test - fun testSameContextWithSuspend() = runBlocking { - expect(1) - launch(context) { // make sure there is not early dispatch here - expect(4) - } - expect(2) - val result = run(context) { // same context! - expect(3) // still here - yield() // now yields to launch! - expect(5) - "OK" - } - assertThat(result, IsEqual("OK")) - finish(6) - } - - @Test - fun testCancelWithJobNoSuspend() = runBlocking { - expect(1) - launch(context) { // make sure there is not early dispatch to here - finish(6) - } - expect(2) - val job = Job() - val result = run(context + job) { // same context + new job - expect(3) // still here - job.cancel() // cancel out job! - try { - yield() // shall throw CancellationException - expectUnreached() - } catch (e: CancellationException) { - expect(4) - } - "OK" - } - assertThat(result, IsEqual("OK")) - expect(5) - } - - @Test - fun testCancelWithJobWithSuspend() = runBlocking { - expect(1) - launch(context) { // make sure there is not early dispatch to here - expect(4) - } - expect(2) - val job = Job() - val result = run(context + job) { // same context + new job - expect(3) // still here - yield() // now yields to launch! - expect(5) - job.cancel() // cancel out job! - try { - yield() // shall throw CancellationExpcetion - expectUnreached() - } catch (e: CancellationException) { - expect(6) - } - "OK" - } - assertThat(result, IsEqual("OK")) - finish(7) - } - - @Test - fun testCommonPoolNoSuspend() = runBlocking { - expect(1) - val result = run(CommonPool) { - expect(2) - "OK" - } - assertThat(result, IsEqual("OK")) - finish(3) - } - - @Test - fun testCommonPoolWithSuspend() = runBlocking { - expect(1) - val result = run(CommonPool) { - expect(2) - delay(100) - expect(3) - "OK" - } - assertThat(result, IsEqual("OK")) - finish(4) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/TestBase.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/TestBase.kt deleted file mode 100644 index 712498b3c5..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/TestBase.kt +++ /dev/null @@ -1,62 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.junit.After -import java.util.concurrent.atomic.AtomicBoolean -import java.util.concurrent.atomic.AtomicInteger -import java.util.concurrent.atomic.AtomicReference - -open class TestBase { - val isStressTest = System.getProperty("stressTest") != null - val stressTestMultiplier = if (isStressTest) 30 else 1 - - var actionIndex = AtomicInteger() - var finished = AtomicBoolean() - var error = AtomicReference() - - public fun error(message: Any): Nothing { - val exception = IllegalStateException(message.toString()) - error.compareAndSet(null, exception) - throw exception - } - - public inline fun check(value: Boolean, lazyMessage: () -> Any): Unit { - if (!value) error(lazyMessage()) - } - - fun expect(index: Int) { - val wasIndex = actionIndex.incrementAndGet() - check(index == wasIndex) { "Expecting action index $index but it is actually $wasIndex" } - } - - fun expectUnreached() { - error("Should not be reached") - } - - fun finish(index: Int) { - expect(index) - check(!finished.getAndSet(true)) { "Should call 'finish(...)' at most once" } - } - - @After - fun onCompletion() { - error.get()?.let { throw it } - check(actionIndex.get() == 0 || finished.get()) { "Expecting that 'finish(...)' was invoked, but it was not" } - } - -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/Try.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/Try.kt deleted file mode 100644 index 63f29c16ca..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/Try.kt +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -public class Try private constructor(private val _value: Any?) { - private class Fail(val exception: Throwable) { - override fun toString(): String = "Failure[$exception]" - } - - public companion object { - public operator fun invoke(block: () -> T): Try = - try { Success(block()) } catch(e: Throwable) { Failure(e) } - public fun Success(value: T) = Try(value) - public fun Failure(exception: Throwable) = Try(Fail(exception)) - } - - @Suppress("UNCHECKED_CAST") - public val value: T get() = if (_value is Fail) throw _value.exception else _value as T - - public val exception: Throwable? get() = (_value as? Fail)?.exception - - override fun toString(): String = _value.toString() -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutOrNullTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutOrNullTest.kt deleted file mode 100644 index 84109ac077..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutOrNullTest.kt +++ /dev/null @@ -1,110 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsNull -import org.junit.Assert.assertThat -import org.junit.Test -import java.io.IOException - -class WithTimeoutOrNullTest : TestBase() { - /** - * Tests property dispatching of `withTimeoutOrNull` blocks - */ - @Test - fun testDispatch() = runBlocking { - expect(1) - launch(context) { - expect(4) - yield() // back to main - expect(7) - } - expect(2) - // test that it does not yield to the above job when started - val result = withTimeoutOrNull(1000) { - expect(3) - yield() // yield only now - expect(5) - "OK" - } - assertThat(result, IsEqual("OK")) - expect(6) - yield() // back to launch - finish(8) - } - - @Test - fun testNullOnTimeout() = runBlocking { - expect(1) - val result = withTimeoutOrNull(100) { - expect(2) - delay(1000) - expectUnreached() - "OK" - } - assertThat(result, IsNull()) - finish(3) - } - - @Test - fun testSuppressException() = runBlocking { - expect(1) - val result = withTimeoutOrNull(100) { - expect(2) - try { - delay(1000) - } catch (e: CancellationException) { - expect(3) - } - "OK" - } - assertThat(result, IsEqual("OK")) - finish(4) - } - - @Test(expected = IOException::class) - fun testReplaceException() = runBlocking { - expect(1) - withTimeoutOrNull(100) { - expect(2) - try { - delay(1000) - } catch (e: CancellationException) { - finish(3) - throw IOException(e) - } - "OK" - } - expectUnreached() - } - - /** - * Tests that a 100% CPU-consuming loop will react on timeout if it has yields. - */ - @Test - fun testYieldBlockingWithTimeout() = runBlocking { - expect(1) - val result = withTimeoutOrNull(100) { - while (true) { - yield() - } - } - assertThat(result, IsNull()) - finish(2) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutOrNullThreadDispatchTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutOrNullThreadDispatchTest.kt deleted file mode 100644 index e7a1f77913..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutOrNullThreadDispatchTest.kt +++ /dev/null @@ -1,99 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsNull -import org.junit.After -import org.junit.Assert -import org.junit.Test -import java.util.concurrent.ExecutorService -import java.util.concurrent.Executors -import java.util.concurrent.ThreadFactory -import java.util.concurrent.atomic.AtomicInteger -import kotlin.coroutines.experimental.CoroutineContext - -class WithTimeoutOrNullThreadDispatchTest : TestBase() { - var executor: ExecutorService? = null - - @After - fun tearDown() { - executor?.shutdown() - } - - @Test - fun testCancellationDispatchScheduled() { - checkCancellationDispatch { - executor = Executors.newScheduledThreadPool(1, it) - executor!!.asCoroutineDispatcher() - } - } - - @Test - fun testCancellationDispatchNonScheduled() { - checkCancellationDispatch { - executor = Executors.newSingleThreadExecutor(it) - executor!!.asCoroutineDispatcher() - } - } - - - @Test - fun testCancellationDispatchCustomNoDelay() { - // it also checks that there is at most once scheduled request in flight (no spurious concurrency) - var error: String? = null - checkCancellationDispatch { - executor = Executors.newSingleThreadExecutor(it) - val scheduled = AtomicInteger(0) - object : CoroutineDispatcher() { - override fun dispatch(context: CoroutineContext, block: Runnable) { - if (scheduled.incrementAndGet() > 1) error = "Two requests are scheduled concurrently" - executor!!.execute { - scheduled.decrementAndGet() - block.run() - } - } - } - } - error?.let { error(it) } - } - - private fun checkCancellationDispatch(factory: (ThreadFactory) -> CoroutineDispatcher) = runBlocking { - expect(1) - var thread: Thread? = null - val dispatcher = factory(ThreadFactory { Thread(it).also { thread = it } }) - run(dispatcher) { - expect(2) - Assert.assertThat(Thread.currentThread(), IsEqual(thread)) - val result = withTimeoutOrNull(100) { - try { - expect(3) - delay(1000) - expectUnreached() - } catch (e: CancellationException) { - expect(4) - Assert.assertThat(Thread.currentThread(), IsEqual(thread)) - throw e // rethrow - } - } - Assert.assertThat(Thread.currentThread(), IsEqual(thread)) - Assert.assertThat(result, IsNull()) - expect(5) - } - finish(6) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutTest.kt deleted file mode 100644 index 785570c7b8..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutTest.kt +++ /dev/null @@ -1,110 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.hamcrest.core.IsEqual -import org.junit.Assert.assertThat -import org.junit.Test -import java.io.IOException - -class WithTimeoutTest : TestBase() { - /** - * Tests proper dispatching of `withTimeout` blocks - */ - @Test - fun testDispatch() = runBlocking { - expect(1) - launch(context) { - expect(4) - yield() // back to main - expect(7) - } - expect(2) - // test that it does not yield to the above job when started - val result = withTimeout(1000) { - expect(3) - yield() // yield only now - expect(5) - "OK" - } - assertThat(result, IsEqual("OK")) - expect(6) - yield() // back to launch - finish(8) - } - - - @Test - fun testExceptionOnTimeout() = runBlocking { - expect(1) - try { - withTimeout(100) { - expect(2) - delay(1000) - expectUnreached() - "OK" - } - } catch (e: CancellationException) { - assertThat(e.message, IsEqual("Timed out waiting for 100 MILLISECONDS")) - finish(3) - } - } - - @Test - fun testSuppressException() = runBlocking { - expect(1) - val result = withTimeout(100) { - expect(2) - try { - delay(1000) - } catch (e: CancellationException) { - expect(3) - } - "OK" - } - assertThat(result, IsEqual("OK")) - finish(4) - } - - @Test(expected = IOException::class) - fun testReplaceException() = runBlocking { - expect(1) - withTimeout(100) { - expect(2) - try { - delay(1000) - } catch (e: CancellationException) { - finish(3) - throw IOException(e) - } - "OK" - } - expectUnreached() - } - - /** - * Tests that a 100% CPU-consuming loop will react on timeout if it has yields. - */ - @Test(expected = CancellationException::class) - fun testYieldBlockingWithTimeout() = runBlocking { - withTimeout(100) { - while (true) { - yield() - } - } - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutThreadDispatchTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutThreadDispatchTest.kt deleted file mode 100644 index cb065af63b..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/WithTimeoutThreadDispatchTest.kt +++ /dev/null @@ -1,100 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental - -import org.hamcrest.core.IsEqual -import org.junit.After -import org.junit.Assert -import org.junit.Test -import java.util.concurrent.ExecutorService -import java.util.concurrent.Executors -import java.util.concurrent.ThreadFactory -import java.util.concurrent.atomic.AtomicInteger -import kotlin.coroutines.experimental.CoroutineContext - -class WithTimeoutThreadDispatchTest : TestBase() { - var executor: ExecutorService? = null - - @After - fun tearDown() { - executor?.shutdown() - } - - @Test - fun testCancellationDispatchScheduled() { - checkCancellationDispatch { - executor = Executors.newScheduledThreadPool(1, it) - executor!!.asCoroutineDispatcher() - } - } - - @Test - fun testCancellationDispatchNonScheduled() { - checkCancellationDispatch { - executor = Executors.newSingleThreadExecutor(it) - executor!!.asCoroutineDispatcher() - } - } - - @Test - fun testCancellationDispatchCustomNoDelay() { - // it also checks that there is at most once scheduled request in flight (no spurious concurrency) - var error: String? = null - checkCancellationDispatch { - executor = Executors.newSingleThreadExecutor(it) - val scheduled = AtomicInteger(0) - object : CoroutineDispatcher() { - override fun dispatch(context: CoroutineContext, block: Runnable) { - if (scheduled.incrementAndGet() > 1) error = "Two requests are scheduled concurrently" - executor!!.execute { - scheduled.decrementAndGet() - block.run() - } - } - } - } - error?.let { error(it) } - } - - private fun checkCancellationDispatch(factory: (ThreadFactory) -> CoroutineDispatcher) = runBlocking { - expect(1) - var thread: Thread? = null - val dispatcher = factory(ThreadFactory { Thread(it).also { thread = it } }) - run(dispatcher) { - expect(2) - Assert.assertThat(Thread.currentThread(), IsEqual(thread)) - try { - withTimeout(100) { - try { - expect(3) - delay(1000) - expectUnreached() - } catch (e: CancellationException) { - expect(4) - Assert.assertThat(Thread.currentThread(), IsEqual(thread)) - throw e // rethrow - } - } - } catch (e: CancellationException) { - expect(5) - Assert.assertThat(Thread.currentThread(), IsEqual(thread)) - } - expect(6) - } - finish(7) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ActorLazyTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ActorLazyTest.kt deleted file mode 100644 index a292f5c8b3..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ActorLazyTest.kt +++ /dev/null @@ -1,77 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.CoroutineStart -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.junit.Assert.assertThat -import org.junit.Test - -class ActorLazyTest : TestBase() { - @Test - fun testEmptyStart() = runBlocking { - expect(1) - val actor = actor(context, start = CoroutineStart.LAZY) { - expect(5) - } - assertThat(actor.isActive, IsEqual(false)) - assertThat(actor.isCompleted, IsEqual(false)) - assertThat(actor.channel.isClosedForSend, IsEqual(false)) - expect(2) - yield() // to actor code --> nothing happens (not started!) - assertThat(actor.isActive, IsEqual(false)) - assertThat(actor.isCompleted, IsEqual(false)) - assertThat(actor.channel.isClosedForSend, IsEqual(false)) - expect(3) - // start actor explicitly - actor.start() - expect(4) - yield() // to started actor - assertThat(actor.isActive, IsEqual(false)) - assertThat(actor.isCompleted, IsEqual(true)) - assertThat(actor.channel.isClosedForSend, IsEqual(true)) - finish(6) - } - - @Test - fun testOne() = runBlocking { - expect(1) - val actor = actor(context, start = CoroutineStart.LAZY) { - expect(4) - assertThat(receive(), IsEqual("OK")) - expect(5) - } - assertThat(actor.isActive, IsEqual(false)) - assertThat(actor.isCompleted, IsEqual(false)) - assertThat(actor.channel.isClosedForSend, IsEqual(false)) - expect(2) - yield() // to actor code --> nothing happens (not started!) - assertThat(actor.isActive, IsEqual(false)) - assertThat(actor.isCompleted, IsEqual(false)) - assertThat(actor.channel.isClosedForSend, IsEqual(false)) - expect(3) - // send message to actor --> should start it - actor.send("OK") - assertThat(actor.isActive, IsEqual(false)) - assertThat(actor.isCompleted, IsEqual(true)) - assertThat(actor.channel.isClosedForSend, IsEqual(true)) - finish(6) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ActorTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ActorTest.kt deleted file mode 100644 index ce637b2da6..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ActorTest.kt +++ /dev/null @@ -1,70 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.junit.Assert.assertThat -import org.junit.Test - -class ActorTest : TestBase() { - @Test - fun testEmpty() = runBlocking { - expect(1) - val actor = actor(context) { - expect(3) - } - assertThat(actor.isActive, IsEqual(true)) - assertThat(actor.isCompleted, IsEqual(false)) - assertThat(actor.channel.isClosedForSend, IsEqual(false)) - expect(2) - yield() // to actor code - assertThat(actor.isActive, IsEqual(false)) - assertThat(actor.isCompleted, IsEqual(true)) - assertThat(actor.channel.isClosedForSend, IsEqual(true)) - finish(4) - } - - @Test - fun testOne() = runBlocking { - expect(1) - val actor = actor(context) { - expect(3) - assertThat(receive(), IsEqual("OK")) - expect(6) - } - assertThat(actor.isActive, IsEqual(true)) - assertThat(actor.isCompleted, IsEqual(false)) - assertThat(actor.channel.isClosedForSend, IsEqual(false)) - expect(2) - yield() // to actor code - assertThat(actor.isActive, IsEqual(true)) - assertThat(actor.isCompleted, IsEqual(false)) - assertThat(actor.channel.isClosedForSend, IsEqual(false)) - expect(4) - // send message to actor - actor.send("OK") - expect(5) - yield() // to actor code - assertThat(actor.isActive, IsEqual(false)) - assertThat(actor.isCompleted, IsEqual(true)) - assertThat(actor.channel.isClosedForSend, IsEqual(true)) - finish(7) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ArrayBroadcastChannelTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ArrayBroadcastChannelTest.kt deleted file mode 100644 index bc9e533b69..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ArrayBroadcastChannelTest.kt +++ /dev/null @@ -1,115 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.* -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsNull -import org.junit.Assert.* -import org.junit.Test - -class ArrayBroadcastChannelTest : TestBase() { - @Test - fun testBasic() = runBlocking { - expect(1) - val broadcast = ArrayBroadcastChannel(1) - assertThat(broadcast.isClosedForSend, IsEqual(false)) - val first = broadcast.open() - launch(context, CoroutineStart.UNDISPATCHED) { - expect(2) - assertThat(first.receive(), IsEqual(1)) // suspends - assertThat(first.isClosedForReceive, IsEqual(false)) - expect(5) - assertThat(first.receive(), IsEqual(2)) // suspends - assertThat(first.isClosedForReceive, IsEqual(false)) - expect(10) - assertThat(first.receiveOrNull(), IsNull()) // suspends - assertThat(first.isClosedForReceive, IsEqual(true)) - expect(14) - } - expect(3) - broadcast.send(1) - expect(4) - yield() // to the first receiver - expect(6) - val second = broadcast.open() - launch(context, CoroutineStart.UNDISPATCHED) { - expect(7) - assertThat(second.receive(), IsEqual(2)) // suspends - assertThat(second.isClosedForReceive, IsEqual(false)) - expect(11) - assertThat(second.receiveOrNull(), IsNull()) // suspends - assertThat(second.isClosedForReceive, IsEqual(true)) - expect(15) - } - expect(8) - broadcast.send(2) - expect(9) - yield() // to first & second receivers - expect(12) - broadcast.close() - expect(13) - assertThat(broadcast.isClosedForSend, IsEqual(true)) - yield() // to first & second receivers - finish(16) - } - - @Test - fun testSendSuspend() = runBlocking { - expect(1) - val broadcast = ArrayBroadcastChannel(1) - val first = broadcast.open() - launch(context) { - expect(4) - assertThat(first.receive(), IsEqual(1)) - expect(5) - assertThat(first.receive(), IsEqual(2)) - expect(6) - } - expect(2) - broadcast.send(1) // puts to buffer, receiver not running yet - expect(3) - broadcast.send(2) // suspends - finish(7) - } - - @Test - fun testConcurrentSendCompletion() = runBlocking { - expect(1) - val broadcast = ArrayBroadcastChannel(1) - val sub = broadcast.open() - // launch 3 concurrent senders (one goes buffer, two other suspend) - for (x in 1..3) { - launch(context, CoroutineStart.UNDISPATCHED) { - expect(x + 1) - broadcast.send(x) - } - } - // and close it for send - expect(5) - broadcast.close() - // now must receive all 3 items - expect(6) - assertThat(sub.isClosedForReceive, IsEqual(false)) - for (x in 1..3) - assertThat(sub.receiveOrNull(), IsEqual(x)) - // and receive close signal - assertThat(sub.receiveOrNull(), IsNull()) - assertThat(sub.isClosedForReceive, IsEqual(true)) - finish(7) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ArrayChannelTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ArrayChannelTest.kt deleted file mode 100644 index 0c26c5e7c8..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ArrayChannelTest.kt +++ /dev/null @@ -1,147 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.* -import org.junit.Test -import org.junit.Assert.* - -class ArrayChannelTest : TestBase() { - @Test - fun testSimple() = runBlocking { - val q = ArrayChannel(1) - check(q.isEmpty && !q.isFull) - expect(1) - val sender = launch(context) { - expect(4) - q.send(1) // success -- buffered - check(!q.isEmpty && q.isFull) - expect(5) - q.send(2) // suspends (buffer full) - expect(9) - } - expect(2) - val receiver = launch(context) { - expect(6) - check(q.receive() == 1) // does not suspend -- took from buffer - check(!q.isEmpty && q.isFull) // waiting sender's element moved to buffer - expect(7) - check(q.receive() == 2) // does not suspend (takes from sender) - expect(8) - } - expect(3) - sender.join() - receiver.join() - check(q.isEmpty && !q.isFull) - finish(10) - } - - @Test - fun testStress() = runBlocking { - val n = 100_000 - val q = ArrayChannel(1) - val sender = launch(context) { - for (i in 1..n) q.send(i) - expect(2) - } - val receiver = launch(context) { - for (i in 1..n) check(q.receive() == i) - expect(3) - } - expect(1) - sender.join() - receiver.join() - finish(4) - } - - @Test - fun testClosedBufferedReceiveOrNull() = runBlocking { - val q = ArrayChannel(1) - check(q.isEmpty && !q.isFull && !q.isClosedForSend && !q.isClosedForReceive) - expect(1) - launch(context) { - expect(5) - check(!q.isEmpty && !q.isFull && q.isClosedForSend && !q.isClosedForReceive) - assertEquals(42, q.receiveOrNull()) - expect(6) - check(!q.isEmpty && !q.isFull && q.isClosedForSend && q.isClosedForReceive) - assertEquals(null, q.receiveOrNull()) - expect(7) - } - expect(2) - q.send(42) // buffers - expect(3) - q.close() // goes on - expect(4) - check(!q.isEmpty && !q.isFull && q.isClosedForSend && !q.isClosedForReceive) - yield() - check(!q.isEmpty && !q.isFull && q.isClosedForSend && q.isClosedForReceive) - finish(8) - } - - @Test - fun testClosedExceptions() = runBlocking { - val q = ArrayChannel(1) - expect(1) - launch(context) { - expect(4) - try { q.receive() } - catch (e: ClosedReceiveChannelException) { - expect(5) - } - } - expect(2) - q.close() - expect(3) - yield() - expect(6) - try { q.send(42) } - catch (e: ClosedSendChannelException) { - finish(7) - } - } - - @Test - fun testOfferAndPool() = runBlocking { - val q = ArrayChannel(1) - assertTrue(q.offer(1)) - expect(1) - launch(context) { - expect(3) - assertEquals(1, q.poll()) - expect(4) - assertEquals(null, q.poll()) - expect(5) - assertEquals(2, q.receive()) // suspends - expect(9) - assertEquals(3, q.poll()) - expect(10) - assertEquals(null, q.poll()) - expect(11) - } - expect(2) - yield() - expect(6) - assertTrue(q.offer(2)) - expect(7) - assertTrue(q.offer(3)) - expect(8) - assertFalse(q.offer(4)) - yield() - finish(12) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ChannelAtomicCancelStressTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ChannelAtomicCancelStressTest.kt deleted file mode 100644 index e595f71fce..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ChannelAtomicCancelStressTest.kt +++ /dev/null @@ -1,159 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.selects.select -import org.junit.Assert.assertEquals -import org.junit.Test -import org.junit.runner.RunWith -import org.junit.runners.Parameterized -import java.util.* -import java.util.concurrent.atomic.AtomicReference - -/** - * Tests cancel atomicity for channel send & receive operations, including their select versions. - */ -@RunWith(Parameterized::class) -class ChannelAtomicCancelStressTest(val kind: TestChannelKind) : TestBase() { - companion object { - @Parameterized.Parameters(name = "{0}") - @JvmStatic - fun params(): Collection> = TestChannelKind.values().map { arrayOf(it) } - } - - val TEST_DURATION = 3000L * stressTestMultiplier - - val channel = kind.create() - val senderDone = ArrayChannel(1) - val receiverDone = ArrayChannel(1) - - var lastSent = 0 - var lastReceived = 0 - - var stoppedSender = 0 - var stoppedReceiver = 0 - - var missedCnt = 0 - var dupCnt = 0 - - val failed = AtomicReference() - - lateinit var sender: Job - lateinit var receiver: Job - - fun fail(e: Throwable) = failed.compareAndSet(null, e) - - inline fun cancellable(done: ArrayChannel, block: () -> Unit) { - try { - block() - } catch (e: Throwable) { - if (e !is CancellationException) fail(e) - throw e - } finally { - if (!done.offer(true)) - fail(IllegalStateException("failed to offer to done channel")) - } - } - - @Test - fun testAtomicCancelStress() = runBlocking { - val deadline = System.currentTimeMillis() + TEST_DURATION - launchSender() - launchReceiver() - val rnd = Random() - while (System.currentTimeMillis() < deadline && failed.get() == null) { - when (rnd.nextInt(3)) { - 0 -> { // cancel & restart sender - stopSender() - launchSender() - } - 1 -> { // cancel & restart receiver - stopReceiver() - launchReceiver() - } - 2 -> yield() // just yield (burn a little time) - } - } - stopSender() - stopReceiver() - println(" Sent $lastSent ints to channel") - println(" Received $lastReceived ints from channel") - println(" Stopped sender $stoppedSender times") - println("Stopped receiver $stoppedReceiver times") - println(" Missed $missedCnt ints") - println(" Duplicated $dupCnt ints") - failed.get()?.let { throw it } - assertEquals(0, dupCnt) - if (!kind.isConflated) { - assertEquals(0, missedCnt) - assertEquals(lastSent, lastReceived) - } - } - - fun launchSender() { - sender = launch(CommonPool) { - val rnd = Random() - cancellable(senderDone) { - var counter = 0 - while (true) { - val trySend = lastSent + 1 - when (rnd.nextInt(2)) { - 0 -> channel.send(trySend) - 1 -> select { channel.onSend(trySend) {} } - else -> error("cannot happen") - } - lastSent = trySend // update on success - if (counter++ % 1000 == 0) yield() // yield periodically to check cancellation on LinkedListChannel - } - } - } - } - - suspend fun stopSender() { - stoppedSender++ - sender.cancel() - senderDone.receive() - } - - fun launchReceiver() { - receiver = launch(CommonPool) { - val rnd = Random() - cancellable(receiverDone) { - while (true) { - val received = when (rnd.nextInt(2)) { - 0 -> channel.receive() - 1 -> select { channel.onReceive { it } } - else -> error("cannot happen") - } - val expected = lastReceived + 1 - if (received > expected) - missedCnt++ - if (received < expected) - dupCnt++ - lastReceived = received - } - } - } - } - - suspend fun stopReceiver() { - stoppedReceiver++ - receiver.cancel() - receiverDone.receive() - } -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ChannelSendReceiveStressTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ChannelSendReceiveStressTest.kt deleted file mode 100644 index f3e9a89006..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ChannelSendReceiveStressTest.kt +++ /dev/null @@ -1,186 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.selects.select -import org.junit.Assert.assertEquals -import org.junit.Assert.assertTrue -import org.junit.Test -import org.junit.runner.RunWith -import org.junit.runners.Parameterized -import java.util.concurrent.atomic.AtomicInteger -import java.util.concurrent.atomic.AtomicIntegerArray - -@RunWith(Parameterized::class) -class ChannelSendReceiveStressTest( - val kind: TestChannelKind, - val nSenders: Int, - val nReceivers: Int -) : TestBase() { - companion object { - @Parameterized.Parameters(name = "{0}, nSenders={1}, nReceivers={2}") - @JvmStatic - fun params(): Collection> = - listOf(1, 2, 10).flatMap { nSenders -> - listOf(1, 10).flatMap { nReceivers -> - TestChannelKind.values().map { arrayOf(it, nSenders, nReceivers) } - } - } - } - - val timeLimit = 30_000L * stressTestMultiplier // 30 sec - val nEvents = 1_000_000 * stressTestMultiplier - - val maxBuffer = 10_000 // artificial limit for LinkedListChannel - - val channel = kind.create() - val sendersCompleted = AtomicInteger() - val receiversCompleted = AtomicInteger() - val dupes = AtomicInteger() - val sentTotal = AtomicInteger() - val received = AtomicIntegerArray(nEvents) - val receivedTotal = AtomicInteger() - val receivedBy = IntArray(nReceivers) - - @Test - fun testSendReceiveStress() = runBlocking { - val receivers = List(nReceivers) { receiverIndex -> - // different event receivers use different code - launch(CommonPool + CoroutineName("receiver$receiverIndex")) { - when (receiverIndex % 5) { - 0 -> doReceive(receiverIndex) - 1 -> doReceiveOrNull(receiverIndex) - 2 -> doIterator(receiverIndex) - 3 -> doReceiveSelect(receiverIndex) - 4 -> doReceiveSelectOrNull(receiverIndex) - } - receiversCompleted.incrementAndGet() - } - } - val senders = List(nSenders) { senderIndex -> - launch(CommonPool + CoroutineName("sender$senderIndex")) { - when (senderIndex % 2) { - 0 -> doSend(senderIndex) - 1 -> doSendSelect(senderIndex) - } - sendersCompleted.incrementAndGet() - } - } - // print progress - val progressJob = launch(context) { - var seconds = 0 - while (true) { - delay(1000) - println("${++seconds}: Sent ${sentTotal.get()}, received ${receivedTotal.get()}") - } - } - try { - withTimeout(timeLimit) { - senders.forEach { it.join() } - channel.close() - receivers.forEach { it.join() } - } - } catch (e: CancellationException) { - println("!!! Test timed out $e") - } - progressJob.cancel() - println("Tested $kind with nSenders=$nSenders, nReceivers=$nReceivers") - println("Completed successfully ${sendersCompleted.get()} sender coroutines") - println("Completed successfully ${receiversCompleted.get()} receiver coroutines") - println(" Sent ${sentTotal.get()} events") - println(" Received ${receivedTotal.get()} events") - println(" Received dupes ${dupes.get()}") - repeat(nReceivers) { receiveIndex -> - println(" Received by #$receiveIndex ${receivedBy[receiveIndex]}") - } - assertEquals(nSenders, sendersCompleted.get()) - assertEquals(nReceivers, receiversCompleted.get()) - assertEquals(0, dupes.get()) - assertEquals(nEvents, sentTotal.get()) - if (!kind.isConflated) assertEquals(nEvents, receivedTotal.get()) - repeat(nReceivers) { receiveIndex -> - assertTrue("Each receiver should have received something", receivedBy[receiveIndex] > 0) - } - } - - private suspend fun doSent() { - sentTotal.incrementAndGet() - if (!kind.isConflated) { - while (sentTotal.get() > receivedTotal.get() + maxBuffer) - yield() // throttle fast senders to prevent OOM with LinkedListChannel - } - } - - private suspend fun doSend(senderIndex: Int) { - for (i in senderIndex until nEvents step nSenders) { - channel.send(i) - doSent() - } - } - - private suspend fun doSendSelect(senderIndex: Int) { - for (i in senderIndex until nEvents step nSenders) { - select { channel.onSend(i) { Unit } } - doSent() - } - } - - private fun doReceived(receiverIndex: Int, event: Int) { - if (!received.compareAndSet(event, 0, 1)) { - println("Duplicate event $event at $receiverIndex") - dupes.incrementAndGet() - } - receivedTotal.incrementAndGet() - receivedBy[receiverIndex]++ - } - - private suspend fun doReceive(receiverIndex: Int) { - while (true) { - try { doReceived(receiverIndex, channel.receive()) } - catch (ex: ClosedReceiveChannelException) { break } - } - } - - private suspend fun doReceiveOrNull(receiverIndex: Int) { - while (true) { - doReceived(receiverIndex, channel.receiveOrNull() ?: break) - } - } - - private suspend fun doIterator(receiverIndex: Int) { - for (event in channel) { - doReceived(receiverIndex, event) - } - } - - private suspend fun doReceiveSelect(receiverIndex: Int) { - while (true) { - try { - val event = select { channel.onReceive { it } } - doReceived(receiverIndex, event) - } catch (ex: ClosedReceiveChannelException) { break } - } - } - - private suspend fun doReceiveSelectOrNull(receiverIndex: Int) { - while (true) { - val event = select { channel.onReceiveOrNull { it } } ?: break - doReceived(receiverIndex, event) - } - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ConflatedBroadcastChannelTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ConflatedBroadcastChannelTest.kt deleted file mode 100644 index ebe7c25907..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ConflatedBroadcastChannelTest.kt +++ /dev/null @@ -1,117 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.* -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.hamcrest.core.IsNull -import org.junit.Assert.* -import org.junit.Test - -class ConflatedBroadcastChannelTest : TestBase() { - @Test - fun testBasicScenario() = runBlocking { - expect(1) - val broadcast = ConflatedBroadcastChannel() - assertThat(exceptionFrom { broadcast.value }, IsInstanceOf(IllegalStateException::class.java)) - assertThat(broadcast.valueOrNull, IsNull()) - launch(context, CoroutineStart.UNDISPATCHED) { - expect(2) - val sub = broadcast.open() - assertThat(sub.poll(), IsNull()) - expect(3) - assertThat(sub.receive(), IsEqual("one")) // suspends - expect(6) - assertThat(sub.receive(), IsEqual("two")) // suspends - expect(12) - sub.close() - expect(13) - } - expect(4) - broadcast.send("one") // does not suspend - assertThat(broadcast.value, IsEqual("one")) - assertThat(broadcast.valueOrNull, IsEqual("one")) - expect(5) - yield() // to receiver - expect(7) - launch(context, CoroutineStart.UNDISPATCHED) { - expect(8) - val sub = broadcast.open() - assertThat(sub.receive(), IsEqual("one")) // does not suspend - expect(9) - assertThat(sub.receive(), IsEqual("two")) // suspends - expect(14) - assertThat(sub.receive(), IsEqual("three")) // suspends - expect(17) - assertThat(sub.receiveOrNull(), IsNull()) // suspends until closed - expect(20) - sub.close() - expect(21) - } - expect(10) - broadcast.send("two") // does not suspend - assertThat(broadcast.value, IsEqual("two")) - assertThat(broadcast.valueOrNull, IsEqual("two")) - expect(11) - yield() // to both receivers - expect(15) - broadcast.send("three") // does not suspend - assertThat(broadcast.value, IsEqual("three")) - assertThat(broadcast.valueOrNull, IsEqual("three")) - expect(16) - yield() // to second receiver - expect(18) - broadcast.close() - assertThat(exceptionFrom { broadcast.value }, IsInstanceOf(IllegalStateException::class.java)) - assertThat(broadcast.valueOrNull, IsNull()) - expect(19) - yield() // to second receiver - assertThat(exceptionFrom { broadcast.send("four") }, IsInstanceOf(ClosedSendChannelException::class.java)) - finish(22) - } - - @Test - fun testInitialValueAndReceiveClosed() = runBlocking { - expect(1) - val broadcast = ConflatedBroadcastChannel(1) - assertThat(broadcast.value, IsEqual(1)) - assertThat(broadcast.valueOrNull, IsEqual(1)) - launch(context, CoroutineStart.UNDISPATCHED) { - expect(2) - val sub = broadcast.open() - assertThat(sub.receive(), IsEqual(1)) - expect(3) - assertThat(exceptionFrom { sub.receive() }, IsInstanceOf(ClosedReceiveChannelException::class.java)) // suspends - expect(6) - } - expect(4) - broadcast.close() - expect(5) - yield() // to child - finish(7) - } - - inline fun exceptionFrom(block: () -> Unit): Throwable? { - try { - block() - return null - } catch (e: Throwable) { - return e - } - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ConflatedChannelTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ConflatedChannelTest.kt deleted file mode 100644 index 6f67df93b4..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/ConflatedChannelTest.kt +++ /dev/null @@ -1,67 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsNull -import org.junit.Assert.assertThat -import org.junit.Test - -class ConflatedChannelTest : TestBase() { - @Test - fun testBasicConflationOfferPoll() { - val q = ConflatedChannel() - assertThat(q.poll(), IsNull()) - assertThat(q.offer(1), IsEqual(true)) - assertThat(q.offer(2), IsEqual(true)) - assertThat(q.offer(3), IsEqual(true)) - assertThat(q.poll(), IsEqual(3)) - assertThat(q.poll(), IsNull()) - } - - @Test - fun testConflationSendReceive() = runBlocking { - val q = ConflatedChannel() - expect(1) - launch(context) { // receiver coroutine - expect(4) - assertThat(q.receive(), IsEqual(2)) - expect(5) - assertThat(q.receive(), IsEqual(3)) // this receive suspends - expect(8) - assertThat(q.receive(), IsEqual(6)) // last conflated value - expect(9) - } - expect(2) - q.send(1) - q.send(2) // shall conflate - expect(3) - yield() // to receiver - expect(6) - q.send(3) // send to the waiting receiver - q.send(4) // buffer - q.send(5) // conflate - q.send(6) // conflate again - expect(7) - yield() // to receiver - finish(10) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/RendezvousChannelTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/RendezvousChannelTest.kt deleted file mode 100644 index 7b5bb1c8bc..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/RendezvousChannelTest.kt +++ /dev/null @@ -1,296 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsNull -import org.junit.Assert.* -import org.junit.Test - -class RendezvousChannelTest : TestBase() { - @Test - fun testSimple() = runBlocking { - val q = RendezvousChannel() - check(q.isEmpty && q.isFull) - expect(1) - val sender = launch(context) { - expect(4) - q.send(1) // suspend -- the first to come to rendezvous - expect(7) - q.send(2) // does not suspend -- receiver is there - expect(8) - } - expect(2) - val receiver = launch(context) { - expect(5) - check(q.receive() == 1) // does not suspend -- sender was there - expect(6) - check(q.receive() == 2) // suspends - expect(9) - } - expect(3) - sender.join() - receiver.join() - check(q.isEmpty && q.isFull) - finish(10) - } - - @Test - fun testStress() = runBlocking { - val n = 100_000 - val q = RendezvousChannel() - val sender = launch(context) { - for (i in 1..n) q.send(i) - expect(2) - } - val receiver = launch(context) { - for (i in 1..n) check(q.receive() == i) - expect(3) - } - expect(1) - sender.join() - receiver.join() - finish(4) - } - - @Test - fun testClosedReceiveOrNull() = runBlocking { - val q = RendezvousChannel() - check(q.isEmpty && q.isFull && !q.isClosedForSend && !q.isClosedForReceive) - expect(1) - launch(context) { - expect(3) - assertEquals(42, q.receiveOrNull()) - expect(4) - assertEquals(null, q.receiveOrNull()) - expect(6) - } - expect(2) - q.send(42) - expect(5) - q.close() - check(!q.isEmpty && !q.isFull && q.isClosedForSend && q.isClosedForReceive) - yield() - check(!q.isEmpty && !q.isFull && q.isClosedForSend && q.isClosedForReceive) - finish(7) - } - - @Test - fun testClosedExceptions() = runBlocking { - val q = RendezvousChannel() - expect(1) - launch(context) { - expect(4) - try { q.receive() } - catch (e: ClosedReceiveChannelException) { - expect(5) - } - } - expect(2) - q.close() - expect(3) - yield() - expect(6) - try { q.send(42) } - catch (e: ClosedSendChannelException) { - finish(7) - } - } - - @Test - fun testOfferAndPool() = runBlocking { - val q = RendezvousChannel() - assertFalse(q.offer(1)) - expect(1) - launch(context) { - expect(3) - assertEquals(null, q.poll()) - expect(4) - assertEquals(2, q.receive()) - expect(7) - assertEquals(null, q.poll()) - yield() - expect(9) - assertEquals(3, q.poll()) - expect(10) - } - expect(2) - yield() - expect(5) - assertTrue(q.offer(2)) - expect(6) - yield() - expect(8) - q.send(3) - finish(11) - } - - @Test - fun testIteratorClosed() = runBlocking { - val q = RendezvousChannel() - expect(1) - launch(context) { - expect(3) - q.close() - expect(4) - } - expect(2) - for (x in q) { - expectUnreached() - } - finish(5) - } - - @Test - fun testIteratorOne() = runBlocking { - val q = RendezvousChannel() - expect(1) - launch(context) { - expect(3) - q.send(1) - expect(4) - q.close() - expect(5) - } - expect(2) - for (x in q) { - expect(6) - assertEquals(1, x) - } - finish(7) - } - - @Test - fun testIteratorOneWithYield() = runBlocking { - val q = RendezvousChannel() - expect(1) - launch(context) { - expect(3) - q.send(1) // will suspend - expect(6) - q.close() - expect(7) - } - expect(2) - yield() // yield to sender coroutine right before starting for loop - expect(4) - for (x in q) { - expect(5) - assertEquals(1, x) - } - finish(8) - } - - @Test - fun testIteratorTwo() = runBlocking { - val q = RendezvousChannel() - expect(1) - launch(context) { - expect(3) - q.send(1) - expect(4) - q.send(2) - expect(7) - q.close() - expect(8) - } - expect(2) - for (x in q) { - when (x) { - 1 -> expect(5) - 2 -> expect(6) - else -> expectUnreached() - } - } - finish(9) - } - - @Test - fun testIteratorTwoWithYield() = runBlocking { - val q = RendezvousChannel() - expect(1) - launch(context) { - expect(3) - q.send(1) // will suspend - expect(6) - q.send(2) - expect(7) - q.close() - expect(8) - } - expect(2) - yield() // yield to sender coroutine right before starting for loop - expect(4) - for (x in q) { - when (x) { - 1 -> expect(5) - 2 -> expect(9) - else -> expectUnreached() - } - } - finish(10) - } - - @Test - fun testSuspendSendOnClosedChannel() = runBlocking { - val q = RendezvousChannel() - expect(1) - launch(context) { - expect(4) - q.send(42) // suspend - expect(11) - } - expect(2) - launch(context) { - expect(5) - q.close() - expect(6) - } - expect(3) - yield() // to sender - expect(7) - yield() // try to resume sender (it will not resume despite the close!) - expect(8) - assertThat(q.receiveOrNull(), IsEqual(42)) - expect(9) - assertThat(q.receiveOrNull(), IsNull()) - expect(10) - yield() // to sender, it was resumed! - finish(12) - } - - class BadClass { - override fun equals(other: Any?): Boolean = error("equals") - override fun hashCode(): Int = error("hashCode") - override fun toString(): String = error("toString") - } - - @Test - fun testProduceBadClass() = runBlocking { - val bad = BadClass() - val c = produce(context) { - expect(1) - send(bad) - } - assertTrue(c.receive() === bad) - finish(2) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/SimpleSendReceiveTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/SimpleSendReceiveTest.kt deleted file mode 100644 index 32cde4a407..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/SimpleSendReceiveTest.kt +++ /dev/null @@ -1,71 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import org.hamcrest.core.IsEqual -import org.junit.Assert.assertThat -import org.junit.Assert.assertTrue -import org.junit.Test -import org.junit.runner.RunWith -import org.junit.runners.Parameterized - -@RunWith(Parameterized::class) -class SimpleSendReceiveTest( - val kind: TestChannelKind, - val n: Int, - val concurrent: Boolean -) { - companion object { - @Parameterized.Parameters(name = "{0}, n={1}, concurrent={2}") - @JvmStatic - fun params(): Collection> = TestChannelKind.values().flatMap { kind -> - listOf(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 100, 1000).flatMap { n -> - listOf(false, true).map { concurrent -> - arrayOf(kind, n, concurrent) - } - } - } - } - - val channel = kind.create() - - @Test - fun testSimpleSendReceive() = runBlocking { - val ctx = if (concurrent) CommonPool else context - launch(ctx) { - repeat(n) { channel.send(it) } - channel.close() - } - var expected = 0 - for (x in channel) { - if (!kind.isConflated) { - assertThat(x, IsEqual(expected++)) - } else { - assertTrue(x >= expected) - expected = x + 1 - } - } - if (kind.isConflated) { - if (n > 0) assertTrue(expected > 0) - } else { - assertThat(expected, IsEqual(n)) - } - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/TestChannelKind.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/TestChannelKind.kt deleted file mode 100644 index 81e8f7da1b..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/channels/TestChannelKind.kt +++ /dev/null @@ -1,77 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.channels - -import kotlinx.coroutines.experimental.selects.SelectInstance - -enum class TestChannelKind { - RENDEZVOUS { - override fun create(): Channel = RendezvousChannel() - override fun toString(): String = "RendezvousChannel" - }, - ARRAY_1 { - override fun create(): Channel = ArrayChannel(1) - override fun toString(): String = "ArrayChannel(1)" - }, - ARRAY_10 { - override fun create(): Channel = ArrayChannel(8) - override fun toString(): String = "ArrayChannel(8)" - }, - LINKED_LIST { - override fun create(): Channel = LinkedListChannel() - override fun toString(): String = "LinkedListChannel" - }, - CONFLATED { - override fun create(): Channel = ConflatedChannel() - override fun toString(): String = "ConflatedChannel" - override val isConflated: Boolean get() = true - }, - ARRAY_BROADCAST_1 { - override fun create(): Channel = ChannelViaBroadcast(ArrayBroadcastChannel(1)) - override fun toString(): String = "ArrayBroadcastChannel(1)" - }, - ARRAY_BROADCAST_10 { - override fun create(): Channel = ChannelViaBroadcast(ArrayBroadcastChannel(10)) - override fun toString(): String = "ArrayBroadcastChannel(10)" - }, - CONFLATED_BROADCAST { - override fun create(): Channel = ChannelViaBroadcast(ConflatedBroadcastChannel()) - override fun toString(): String = "ConflatedBroadcastChannel" - override val isConflated: Boolean get() = true - } - ; - - abstract fun create(): Channel - open val isConflated: Boolean get() = false -} - -private class ChannelViaBroadcast( - private val broadcast: BroadcastChannel -): Channel, SendChannel by broadcast { - val sub = broadcast.open() - - override val isClosedForReceive: Boolean get() = sub.isClosedForReceive - override val isEmpty: Boolean get() = sub.isEmpty - suspend override fun receive(): E = sub.receive() - suspend override fun receiveOrNull(): E? = sub.receiveOrNull() - override fun poll(): E? = sub.poll() - override fun iterator(): ChannelIterator = sub.iterator() - override fun registerSelectReceive(select: SelectInstance, block: suspend (E) -> R) = - sub.registerSelectReceive(select, block) - override fun registerSelectReceiveOrNull(select: SelectInstance, block: suspend (E?) -> R) = - sub.registerSelectReceiveOrNull(select, block) -} diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListAtomicStressTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListAtomicStressTest.kt deleted file mode 100644 index d3ac3bc0c0..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListAtomicStressTest.kt +++ /dev/null @@ -1,162 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.internal - -import kotlinx.coroutines.experimental.TestBase -import org.junit.Assert.* -import org.junit.Test -import java.util.* -import java.util.concurrent.atomic.AtomicInteger -import kotlin.concurrent.thread - -/** - * This stress test has 4 threads adding randomly to the list and them immediately undoing - * this addition by remove, and 4 threads trying to remove nodes from two lists simultaneously (atomically). - */ -class LockFreeLinkedListAtomicStressTest : TestBase() { - data class IntNode(val i: Int) : LockFreeLinkedListNode() - - val TEST_DURATION = 5000L * stressTestMultiplier - - val threads = mutableListOf() - val nLists = 4 - val nAdderThreads = 4 - val nRemoverThreads = 4 - val completedAdder = AtomicInteger() - val completedRemover = AtomicInteger() - - val lists = Array(nLists) { LockFreeLinkedListHead() } - - val undone = AtomicInteger() - val missed = AtomicInteger() - val removed = AtomicInteger() - - @Test - fun testStress() { - val deadline = System.currentTimeMillis() + TEST_DURATION - repeat(nAdderThreads) { threadId -> - threads += thread(start = false, name = "adder-$threadId") { - val rnd = Random() - while (System.currentTimeMillis() < deadline) { - when (rnd.nextInt(4)) { - 0 -> { - val list = lists[rnd.nextInt(nLists)] - val node = IntNode(threadId) - list.addLast(node) - burnTime(rnd) - tryRemove(node) - } - 1 -> { - // just to test conditional add - val list = lists[rnd.nextInt(nLists)] - val node = IntNode(threadId) - assertTrue(list.addLastIf(node, { true })) - burnTime(rnd) - tryRemove(node) - } - 2 -> { - // just to test failed conditional add and burn some time - val list = lists[rnd.nextInt(nLists)] - val node = IntNode(threadId) - assertFalse(list.addLastIf(node, { false })) - burnTime(rnd) - } - 3 -> { - // add two atomically - val idx1 = rnd.nextInt(nLists - 1) - val idx2 = idx1 + 1 + rnd.nextInt(nLists - idx1 - 1) - check(idx1 < idx2) // that is our global order - val list1 = lists[idx1] - val list2 = lists[idx2] - val node1 = IntNode(threadId) - val node2 = IntNode(-threadId - 1) - val add1 = list1.describeAddLast(node1) - val add2 = list2.describeAddLast(node2) - val op = object : AtomicOp() { - override fun prepare(): Any? = add1.prepare(this) ?: add2.prepare(this) - override fun complete(affected: Any?, failure: Any?) { - add1.complete(this, failure) - add2.complete(this, failure) - } - } - assertTrue(op.perform(null) == null) - burnTime(rnd) - tryRemove(node1) - tryRemove(node2) - } - else -> error("Cannot happen") - } - } - completedAdder.incrementAndGet() - } - } - repeat(nRemoverThreads) { threadId -> - threads += thread(start = false, name = "remover-$threadId") { - val rnd = Random() - while (System.currentTimeMillis() < deadline) { - val idx1 = rnd.nextInt(nLists - 1) - val idx2 = idx1 + 1 + rnd.nextInt(nLists - idx1 - 1) - check(idx1 < idx2) // that is our global order - val list1 = lists[idx1] - val list2 = lists[idx2] - val remove1 = list1.describeRemoveFirst() - val remove2 = list2.describeRemoveFirst() - val op = object : AtomicOp() { - override fun prepare(): Any? = remove1.prepare(this) ?: remove2.prepare(this) - override fun complete(affected: Any?, failure: Any?) { - remove1.complete(this, failure) - remove2.complete(this, failure) - } - } - val success = op.perform(null) == null - if (success) removed.addAndGet(2) - - } - completedRemover.incrementAndGet() - } - } - threads.forEach(Thread::start) - threads.forEach(Thread::join) - println("Completed successfully ${completedAdder.get()} adder threads") - println("Completed successfully ${completedRemover.get()} remover threads") - println(" Adders undone ${undone.get()} node additions") - println(" Adders missed ${missed.get()} nodes") - println("Remover removed ${removed.get()} nodes") - assertEquals(nAdderThreads, completedAdder.get()) - assertEquals(nRemoverThreads, completedRemover.get()) - assertEquals(missed.get(), removed.get()) - assertTrue(undone.get() > 0) - assertTrue(missed.get() > 0) - lists.forEach { it.validate() } - } - - private fun burnTime(rnd: Random) { - when (rnd.nextInt(3)) { - 0 -> {} // nothing -- be quick - 1 -> Thread.yield() // burn some time - 2 -> Thread.sleep(1) // burn more time - else -> error("Cannot happen") - } - } - - private fun tryRemove(node: IntNode) { - if (node.remove()) - undone.incrementAndGet() - else - missed.incrementAndGet() - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListLongStressTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListLongStressTest.kt deleted file mode 100644 index dc5d931b6d..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListLongStressTest.kt +++ /dev/null @@ -1,85 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.internal - -import kotlinx.coroutines.experimental.TestBase -import org.junit.Test -import java.util.* -import java.util.concurrent.atomic.AtomicInteger -import kotlin.concurrent.thread -import kotlin.coroutines.experimental.buildIterator - -/** - * This stress test has 2 threads adding on one side on list, 2 more threads adding on the other, - * and 6 threads iterating and concurrently removing items. The resulting list that is being - * stressed is long. - */ -class LockFreeLinkedListLongStressTest : TestBase() { - data class IntNode(val i: Int) : LockFreeLinkedListNode() - val list = LockFreeLinkedListHead() - - val threads = mutableListOf() - val nAdded = 10_000_000 // should not stress more, because that'll run out of memory - val nAddThreads = 4 // must be power of 2 (!!!) - val nRemoveThreads = 6 - val removeProbability = 0.2 - val workingAdders = AtomicInteger(nAddThreads) - - fun shallRemove(i: Int) = i and 63 != 42 - - @Test - fun testStress() { - for (j in 0 until nAddThreads) - threads += thread(start = false, name = "adder-$j") { - for (i in j until nAdded step nAddThreads) { - list.addLast(IntNode(i)) - } - println("${Thread.currentThread().name} completed") - workingAdders.decrementAndGet() - } - for (j in 0 until nRemoveThreads) - threads += thread(start = false, name = "remover-$j") { - val rnd = Random() - do { - val lastTurn = workingAdders.get() == 0 - list.forEach { node -> - if (shallRemove(node.i) && (lastTurn || rnd.nextDouble() < removeProbability)) - node.remove() - } - } while (!lastTurn) - println("${Thread.currentThread().name} completed") - } - println("Starting ${threads.size} threads") - for (thread in threads) - thread.start() - println("Joining threads") - for (thread in threads) - thread.join() - // verification - println("Verify result") - list.validate() - val expected = buildIterator { - for (i in 0 until nAdded) - if (!shallRemove(i)) - yield(i) - } - list.forEach { node -> - require(node.i == expected.next()) - } - require(!expected.hasNext()) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListShortStressTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListShortStressTest.kt deleted file mode 100644 index 5a7376ff6a..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListShortStressTest.kt +++ /dev/null @@ -1,97 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.internal - -import kotlinx.coroutines.experimental.TestBase -import org.junit.Assert.* -import org.junit.Test -import java.util.* -import java.util.concurrent.atomic.AtomicInteger -import kotlin.concurrent.thread - -/** - * This stress test has 6 threads adding randomly first to the list and them immediately undoing - * this addition by remove, and 4 threads removing first node. The resulting list that is being - * stressed is very short. - */ -class LockFreeLinkedListShortStressTest : TestBase() { - data class IntNode(val i: Int) : LockFreeLinkedListNode() - val list = LockFreeLinkedListHead() - - val TEST_DURATION = 5000L * stressTestMultiplier - - val threads = mutableListOf() - val nAdderThreads = 6 - val nRemoverThreads = 4 - val completedAdder = AtomicInteger() - val completedRemover = AtomicInteger() - - val undone = AtomicInteger() - val missed = AtomicInteger() - val removed = AtomicInteger() - - @Test - fun testStress() { - val deadline = System.currentTimeMillis() + TEST_DURATION - repeat(nAdderThreads) { threadId -> - threads += thread(start = false, name = "adder-$threadId") { - val rnd = Random() - while (System.currentTimeMillis() < deadline) { - var node: IntNode? = IntNode(threadId) - when (rnd.nextInt(3)) { - 0 -> list.addLast(node!!) - 1 -> assertTrue(list.addLastIf(node!!, { true })) // just to test conditional add - 2 -> { // just to test failed conditional add - assertFalse(list.addLastIf(node!!, { false })) - node = null - } - } - if (node != null) { - if (node.remove()) - undone.incrementAndGet() - else - missed.incrementAndGet() - } - } - completedAdder.incrementAndGet() - } - } - repeat(nRemoverThreads) { threadId -> - threads += thread(start = false, name = "remover-$threadId") { - while (System.currentTimeMillis() < deadline) { - val node = list.removeFirstOrNull() - if (node != null) removed.incrementAndGet() - - } - completedRemover.incrementAndGet() - } - } - threads.forEach { it.start() } - threads.forEach { it.join() } - println("Completed successfully ${completedAdder.get()} adder threads") - println("Completed successfully ${completedRemover.get()} remover threads") - println(" Adders undone ${undone.get()} node additions") - println(" Adders missed ${missed.get()} nodes") - println("Remover removed ${removed.get()} nodes") - assertEquals(nAdderThreads, completedAdder.get()) - assertEquals(nRemoverThreads, completedRemover.get()) - assertEquals(missed.get(), removed.get()) - assertTrue(undone.get() > 0) - assertTrue(missed.get() > 0) - list.validate() - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListTest.kt deleted file mode 100644 index c72bfff5ba..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/internal/LockFreeLinkedListTest.kt +++ /dev/null @@ -1,127 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.internal - -import org.junit.Assert.* -import org.junit.Test - -class LockFreeLinkedListTest { - data class IntNode(val i: Int) : LockFreeLinkedListNode() - - @Test - fun testSimpleAddLast() { - val list = LockFreeLinkedListHead() - assertContents(list) - val n1 = IntNode(1).apply { list.addLast(this) } - assertContents(list, 1) - val n2 = IntNode(2).apply { list.addLast(this) } - assertContents(list, 1, 2) - val n3 = IntNode(3).apply { list.addLast(this) } - assertContents(list, 1, 2, 3) - val n4 = IntNode(4).apply { list.addLast(this) } - assertContents(list, 1, 2, 3, 4) - assertTrue(n1.remove()) - assertContents(list, 2, 3, 4) - assertTrue(n3.remove()) - assertContents(list, 2, 4) - assertTrue(n4.remove()) - assertContents(list, 2) - assertTrue(n2.remove()) - assertFalse(n2.remove()) - assertContents(list) - } - - @Test - fun testCondOps() { - val list = LockFreeLinkedListHead() - assertContents(list) - assertTrue(list.addLastIf(IntNode(1)) { true }) - assertContents(list, 1) - assertFalse(list.addLastIf(IntNode(2)) { false }) - assertContents(list, 1) - assertTrue(list.addLastIf(IntNode(3)) { true }) - assertContents(list, 1, 3) - assertFalse(list.addLastIf(IntNode(4)) { false }) - assertContents(list, 1, 3) - } - - @Test - fun testRemoveTwoAtomic() { - val list = LockFreeLinkedListHead() - val n1 = IntNode(1).apply { list.addLast(this) } - val n2 = IntNode(2).apply { list.addLast(this) } - assertContents(list, 1, 2) - assertFalse(n1.isRemoved) - assertFalse(n2.isRemoved) - val remove1Desc = n1.describeRemove()!! - val remove2Desc = n2.describeRemove()!! - val operation = object : AtomicOp() { - override fun prepare(): Any? = remove1Desc.prepare(this) ?: remove2Desc.prepare(this) - override fun complete(affected: Any?, failure: Any?) { - remove1Desc.complete(this, failure) - remove2Desc.complete(this, failure) - } - } - assertTrue(operation.perform(null) == null) - assertTrue(n1.isRemoved) - assertTrue(n2.isRemoved) - assertContents(list) - } - - @Test - fun testAtomicOpsSingle() { - val list = LockFreeLinkedListHead() - assertContents(list) - val n1 = IntNode(1).also { single(list.describeAddLast(it)) } - assertContents(list, 1) - val n2 = IntNode(2).also { single(list.describeAddLast(it)) } - assertContents(list, 1, 2) - val n3 = IntNode(3).also { single(list.describeAddLast(it)) } - assertContents(list, 1, 2, 3) - val n4 = IntNode(4).also { single(list.describeAddLast(it)) } - assertContents(list, 1, 2, 3, 4) - single(n3.describeRemove()!!) - assertContents(list, 1, 2, 4) - assertTrue(n3.describeRemove() == null) - single(list.describeRemoveFirst()) - assertContents(list, 2, 4) - assertTrue(n1.describeRemove() == null) - assertTrue(n2.remove()) - assertContents(list, 4) - assertTrue(n4.remove()) - assertContents(list) - } - - private fun single(part: AtomicDesc) { - val operation = object : AtomicOp() { - override fun prepare(): Any? = part.prepare(this) - override fun complete(affected: Any?, failure: Any?) = part.complete(this, failure) - } - assertTrue(operation.perform(null) == null) - } - - private fun assertContents(list: LockFreeLinkedListHead, vararg expected: Int) { - list.validate() - val n = expected.size - val actual = IntArray(n) - var index = 0 - list.forEach { actual[index++] = it.i } - assertEquals(n, index) - for (i in 0 until n) assertEquals("item i", expected[i], actual[i]) - assertEquals(expected.isEmpty(), list.isEmpty) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectArrayChannelTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectArrayChannelTest.kt deleted file mode 100644 index 116dc627e5..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectArrayChannelTest.kt +++ /dev/null @@ -1,309 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.channels.ArrayChannel -import kotlinx.coroutines.experimental.channels.ClosedReceiveChannelException -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.junit.Assert.assertEquals -import org.junit.Test - -class SelectArrayChannelTest : TestBase() { - @Test - fun testSelectSendSuccess() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - launch(context) { - expect(2) - assertEquals("OK", channel.receive()) - finish(6) - } - yield() // to launched coroutine - expect(3) - select { - channel.onSend("OK") { - expect(4) - } - } - expect(5) - } - - @Test - fun testSelectSendSuccessWithDefault() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - launch(context) { - expect(2) - assertEquals("OK", channel.receive()) - finish(6) - } - yield() // to launched coroutine - expect(3) - select { - channel.onSend("OK") { - expect(4) - } - default { - expectUnreached() - } - } - expect(5) - } - - @Test - fun testSelectSendReceiveBuf() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - select { - channel.onSend("OK") { - expect(2) - } - } - expect(3) - select { - channel.onReceive { v -> - expect(4) - assertEquals("OK", v) - } - } - finish(5) - } - - @Test - fun testSelectSendWait() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - launch(context) { - expect(4) - assertEquals("BUF", channel.receive()) - expect(5) - assertEquals("OK", channel.receive()) - expect(6) - } - expect(2) - channel.send("BUF") - expect(3) - select { - channel.onSend("OK") { - expect(7) - } - } - finish(8) - } - - @Test - fun testSelectReceiveSuccess() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - channel.send("OK") - expect(2) - select { - channel.onReceive { v -> - expect(3) - assertEquals("OK", v) - } - } - finish(4) - } - - @Test - fun testSelectReceiveSuccessWithDefault() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - channel.send("OK") - expect(2) - select { - channel.onReceive { v -> - expect(3) - assertEquals("OK", v) - } - default { - expectUnreached() - } - } - finish(4) - } - - @Test - fun testSelectReceiveWaitWithDefault() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - select { - channel.onReceive { - expectUnreached() - } - default { - expect(2) - } - } - expect(3) - channel.send("BUF") - expect(4) - // make sure second send blocks (select above is over) - launch(context) { - expect(6) - channel.send("CHK") - finish(10) - } - expect(5) - yield() - expect(7) - assertEquals("BUF", channel.receive()) - expect(8) - assertEquals("CHK", channel.receive()) - expect(9) - } - - @Test - fun testSelectReceiveWait() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - launch(context) { - expect(3) - channel.send("OK") - expect(4) - } - expect(2) - select { - channel.onReceive { v -> - expect(5) - assertEquals("OK", v) - } - } - finish(6) - } - - @Test(expected = ClosedReceiveChannelException::class) - fun testSelectReceiveClosed() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - channel.close() - finish(2) - select { - channel.onReceive { - expectUnreached() - } - } - expectUnreached() - } - - @Test(expected = ClosedReceiveChannelException::class) - fun testSelectReceiveWaitClosed() = runBlocking { - expect(1) - val channel = ArrayChannel(1) - launch(context) { - expect(3) - channel.close() - finish(4) - } - expect(2) - select { - channel.onReceive { - expectUnreached() - } - } - expectUnreached() - } - - @Test - fun testSelectSendResourceCleanup() = runBlocking { - val channel = ArrayChannel(1) - val n = 10_000_000 * stressTestMultiplier - expect(1) - channel.send(-1) // fill the buffer, so all subsequent sends cannot proceed - repeat(n) { i -> - select { - channel.onSend(i) { expectUnreached() } - default { expect(i + 2) } - } - } - finish(n + 2) - } - - @Test - fun testSelectReceiveResourceCleanup() = runBlocking { - val channel = ArrayChannel(1) - val n = 10_000_000 * stressTestMultiplier - expect(1) - repeat(n) { i -> - select { - channel.onReceive { expectUnreached() } - default { expect(i + 2) } - } - } - finish(n + 2) - } - - @Test - fun testSelectReceiveDispatchNonSuspending() = runBlocking { - val channel = ArrayChannel(1) - expect(1) - channel.send(42) - expect(2) - launch(context) { - expect(4) - select { - channel.onReceive { v -> - expect(5) - assertEquals(42, v) - expect(6) - } - } - expect(7) // returns from select without further dispatch - } - expect(3) - yield() // to launched - finish(8) - } - - @Test - fun testSelectReceiveDispatchNonSuspending2() = runBlocking { - val channel = ArrayChannel(1) - expect(1) - channel.send(42) - expect(2) - launch(context) { - expect(4) - select { - channel.onReceive { v -> - expect(5) - assertEquals(42, v) - expect(6) - yield() // back to main - expect(8) - } - } - expect(9) // returns from select without further dispatch - } - expect(3) - yield() // to launched - expect(7) - yield() // again - finish(10) - } - - // only for debugging - internal fun SelectBuilder.default(block: suspend () -> R) { - this as SelectBuilderImpl // type assertion - if (!trySelect(null)) return - block.startCoroutineUndispatched(this) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectBiasTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectBiasTest.kt deleted file mode 100644 index 49e74e4df6..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectBiasTest.kt +++ /dev/null @@ -1,59 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.async -import kotlinx.coroutines.experimental.runBlocking -import org.junit.Assert.assertEquals -import org.junit.Assert.assertTrue -import org.junit.Test - -class SelectBiasTest { - val n = 10_000 - - @Test - fun testBiased() = runBlocking { - val d0 = async(context) { 0 } - val d1 = async(context) { 1 } - val counter = IntArray(2) - repeat(n) { - val selected = select { - d0.onAwait { 0 } - d1.onAwait { 1 } - } - counter[selected]++ - } - assertEquals(n, counter[0]) - assertEquals(0, counter[1]) - } - - @Test - fun testUnbiased() = runBlocking { - val d0 = async(context) { 0 } - val d1 = async(context) { 1 } - val counter = IntArray(2) - repeat(n) { - val selected = selectUnbiased { - d0.onAwait { 0 } - d1.onAwait { 1 } - } - counter[selected]++ - } - assertTrue(counter[0] >= n / 4) - assertTrue(counter[1] >= n / 4) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectDeferredTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectDeferredTest.kt deleted file mode 100644 index b53878b358..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectDeferredTest.kt +++ /dev/null @@ -1,131 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.* -import org.junit.Assert.assertEquals -import org.junit.Test - -class SelectDeferredTest : TestBase() { - @Test - fun testSimpleReturnsImmediately() = runBlocking { - expect(1) - val d1 = async(context) { - expect(3) - 42 - } - expect(2) - val res = select { - d1.onAwait { v -> - expect(4) - assertEquals(42, v) - "OK" - } - } - expect(5) - assertEquals("OK", res) - finish(6) - } - - @Test - fun testSimpleWithYield() = runBlocking { - expect(1) - val d1 = async(context) { - expect(3) - 42 - } - launch(context) { - expect(4) - yield() // back to main - expect(6) - } - expect(2) - val res = select { - d1.onAwait { v -> - expect(5) - assertEquals(42, v) - yield() // to launch - expect(7) - "OK" - } - } - finish(8) - assertEquals("OK", res) - } - - @Test - fun testSelectIncompleteLazy() = runBlocking { - expect(1) - val d1 = async(context, CoroutineStart.LAZY) { - expect(5) - 42 - } - launch(context) { - expect(3) - val res = select { - d1.onAwait { v -> - expect(7) - assertEquals(42, v) - "OK" - } - } - expect(8) - assertEquals("OK", res) - } - expect(2) - yield() // to launch - expect(4) - yield() // to started async - expect(6) - yield() // to triggered select - finish(9) - } - - @Test - fun testSelectTwo() = runBlocking { - expect(1) - val d1 = async(context) { - expect(3) - yield() // to the other deffered - expect(5) - yield() // to fired select - expect(7) - "d1" - } - val d2 = async(context) { - expect(4) - "d2" // returns result - } - expect(2) - val res = select { - d1.onAwait { - expectUnreached() - "FAIL" - } - d2.onAwait { v2 -> - expect(6) - assertEquals("d2", v2) - yield() // to first deferred - expect(8) - "OK" - } - } - assertEquals("OK", res) - finish(9) - } - -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectJobTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectJobTest.kt deleted file mode 100644 index 0231fd1325..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectJobTest.kt +++ /dev/null @@ -1,79 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.* -import org.junit.Assert.assertEquals -import org.junit.Test - -class SelectJobTest : TestBase() { - @Test - fun testSelectCompleted() = runBlocking { - expect(1) - launch(context) { // makes sure we don't yield to it earlier - finish(4) - } - val job = Job() - job.cancel() - select { - job.onJoin { - expect(2) - } - } - expect(3) - } - - @Test - fun testSelectIncomplete() = runBlocking { - expect(1) - val job = Job() - launch(context) { // makes sure we don't yield to it earlier - expect(3) - val res = select { - job.onJoin { - expect(6) - "OK" - } - } - expect(7) - assertEquals("OK", res) - } - expect(2) - yield() - expect(4) - job.cancel() - expect(5) - yield() - finish(8) - } - - @Test - fun testSelectLazy() = runBlocking { - expect(1) - val job = launch(context, CoroutineStart.LAZY) { - expect(2) - } - val res = select { - job.onJoin { - expect(3) - "OK" - } - } - finish(4) - assertEquals("OK", res) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectMutexTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectMutexTest.kt deleted file mode 100644 index 6aff6b156c..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectMutexTest.kt +++ /dev/null @@ -1,95 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.sync.Mutex -import kotlinx.coroutines.experimental.sync.MutexImpl -import kotlinx.coroutines.experimental.yield -import org.junit.Assert.assertEquals -import org.junit.Assert.assertTrue -import org.junit.Test - -class SelectMutexTest : TestBase() { - @Test - fun testSelectLock() = runBlocking { - val mutex = Mutex() - expect(1) - launch(context) { // ensure that it is not scheduled earlier than needed - finish(4) - } - val res = select { - mutex.onLock { - assertTrue(mutex.isLocked) - expect(2) - "OK" - } - } - assertEquals("OK", res) - expect(3) - } - - @Test - fun testSelectLockWait() = runBlocking { - val mutex = Mutex(true) // locked - expect(1) - launch(context) { - expect(3) - val res = select { // will suspended - mutex.onLock { - assertTrue(mutex.isLocked) - expect(6) - "OK" - } - } - assertEquals("OK", res) - expect(7) - } - expect(2) - yield() // to launched coroutine - expect(4) - mutex.unlock() - expect(5) - yield() // to resumed select - finish(8) - } - - @Test - fun testSelectCancelledResourceRelease() = runBlocking { - val n = 1_000 * stressTestMultiplier - val mutex = Mutex(true) as MutexImpl // locked - expect(1) - repeat(n) { i -> - val job = launch(context) { - expect(i + 2) - select { - mutex.onLock { - expectUnreached() // never able to lock - } - } - } - yield() // to the launched job, so that it suspends - job.cancel() // cancel the job and select - yield() // so it can cleanup after itself - } - assertTrue(mutex.isLocked) - assertTrue(mutex.isLockedEmptyQueueState) - finish(n + 2) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectPhilosophersStressTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectPhilosophersStressTest.kt deleted file mode 100644 index 9406ff2ac3..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectPhilosophersStressTest.kt +++ /dev/null @@ -1,77 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.sync.Mutex -import org.junit.Assert.assertTrue -import org.junit.Test - -class SelectPhilosophersStressTest : TestBase() { - val TEST_DURATION = 3000L * stressTestMultiplier - - val n = 10 // number of philosophers - val forks = Array(n) { Mutex() } - - suspend fun eat(id: Int, desc: String) { - val left = forks[id] - val right = forks[(id + 1) % n] - while (true) { - val pair = selectUnbiased> { - left.onLock(desc) { left to right } - right.onLock(desc) { right to left } - } - if (pair.second.tryLock(desc)) break - pair.first.unlock(desc) - pair.second.lock(desc) - if (pair.first.tryLock(desc)) break - pair.second.unlock(desc) - } - assertTrue(left.isLocked && right.isLocked) - // om, nom, nom --> eating!!! - right.unlock(desc) - left.unlock(desc) - } - - @Test - fun testPhilosophers() = runBlocking { - val timeLimit = System.currentTimeMillis() + TEST_DURATION - val philosophers = List>(n) { id -> - async(CommonPool) { - val desc = "Philosopher $id" - var eatsCount = 0 - while (System.currentTimeMillis() < timeLimit) { - eat(id, desc) - eatsCount++ - yield() - } - println("Philosopher $id done, eats $eatsCount times") - eatsCount - } - } - val debugJob = launch(context) { - delay(3 * TEST_DURATION) - println("Test is failing. Lock states are:") - forks.withIndex().forEach { (id, mutex) -> println("$id: $mutex") } - } - val eats = withTimeout(5 * TEST_DURATION) { philosophers.map { it.await() } } - debugJob.cancel() - eats.withIndex().forEach { (id, eats) -> - assertTrue("$id shall not starve", eats > 0) - } - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectRendezvousChannelTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectRendezvousChannelTest.kt deleted file mode 100644 index 66f42dd383..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectRendezvousChannelTest.kt +++ /dev/null @@ -1,329 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.channels.ClosedReceiveChannelException -import kotlinx.coroutines.experimental.channels.RendezvousChannel -import kotlinx.coroutines.experimental.intrinsics.startCoroutineUndispatched -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.junit.Assert.assertEquals -import org.junit.Test - -class SelectRendezvousChannelTest : TestBase() { - @Test - fun testSelectSendSuccess() = runBlocking { - expect(1) - val channel = RendezvousChannel() - launch(context) { - expect(2) - assertEquals("OK", channel.receive()) - finish(6) - } - yield() // to launched coroutine - expect(3) - select { - channel.onSend("OK") { - expect(4) - } - } - expect(5) - } - - @Test - fun testSelectSendSuccessWithDefault() = runBlocking { - expect(1) - val channel = RendezvousChannel() - launch(context) { - expect(2) - assertEquals("OK", channel.receive()) - finish(6) - } - yield() // to launched coroutine - expect(3) - select { - channel.onSend("OK") { - expect(4) - } - default { - expectUnreached() - } - } - expect(5) - } - - @Test - fun testSelectSendWaitWithDefault() = runBlocking { - expect(1) - val channel = RendezvousChannel() - select { - channel.onSend("OK") { - expectUnreached() - } - default { - expect(2) - } - } - expect(3) - // make sure receive blocks (select above is over) - launch(context) { - expect(5) - assertEquals("CHK", channel.receive()) - finish(8) - } - expect(4) - yield() - expect(6) - channel.send("CHK") - expect(7) - } - - @Test - fun testSelectSendWait() = runBlocking { - expect(1) - val channel = RendezvousChannel() - launch(context) { - expect(3) - assertEquals("OK", channel.receive()) - expect(4) - } - expect(2) - select { - channel.onSend("OK") { - expect(5) - } - } - finish(6) - } - - @Test - fun testSelectReceiveSuccess() = runBlocking { - expect(1) - val channel = RendezvousChannel() - launch(context) { - expect(2) - channel.send("OK") - finish(6) - } - yield() // to launched coroutine - expect(3) - select { - channel.onReceive { v -> - expect(4) - assertEquals("OK", v) - } - } - expect(5) - } - - @Test - fun testSelectReceiveSuccessWithDefault() = runBlocking { - expect(1) - val channel = RendezvousChannel() - launch(context) { - expect(2) - channel.send("OK") - finish(6) - } - yield() // to launched coroutine - expect(3) - select { - channel.onReceive { v -> - expect(4) - assertEquals("OK", v) - } - default { - expectUnreached() - } - } - expect(5) - } - - @Test - fun testSelectReceiveWaitWithDefault() = runBlocking { - expect(1) - val channel = RendezvousChannel() - select { - channel.onReceive { - expectUnreached() - } - default { - expect(2) - } - } - expect(3) - // make sure send blocks (select above is over) - launch(context) { - expect(5) - channel.send("CHK") - finish(8) - } - expect(4) - yield() - expect(6) - assertEquals("CHK", channel.receive()) - expect(7) - } - - @Test - fun testSelectReceiveWait() = runBlocking { - expect(1) - val channel = RendezvousChannel() - launch(context) { - expect(3) - channel.send("OK") - expect(4) - } - expect(2) - select { - channel.onReceive { v -> - expect(5) - assertEquals("OK", v) - } - } - finish(6) - } - - @Test(expected = ClosedReceiveChannelException::class) - fun testSelectReceiveClosed() = runBlocking { - expect(1) - val channel = RendezvousChannel() - channel.close() - finish(2) - select { - channel.onReceive { - expectUnreached() - } - } - expectUnreached() - } - - @Test(expected = ClosedReceiveChannelException::class) - fun testSelectReceiveWaitClosed() = runBlocking { - expect(1) - val channel = RendezvousChannel() - launch(context) { - expect(3) - channel.close() - finish(4) - } - expect(2) - select { - channel.onReceive { - expectUnreached() - } - } - expectUnreached() - } - - @Test - fun testSelectSendResourceCleanup() = runBlocking { - val channel = RendezvousChannel() - val n = 10_000_000 * stressTestMultiplier - expect(1) - repeat(n) { i -> - select { - channel.onSend(i) { expectUnreached() } - default { expect(i + 2) } - } - } - finish(n + 2) - } - - @Test - fun testSelectReceiveResourceCleanup() = runBlocking { - val channel = RendezvousChannel() - val n = 10_000_000 * stressTestMultiplier - expect(1) - repeat(n) { i -> - select { - channel.onReceive { expectUnreached() } - default { expect(i + 2) } - } - } - finish(n + 2) - } - - @Test - fun testSelectAtomicFailure() = runBlocking { - val c1 = RendezvousChannel() - val c2 = RendezvousChannel() - expect(1) - launch(context) { - expect(3) - val res = select { - c1.onReceive { v1 -> - expect(4) - assertEquals(42, v1) - yield() // back to main - expect(7) - "OK" - } - c2.onReceive { - "FAIL" - } - } - expect(8) - assertEquals("OK", res) - } - expect(2) - c1.send(42) // send to coroutine, suspends - expect(5) - c2.close() // makes sure that selected expression does not fail! - expect(6) - yield() // back - finish(9) - } - - @Test - fun testSelectWaitDispatch() = runBlocking { - val c = RendezvousChannel() - expect(1) - launch(context) { - expect(3) - val res = select { - c.onReceive { v -> - expect(6) - assertEquals(42, v) - yield() // back to main - expect(8) - "OK" - } - } - expect(9) - assertEquals("OK", res) - } - expect(2) - yield() // to launch - expect(4) - c.send(42) // do not suspend - expect(5) - yield() // to receive - expect(7) - yield() // again - finish(10) - } - - // only for debugging - internal fun SelectBuilder.default(block: suspend () -> R) { - this as SelectBuilderImpl // type assertion - if (!trySelect(null)) return - block.startCoroutineUndispatched(this) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectTimeoutTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectTimeoutTest.kt deleted file mode 100644 index b8842c19b4..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/selects/SelectTimeoutTest.kt +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.selects - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import org.hamcrest.MatcherAssert.assertThat -import org.hamcrest.core.IsEqual -import org.junit.Test - -class SelectTimeoutTest : TestBase() { - @Test - fun testBasic() = runBlocking { - expect(1) - val result = select { - onTimeout(1000) { - expectUnreached() - "FAIL" - } - onTimeout(100) { - expect(2) - "OK" - } - onTimeout(500) { - expectUnreached() - "FAIL" - } - } - assertThat(result, IsEqual("OK")) - finish(3) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/sync/MutexTest.kt b/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/sync/MutexTest.kt deleted file mode 100644 index 7a735df12a..0000000000 --- a/kotlinx-coroutines-core/src/test/kotlin/kotlinx/coroutines/experimental/sync/MutexTest.kt +++ /dev/null @@ -1,94 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.sync - -import guide.sync.example06.mutex -import kotlinx.coroutines.experimental.* -import org.junit.Assert.* -import org.junit.Test - -class MutexTest : TestBase() { - @Test - fun testSimple() = runBlocking { - val mutex = Mutex() - expect(1) - launch(context) { - expect(4) - mutex.lock() // suspends - expect(7) // now got lock - mutex.unlock() - expect(8) - } - expect(2) - mutex.lock() // locked - expect(3) - yield() // yield to child - expect(5) - mutex.unlock() - expect(6) - yield() // now child has lock - finish(9) - } - - @Test - fun tryLockTest() { - val mutex = Mutex() - assertFalse(mutex.isLocked) - assertTrue(mutex.tryLock()) - assertTrue(mutex.isLocked) - assertFalse(mutex.tryLock()) - assertTrue(mutex.isLocked) - mutex.unlock() - assertFalse(mutex.isLocked) - assertTrue(mutex.tryLock()) - assertTrue(mutex.isLocked) - assertFalse(mutex.tryLock()) - assertTrue(mutex.isLocked) - mutex.unlock() - assertFalse(mutex.isLocked) - } - - @Test - fun withLockTest() = runBlocking { - val mutex = Mutex() - assertFalse(mutex.isLocked) - mutex.withLock { - assertTrue(mutex.isLocked) - } - assertFalse(mutex.isLocked) - } - - @Test - fun testStress() = runBlocking { - val n = 1000 * stressTestMultiplier - val k = 100 - var shared = 0 - val mutex = Mutex() - val jobs = List(n) { - launch(CommonPool) { - repeat(k) { - mutex.lock() - shared++ - mutex.unlock() - } - } - } - jobs.forEach { it.join() } - println("Shared value = $shared") - assertEquals(n * k, shared) - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-core/wasmJs/src/CoroutineContext.kt b/kotlinx-coroutines-core/wasmJs/src/CoroutineContext.kt new file mode 100644 index 0000000000..8e41274d7f --- /dev/null +++ b/kotlinx-coroutines-core/wasmJs/src/CoroutineContext.kt @@ -0,0 +1,16 @@ +package kotlinx.coroutines + +internal external interface JsProcess : JsAny { + fun nextTick(handler: () -> Unit) +} + +internal fun tryGetProcess(): JsProcess? = + js("(typeof(process) !== 'undefined' && typeof(process.nextTick) === 'function') ? process : null") + +internal fun tryGetWindow(): W3CWindow? = + js("(typeof(window) !== 'undefined' && window != null && typeof(window.addEventListener) === 'function') ? window : null") + +internal actual fun createDefaultDispatcher(): CoroutineDispatcher = + tryGetProcess()?.let(::NodeDispatcher) + ?: tryGetWindow()?.let(::WindowDispatcher) + ?: SetTimeoutDispatcher diff --git a/kotlinx-coroutines-core/wasmJs/src/Debug.kt b/kotlinx-coroutines-core/wasmJs/src/Debug.kt new file mode 100644 index 0000000000..00fa5bdbce --- /dev/null +++ b/kotlinx-coroutines-core/wasmJs/src/Debug.kt @@ -0,0 +1,16 @@ +package kotlinx.coroutines + +internal actual val DEBUG: Boolean = false + +internal actual val Any.hexAddress: String + get() = this.hashCode().toString() + +internal actual val Any.classSimpleName: String get() = this::class.simpleName ?: "Unknown" + +internal actual inline fun assert(value: () -> Boolean) {} + +internal external interface Console { + fun error(s: String) +} + +internal external val console: Console \ No newline at end of file diff --git a/kotlinx-coroutines-core/wasmJs/src/JSDispatcher.kt b/kotlinx-coroutines-core/wasmJs/src/JSDispatcher.kt new file mode 100644 index 0000000000..d5a95190c5 --- /dev/null +++ b/kotlinx-coroutines-core/wasmJs/src/JSDispatcher.kt @@ -0,0 +1,90 @@ +package kotlinx.coroutines + +import kotlin.js.* + +internal actual abstract external class W3CWindow { + fun clearTimeout(handle: Int) +} + +internal actual fun w3cSetTimeout(window: W3CWindow, handler: () -> Unit, timeout: Int): Int = + setTimeout(window, handler, timeout) + +internal actual fun w3cSetTimeout(handler: () -> Unit, timeout: Int): Int = + setTimeout(handler, timeout) + +internal actual fun w3cClearTimeout(window: W3CWindow, handle: Int) = + window.clearTimeout(handle) + +internal actual fun w3cClearTimeout(handle: Int) = + clearTimeout(handle) + +internal actual class ScheduledMessageQueue actual constructor(private val dispatcher: SetTimeoutBasedDispatcher) : MessageQueue() { + internal val processQueue: () -> Unit = ::process + + actual override fun schedule() { + dispatcher.scheduleQueueProcessing() + } + + actual override fun reschedule() { + setTimeout(processQueue, 0) + } + + internal actual fun setTimeout(timeout: Int) { + setTimeout(processQueue, timeout) + } +} + +internal class NodeDispatcher(private val process: JsProcess) : SetTimeoutBasedDispatcher() { + override fun scheduleQueueProcessing() { + process.nextTick(messageQueue.processQueue) + } +} + +@Suppress("UNUSED_PARAMETER") +private fun subscribeToWindowMessages(window: W3CWindow, process: () -> Unit): Unit = js("""{ + const handler = (event) => { + if (event.source == window && event.data == 'dispatchCoroutine') { + event.stopPropagation(); + process(); + } + } + window.addEventListener('message', handler, true); +}""") + +@Suppress("UNUSED_PARAMETER") +private fun createRescheduleMessagePoster(window: W3CWindow): () -> Unit = + js("() => window.postMessage('dispatchCoroutine', '*')") + +@Suppress("UNUSED_PARAMETER") +private fun createScheduleMessagePoster(process: () -> Unit): () -> Unit = + js("() => Promise.resolve(0).then(process)") + +internal actual class WindowMessageQueue actual constructor(window: W3CWindow) : MessageQueue() { + private val scheduleMessagePoster = createScheduleMessagePoster(::process) + private val rescheduleMessagePoster = createRescheduleMessagePoster(window) + init { + subscribeToWindowMessages(window, ::process) + } + + actual override fun schedule() { + scheduleMessagePoster() + } + + actual override fun reschedule() { + rescheduleMessagePoster() + } +} + +// We need to reference global setTimeout and clearTimeout so that it works on Node.JS as opposed to +// using them via "window" (which only works in browser) +private external fun setTimeout(handler: () -> Unit, timeout: Int): Int + +// d8 doesn't have clearTimeout +@Suppress("UNUSED_PARAMETER") +private fun clearTimeout(handle: Int): Unit = + js("{ if (typeof clearTimeout !== 'undefined') clearTimeout(handle); }") + +@Suppress("UNUSED_PARAMETER") +private fun setTimeout(window: W3CWindow, handler: () -> Unit, timeout: Int): Int = + js("window.setTimeout(handler, timeout)") + diff --git a/kotlinx-coroutines-core/wasmJs/src/Promise.kt b/kotlinx-coroutines-core/wasmJs/src/Promise.kt new file mode 100644 index 0000000000..9099a7c3e9 --- /dev/null +++ b/kotlinx-coroutines-core/wasmJs/src/Promise.kt @@ -0,0 +1,80 @@ +package kotlinx.coroutines + +import kotlin.coroutines.* +import kotlin.js.* + +@Suppress("UNUSED_PARAMETER") +internal fun promiseSetDeferred(promise: Promise, deferred: JsAny): Unit = + js("promise.deferred = deferred") + +@Suppress("UNUSED_PARAMETER") +internal fun promiseGetDeferred(promise: Promise): JsAny? = js("""{ + console.assert(promise instanceof Promise, "promiseGetDeferred must receive a promise, but got ", promise); + return promise.deferred == null ? null : promise.deferred; +}""") + + +/** + * Starts new coroutine and returns its result as an implementation of [Promise]. + * + * Coroutine context is inherited from a [CoroutineScope], additional context elements can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * The parent job is inherited from a [CoroutineScope] as well, but it can also be overridden + * with corresponding [context] element. + * + * By default, the coroutine is immediately scheduled for execution. + * Other options can be specified via `start` parameter. See [CoroutineStart] for details. + * + * @param context additional to [CoroutineScope.coroutineContext] context of the coroutine. + * @param start coroutine start option. The default value is [CoroutineStart.DEFAULT]. + * @param block the coroutine code. + */ +public fun CoroutineScope.promise( + context: CoroutineContext = EmptyCoroutineContext, + start: CoroutineStart = CoroutineStart.DEFAULT, + block: suspend CoroutineScope.() -> T +): Promise = + async(context, start, block).asPromise() + +/** + * Converts this deferred value to the instance of [Promise]. + */ +public fun Deferred.asPromise(): Promise { + val promise = Promise { resolve, reject -> + invokeOnCompletion { + val e = getCompletionExceptionOrNull() + if (e != null) { + reject(e.toJsReference()) + } else { + resolve(getCompleted()?.toJsReference()) + } + } + } + promiseSetDeferred(promise, this.toJsReference()) + return promise +} + +/** + * Converts this promise value to the instance of [Deferred]. + */ +@Suppress("UNCHECKED_CAST_TO_EXTERNAL_INTERFACE", "UNCHECKED_CAST") +public fun Promise.asDeferred(): Deferred { + val deferred = promiseGetDeferred(this) as? JsReference> + return deferred?.get() ?: GlobalScope.async(start = CoroutineStart.UNDISPATCHED) { await() } +} + +/** + * Awaits for completion of the promise without blocking. + * + * This suspending function is cancellable: if the [Job] of the current coroutine is cancelled while this + * suspending function is waiting on the promise, this function immediately resumes with [CancellationException]. + * There is a **prompt cancellation guarantee**: even if this function is ready to return the result, but was cancelled + * while suspended, [CancellationException] will be thrown. See [suspendCancellableCoroutine] for low-level details. + */ +@Suppress("UNCHECKED_CAST") +public suspend fun Promise.await(): T = suspendCancellableCoroutine { cont: CancellableContinuation -> + this@await.then( + onFulfilled = { cont.resume(it as T); null }, + onRejected = { cont.resumeWithException(it.toThrowableOrNull() ?: error("Unexpected non-Kotlin exception $it")); null } + ) +} diff --git a/kotlinx-coroutines-core/wasmJs/src/internal/CopyOnWriteList.kt b/kotlinx-coroutines-core/wasmJs/src/internal/CopyOnWriteList.kt new file mode 100644 index 0000000000..43496a5c10 --- /dev/null +++ b/kotlinx-coroutines-core/wasmJs/src/internal/CopyOnWriteList.kt @@ -0,0 +1,69 @@ +package kotlinx.coroutines.internal + +@Suppress("UNCHECKED_CAST") +internal class CopyOnWriteList : AbstractMutableList() { + private var array: Array = arrayOfNulls(0) + + override val size: Int + get() = array.size + + override fun add(element: E): Boolean { + val n = size + val update = array.copyOf(n + 1) + update[n] = element + array = update + return true + } + + override fun add(index: Int, element: E) { + rangeCheck(index) + val n = size + val update = arrayOfNulls(n + 1) + array.copyInto(destination = update, endIndex = index) + update[index] = element + array.copyInto(destination = update, destinationOffset = index + 1, startIndex = index, endIndex = n + 1) + array = update + } + + override fun remove(element: E): Boolean { + val index = array.indexOf(element as Any) + if (index == -1) return false + removeAt(index) + return true + } + + override fun removeAt(index: Int): E { + rangeCheck(index) + val n = size + val element = array[index] + val update = arrayOfNulls(n - 1) + array.copyInto(destination = update, endIndex = index) + array.copyInto(destination = update, destinationOffset = index, startIndex = index + 1, endIndex = n) + array = update + return element as E + } + + override fun iterator(): MutableIterator = IteratorImpl(array as Array) + override fun listIterator(): MutableListIterator = throw UnsupportedOperationException("Operation is not supported") + override fun listIterator(index: Int): MutableListIterator = throw UnsupportedOperationException("Operation is not supported") + override fun isEmpty(): Boolean = size == 0 + override fun set(index: Int, element: E): E = throw UnsupportedOperationException("Operation is not supported") + override fun get(index: Int): E = array[rangeCheck(index)] as E + + private class IteratorImpl(private val array: Array) : MutableIterator { + private var current = 0 + + override fun hasNext(): Boolean = current != array.size + + override fun next(): E { + if (!hasNext()) throw NoSuchElementException() + return array[current++] + } + + override fun remove() = throw UnsupportedOperationException("Operation is not supported") + } + + private fun rangeCheck(index: Int) = index.apply { + if (index < 0 || index >= size) throw IndexOutOfBoundsException("index: $index, size: $size") + } +} diff --git a/kotlinx-coroutines-core/wasmJs/src/internal/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/wasmJs/src/internal/CoroutineExceptionHandlerImpl.kt new file mode 100644 index 0000000000..b3c09e7c38 --- /dev/null +++ b/kotlinx-coroutines-core/wasmJs/src/internal/CoroutineExceptionHandlerImpl.kt @@ -0,0 +1,8 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* + +internal actual fun propagateExceptionFinalResort(exception: Throwable) { + // log exception + console.error(exception.toString()) +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/wasmJs/test/PromiseTest.kt b/kotlinx-coroutines-core/wasmJs/test/PromiseTest.kt new file mode 100644 index 0000000000..e72e661517 --- /dev/null +++ b/kotlinx-coroutines-core/wasmJs/test/PromiseTest.kt @@ -0,0 +1,110 @@ +package kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlin.js.* +import kotlin.test.* + +class PromiseTest : TestBase() { + @Test + fun testPromiseResolvedAsDeferred() = GlobalScope.promise { + val promise = Promise> { resolve, _ -> + resolve("OK".toJsReference()) + } + val deferred = promise.asDeferred>() + assertEquals("OK", deferred.await().get()) + } + + @Test + fun testPromiseRejectedAsDeferred() = GlobalScope.promise { + lateinit var promiseReject: (JsAny) -> Unit + val promise = Promise { _, reject -> + promiseReject = reject + } + val deferred = promise.asDeferred>() + // reject after converting to deferred to avoid "Unhandled promise rejection" warnings + promiseReject(TestException("Rejected").toJsReference()) + try { + deferred.await() + expectUnreached() + } catch (e: Throwable) { + assertIs(e) + assertEquals("Rejected", e.message) + } + } + + @Test + fun testCompletedDeferredAsPromise() = GlobalScope.promise { + val deferred = async(start = CoroutineStart.UNDISPATCHED) { + // completed right away + "OK" + } + val promise = deferred.asPromise() + assertEquals("OK", promise.await()) + } + + @Test + fun testWaitForDeferredAsPromise() = GlobalScope.promise { + val deferred = async { + // will complete later + "OK" + } + val promise = deferred.asPromise() + assertEquals("OK", promise.await()) // await yields main thread to deferred coroutine + } + + @Test + fun testCancellableAwaitPromise() = GlobalScope.promise { + lateinit var r: (JsAny) -> Unit + val toAwait = Promise { resolve, _ -> r = resolve } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + toAwait.await() // suspends + } + job.cancel() // cancel the job + r("fail".toJsString()) // too late, the waiting job was already cancelled + } + + @Test + fun testAsPromiseAsDeferred() = GlobalScope.promise { + val deferred = async { "OK" } + val promise = deferred.asPromise() + val d2 = promise.asDeferred() + assertSame(d2, deferred) + assertEquals("OK", d2.await()) + } + + @Test + fun testLeverageTestResult(): TestResult { + // Cannot use expect(..) here + var seq = 0 + val result = runTest { + ++seq + } + return result.then { + if (seq != 1) error("Unexpected result: $seq") + null + } + } + + @Test + fun testAwaitPromiseRejectedWithNonKotlinException() = GlobalScope.promise { + lateinit var r: (JsAny) -> Unit + val toAwait = Promise { _, reject -> r = reject } + val throwable = async(start = CoroutineStart.UNDISPATCHED) { + assertFails { toAwait.await() } + } + r("Rejected".toJsString()) + assertIs(throwable.await()) + } + + @Test + fun testAwaitPromiseRejectedWithKotlinException() = GlobalScope.promise { + lateinit var r: (JsAny) -> Unit + val toAwait = Promise { _, reject -> r = reject } + val throwable = async(start = CoroutineStart.UNDISPATCHED) { + assertFails { toAwait.await() } + } + r(RuntimeException("Rejected").toJsReference()) + assertIs(throwable.await()) + assertEquals("Rejected", throwable.await().message) + } +} diff --git a/kotlinx-coroutines-core/wasmWasi/src/Debug.kt b/kotlinx-coroutines-core/wasmWasi/src/Debug.kt new file mode 100644 index 0000000000..740265ac84 --- /dev/null +++ b/kotlinx-coroutines-core/wasmWasi/src/Debug.kt @@ -0,0 +1,10 @@ +package kotlinx.coroutines + +internal actual val DEBUG: Boolean = false + +internal actual val Any.hexAddress: String + get() = this.hashCode().toString() + +internal actual val Any.classSimpleName: String get() = this::class.simpleName ?: "Unknown" + +internal actual inline fun assert(value: () -> Boolean) {} \ No newline at end of file diff --git a/kotlinx-coroutines-core/wasmWasi/src/EventLoop.kt b/kotlinx-coroutines-core/wasmWasi/src/EventLoop.kt new file mode 100644 index 0000000000..a0f392e5b0 --- /dev/null +++ b/kotlinx-coroutines-core/wasmWasi/src/EventLoop.kt @@ -0,0 +1,120 @@ +@file:OptIn(UnsafeWasmMemoryApi::class) +package kotlinx.coroutines + +import kotlin.coroutines.CoroutineContext +import kotlin.wasm.* +import kotlin.wasm.unsafe.* + +@WasmImport("wasi_snapshot_preview1", "poll_oneoff") +private external fun wasiPollOneOff(ptrToSubscription: Int, eventPtr: Int, nsubscriptions: Int, resultPtr: Int): Int + +@WasmImport("wasi_snapshot_preview1", "clock_time_get") +private external fun wasiRawClockTimeGet(clockId: Int, precision: Long, resultPtr: Int): Int + +private const val CLOCKID_MONOTONIC = 1 + +internal actual fun createEventLoop(): EventLoop = DefaultExecutor + +internal actual fun nanoTime(): Long = withScopedMemoryAllocator { allocator: MemoryAllocator -> + val ptrTo8Bytes = allocator.allocate(8) + val returnCode = wasiRawClockTimeGet( + clockId = CLOCKID_MONOTONIC, + precision = 1, + resultPtr = ptrTo8Bytes.address.toInt() + ) + check(returnCode == 0) { "clock_time_get failed with the return code $returnCode" } + ptrTo8Bytes.loadLong() +} + +private fun sleep(nanos: Long, ptrTo32Bytes: Pointer, ptrTo8Bytes: Pointer, ptrToSubscription: Pointer) { + //__wasi_timestamp_t timeout; + (ptrToSubscription + 24).storeLong(nanos) + val returnCode = wasiPollOneOff( + ptrToSubscription = ptrToSubscription.address.toInt(), + eventPtr = ptrTo32Bytes.address.toInt(), + nsubscriptions = 1, + resultPtr = ptrTo8Bytes.address.toInt() + ) + check(returnCode == 0) { "poll_oneoff failed with the return code $returnCode" } +} + +internal actual object DefaultExecutor : EventLoopImplBase() { + + init { + if (kotlin.wasm.internal.onExportedFunctionExit == null) { + kotlin.wasm.internal.onExportedFunctionExit = ::runEventLoop + } + } + + override fun shutdown() { + // don't do anything: on WASI, the event loop is the default executor, we can't shut it down + } + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + scheduleInvokeOnTimeout(timeMillis, block) +} + +internal actual abstract class EventLoopImplPlatform : EventLoop() { + protected actual fun unpark() { + // do nothing: in WASI, no external callbacks can be invoked while `poll_oneoff` is running, + // so it is both impossible and unnecessary to unpark the event loop + } + + protected actual fun reschedule(now: Long, delayedTask: EventLoopImplBase.DelayedTask) { + // throw; on WASI, the event loop is the default executor, we can't shut it down or reschedule tasks + // to anyone else + throw UnsupportedOperationException("runBlocking event loop is not supported") + } +} + +internal actual inline fun platformAutoreleasePool(crossinline block: () -> Unit) = block() + +internal fun runEventLoop() { + withScopedMemoryAllocator { allocator -> + val ptrToSubscription = initializeSubscriptionPtr(allocator) + val ptrTo32Bytes = allocator.allocate(32) + val ptrTo8Bytes = allocator.allocate(8) + val eventLoop = DefaultExecutor + eventLoop.incrementUseCount() + try { + while (true) { + val parkNanos = eventLoop.processNextEvent() + if (parkNanos == Long.MAX_VALUE) { + // no more events + break + } + if (parkNanos > 0) { + // sleep until the next event + sleep( + parkNanos, + ptrTo32Bytes = ptrTo32Bytes, + ptrTo8Bytes = ptrTo8Bytes, + ptrToSubscription = ptrToSubscription + ) + } + } + } finally { // paranoia + eventLoop.decrementUseCount() + } + } +} + +private fun initializeSubscriptionPtr(allocator: MemoryAllocator): Pointer { + val ptrToSubscription = allocator.allocate(48) + //userdata + ptrToSubscription.storeLong(0) + //uint8_t tag; + (ptrToSubscription + 8).storeByte(0) //EVENTTYPE_CLOCK + //__wasi_clockid_t id; + (ptrToSubscription + 16).storeInt(CLOCKID_MONOTONIC) //CLOCKID_MONOTONIC + //__wasi_timestamp_t timeout; + //(ptrToSubscription + 24).storeLong(timeout) + //__wasi_timestamp_t precision; + (ptrToSubscription + 32).storeLong(0) + //__wasi_subclockflags_t + (ptrToSubscription + 40).storeShort(0) //ABSOLUTE_TIME=1/RELATIVE=0 + + return ptrToSubscription +} + +internal actual fun createDefaultDispatcher(): CoroutineDispatcher = DefaultExecutor diff --git a/kotlinx-coroutines-core/wasmWasi/src/internal/CoroutineExceptionHandlerImpl.kt b/kotlinx-coroutines-core/wasmWasi/src/internal/CoroutineExceptionHandlerImpl.kt new file mode 100644 index 0000000000..ba75a7ff65 --- /dev/null +++ b/kotlinx-coroutines-core/wasmWasi/src/internal/CoroutineExceptionHandlerImpl.kt @@ -0,0 +1,57 @@ +@file:OptIn(UnsafeWasmMemoryApi::class) + +package kotlinx.coroutines.internal + +import kotlin.wasm.WasmImport +import kotlin.wasm.unsafe.UnsafeWasmMemoryApi +import kotlin.wasm.unsafe.withScopedMemoryAllocator + +private const val STDERR = 2 + +/** + * Write to a file descriptor. Note: This is similar to `writev` in POSIX. + */ +@WasmImport("wasi_snapshot_preview1", "fd_write") +private external fun wasiRawFdWrite(descriptor: Int, scatterPtr: Int, scatterSize: Int, errorPtr: Int): Int + +@OptIn(UnsafeWasmMemoryApi::class) +private fun printlnErrorStream(message: String): Int = withScopedMemoryAllocator { allocator -> + val data = message.encodeToByteArray() + val dataSize = data.size + val memorySize = dataSize + 1 + + val ptr = allocator.allocate(memorySize) + var currentPtr = ptr + for (el in data) { + currentPtr.storeByte(el) + currentPtr += 1 + } + (ptr + dataSize).storeByte(0x0A) + + val scatterPtr = allocator.allocate(8) + (scatterPtr + 0).storeInt(ptr.address.toInt()) + (scatterPtr + 4).storeInt(memorySize) + + val rp0 = allocator.allocate(4) + + val ret = wasiRawFdWrite( + descriptor = STDERR, + scatterPtr = scatterPtr.address.toInt(), + scatterSize = 1, + errorPtr = rp0.address.toInt() + ) + + if (ret != 0) rp0.loadInt() else 0 +} + +/* +* Terminate the process normally with an exit code. + */ +@WasmImport("wasi_snapshot_preview1", "proc_exit") +private external fun wasiProcExit(exitCode: Int) + +internal actual fun propagateExceptionFinalResort(exception: Throwable) { + val errorCode = printlnErrorStream("!!!") + val returnCode = if (errorCode != 0) errorCode else 1 + wasiProcExit(returnCode) +} \ No newline at end of file diff --git a/kotlinx-coroutines-core/wasmWasi/src/internal/CoroutineRunner.kt b/kotlinx-coroutines-core/wasmWasi/src/internal/CoroutineRunner.kt new file mode 100644 index 0000000000..081ce0ed09 --- /dev/null +++ b/kotlinx-coroutines-core/wasmWasi/src/internal/CoroutineRunner.kt @@ -0,0 +1,15 @@ +package kotlinx.coroutines.internal + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** @suppress **This is internal API and it is subject to change.** */ +@InternalCoroutinesApi +public fun runTestCoroutine(context: CoroutineContext, block: suspend CoroutineScope.() -> Unit) { + val newContext = GlobalScope.newCoroutineContext(context) + val coroutine = object: AbstractCoroutine(newContext, initParentJob = true, active = true) {} + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) + runEventLoop() + check(coroutine.isCompleted) { "Coroutine $coroutine did not complete, but the system reached quiescence" } + coroutine.getCompletionExceptionOrNull()?.let { throw it } +} diff --git a/kotlinx-coroutines-debug/README.md b/kotlinx-coroutines-debug/README.md new file mode 100644 index 0000000000..de288520d6 --- /dev/null +++ b/kotlinx-coroutines-debug/README.md @@ -0,0 +1,272 @@ +# Module kotlinx-coroutines-debug + +Debugging facilities for `kotlinx.coroutines` on JVM. + +### Overview + +This module provides a debug JVM agent that allows to track and trace existing coroutines. +The main entry point to debug facilities is [DebugProbes] API. +Call to [DebugProbes.install] installs debug agent via ByteBuddy and starts spying on coroutines when they are created, suspended and resumed. + +After that, you can use [DebugProbes.dumpCoroutines] to print all active (suspended or running) coroutines, including their state, creation and +suspension stacktraces. +Additionally, it is possible to process the list of such coroutines via [DebugProbes.dumpCoroutinesInfo] or dump isolated parts +of coroutines hierarchy referenced by a [Job] or [CoroutineScope] instances using [DebugProbes.printJob] and [DebugProbes.printScope] respectively. + +This module also provides an automatic [BlockHound](https://github.com/reactor/BlockHound) integration +that detects when a blocking operation was called in a coroutine context that prohibits it. In order to use it, +please follow the BlockHound [quick start guide]( +https://github.com/reactor/BlockHound/blob/1.0.8.RELEASE/docs/quick_start.md). + +### Using in your project + +Add `kotlinx-coroutines-debug` to your project test dependencies: +``` +dependencies { + testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-debug:1.4.0' +} +``` + +### Using in unit tests + +For JUnit4 debug module provides special test rule, [CoroutinesTimeout], for installing debug probes +and to dump coroutines on timeout to simplify tests debugging. + +Its usage is better demonstrated by the example (runnable code is [here](test/TestRuleExample.kt)): + +```kotlin +class TestRuleExample { + @get:Rule + public val timeout = CoroutinesTimeout.seconds(1) + + private suspend fun someFunctionDeepInTheStack() { + withContext(Dispatchers.IO) { + delay(Long.MAX_VALUE) // Hang method + } + } + + @Test + fun hangingTest() = runBlocking { + val job = launch { + someFunctionDeepInTheStack() + } + job.join() // Join will hang + } +} +``` + +After 1 second, test will fail with `TestTimeoutException` and all coroutines (`runBlocking` and `launch`) and their +stacktraces will be dumped to the console. + +### Using as JVM agent + +Debug module can also be used as a standalone JVM agent to enable debug probes on the application startup. +You can run your application with an additional argument: `-javaagent:kotlinx-coroutines-debug-1.10.2.jar`. +Additionally, on Linux and Mac OS X you can use `kill -5 $pid` command in order to force your application to print all alive coroutines. +When used as Java agent, `"kotlinx.coroutines.debug.enable.creation.stack.trace"` system property can be used to control +[DebugProbes.enableCreationStackTraces] along with agent startup. + +### Using in production environment + +It is possible to run an application in production environments with debug probes in order to monitor its +state and improve its observability. +For that, it is strongly recommended not to enable [DebugProbes.enableCreationStackTraces], as enabling it makes +the performance overhead of the debug probes non-negligible. +With creation stack-traces disabled, the typical overhead of enabled debug probes is a single-digit percentage of the total +application throughput. + + +### Example of usage + +Capabilities of this module can be demonstrated by the following example +(runnable code is [here](test/Example.kt)): + +```kotlin +suspend fun computeValue(): String = coroutineScope { + val one = async { computeOne() } + val two = async { computeTwo() } + combineResults(one, two) +} + +suspend fun combineResults(one: Deferred, two: Deferred): String = + one.await() + two.await() + +suspend fun computeOne(): String { + delay(5000) + return "4" +} + +suspend fun computeTwo(): String { + delay(5000) + return "2" +} + +fun main() = runBlocking { + DebugProbes.install() + val deferred = async { computeValue() } + // Delay for some time + delay(1000) + // Dump running coroutines + DebugProbes.dumpCoroutines() + println("\nDumping only deferred") + DebugProbes.printJob(deferred) +} +``` + +Printed result will be: + +``` +Coroutines dump 2018/11/12 21:44:02 + +Coroutine "coroutine#2":DeferredCoroutine{Active}@289d1c02, state: SUSPENDED + at kotlinx.coroutines.DeferredCoroutine.await$suspendImpl(Builders.common.kt:99) + at ExampleKt.combineResults(Example.kt:11) + at ExampleKt$computeValue$2.invokeSuspend(Example.kt:7) + at ExampleKt$main$1$deferred$1.invokeSuspend(Example.kt:25) + +... More coroutines here ... + +Dumping only deferred +"coroutine#2":DeferredCoroutine{Active}, continuation is SUSPENDED at line kotlinx.coroutines.DeferredCoroutine.await$suspendImpl(Builders.common.kt:99) + "coroutine#3":DeferredCoroutine{Active}, continuation is SUSPENDED at line ExampleKt.computeOne(Example.kt:14) + "coroutine#4":DeferredCoroutine{Active}, continuation is SUSPENDED at line ExampleKt.computeTwo(Example.kt:19) +``` + +### Status of the API + +API is experimental, and it is not guaranteed it won't be changed (while it is marked as `@ExperimentalCoroutinesApi`). +Like the rest of experimental API, `DebugProbes` is carefully designed, tested and ready to use in both test and production +environments. It is marked as experimental to leave us the room to enrich the output data in a potentially backwards incompatible manner +to further improve diagnostics and debugging experience. + +The output format of [DebugProbes] can be changed in the future and it is not recommended to rely on the string representation +of the dump programmatically. + +### Debug agent and Android + +Android runtime does not support Instrument API necessary for `kotlinx-coroutines-debug` to function, triggering `java.lang.NoClassDefFoundError: Failed resolution of: Ljava/lang/management/ManagementFactory;`, +and it is not possible to use coroutine debugger along with Android emulator. + + + +#### Build failures due to duplicate resource files + +Building an Android project that depends on `kotlinx-coroutines-debug` (usually introduced by being a transitive +dependency of `kotlinx-coroutines-test`) may fail with `DuplicateRelativeFileException` for `META-INF/AL2.0`, +`META-INF/LGPL2.1`, or `win32-x86/attach_hotspot_windows.dll` when trying to merge the Android resource. + +The problem is that Android merges the resources of all its dependencies into a single directory and complains about +conflicts, but: +`kotlinx-coroutines-debug` transitively depends on JNA and JNA-platform, byte-buddy and byte-buddy-agent, all of them include license files in their +META-INF directories. Trying to merge these files leads to conflicts, which means that any Android project that +depends on JNA and JNA-platform will experience build failures. + +One possible workaround for these issues is to add the following to the `android` block in your gradle file for the +application subproject: +```groovy + packagingOptions { + // for JNA and JNA-platform + exclude "META-INF/AL2.0" + exclude "META-INF/LGPL2.1" + // for byte-buddy + exclude "META-INF/licenses/ASM" + pickFirst "win32-x86-64/attach_hotspot_windows.dll" + pickFirst "win32-x86/attach_hotspot_windows.dll" + } +``` +This will cause the resource merge algorithm to exclude the problematic license files altogether and only leave a single +copy of the files needed for `byte-buddy-agent` to work. + +Alternatively, avoid depending on `kotlinx-coroutines-debug`. In particular, if the only reason why this library a +dependency of your project is that `kotlinx-coroutines-test` in turn depends on it, you may change your dependency on +`kotlinx.coroutines.test` to exclude `kotlinx-coroutines-debug`. For example, you could replace +```kotlin +androidTestImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:$coroutines_version") +``` +with +```groovy +androidTestImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:$coroutines_version") { + exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-debug" +} +``` + + + + +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html + + + + +[DebugProbes]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug/-debug-probes/index.html +[DebugProbes.install]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug/-debug-probes/install.html +[DebugProbes.dumpCoroutines]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug/-debug-probes/dump-coroutines.html +[DebugProbes.dumpCoroutinesInfo]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug/-debug-probes/dump-coroutines-info.html +[DebugProbes.printJob]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug/-debug-probes/print-job.html +[DebugProbes.printScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug/-debug-probes/print-scope.html +[DebugProbes.enableCreationStackTraces]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug/-debug-probes/enable-creation-stack-traces.html + + + +[CoroutinesTimeout]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug.junit4/-coroutines-timeout/index.html + + diff --git a/kotlinx-coroutines-debug/api/kotlinx-coroutines-debug.api b/kotlinx-coroutines-debug/api/kotlinx-coroutines-debug.api new file mode 100644 index 0000000000..11131fad42 --- /dev/null +++ b/kotlinx-coroutines-debug/api/kotlinx-coroutines-debug.api @@ -0,0 +1,71 @@ +public final class kotlinx/coroutines/debug/CoroutineInfo { + public final fun getContext ()Lkotlin/coroutines/CoroutineContext; + public final fun getCreationStackTrace ()Ljava/util/List; + public final fun getJob ()Lkotlinx/coroutines/Job; + public final fun getState ()Lkotlinx/coroutines/debug/State; + public final fun lastObservedStackTrace ()Ljava/util/List; + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/debug/CoroutinesBlockHoundIntegration : reactor/blockhound/integration/BlockHoundIntegration { + public fun ()V + public fun applyTo (Lreactor/blockhound/BlockHound$Builder;)V +} + +public final class kotlinx/coroutines/debug/DebugProbes { + public static final field INSTANCE Lkotlinx/coroutines/debug/DebugProbes; + public final fun dumpCoroutines (Ljava/io/PrintStream;)V + public static synthetic fun dumpCoroutines$default (Lkotlinx/coroutines/debug/DebugProbes;Ljava/io/PrintStream;ILjava/lang/Object;)V + public final fun dumpCoroutinesInfo ()Ljava/util/List; + public final fun getEnableCreationStackTraces ()Z + public final fun getIgnoreCoroutinesWithEmptyContext ()Z + public final fun getSanitizeStackTraces ()Z + public final fun install ()V + public final fun isInstalled ()Z + public final fun jobToString (Lkotlinx/coroutines/Job;)Ljava/lang/String; + public final fun printJob (Lkotlinx/coroutines/Job;Ljava/io/PrintStream;)V + public static synthetic fun printJob$default (Lkotlinx/coroutines/debug/DebugProbes;Lkotlinx/coroutines/Job;Ljava/io/PrintStream;ILjava/lang/Object;)V + public final fun printScope (Lkotlinx/coroutines/CoroutineScope;Ljava/io/PrintStream;)V + public static synthetic fun printScope$default (Lkotlinx/coroutines/debug/DebugProbes;Lkotlinx/coroutines/CoroutineScope;Ljava/io/PrintStream;ILjava/lang/Object;)V + public final fun scopeToString (Lkotlinx/coroutines/CoroutineScope;)Ljava/lang/String; + public final fun setEnableCreationStackTraces (Z)V + public final fun setIgnoreCoroutinesWithEmptyContext (Z)V + public final fun setSanitizeStackTraces (Z)V + public final fun uninstall ()V + public final fun withDebugProbes (Lkotlin/jvm/functions/Function0;)V +} + +public final class kotlinx/coroutines/debug/State : java/lang/Enum { + public static final field CREATED Lkotlinx/coroutines/debug/State; + public static final field RUNNING Lkotlinx/coroutines/debug/State; + public static final field SUSPENDED Lkotlinx/coroutines/debug/State; + public static fun getEntries ()Lkotlin/enums/EnumEntries; + public static fun valueOf (Ljava/lang/String;)Lkotlinx/coroutines/debug/State; + public static fun values ()[Lkotlinx/coroutines/debug/State; +} + +public final class kotlinx/coroutines/debug/junit4/CoroutinesTimeout : org/junit/rules/TestRule { + public static final field Companion Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout$Companion; + public fun (JZ)V + public synthetic fun (JZILkotlin/jvm/internal/DefaultConstructorMarker;)V + public fun (JZZ)V + public synthetic fun (JZZILkotlin/jvm/internal/DefaultConstructorMarker;)V + public fun apply (Lorg/junit/runners/model/Statement;Lorg/junit/runner/Description;)Lorg/junit/runners/model/Statement; +} + +public final class kotlinx/coroutines/debug/junit4/CoroutinesTimeout$Companion { + public final fun seconds (I)Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout; + public final fun seconds (IZ)Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout; + public final fun seconds (IZZ)Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout; + public final fun seconds (J)Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout; + public final fun seconds (JZ)Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout; + public final fun seconds (JZZ)Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout; + public static synthetic fun seconds$default (Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout$Companion;IZZILjava/lang/Object;)Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout; + public static synthetic fun seconds$default (Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout$Companion;JZZILjava/lang/Object;)Lkotlinx/coroutines/debug/junit4/CoroutinesTimeout; +} + +public abstract interface annotation class kotlinx/coroutines/debug/junit5/CoroutinesTimeout : java/lang/annotation/Annotation { + public abstract fun cancelOnTimeout ()Z + public abstract fun testTimeoutMs ()J +} + diff --git a/kotlinx-coroutines-debug/build.gradle.kts b/kotlinx-coroutines-debug/build.gradle.kts new file mode 100644 index 0000000000..c6276073aa --- /dev/null +++ b/kotlinx-coroutines-debug/build.gradle.kts @@ -0,0 +1,63 @@ +import org.gradle.api.JavaVersion +import org.gradle.api.tasks.bundling.Jar +import org.gradle.api.tasks.testing.Test + +plugins { + id("org.jetbrains.kotlinx.kover") // apply plugin to use autocomplete for Kover DSL +} + +val junit_version by properties +val junit5_version by properties +val byte_buddy_version by properties +val blockhound_version by properties +val jna_version by properties + +dependencies { + compileOnly("junit:junit:$junit_version") + compileOnly("org.junit.jupiter:junit-jupiter-api:$junit5_version") + testImplementation("org.junit.jupiter:junit-jupiter-engine:$junit5_version") + testImplementation("org.junit.platform:junit-platform-testkit:1.7.0") + implementation("net.bytebuddy:byte-buddy:$byte_buddy_version") + implementation("net.bytebuddy:byte-buddy-agent:$byte_buddy_version") + compileOnly("io.projectreactor.tools:blockhound:$blockhound_version") + testImplementation("io.projectreactor.tools:blockhound:$blockhound_version") + testImplementation("com.google.code.gson:gson:2.8.6") + api("net.java.dev.jna:jna:$jna_version") + api("net.java.dev.jna:jna-platform:$jna_version") +} + +java { + /* This is needed to be able to run JUnit5 tests. Otherwise, Gradle complains that it can't find the + JVM1.6-compatible version of the `junit-jupiter-api` artifact. */ + disableAutoTargetJvm() +} + +// This is required for BlockHound tests to work, see https://github.com/Kotlin/kotlinx.coroutines/issues/3701 +tasks.withType().configureEach { + if (JavaVersion.toVersion(jdkToolchainVersion).isCompatibleWith(JavaVersion.VERSION_13)) { + jvmArgs("-XX:+AllowRedefinitionToAddDeleteMethods") + } +} + +tasks.named("jar") { + manifest { + attributes( + mapOf( + "Premain-Class" to "kotlinx.coroutines.debug.internal.AgentPremain", + "Can-Redefine-Classes" to "true", + "Multi-Release" to "true" + ) + ) + } +} + +kover { + reports { + filters { + excludes { + // Never used, safety mechanism + classes("kotlinx.coroutines.debug.NoOpProbesKt") + } + } + } +} diff --git a/kotlinx-coroutines-debug/resources/META-INF/services/reactor.blockhound.integration.BlockHoundIntegration b/kotlinx-coroutines-debug/resources/META-INF/services/reactor.blockhound.integration.BlockHoundIntegration new file mode 100644 index 0000000000..c2f1e9cf38 --- /dev/null +++ b/kotlinx-coroutines-debug/resources/META-INF/services/reactor.blockhound.integration.BlockHoundIntegration @@ -0,0 +1 @@ +kotlinx.coroutines.debug.CoroutinesBlockHoundIntegration \ No newline at end of file diff --git a/kotlinx-coroutines-debug/src/Attach.kt b/kotlinx-coroutines-debug/src/Attach.kt new file mode 100644 index 0000000000..291d913f3e --- /dev/null +++ b/kotlinx-coroutines-debug/src/Attach.kt @@ -0,0 +1,38 @@ +@file:Suppress("unused") +package kotlinx.coroutines.debug + +import net.bytebuddy.* +import net.bytebuddy.agent.* +import net.bytebuddy.dynamic.loading.* + +/* + * This class is used reflectively from kotlinx-coroutines-core when this module is present in the classpath. + * It is a substitute for service loading. + */ +internal class ByteBuddyDynamicAttach : Function1 { + override fun invoke(value: Boolean) { + if (value) attach() else detach() + } + + private fun attach() { + ByteBuddyAgent.install(ByteBuddyAgent.AttachmentProvider.ForEmulatedAttachment.INSTANCE) + val cl = Class.forName("kotlin.coroutines.jvm.internal.DebugProbesKt") + val cl2 = Class.forName("kotlinx.coroutines.debug.internal.DebugProbesKt") + + ByteBuddy() + .redefine(cl2) + .name(cl.name) + .make() + .load(cl.classLoader, ClassReloadingStrategy.fromInstalledAgent()) + } + + private fun detach() { + val cl = Class.forName("kotlin.coroutines.jvm.internal.DebugProbesKt") + val cl2 = Class.forName("kotlinx.coroutines.debug.NoOpProbesKt") + ByteBuddy() + .redefine(cl2) + .name(cl.name) + .make() + .load(cl.classLoader, ClassReloadingStrategy.fromInstalledAgent()) + } +} diff --git a/kotlinx-coroutines-debug/src/CoroutineInfo.kt b/kotlinx-coroutines-debug/src/CoroutineInfo.kt new file mode 100644 index 0000000000..cb89405a30 --- /dev/null +++ b/kotlinx-coroutines-debug/src/CoroutineInfo.kt @@ -0,0 +1,89 @@ +@file:Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE", "UNUSED") +package kotlinx.coroutines.debug + +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* +import kotlin.coroutines.* +import kotlin.coroutines.jvm.internal.* + +/** + * Class describing coroutine info such as its context, state and stacktrace. + */ +@ExperimentalCoroutinesApi +public class CoroutineInfo internal constructor(delegate: DebugCoroutineInfo) { + /** + * [Coroutine context][coroutineContext] of the coroutine + */ + public val context: CoroutineContext = delegate.context + + /** + * Last observed state of the coroutine + */ + public val state: State = State.valueOf(delegate.state) + + private val creationStackBottom: CoroutineStackFrame? = delegate.creationStackBottom + + /** + * [Job] associated with a current coroutine or null. + * May be later used in [DebugProbes.printJob]. + */ + public val job: Job? get() = context[Job] + + /** + * Creation stacktrace of the coroutine. + * Can be empty if [DebugProbes.enableCreationStackTraces] is not set. + */ + public val creationStackTrace: List get() = creationStackTrace() + + private val lastObservedFrame: CoroutineStackFrame? = delegate.lastObservedFrame + + /** + * Last observed stacktrace of the coroutine captured on its suspension or resumption point. + * It means that for [running][State.RUNNING] coroutines resulting stacktrace is inaccurate and + * reflects stacktrace of the resumption point, not the actual current stacktrace. + */ + public fun lastObservedStackTrace(): List { + var frame: CoroutineStackFrame? = lastObservedFrame ?: return emptyList() + val result = ArrayList() + while (frame != null) { + frame.getStackTraceElement()?.let { result.add(it) } + frame = frame.callerFrame + } + return result + } + + private fun creationStackTrace(): List { + val bottom = creationStackBottom ?: return emptyList() + // Skip "Coroutine creation stacktrace" frame + return sequence { yieldFrames(bottom.callerFrame) }.toList() + } + + private tailrec suspend fun SequenceScope.yieldFrames(frame: CoroutineStackFrame?) { + if (frame == null) return + frame.getStackTraceElement()?.let { yield(it) } + val caller = frame.callerFrame + if (caller != null) { + yieldFrames(caller) + } + } + + override fun toString(): String = "CoroutineInfo(state=$state,context=$context)" +} + +/** + * Current state of the coroutine. + */ +public enum class State { + /** + * Created, but not yet started. + */ + CREATED, + /** + * Started and running. + */ + RUNNING, + /** + * Suspended. + */ + SUSPENDED +} diff --git a/kotlinx-coroutines-debug/src/CoroutinesBlockHoundIntegration.kt b/kotlinx-coroutines-debug/src/CoroutinesBlockHoundIntegration.kt new file mode 100644 index 0000000000..f8213eae22 --- /dev/null +++ b/kotlinx-coroutines-debug/src/CoroutinesBlockHoundIntegration.kt @@ -0,0 +1,186 @@ +@file:Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + +package kotlinx.coroutines.debug + +import kotlinx.coroutines.scheduling.* +import reactor.blockhound.* +import reactor.blockhound.integration.* + +/** + * @suppress + */ +public class CoroutinesBlockHoundIntegration : BlockHoundIntegration { + + override fun applyTo(builder: BlockHound.Builder): Unit = with(builder) { + allowBlockingCallsInPrimitiveImplementations() + allowBlockingWhenEnqueuingTasks() + allowServiceLoaderInvocationsOnInit() + allowBlockingCallsInReflectionImpl() + allowBlockingCallsInDebugProbes() + allowBlockingCallsInWorkQueue() + // Stacktrace recovery cache is guarded by lock + allowBlockingCallsInside("kotlinx.coroutines.internal.ExceptionsConstructorKt", "tryCopyException") + /* The predicates that define that BlockHound should only report blocking calls from threads that are part of + the coroutine thread pool and currently execute a CPU-bound coroutine computation. */ + addDynamicThreadPredicate { isSchedulerWorker(it) } + nonBlockingThreadPredicate { p -> p.or { mayNotBlock(it) } } + } + + /** + * Allows blocking calls in various coroutine structures, such as flows and channels. + * + * They use locks in implementations, though only for protecting short pieces of fast and well-understood code, so + * locking in such places doesn't affect the program liveness. + */ + private fun BlockHound.Builder.allowBlockingCallsInPrimitiveImplementations() { + allowBlockingCallsInJobSupport() + allowBlockingCallsInThreadSafeHeap() + allowBlockingCallsInFlow() + allowBlockingCallsInChannels() + } + + /** + * Allows blocking inside [kotlinx.coroutines.JobSupport]. + */ + private fun BlockHound.Builder.allowBlockingCallsInJobSupport() { + for (method in listOf("finalizeFinishingState", "invokeOnCompletion", "makeCancelling", + "tryMakeCompleting")) + { + allowBlockingCallsInside("kotlinx.coroutines.JobSupport", method) + } + } + + /** + * Allow blocking calls inside [kotlinx.coroutines.debug.internal.DebugProbesImpl]. + */ + private fun BlockHound.Builder.allowBlockingCallsInDebugProbes() { + for (method in listOf("install", "uninstall", "hierarchyToString", "dumpCoroutinesInfo", "dumpDebuggerInfo", + "dumpCoroutinesSynchronized", "updateRunningState", "updateState")) + { + allowBlockingCallsInside("kotlinx.coroutines.debug.internal.DebugProbesImpl", method) + } + } + + /** + * Allow blocking calls inside [kotlinx.coroutines.scheduling.WorkQueue] + */ + private fun BlockHound.Builder.allowBlockingCallsInWorkQueue() { + /** uses [Thread.yield] in a benign way. */ + allowBlockingCallsInside("kotlinx.coroutines.scheduling.WorkQueue", "addLast") + } + + /** + * Allows blocking inside [kotlinx.coroutines.internal.ThreadSafeHeap]. + */ + private fun BlockHound.Builder.allowBlockingCallsInThreadSafeHeap() { + for (method in listOf("clear", "peek", "removeFirstOrNull", "addLast")) { + allowBlockingCallsInside("kotlinx.coroutines.internal.ThreadSafeHeap", method) + } + } + + private fun BlockHound.Builder.allowBlockingCallsInFlow() { + allowBlockingCallsInsideStateFlow() + allowBlockingCallsInsideSharedFlow() + } + + /** + * Allows blocking inside the implementation of [kotlinx.coroutines.flow.StateFlow]. + */ + private fun BlockHound.Builder.allowBlockingCallsInsideStateFlow() { + allowBlockingCallsInside("kotlinx.coroutines.flow.StateFlowImpl", "updateState") + } + + /** + * Allows blocking inside the implementation of [kotlinx.coroutines.flow.SharedFlow]. + */ + private fun BlockHound.Builder.allowBlockingCallsInsideSharedFlow() { + for (method in listOf("emitSuspend", "awaitValue", "getReplayCache", "tryEmit", "cancelEmitter", + "tryTakeValue", "resetReplayCache")) + { + allowBlockingCallsInside("kotlinx.coroutines.flow.SharedFlowImpl", method) + } + for (method in listOf("getSubscriptionCount", "allocateSlot", "freeSlot")) { + allowBlockingCallsInside("kotlinx.coroutines.flow.internal.AbstractSharedFlow", method) + } + } + + private fun BlockHound.Builder.allowBlockingCallsInChannels() { + allowBlockingCallsInBroadcastChannels() + allowBlockingCallsInConflatedChannels() + } + + /** + * Allows blocking inside [kotlinx.coroutines.channels.BroadcastChannel]. + */ + private fun BlockHound.Builder.allowBlockingCallsInBroadcastChannels() { + for (method in listOf("openSubscription", "removeSubscriber", "send", "trySend", "registerSelectForSend", + "close", "cancelImpl", "isClosedForSend", "value", "valueOrNull")) + { + allowBlockingCallsInside("kotlinx.coroutines.channels.BroadcastChannelImpl", method) + } + for (method in listOf("cancelImpl")) { + allowBlockingCallsInside("kotlinx.coroutines.channels.BroadcastChannelImpl\$SubscriberConflated", method) + } + for (method in listOf("cancelImpl")) { + allowBlockingCallsInside("kotlinx.coroutines.channels.BroadcastChannelImpl\$SubscriberBuffered", method) + } + } + + /** + * Allows blocking inside [kotlinx.coroutines.channels.ConflatedBufferedChannel]. + */ + private fun BlockHound.Builder.allowBlockingCallsInConflatedChannels() { + for (method in listOf("receive", "receiveCatching", "tryReceive", "registerSelectForReceive", + "send", "trySend", "sendBroadcast", "registerSelectForSend", + "close", "cancelImpl", "isClosedForSend", "isClosedForReceive", "isEmpty")) + { + allowBlockingCallsInside("kotlinx.coroutines.channels.ConflatedBufferedChannel", method) + } + for (method in listOf("hasNext")) { + allowBlockingCallsInside("kotlinx.coroutines.channels.ConflatedBufferedChannel\$ConflatedChannelIterator", method) + } + } + + /** + * Allows blocking when enqueuing tasks into a thread pool. + * + * Without this, the following code breaks: + * ``` + * withContext(Dispatchers.Default) { + * withContext(newSingleThreadContext("singleThreadedContext")) { + * } + * } + * ``` + */ + private fun BlockHound.Builder.allowBlockingWhenEnqueuingTasks() { + /* This method may block as part of its implementation, but is probably safe. */ + allowBlockingCallsInside("java.util.concurrent.ScheduledThreadPoolExecutor", "execute") + } + + /** + * Allows instances of [java.util.ServiceLoader] being called. + * + * Each instance is listed separately; another approach could be to generally allow the operations performed by + * service loaders, as they can generally be considered safe. This was not done here because ServiceLoader has a + * large API surface, with some methods being hidden as implementation details (in particular, the implementation of + * its iterator is completely opaque). Relying on particular names being used in ServiceLoader's implementation + * would be brittle, so here we only provide clearance rules for some specific instances. + */ + private fun BlockHound.Builder.allowServiceLoaderInvocationsOnInit() { + allowBlockingCallsInside("kotlinx.coroutines.reactive.ReactiveFlowKt", "") + allowBlockingCallsInside("kotlinx.coroutines.CoroutineExceptionHandlerImplKt", "") + // not part of the coroutines library, but it would be nice if reflection also wasn't considered blocking + allowBlockingCallsInside("kotlin.reflect.jvm.internal.impl.resolve.OverridingUtil", "") + } + + /** + * Allows some blocking calls from the reflection API. + * + * The API is big, so surely some other blocking calls will show up, but with these rules in place, at least some + * simple examples work without problems. + */ + private fun BlockHound.Builder.allowBlockingCallsInReflectionImpl() { + allowBlockingCallsInside("kotlin.reflect.jvm.internal.impl.builtins.jvm.JvmBuiltInsPackageFragmentProvider", "findPackage") + } + +} diff --git a/kotlinx-coroutines-debug/src/DebugProbes.kt b/kotlinx-coroutines-debug/src/DebugProbes.kt new file mode 100644 index 0000000000..104212b728 --- /dev/null +++ b/kotlinx-coroutines-debug/src/DebugProbes.kt @@ -0,0 +1,182 @@ +@file:Suppress("UNUSED", "INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") + +package kotlinx.coroutines.debug + +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* +import java.io.* +import java.lang.management.* +import kotlin.coroutines.* + +/** + * Kotlin debug probes support. + * + * Debug probes is a dynamic attach mechanism which installs multiple hooks into coroutines machinery. + * It slows down all coroutine-related code, but in return provides diagnostic information, including + * asynchronous stacktraces, coroutine dumps (similar to [ThreadMXBean.dumpAllThreads] and `jstack`) via [DebugProbes.dumpCoroutines], + * and programmatic introspection of all alive coroutines. + * All introspecting methods throw [IllegalStateException] if debug probes were not installed. + * + * ### Consistency guarantees + * + * All snapshotting operations (e.g. [dumpCoroutines]) are *weakly-consistent*, meaning that they happen + * concurrently with coroutines progressing their own state. These operations are guaranteed to observe + * each coroutine's state exactly once, but the state is not guaranteed to be the most recent before the operation. + * In practice, it means that for snapshotting operations in progress, for each concurrent coroutine either + * the state prior to the operation or the state that was reached during the current operation is observed. + * + * ### Overhead + * + * - Every created coroutine is stored in a concurrent hash map, and the hash map is looked up in and + * updated on each suspension and resumption. + * - If [DebugProbes.enableCreationStackTraces] is enabled, stack trace of the current thread is captured on + * each created coroutine that is a rough equivalent of throwing an exception per each created coroutine. + * + * ### Internal machinery and classloading. + * + * Under the hood, debug probes replace internal `kotlin.coroutines.jvm.internal.DebugProbesKt` class that has the following + * empty static methods: + * + * - `probeCoroutineResumed` that is invoked on every [Continuation.resume]. + * - `probeCoroutineSuspended` that is invoked on every continuation suspension. + * - `probeCoroutineCreated` that is invoked on every coroutine creation. + * + * with a `kotlinx-coroutines`-specific class to keep track of all the coroutines machinery. + * + * The new class is located in the `kotlinx-coroutines-core` module, meaning that all target application classes that use + * coroutines and `suspend` functions have to be loaded by the classloader in which `kotlinx-coroutines-core` classes are available. + */ +@ExperimentalCoroutinesApi +public object DebugProbes { + + /** + * Whether coroutine creation stack traces should be sanitized. + * Sanitization removes all frames from `kotlinx.coroutines` package except + * the first one and the last one to simplify diagnostic. + * + * `true` by default. + */ + public var sanitizeStackTraces: Boolean + get() = DebugProbesImpl.sanitizeStackTraces + @Suppress("INVISIBLE_SETTER") // do not remove the INVISIBLE_SETTER suppression: required in k2 + set(value) { + DebugProbesImpl.sanitizeStackTraces = value + } + + /** + * Whether coroutine creation stack traces should be captured. + * When enabled, for each created coroutine a stack trace of the current thread is captured and attached to the coroutine. + * This option can be useful during local debug sessions, but is recommended + * to be disabled in production environments to avoid performance overhead of capturing real stacktraces. + * + * `false` by default. + */ + public var enableCreationStackTraces: Boolean + get() = DebugProbesImpl.enableCreationStackTraces + @Suppress("INVISIBLE_SETTER") // do not remove the INVISIBLE_SETTER suppression: required in k2 + set(value) { + DebugProbesImpl.enableCreationStackTraces = value + } + + /** + * Whether to ignore coroutines whose context is [EmptyCoroutineContext]. + * + * Coroutines with empty context are considered to be irrelevant for the concurrent coroutines' observability: + * - They do not contribute to any concurrent executions + * - They do not contribute to the (concurrent) system's liveness and/or deadlocks, as no other coroutines might wait for them + * - The typical usage of such coroutines is a combinator/builder/lookahead parser that can be debugged using more convenient tools. + * + * `true` by default. + */ + public var ignoreCoroutinesWithEmptyContext: Boolean + get() = DebugProbesImpl.ignoreCoroutinesWithEmptyContext + @Suppress("INVISIBLE_SETTER") // do not remove the INVISIBLE_SETTER suppression: required in k2 + set(value) { + DebugProbesImpl.ignoreCoroutinesWithEmptyContext = value + } + + /** + * Determines whether debug probes were [installed][DebugProbes.install]. + */ + public val isInstalled: Boolean get() = DebugProbesImpl.isInstalled + + /** + * Installs a [DebugProbes] instead of no-op stdlib probes by redefining + * debug probes class using the same class loader as one loaded [DebugProbes] class. + */ + public fun install() { + DebugProbesImpl.install() + } + + /** + * Uninstall debug probes. + */ + public fun uninstall() { + DebugProbesImpl.uninstall() + } + + /** + * Invokes given block of code with installed debug probes and uninstall probes in the end. + */ + public inline fun withDebugProbes(block: () -> Unit) { + install() + try { + block() + } finally { + uninstall() + } + } + + /** + * Returns string representation of the coroutines [job] hierarchy with additional debug information. + * Hierarchy is printed from the [job] as a root transitively to all children. + */ + public fun jobToString(job: Job): String = DebugProbesImpl.hierarchyToString(job) + + /** + * Returns string representation of all coroutines launched within the given [scope]. + * Throws [IllegalStateException] if the scope has no a job in it. + */ + public fun scopeToString(scope: CoroutineScope): String = + jobToString(scope.coroutineContext[Job] ?: error("Job is not present in the scope")) + + /** + * Prints [job] hierarchy representation from [jobToString] to the given [out]. + */ + public fun printJob(job: Job, out: PrintStream = System.out): Unit = + out.println(DebugProbesImpl.hierarchyToString(job)) + + /** + * Prints all coroutines launched within the given [scope]. + * Throws [IllegalStateException] if the scope has no a job in it. + */ + public fun printScope(scope: CoroutineScope, out: PrintStream = System.out): Unit = + printJob(scope.coroutineContext[Job] ?: error("Job is not present in the scope"), out) + + /** + * Returns all existing coroutines' info. + * The resulting collection represents a consistent snapshot of all existing coroutines at the moment of invocation. + */ + public fun dumpCoroutinesInfo(): List = + DebugProbesImpl.dumpCoroutinesInfo().map { CoroutineInfo(it) } + + /** + * Dumps all active coroutines into the given output stream, providing a consistent snapshot of all existing coroutines at the moment of invocation. + * The output of this method is similar to `jstack` or a full thread dump. It can be used as the replacement to + * "Dump threads" action. + * + * Example of the output: + * ``` + * Coroutines dump 2018/11/12 19:45:14 + * + * Coroutine "coroutine#42":StandaloneCoroutine{Active}@58fdd99, state: SUSPENDED + * at MyClass$awaitData.invokeSuspend(MyClass.kt:37) + * at _COROUTINE._CREATION._(CoroutineDebugging.kt) + * at MyClass.createIoRequest(MyClass.kt:142) + * at MyClass.fetchData(MyClass.kt:154) + * at MyClass.showData(MyClass.kt:31) + * ... + * ``` + */ + public fun dumpCoroutines(out: PrintStream = System.out): Unit = DebugProbesImpl.dumpCoroutines(out) +} diff --git a/kotlinx-coroutines-debug/src/NoOpProbes.kt b/kotlinx-coroutines-debug/src/NoOpProbes.kt new file mode 100644 index 0000000000..927936eaa9 --- /dev/null +++ b/kotlinx-coroutines-debug/src/NoOpProbes.kt @@ -0,0 +1,15 @@ +@file:Suppress("unused", "UNUSED_PARAMETER") + +package kotlinx.coroutines.debug + +import kotlin.coroutines.* + +/* + * Empty class used to replace installed agent in the end of debug session + */ +@JvmName("probeCoroutineResumed") +internal fun probeCoroutineResumedNoOp(frame: Continuation<*>) = Unit +@JvmName("probeCoroutineSuspended") +internal fun probeCoroutineSuspendedNoOp(frame: Continuation<*>) = Unit +@JvmName("probeCoroutineCreated") +internal fun probeCoroutineCreatedNoOp(completion: kotlin.coroutines.Continuation): kotlin.coroutines.Continuation = completion diff --git a/kotlinx-coroutines-debug/src/junit/CoroutinesTimeoutImpl.kt b/kotlinx-coroutines-debug/src/junit/CoroutinesTimeoutImpl.kt new file mode 100644 index 0000000000..ba5804d5bb --- /dev/null +++ b/kotlinx-coroutines-debug/src/junit/CoroutinesTimeoutImpl.kt @@ -0,0 +1,77 @@ +package kotlinx.coroutines.debug + +import java.util.concurrent.* + +/** + * Run [invocation] in a separate thread with the given timeout in ms, after which the coroutines info is dumped and, if + * [cancelOnTimeout] is set, the execution is interrupted. + * + * Assumes that [DebugProbes] are installed. Does not deinstall them. + */ +internal inline fun runWithTimeoutDumpingCoroutines( + methodName: String, + testTimeoutMs: Long, + cancelOnTimeout: Boolean, + initCancellationException: () -> Throwable, + crossinline invocation: () -> T +): T { + val testStartedLatch = CountDownLatch(1) + val testResult = FutureTask { + testStartedLatch.countDown() + invocation() + } + /* + * We are using hand-rolled thread instead of single thread executor + * in order to be able to safely interrupt thread in the end of a test + */ + val testThread = Thread(testResult, "Timeout test thread").apply { isDaemon = true } + try { + testThread.start() + // Await until test is started to take only test execution time into account + testStartedLatch.await() + return testResult.get(testTimeoutMs, TimeUnit.MILLISECONDS) + } catch (e: TimeoutException) { + handleTimeout(testThread, methodName, testTimeoutMs, cancelOnTimeout, initCancellationException()) + } catch (e: ExecutionException) { + throw e.cause ?: e + } +} + +private fun handleTimeout(testThread: Thread, methodName: String, testTimeoutMs: Long, cancelOnTimeout: Boolean, + cancellationException: Throwable): Nothing { + val units = + if (testTimeoutMs % 1000 == 0L) + "${testTimeoutMs / 1000} seconds" + else "$testTimeoutMs milliseconds" + + System.err.println("\nTest $methodName timed out after $units\n") + System.err.flush() + + DebugProbes.dumpCoroutines() + System.out.flush() // Synchronize serr/sout + + /* + * Order is important: + * 1) Create exception with a stacktrace of hang test + * 2) Cancel all coroutines via debug agent API (changing system state!) + * 3) Throw created exception + */ + cancellationException.attachStacktraceFrom(testThread) + testThread.interrupt() + cancelIfNecessary(cancelOnTimeout) + // If timed out test throws an exception, we can't do much except ignoring it + throw cancellationException +} + +private fun cancelIfNecessary(cancelOnTimeout: Boolean) { + if (cancelOnTimeout) { + DebugProbes.dumpCoroutinesInfo().forEach { + it.job?.cancel() + } + } +} + +private fun Throwable.attachStacktraceFrom(thread: Thread) { + val stackTrace = thread.stackTrace + this.stackTrace = stackTrace +} diff --git a/kotlinx-coroutines-debug/src/junit/junit4/CoroutinesTimeout.kt b/kotlinx-coroutines-debug/src/junit/junit4/CoroutinesTimeout.kt new file mode 100644 index 0000000000..f9f2a74d48 --- /dev/null +++ b/kotlinx-coroutines-debug/src/junit/junit4/CoroutinesTimeout.kt @@ -0,0 +1,91 @@ +package kotlinx.coroutines.debug.junit4 + +import kotlinx.coroutines.debug.* +import org.junit.rules.* +import org.junit.runner.* +import org.junit.runners.model.* +import java.util.concurrent.* + +/** + * Coroutines timeout rule for JUnit4 that is applied to all methods in the class. + * This rule is very similar to [Timeout] rule: it runs tests in a separate thread, + * fails tests after the given timeout and interrupts test thread. + * + * Additionally, this rule installs [DebugProbes] and dumps all coroutines at the moment of the timeout. + * It may cancel coroutines on timeout if [cancelOnTimeout] set to `true`. + * [enableCoroutineCreationStackTraces] controls the corresponding [DebugProbes.enableCreationStackTraces] property + * and can be optionally enabled if the creation stack traces are necessary. + * + * Example of usage: + * ``` + * class HangingTest { + * @get:Rule + * val timeout = CoroutinesTimeout.seconds(5) + * + * @Test + * fun testThatHangs() = runBlocking { + * ... + * delay(Long.MAX_VALUE) // somewhere deep in the stack + * ... + * } + * } + * ``` + */ +public class CoroutinesTimeout( + private val testTimeoutMs: Long, + private val cancelOnTimeout: Boolean = false, + private val enableCoroutineCreationStackTraces: Boolean = false +) : TestRule { + + @Suppress("UNUSED") // Binary compatibility + public constructor(testTimeoutMs: Long, cancelOnTimeout: Boolean = false) : this( + testTimeoutMs, + cancelOnTimeout, + true + ) + + init { + require(testTimeoutMs > 0) { "Expected positive test timeout, but had $testTimeoutMs" } + /* + * Install probes in the constructor, so all the coroutines launched from within + * target test constructor will be captured + */ + // Do not preserve previous state for unit-test environment + DebugProbes.enableCreationStackTraces = enableCoroutineCreationStackTraces + DebugProbes.install() + } + + public companion object { + /** + * Creates [CoroutinesTimeout] rule with the given timeout in seconds. + */ + @JvmOverloads + public fun seconds( + seconds: Int, + cancelOnTimeout: Boolean = false, + enableCoroutineCreationStackTraces: Boolean = true + ): CoroutinesTimeout = + seconds(seconds.toLong(), cancelOnTimeout, enableCoroutineCreationStackTraces) + + /** + * Creates [CoroutinesTimeout] rule with the given timeout in seconds. + */ + @JvmOverloads + public fun seconds( + seconds: Long, + cancelOnTimeout: Boolean = false, + enableCoroutineCreationStackTraces: Boolean = true + ): CoroutinesTimeout = + CoroutinesTimeout( + TimeUnit.SECONDS.toMillis(seconds), // Overflow is properly handled by TimeUnit + cancelOnTimeout, + enableCoroutineCreationStackTraces + ) + } + + /** + * @suppress suppress from Dokka + */ + override fun apply(base: Statement, description: Description): Statement = + CoroutinesTimeoutStatement(base, description, testTimeoutMs, cancelOnTimeout) +} diff --git a/kotlinx-coroutines-debug/src/junit/junit4/CoroutinesTimeoutStatement.kt b/kotlinx-coroutines-debug/src/junit/junit4/CoroutinesTimeoutStatement.kt new file mode 100644 index 0000000000..a51ea6aeee --- /dev/null +++ b/kotlinx-coroutines-debug/src/junit/junit4/CoroutinesTimeoutStatement.kt @@ -0,0 +1,26 @@ +package kotlinx.coroutines.debug.junit4 + +import kotlinx.coroutines.debug.* +import org.junit.runner.* +import org.junit.runners.model.* +import java.util.concurrent.* + +internal class CoroutinesTimeoutStatement( + private val testStatement: Statement, + private val testDescription: Description, + private val testTimeoutMs: Long, + private val cancelOnTimeout: Boolean = false +) : Statement() { + + override fun evaluate() { + try { + runWithTimeoutDumpingCoroutines(testDescription.methodName, testTimeoutMs, cancelOnTimeout, + { TestTimedOutException(testTimeoutMs, TimeUnit.MILLISECONDS) }) + { + testStatement.evaluate() + } + } finally { + DebugProbes.uninstall() + } + } +} diff --git a/kotlinx-coroutines-debug/src/junit/junit5/CoroutinesTimeout.kt b/kotlinx-coroutines-debug/src/junit/junit5/CoroutinesTimeout.kt new file mode 100644 index 0000000000..c2ac0599a5 --- /dev/null +++ b/kotlinx-coroutines-debug/src/junit/junit5/CoroutinesTimeout.kt @@ -0,0 +1,59 @@ +package kotlinx.coroutines.debug.junit5 +import kotlinx.coroutines.debug.* +import org.junit.jupiter.api.* +import org.junit.jupiter.api.extension.* +import org.junit.jupiter.api.parallel.* +import java.lang.annotation.* + +/** + * Coroutines timeout annotation that is similar to JUnit5's [Timeout] annotation. It allows running test methods in a + * separate thread, failing them after the provided time limit and interrupting the thread. + * + * Additionally, it installs [DebugProbes] and dumps all coroutines at the moment of the timeout. It also cancels + * coroutines on timeout if [cancelOnTimeout] set to `true`. The dump contains the coroutine creation stack traces. + * + * This annotation has an effect on test, test factory, test template, and lifecycle methods and test classes that are + * annotated with it. + * + * Annotating a class is the same as annotating every test, test factory, and test template method (but not lifecycle + * methods) of that class and its inner test classes, unless any of them is annotated with [CoroutinesTimeout], in which + * case their annotation overrides the one on the containing class. + * + * Declaring [CoroutinesTimeout] on a test factory checks that it finishes in the specified time, but does not check + * whether the methods that it produces obey the timeout as well. + * + * Example usage: + * ``` + * @CoroutinesTimeout(100) + * class CoroutinesTimeoutSimpleTest { + * // does not time out, as the annotation on the method overrides the class-level one + * @CoroutinesTimeout(1000) + * @Test + * fun classTimeoutIsOverridden() { + * runBlocking { + * delay(150) + * } + * } + * + * // times out in 100 ms, timeout value is taken from the class-level annotation + * @Test + * fun classTimeoutIsUsed() { + * runBlocking { + * delay(150) + * } + * } + * } + * ``` + * + * @see Timeout + */ +@ExtendWith(CoroutinesTimeoutExtension::class) +@Inherited +@MustBeDocumented +@ResourceLock("coroutines timeout", mode = ResourceAccessMode.READ) +@Retention(value = AnnotationRetention.RUNTIME) +@Target(AnnotationTarget.CLASS, AnnotationTarget.FUNCTION) +public annotation class CoroutinesTimeout( + val testTimeoutMs: Long, + val cancelOnTimeout: Boolean = false +) diff --git a/kotlinx-coroutines-debug/src/junit/junit5/CoroutinesTimeoutExtension.kt b/kotlinx-coroutines-debug/src/junit/junit5/CoroutinesTimeoutExtension.kt new file mode 100644 index 0000000000..edb72815ad --- /dev/null +++ b/kotlinx-coroutines-debug/src/junit/junit5/CoroutinesTimeoutExtension.kt @@ -0,0 +1,279 @@ +package kotlinx.coroutines.debug.junit5 + +import kotlinx.coroutines.debug.* +import kotlinx.coroutines.debug.runWithTimeoutDumpingCoroutines +import org.junit.jupiter.api.extension.* +import org.junit.platform.commons.support.AnnotationSupport +import java.lang.reflect.* +import java.util.* +import java.util.concurrent.atomic.* + +internal class CoroutinesTimeoutException(val timeoutMs: Long): Exception("test timed out after $timeoutMs ms") + +/** + * This JUnit5 extension allows running test, test factory, test template, and lifecycle methods in a separate thread, + * failing them after the provided time limit and interrupting the thread. + * + * Additionally, it installs [DebugProbes] and dumps all coroutines at the moment of the timeout. It also cancels + * coroutines on timeout if [cancelOnTimeout] set to `true`. + * [enableCoroutineCreationStackTraces] controls the corresponding [DebugProbes.enableCreationStackTraces] property + * and can be optionally enabled if the creation stack traces are necessary. + * + * Beware that if several tests that use this extension set [enableCoroutineCreationStackTraces] to different values and + * execute in parallel, the behavior is ill-defined. In order to avoid conflicts between different instances of this + * extension when using JUnit5 in parallel, use [ResourceLock] with resource name `coroutines timeout` on tests that use + * it. Note that the tests annotated with [CoroutinesTimeout] already use this [ResourceLock], so there is no need to + * annotate them additionally. + * + * Note that while calls to test factories are verified to finish in the specified time, but the methods that they + * produce are not affected by this extension. + * + * Beware that registering the extension via [CoroutinesTimeout] annotation conflicts with manually registering it on + * the same tests via other methods (most notably, [RegisterExtension]) and is prohibited. + * + * Example of usage: + * ``` + * class HangingTest { + * @JvmField + * @RegisterExtension + * val timeout = CoroutinesTimeoutExtension.seconds(5) + * + * @Test + * fun testThatHangs() = runBlocking { + * ... + * delay(Long.MAX_VALUE) // somewhere deep in the stack + * ... + * } + * } + * ``` + * + * @see [CoroutinesTimeout] + * */ +// NB: the constructor is not private so that JUnit is able to call it via reflection. +internal class CoroutinesTimeoutExtension internal constructor( + private val enableCoroutineCreationStackTraces: Boolean = false, + private val timeoutMs: Long? = null, + private val cancelOnTimeout: Boolean? = null): InvocationInterceptor +{ + /** + * Creates the [CoroutinesTimeoutExtension] extension with the given timeout in milliseconds. + */ + public constructor(timeoutMs: Long, cancelOnTimeout: Boolean = false, + enableCoroutineCreationStackTraces: Boolean = true): + this(enableCoroutineCreationStackTraces, timeoutMs, cancelOnTimeout) + + public companion object { + /** + * Creates the [CoroutinesTimeoutExtension] extension with the given timeout in seconds. + */ + @JvmOverloads + public fun seconds(timeout: Int, cancelOnTimeout: Boolean = false, + enableCoroutineCreationStackTraces: Boolean = true): CoroutinesTimeoutExtension = + CoroutinesTimeoutExtension(enableCoroutineCreationStackTraces, timeout.toLong() * 1000, cancelOnTimeout) + } + + /** @see [initialize] */ + private val debugProbesOwnershipPassed = AtomicBoolean(false) + + private fun tryPassDebugProbesOwnership() = debugProbesOwnershipPassed.compareAndSet(false, true) + + /* We install the debug probes early so that the coroutines launched from the test constructor are captured as well. + However, this is not enough as the same extension instance may be reused several times, even cleaning up its + resources from the store. */ + init { + DebugProbes.enableCreationStackTraces = enableCoroutineCreationStackTraces + DebugProbes.install() + } + + // This is needed so that a class with no tests still successfully passes the ownership of DebugProbes to JUnit5. + override fun interceptTestClassConstructor( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext>, + extensionContext: ExtensionContext + ): T { + initialize(extensionContext) + return invocation.proceed() + } + + /** + * Initialize this extension instance and/or the extension value store. + * + * It seems that the only way to reliably have JUnit5 clean up after its extensions is to put an instance of + * [ExtensionContext.Store.CloseableResource] into the value store corresponding to the extension instance, which + * means that [DebugProbes.uninstall] must be placed into the value store. [debugProbesOwnershipPassed] is `true` + * if the call to [DebugProbes.install] performed in the constructor of the extension instance was matched with a + * placing of [DebugProbes.uninstall] into the value store. We call the process of placing the cleanup procedure + * "passing the ownership", as now JUnit5 (and not our code) has to worry about uninstalling the debug probes. + * + * However, extension instances can be reused with different value stores, and value stores can be reused across + * extension instances. This leads to a tricky scheme of performing [DebugProbes.uninstall]: + * + * - If neither the ownership of this instance's [DebugProbes] was yet passed nor there is any cleanup procedure + * stored, it means that we can just store our cleanup procedure, passing the ownership. + * - If the ownership was not yet passed, but a cleanup procedure is already stored, we can't just replace it with + * another one, as this would lead to imbalance between [DebugProbes.install] and [DebugProbes.uninstall]. + * Instead, we know that this extension context will at least outlive this use of this instance, so some debug + * probes other than the ones from our constructor are already installed and won't be uninstalled during our + * operation. We simply uninstall the debug probes that were installed in our constructor. + * - If the ownership was passed, but the store is empty, it means that this test instance is reused and, possibly, + * the debug probes installed in its constructor were already uninstalled. This means that we have to install them + * anew and store an uninstaller. + */ + private fun initialize(extensionContext: ExtensionContext) { + val store: ExtensionContext.Store = extensionContext.getStore( + ExtensionContext.Namespace.create(CoroutinesTimeoutExtension::class, extensionContext.uniqueId)) + /** It seems that the JUnit5 documentation does not specify the relationship between the extension instances and + * the corresponding [ExtensionContext] (in which the value stores are managed), so it is unclear whether it's + * theoretically possible for two extension instances that run concurrently to share an extension context. So, + * just in case this risk exists, we synchronize here. */ + synchronized(store) { + if (store["debugProbes"] == null) { + if (!tryPassDebugProbesOwnership()) { + /** This means that the [DebugProbes.install] call from the constructor of this extensions has + * already been matched with a corresponding cleanup procedure for JUnit5, but then JUnit5 cleaned + * everything up and later reused the same extension instance for other tests. Therefore, we need to + * install the [DebugProbes] anew. */ + DebugProbes.enableCreationStackTraces = enableCoroutineCreationStackTraces + DebugProbes.install() + } + /** put a fake resource into this extensions's store so that JUnit cleans it up, uninstalling the + * [DebugProbes] after this extension instance is no longer needed. **/ + store.put("debugProbes", ExtensionContext.Store.CloseableResource { DebugProbes.uninstall() }) + } else if (!debugProbesOwnershipPassed.get()) { + /** This instance shares its store with other ones. Because of this, there was no need to install + * [DebugProbes], they are already installed, and this fact will outlive this use of this instance of + * the extension. */ + if (tryPassDebugProbesOwnership()) { + // We successfully marked the ownership as passed and now may uninstall the extraneous debug probes. + DebugProbes.uninstall() + } + } + } + } + + override fun interceptTestMethod( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ) { + interceptNormalMethod(invocation, invocationContext, extensionContext) + } + + override fun interceptAfterAllMethod( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ) { + interceptLifecycleMethod(invocation, invocationContext, extensionContext) + } + + override fun interceptAfterEachMethod( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ) { + interceptLifecycleMethod(invocation, invocationContext, extensionContext) + } + + override fun interceptBeforeAllMethod( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ) { + interceptLifecycleMethod(invocation, invocationContext, extensionContext) + } + + override fun interceptBeforeEachMethod( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ) { + interceptLifecycleMethod(invocation, invocationContext, extensionContext) + } + + override fun interceptTestFactoryMethod( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ): T = interceptNormalMethod(invocation, invocationContext, extensionContext) + + override fun interceptTestTemplateMethod( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ) { + interceptNormalMethod(invocation, invocationContext, extensionContext) + } + + private fun Class.coroutinesTimeoutAnnotation(): Optional = + AnnotationSupport.findAnnotation(this, CoroutinesTimeout::class.java).let { + when { + it.isPresent -> it + enclosingClass != null -> enclosingClass.coroutinesTimeoutAnnotation() + else -> Optional.empty() + } + } + + private fun interceptMethod( + useClassAnnotation: Boolean, + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ): T { + initialize(extensionContext) + val testAnnotationOptional = + AnnotationSupport.findAnnotation(invocationContext.executable, CoroutinesTimeout::class.java) + val classAnnotationOptional = extensionContext.testClass.flatMap { it.coroutinesTimeoutAnnotation() } + if (timeoutMs != null && cancelOnTimeout != null) { + // this means we @RegisterExtension was used in order to register this extension. + if (testAnnotationOptional.isPresent || classAnnotationOptional.isPresent) { + /* Using annotations creates a separate instance of the extension, which composes in a strange way: both + timeouts are applied. This is at odds with the concept that method-level annotations override the outer + rules and may lead to unexpected outcomes, so we prohibit this. */ + throw UnsupportedOperationException("Using CoroutinesTimeout along with instance field-registered CoroutinesTimeout is prohibited; please use either @RegisterExtension or @CoroutinesTimeout, but not both") + } + return interceptInvocation(invocation, invocationContext.executable.name, timeoutMs, cancelOnTimeout) + } + /* The extension was registered via an annotation; check that we succeeded in finding the annotation that led to + the extension being registered and taking its parameters. */ + if (!testAnnotationOptional.isPresent && !classAnnotationOptional.isPresent) { + throw UnsupportedOperationException("Timeout was registered with a CoroutinesTimeout annotation, but we were unable to find it. Please report this.") + } + return when { + testAnnotationOptional.isPresent -> { + val annotation = testAnnotationOptional.get() + interceptInvocation(invocation, invocationContext.executable.name, annotation.testTimeoutMs, + annotation.cancelOnTimeout) + } + useClassAnnotation && classAnnotationOptional.isPresent -> { + val annotation = classAnnotationOptional.get() + interceptInvocation(invocation, invocationContext.executable.name, annotation.testTimeoutMs, + annotation.cancelOnTimeout) + } + else -> { + invocation.proceed() + } + } + } + + private fun interceptNormalMethod( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ): T = interceptMethod(true, invocation, invocationContext, extensionContext) + + private fun interceptLifecycleMethod( + invocation: InvocationInterceptor.Invocation, + invocationContext: ReflectiveInvocationContext, + extensionContext: ExtensionContext + ) = interceptMethod(false, invocation, invocationContext, extensionContext) + + private fun interceptInvocation( + invocation: InvocationInterceptor.Invocation, + methodName: String, + testTimeoutMs: Long, + cancelOnTimeout: Boolean + ): T = + runWithTimeoutDumpingCoroutines(methodName, testTimeoutMs, cancelOnTimeout, + { CoroutinesTimeoutException(testTimeoutMs) }, { invocation.proceed() }) +} diff --git a/kotlinx-coroutines-debug/src/module-info.java b/kotlinx-coroutines-debug/src/module-info.java new file mode 100644 index 0000000000..04d321c7c1 --- /dev/null +++ b/kotlinx-coroutines-debug/src/module-info.java @@ -0,0 +1,14 @@ +module kotlinx.coroutines.debug { + requires java.management; + requires java.instrument; + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires static net.bytebuddy; + requires static net.bytebuddy.agent; + requires static org.junit.jupiter.api; + requires static org.junit.platform.commons; + + exports kotlinx.coroutines.debug; + exports kotlinx.coroutines.debug.junit4; + exports kotlinx.coroutines.debug.junit5; +} diff --git a/kotlinx-coroutines-debug/test/BlockHoundTest.kt b/kotlinx-coroutines-debug/test/BlockHoundTest.kt new file mode 100644 index 0000000000..685dbb373e --- /dev/null +++ b/kotlinx-coroutines-debug/test/BlockHoundTest.kt @@ -0,0 +1,118 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.junit.* +import reactor.blockhound.* + +@Suppress("UnusedEquals", "DeferredResultUnused", "BlockingMethodInNonBlockingContext") +class BlockHoundTest : TestBase() { + + @Before + fun init() { + BlockHound.install() + } + + @Test(expected = BlockingOperationError::class) + fun testShouldDetectBlockingInDefault() = runTest { + withContext(Dispatchers.Default) { + Thread.sleep(1) + } + } + + @Test + fun testShouldNotDetectBlockingInIO() = runTest { + withContext(Dispatchers.IO) { + Thread.sleep(1) + } + } + + @Test + fun testShouldNotDetectNonblocking() = runTest { + withContext(Dispatchers.Default) { + val a = 1 + val b = 2 + assert(a + b == 3) + } + } + + @Test + fun testReusingThreads() = runTest { + val n = 100 + repeat(n) { + async(Dispatchers.IO) { + Thread.sleep(1) + } + } + repeat(n) { + async(Dispatchers.Default) { + } + } + repeat(n) { + async(Dispatchers.IO) { + Thread.sleep(1) + } + } + } + + @Suppress("DEPRECATION_ERROR") + @Test + fun testBroadcastChannelNotBeingConsideredBlocking() = runTest { + withContext(Dispatchers.Default) { + // Copy of kotlinx.coroutines.channels.BufferedChannelTest.testSimple + val q = BroadcastChannel(1) + val s = q.openSubscription() + check(!q.isClosedForSend) + check(s.isEmpty) + check(!s.isClosedForReceive) + val sender = launch { + q.send(1) + q.send(2) + } + val receiver = launch { + s.receive() == 1 + s.receive() == 2 + s.cancel() + } + sender.join() + receiver.join() + } + } + + @Test + fun testConflatedChannelNotBeingConsideredBlocking() = runTest { + withContext(Dispatchers.Default) { + val q = Channel(Channel.CONFLATED) + check(q.isEmpty) + check(!q.isClosedForReceive) + check(!q.isClosedForSend) + val sender = launch { + q.send(1) + } + val receiver = launch { + q.receive() == 1 + } + sender.join() + receiver.join() + } + } + + @Test(expected = BlockingOperationError::class) + fun testReusingThreadsFailure() = runTest { + val n = 100 + repeat(n) { + async(Dispatchers.IO) { + Thread.sleep(1) + } + } + async(Dispatchers.Default) { + Thread.sleep(1) + } + repeat(n) { + async(Dispatchers.IO) { + Thread.sleep(1) + } + } + } +} diff --git a/kotlinx-coroutines-debug/test/CoroutinesDumpTest.kt b/kotlinx-coroutines-debug/test/CoroutinesDumpTest.kt new file mode 100644 index 0000000000..f1b9a6d498 --- /dev/null +++ b/kotlinx-coroutines-debug/test/CoroutinesDumpTest.kt @@ -0,0 +1,216 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import kotlin.coroutines.* +import kotlin.test.* + +class CoroutinesDumpTest : DebugTestBase() { + private val monitor = Any() + private var coroutineThread: Thread? = null // guarded by monitor + + @Before + override fun setUp() { + super.setUp() + DebugProbes.enableCreationStackTraces = true + } + + @Test + fun testSuspendedCoroutine() = runBlocking { + val deferred = async(Dispatchers.Default) { + sleepingOuterMethod() + } + + awaitCoroutine() + val found = DebugProbes.dumpCoroutinesInfo().single { it.job === deferred } + verifyDump( + "Coroutine \"coroutine#1\":DeferredCoroutine{Active}@1e4a7dd4, state: SUSPENDED\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.sleepingNestedMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.sleepingOuterMethod(CoroutinesDumpTest.kt)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt)\n" + + "\tat kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt)\n", + ignoredCoroutine = "BlockingCoroutine" + ) { + deferred.cancel() + coroutineThread!!.interrupt() + } + assertSame(deferred, found.job) + } + + @Test + fun testRunningCoroutine() = runBlocking { + val deferred = async(Dispatchers.IO) { + activeMethod(shouldSuspend = false) + assertTrue(true) + } + + awaitCoroutine() + verifyDump( + "Coroutine \"coroutine#1\":DeferredCoroutine{Active}@227d9994, state: RUNNING\n" + + "\tat java.lang.Thread.sleep(Native Method)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.nestedActiveMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.activeMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.access\$activeMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest\$testRunningCoroutine\$1\$deferred\$1.invokeSuspend(CoroutinesDumpTest.kt)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt)\n" + + "\tat kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt)\n" + + "\tat kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt)\n" + + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async(Builders.common.kt)\n" + + "\tat kotlinx.coroutines.BuildersKt.async(Unknown Source)\n" + + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async\$default(Builders.common.kt)\n" + + "\tat kotlinx.coroutines.BuildersKt.async\$default(Unknown Source)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest\$testRunningCoroutine\$1.invokeSuspend(CoroutinesDumpTest.kt)", + ignoredCoroutine = "BlockingCoroutine" + ) { + deferred.cancel() + coroutineThread?.interrupt() + } + } + + @Test + fun testRunningCoroutineWithSuspensionPoint() = runBlocking { + val deferred = async(Dispatchers.IO) { + activeMethod(shouldSuspend = true) + yield() // tail-call + } + + awaitCoroutine() + verifyDump( + "Coroutine \"coroutine#1\":DeferredCoroutine{Active}@1e4a7dd4, state: RUNNING\n" + + "\tat java.lang.Thread.sleep(Native Method)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.nestedActiveMethod(CoroutinesDumpTest.kt)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest.activeMethod(CoroutinesDumpTest.kt)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt)\n" + + "\tat kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt)\n" + + "\tat kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt)\n" + + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async(Builders.common.kt)\n" + + "\tat kotlinx.coroutines.BuildersKt.async(Unknown Source)\n" + + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async\$default(Builders.common.kt)\n" + + "\tat kotlinx.coroutines.BuildersKt.async\$default(Unknown Source)\n" + + "\tat kotlinx.coroutines.debug.CoroutinesDumpTest\$testRunningCoroutineWithSuspensionPoint\$1.invokeSuspend(CoroutinesDumpTest.kt)", + ignoredCoroutine = "BlockingCoroutine" + ) { + deferred.cancel() + coroutineThread!!.interrupt() + } + } + + /** + * Tests that a coroutine started with [CoroutineStart.UNDISPATCHED] is considered running. + */ + @Test + fun testUndispatchedCoroutineIsRunning() = runBlocking { + val job = launch(Dispatchers.IO, start = CoroutineStart.UNDISPATCHED) { // or launch(Dispatchers.Unconfined) + verifyDump( + "Coroutine \"coroutine#1\":StandaloneCoroutine{Active}@1e4a7dd4, state: RUNNING\n", + ignoredCoroutine = "BlockingCoroutine" + ) + delay(Long.MAX_VALUE) + } + verifyDump( + "Coroutine \"coroutine#1\":StandaloneCoroutine{Active}@1e4a7dd4, state: SUSPENDED\n", + ignoredCoroutine = "BlockingCoroutine" + ) { + job.cancel() + } + } + + @Test + fun testCreationStackTrace() = runBlocking { + val deferred = async(Dispatchers.IO) { + activeMethod(shouldSuspend = true) + } + + awaitCoroutine() + val coroutine = DebugProbes.dumpCoroutinesInfo().first { it.job is Deferred<*> } + val result = coroutine.creationStackTrace.fold(StringBuilder()) { acc, element -> + acc.append(element.toString()) + acc.append('\n') + }.toString().trimStackTrace() + + deferred.cancel() + coroutineThread!!.interrupt() + + val expected = + "kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)\n" + + "kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt)\n" + + "kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt)\n" + + "kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt)\n" + + "kotlinx.coroutines.BuildersKt__Builders_commonKt.async(Builders.common.kt)\n" + + "kotlinx.coroutines.BuildersKt.async(Unknown Source)\n" + + "kotlinx.coroutines.BuildersKt__Builders_commonKt.async\$default(Builders.common.kt)\n" + + "kotlinx.coroutines.BuildersKt.async\$default(Unknown Source)\n" + + "kotlinx.coroutines.debug.CoroutinesDumpTest\$testCreationStackTrace\$1.invokeSuspend(CoroutinesDumpTest.kt)" + if (!result.startsWith(expected)) { + error(""" + |Does not start with expected lines + |=== Actual result: + |$result + """.trimMargin() + ) + } + + } + + @Test + fun testFinishedCoroutineRemoved() = runBlocking { + val deferred = async(Dispatchers.IO) { + activeMethod(shouldSuspend = true) + } + + awaitCoroutine() + deferred.cancel() + coroutineThread!!.interrupt() + deferred.join() + verifyDump(ignoredCoroutine = "BlockingCoroutine") + } + + private suspend fun activeMethod(shouldSuspend: Boolean) { + nestedActiveMethod(shouldSuspend) + assertTrue(true) // tail-call + } + + private suspend fun nestedActiveMethod(shouldSuspend: Boolean) { + if (shouldSuspend) yield() + notifyCoroutineStarted() + while (coroutineContext[Job]!!.isActive) { + try { + Thread.sleep(60_000) + } catch (_ : InterruptedException) { + } + } + } + + private suspend fun sleepingOuterMethod() { + sleepingNestedMethod() + yield() // TCE + } + + private suspend fun sleepingNestedMethod() { + yield() // Suspension point + notifyCoroutineStarted() + delay(Long.MAX_VALUE) + } + + private fun awaitCoroutine() = synchronized(monitor) { + while (coroutineThread == null) (monitor as Object).wait() + while (coroutineThread!!.state != Thread.State.TIMED_WAITING) { + // Wait until thread sleeps to have a consistent stacktrace + } + } + + private fun notifyCoroutineStarted() { + synchronized(monitor) { + coroutineThread = Thread.currentThread() + (monitor as Object).notifyAll() + } + } +} diff --git a/kotlinx-coroutines-debug/test/DebugLeaksTest.kt b/kotlinx-coroutines-debug/test/DebugLeaksTest.kt new file mode 100644 index 0000000000..7a59783c58 --- /dev/null +++ b/kotlinx-coroutines-debug/test/DebugLeaksTest.kt @@ -0,0 +1,54 @@ +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* +import kotlinx.coroutines.debug.internal.* +import org.junit.* + +/** + * This is fast but fragile version of [DebugLeaksStressTest] that check reachability of a captured object + * in [DebugProbesImpl] via [FieldWalker]. + */ +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +class DebugLeaksTest : DebugTestBase() { + private class Captured + + @Test + fun testIteratorLeak() { + val captured = Captured() + iterator { yield(captured) } + assertNoCapturedReference() + } + + @Test + fun testLazyGlobalCoroutineLeak() { + val captured = Captured() + GlobalScope.launch(start = CoroutineStart.LAZY) { println(captured) } + assertNoCapturedReference() + } + + @Test + fun testLazyCancelledChildCoroutineLeak() = runTest { + val captured = Captured() + coroutineScope { + val child = launch(start = CoroutineStart.LAZY) { println(captured) } + child.cancel() + } + assertNoCapturedReference() + } + + @Test + fun testAbandonedGlobalCoroutineLeak() { + val captured = Captured() + GlobalScope.launch { + suspendForever() + println(captured) + } + assertNoCapturedReference() + } + + private suspend fun suspendForever() = suspendCancellableCoroutine { } + + private fun assertNoCapturedReference() { + FieldWalker.assertReachableCount(0, DebugProbesImpl, rootStatics = true) { it is Captured } + } +} diff --git a/kotlinx-coroutines-debug/test/DebugProbesTest.kt b/kotlinx-coroutines-debug/test/DebugProbesTest.kt new file mode 100644 index 0000000000..3994bb4967 --- /dev/null +++ b/kotlinx-coroutines-debug/test/DebugProbesTest.kt @@ -0,0 +1,144 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicBoolean +import kotlin.test.* + +class DebugProbesTest : DebugTestBase() { + + private fun CoroutineScope.createDeferred(): Deferred<*> = async(NonCancellable) { + throw ExecutionException(null) + } + + @Test + fun testAsync() = runTest { + val deferred = createDeferred() + val traces = listOf( + "java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest\$createDeferred\$1.invokeSuspend(DebugProbesTest.kt:14)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest.oneMoreNestedMethod(DebugProbesTest.kt:49)\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest.nestedMethod(DebugProbesTest.kt:44)\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest\$testAsync\$1.invokeSuspend(DebugProbesTest.kt:17)\n", + "Caused by: java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest\$createDeferred\$1.invokeSuspend(DebugProbesTest.kt:14)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)" + ) + nestedMethod(deferred, traces) + deferred.join() + } + + @Test + fun testAsyncWithProbes() = DebugProbes.withDebugProbes { + DebugProbes.sanitizeStackTraces = false + runTest { + val deferred = createDeferred() + val traces = listOf( + "java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest\$createDeferred\$1.invokeSuspend(DebugProbesTest.kt)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest.oneMoreNestedMethod(DebugProbesTest.kt)\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest.nestedMethod(DebugProbesTest.kt)\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest\$testAsyncWithProbes\$1\$1.invokeSuspend(DebugProbesTest.kt:62)", + "Caused by: java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest\$createDeferred\$1.invokeSuspend(DebugProbesTest.kt)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt)\n" + ) + nestedMethod(deferred, traces) + deferred.join() + } + } + + @Test + fun testAsyncWithSanitizedProbes() = DebugProbes.withDebugProbes { + DebugProbes.sanitizeStackTraces = true + runTest { + val deferred = createDeferred() + val traces = listOf( + "java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest\$createDeferred\$1.invokeSuspend(DebugProbesTest.kt:16)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest.oneMoreNestedMethod(DebugProbesTest.kt:71)\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest.nestedMethod(DebugProbesTest.kt:66)\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest\$testAsyncWithSanitizedProbes\$1\$1.invokeSuspend(DebugProbesTest.kt:87)", + "Caused by: java.util.concurrent.ExecutionException\n" + + "\tat kotlinx.coroutines.debug.DebugProbesTest\$createDeferred\$1.invokeSuspend(DebugProbesTest.kt:16)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)\n" + ) + nestedMethod(deferred, traces) + deferred.join() + } + } + + private suspend fun nestedMethod(deferred: Deferred<*>, traces: List) { + oneMoreNestedMethod(deferred, traces) + assertTrue(true) // Prevent tail-call optimization + } + + private suspend fun oneMoreNestedMethod(deferred: Deferred<*>, traces: List) { + try { + deferred.await() + expectUnreached() + } catch (e: ExecutionException) { + verifyStackTrace(e, traces) + } + } + + @Test + fun testMultipleConsecutiveProbeResumed() = runTest { + val job = launch { + expect(1) + foo() + expect(4) + delay(Long.MAX_VALUE) + expectUnreached() + } + yield() + yield() + expect(5) + val infos = DebugProbes.dumpCoroutinesInfo() + assertEquals(2, infos.size) + assertEquals(setOf(State.RUNNING, State.SUSPENDED), infos.map { it.state }.toSet()) + job.cancel() + finish(6) + } + + @Test + fun testMultipleConsecutiveProbeResumedAndLaterRunning() = runTest { + val reachedActiveStage = AtomicBoolean(false) + val job = launch(Dispatchers.Default) { + expect(1) + foo() + expect(4) + yield() + reachedActiveStage.set(true) + while (isActive) { + // Spin until test is done + } + } + while (!reachedActiveStage.get()) { + delay(10) + } + expect(5) + val infos = DebugProbes.dumpCoroutinesInfo() + assertEquals(2, infos.size) + assertEquals(setOf(State.RUNNING, State.RUNNING), infos.map { it.state }.toSet()) + job.cancel() + finish(6) + } + + private suspend fun foo() { + bar() + // Kill TCO + expect(3) + } + + + private suspend fun bar() { + yield() + expect(2) + } +} diff --git a/kotlinx-coroutines-debug/test/DebugTestBase.kt b/kotlinx-coroutines-debug/test/DebugTestBase.kt new file mode 100644 index 0000000000..93cb2f60bf --- /dev/null +++ b/kotlinx-coroutines-debug/test/DebugTestBase.kt @@ -0,0 +1,26 @@ +package kotlinx.coroutines.debug + + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.junit4.* +import org.junit.* + +open class DebugTestBase : TestBase() { + + @JvmField + @Rule + val timeout = CoroutinesTimeout.seconds(60) + + @Before + open fun setUp() { + DebugProbes.sanitizeStackTraces = false + DebugProbes.enableCreationStackTraces = false + DebugProbes.install() + } + + @After + fun tearDown() { + DebugProbes.uninstall() + } +} diff --git a/kotlinx-coroutines-debug/test/DumpCoroutineInfoAsJsonAndReferencesTest.kt b/kotlinx-coroutines-debug/test/DumpCoroutineInfoAsJsonAndReferencesTest.kt new file mode 100644 index 0000000000..afd7a50227 --- /dev/null +++ b/kotlinx-coroutines-debug/test/DumpCoroutineInfoAsJsonAndReferencesTest.kt @@ -0,0 +1,99 @@ +@file:Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import com.google.gson.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* +import org.junit.Test +import kotlin.coroutines.* +import kotlin.test.* + +@ExperimentalStdlibApi +class DumpCoroutineInfoAsJsonAndReferencesTest : DebugTestBase() { + private data class CoroutineInfoFromJson( + val name: String?, + val id: Long?, + val dispatcher: String?, + val sequenceNumber: Long?, + val state: String? + ) + + @Test + fun testDumpOfUnnamedCoroutine() = + runTestWithNamedDeferred(name = null) + + @Test + fun testDumpOfNamedCoroutine() = + runTestWithNamedDeferred("Name") + + @Test + fun testDumpOfNamedCoroutineWithSpecialCharacters() = + runTestWithNamedDeferred("Name with\n \"special\" characters\\/\t\b") + + @Test + fun testDumpWithNoCoroutines() { + val dumpResult = DebugProbesImpl.dumpCoroutinesInfoAsJsonAndReferences() + assertEquals(dumpResult.size, 4) + assertIsEmptyArray(dumpResult[1]) + assertIsEmptyArray(dumpResult[2]) + assertIsEmptyArray(dumpResult[3]) + } + + private fun assertIsEmptyArray(obj: Any) = + assertTrue(obj is Array<*> && obj.isEmpty()) + + private fun runTestWithNamedDeferred(name: String?) = runTest { + val context = if (name == null) EmptyCoroutineContext else CoroutineName(name) + val deferred = async(context) { + suspendingMethod() + assertTrue(true) + } + yield() + verifyDump() + deferred.cancelAndJoin() + } + + private suspend fun suspendingMethod() { + delay(Long.MAX_VALUE) + } + + private fun verifyDump() { + val dumpResult = DebugProbesImpl.dumpCoroutinesInfoAsJsonAndReferences() + + assertEquals(dumpResult.size, 4) + + val coroutinesInfoAsJsonString = dumpResult[0] + val lastObservedThreads = dumpResult[1] + val lastObservedFrames = dumpResult[2] + val coroutinesInfo = dumpResult[3] + + assertIs(coroutinesInfoAsJsonString) + assertIs>(lastObservedThreads) + assertIs>(lastObservedFrames) + assertIs>(coroutinesInfo) + + val coroutinesInfoFromJson = + Gson().fromJson(coroutinesInfoAsJsonString, Array::class.java) + + val size = coroutinesInfo.size + assertTrue(size != 0) + assertEquals(size, coroutinesInfoFromJson.size) + assertEquals(size, lastObservedFrames.size) + assertEquals(size, lastObservedThreads.size) + + for (i in 0 until size) { + val info = coroutinesInfo[i] + val infoFromJson = coroutinesInfoFromJson[i] + assertIs(info) + assertEquals(info.lastObservedThread, lastObservedThreads[i]) + assertEquals(info.lastObservedFrame, lastObservedFrames[i]) + assertEquals(info.sequenceNumber, infoFromJson.sequenceNumber) + assertEquals(info.state, infoFromJson.state) + val context = info.context + assertEquals(context[CoroutineName.Key]?.name, infoFromJson.name) + assertEquals(context[CoroutineId.Key]?.id, infoFromJson.id) + assertEquals(context[CoroutineDispatcher.Key]?.toString(), infoFromJson.dispatcher) + } + } +} diff --git a/kotlinx-coroutines-debug/test/DumpWithCreationStackTraceTest.kt b/kotlinx-coroutines-debug/test/DumpWithCreationStackTraceTest.kt new file mode 100644 index 0000000000..cb0ae3844e --- /dev/null +++ b/kotlinx-coroutines-debug/test/DumpWithCreationStackTraceTest.kt @@ -0,0 +1,47 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class DumpWithCreationStackTraceTest : DebugTestBase() { + @Before + override fun setUp() { + super.setUp() + DebugProbes.enableCreationStackTraces = true + } + + @Test + fun testCoroutinesDump() = runTest { + val deferred = createActiveDeferred() + yield() + verifyDump( + "Coroutine \"coroutine#1\":BlockingCoroutine{Active}@70d1cb56, state: RUNNING\n" + + "\tat java.lang.Thread.getStackTrace(Thread.java)\n" + + "\tat kotlinx.coroutines.debug.internal.DebugProbesImpl.enhanceStackTraceWithThreadDumpImpl(DebugProbesImpl.kt)\n" + + "\tat kotlinx.coroutines.debug.internal.DebugProbesImpl.dumpCoroutinesSynchronized(DebugProbesImpl.kt)\n" + + "\tat kotlinx.coroutines.debug.internal.DebugProbesImpl.dumpCoroutines(DebugProbesImpl.kt)\n" + + "\tat kotlinx.coroutines.debug.DebugProbes.dumpCoroutines(DebugProbes.kt:182)\n" + + "\tat kotlinx.coroutines.debug.StacktraceUtilsKt.verifyDump(StacktraceUtils.kt)\n" + + "\tat kotlinx.coroutines.debug.StacktraceUtilsKt.verifyDump\$default(StacktraceUtils.kt)\n" + + "\tat kotlinx.coroutines.debug.DumpWithCreationStackTraceTest\$testCoroutinesDump\$1.invokeSuspend(DumpWithCreationStackTraceTest.kt)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt)", + + "Coroutine \"coroutine#2\":DeferredCoroutine{Active}@383fa309, state: SUSPENDED\n" + + "\tat kotlinx.coroutines.debug.DumpWithCreationStackTraceTest\$createActiveDeferred\$1.invokeSuspend(DumpWithCreationStackTraceTest.kt)" + ) + deferred.cancelAndJoin() + } + + + private fun CoroutineScope.createActiveDeferred(): Deferred<*> = async { + suspendingMethod() + assertTrue(true) + } + + private suspend fun suspendingMethod() { + delay(Long.MAX_VALUE) + } +} diff --git a/kotlinx-coroutines-debug/test/EnhanceStackTraceWithTreadDumpAsJsonTest.kt b/kotlinx-coroutines-debug/test/EnhanceStackTraceWithTreadDumpAsJsonTest.kt new file mode 100644 index 0000000000..72e61d6bb5 --- /dev/null +++ b/kotlinx-coroutines-debug/test/EnhanceStackTraceWithTreadDumpAsJsonTest.kt @@ -0,0 +1,47 @@ +@file:Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import com.google.gson.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* +import org.junit.Test +import kotlin.test.* + +class EnhanceStackTraceWithTreadDumpAsJsonTest : DebugTestBase() { + private data class StackTraceElementInfoFromJson( + val declaringClass: String, + val methodName: String, + val fileName: String?, + val lineNumber: Int + ) + + @Test + fun testEnhancedStackTraceFormatWithDeferred() = runTest { + val deferred = async { + suspendingMethod() + assertTrue(true) + } + yield() + + val coroutineInfo = DebugProbesImpl.dumpCoroutinesInfo() + assertEquals(coroutineInfo.size, 2) + val info = coroutineInfo[1] + val enhancedStackTraceAsJsonString = DebugProbesImpl.enhanceStackTraceWithThreadDumpAsJson(info) + val enhancedStackTraceFromJson = Gson().fromJson(enhancedStackTraceAsJsonString, Array::class.java) + val enhancedStackTrace = DebugProbesImpl.enhanceStackTraceWithThreadDump(info, info.lastObservedStackTrace) + assertEquals(enhancedStackTrace.size, enhancedStackTraceFromJson.size) + for ((frame, frameFromJson) in enhancedStackTrace.zip(enhancedStackTraceFromJson)) { + assertEquals(frame.className, frameFromJson.declaringClass) + assertEquals(frame.methodName, frameFromJson.methodName) + assertEquals(frame.fileName, frameFromJson.fileName) + assertEquals(frame.lineNumber, frameFromJson.lineNumber) + } + + deferred.cancelAndJoin() + } + + private suspend fun suspendingMethod() { + delay(Long.MAX_VALUE) + } +} diff --git a/kotlinx-coroutines-debug/test/Example.kt b/kotlinx-coroutines-debug/test/Example.kt new file mode 100644 index 0000000000..8a0944cbd1 --- /dev/null +++ b/kotlinx-coroutines-debug/test/Example.kt @@ -0,0 +1,32 @@ +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* + +suspend fun computeValue(): String = coroutineScope { + val one = async { computeOne() } + val two = async { computeTwo() } + combineResults(one, two) +} + +suspend fun combineResults(one: Deferred, two: Deferred): String = + one.await() + two.await() + +suspend fun computeOne(): String { + delay(5000) + return "4" +} + +suspend fun computeTwo(): String { + delay(5000) + return "2" +} + +fun main() = runBlocking { + DebugProbes.install() + val deferred = async { computeValue() } + // Delay for some time + delay(1000) + // Dump running coroutines + DebugProbes.dumpCoroutines() + println("\nDumping only deferred") + DebugProbes.printJob(deferred) +} \ No newline at end of file diff --git a/kotlinx-coroutines-debug/test/LazyCoroutineTest.kt b/kotlinx-coroutines-debug/test/LazyCoroutineTest.kt new file mode 100644 index 0000000000..8cf1f86a7d --- /dev/null +++ b/kotlinx-coroutines-debug/test/LazyCoroutineTest.kt @@ -0,0 +1,21 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class LazyCoroutineTest : DebugTestBase() { + + @Test + fun testLazyCompletedCoroutine() = runTest { + val job = launch(start = CoroutineStart.LAZY) {} + job.invokeOnCompletion { expect(2) } + expect(1) + job.cancelAndJoin() + expect(3) + assertEquals(1, DebugProbes.dumpCoroutinesInfo().size) // Outer runBlocking + verifyPartialDump(1, "BlockingCoroutine{Active}") + finish(4) + } +} diff --git a/kotlinx-coroutines-debug/test/RecoveryExample.kt b/kotlinx-coroutines-debug/test/RecoveryExample.kt new file mode 100644 index 0000000000..eeadcf5618 --- /dev/null +++ b/kotlinx-coroutines-debug/test/RecoveryExample.kt @@ -0,0 +1,28 @@ +@file:Suppress("PackageDirectoryMismatch") +package example + +import kotlinx.coroutines.* + +object PublicApiImplementation : CoroutineScope by CoroutineScope(CoroutineName("Example")) { + + private fun doWork(): Int { + error("Internal invariant failed") + } + + private fun asynchronousWork(): Int { + return doWork() + 1 + } + + public suspend fun awaitAsynchronousWorkInMainThread() { + val task = async(Dispatchers.Default) { + asynchronousWork() + } + + task.await() + } +} + +suspend fun main() { + // Try to switch debug mode on and off to see the difference + PublicApiImplementation.awaitAsynchronousWorkInMainThread() +} diff --git a/kotlinx-coroutines-debug/test/RunningThreadStackMergeTest.kt b/kotlinx-coroutines-debug/test/RunningThreadStackMergeTest.kt new file mode 100644 index 0000000000..bafea1f031 --- /dev/null +++ b/kotlinx-coroutines-debug/test/RunningThreadStackMergeTest.kt @@ -0,0 +1,174 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class RunningThreadStackMergeTest : DebugTestBase() { + + private val testMainBlocker = CountDownLatch(1) // Test body blocks on it + private val coroutineBlocker = CyclicBarrier(2) // Launched coroutine blocks on it + + @Test + fun testStackMergeWithContext() = runTest { + launchCoroutine() + awaitCoroutineStarted() + verifyDump( + "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@50284dc4, state: RUNNING\n" + + "\tat jdk.internal.misc.Unsafe.park(Native Method)\n" + + "\tat java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)\n" + + "\tat java.util.concurrent.locks.AbstractQueuedSynchronizer\$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)\n" + + "\tat java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:234)\n" + + "\tat java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.nonSuspendingFun(RunningThreadStackMergeTest.kt:86)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.access\$nonSuspendingFun(RunningThreadStackMergeTest.kt:12)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$suspendingFunction\$2.invokeSuspend(RunningThreadStackMergeTest.kt:77)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.suspendingFunction(RunningThreadStackMergeTest.kt:75)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$launchCoroutine\$1.invokeSuspend(RunningThreadStackMergeTest.kt:68)", + ignoredCoroutine = "BlockingCoroutine" + ) { + coroutineBlocker.await() + } + } + + private fun awaitCoroutineStarted() { + testMainBlocker.await() + while (coroutineBlocker.numberWaiting != 1) { + Thread.sleep(10) + } + } + + private fun CoroutineScope.launchCoroutine() { + launch(Dispatchers.Default) { + suspendingFunction() + assertTrue(true) + } + } + + private suspend fun suspendingFunction() { + // Typical use-case + withContext(Dispatchers.IO) { + yield() + nonSuspendingFun() + } + + assertTrue(true) + } + + private fun nonSuspendingFun() { + testMainBlocker.countDown() + coroutineBlocker.await() + } + + @Test + fun testStackMergeEscapeSuspendMethod() = runTest { + launchEscapingCoroutine() + awaitCoroutineStarted() + Thread.sleep(10) + verifyDump( + "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@3aea3c67, state: RUNNING\n" + + "\tat jdk.internal.misc.Unsafe.park(Native Method)\n" + + "\tat java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)\n" + + "\tat java.util.concurrent.locks.AbstractQueuedSynchronizer\$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)\n" + + "\tat java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:234)\n" + + "\tat java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.nonSuspendingFun(RunningThreadStackMergeTest.kt:83)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.access\$nonSuspendingFun(RunningThreadStackMergeTest.kt:12)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$suspendingFunctionWithContext\$2.invokeSuspend(RunningThreadStackMergeTest.kt:124)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.suspendingFunctionWithContext(RunningThreadStackMergeTest.kt:122)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$launchEscapingCoroutine\$1.invokeSuspend(RunningThreadStackMergeTest.kt:116)", + ignoredCoroutine = "BlockingCoroutine" + ) { + coroutineBlocker.await() + } + } + + private fun CoroutineScope.launchEscapingCoroutine() { + launch(Dispatchers.Default) { + suspendingFunctionWithContext() + assertTrue(true) + } + } + + private suspend fun suspendingFunctionWithContext() { + withContext(Dispatchers.IO) { + actualSuspensionPoint() + nonSuspendingFun() + } + + assertTrue(true) + } + + @Test + fun testMergeThroughInvokeSuspend() = runTest { + launchEscapingCoroutineWithoutContext() + awaitCoroutineStarted() + verifyDump( + "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@3aea3c67, state: RUNNING\n" + + "\tat jdk.internal.misc.Unsafe.park(Native Method)\n" + + "\tat java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)\n" + + "\tat java.util.concurrent.locks.AbstractQueuedSynchronizer\$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)\n" + + "\tat java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:234)\n" + + "\tat java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.nonSuspendingFun(RunningThreadStackMergeTest.kt:83)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest.suspendingFunctionWithoutContext(RunningThreadStackMergeTest.kt:160)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$launchEscapingCoroutineWithoutContext\$1.invokeSuspend(RunningThreadStackMergeTest.kt:153)", + ignoredCoroutine = "BlockingCoroutine" + ) { + coroutineBlocker.await() + } + } + + private fun CoroutineScope.launchEscapingCoroutineWithoutContext() { + launch(Dispatchers.IO) { + suspendingFunctionWithoutContext() + assertTrue(true) + } + } + + private suspend fun suspendingFunctionWithoutContext() { + actualSuspensionPoint() + nonSuspendingFun() + assertTrue(true) + } + + @Test + fun testRunBlocking() = runBlocking { + verifyDump( + "Coroutine \"coroutine#1\":BlockingCoroutine{Active}@4bcd176c, state: RUNNING\n" + + "\tat java.lang.Thread.getStackTrace(Thread.java)\n" + + "\tat kotlinx.coroutines.debug.internal.DebugProbesImpl.enhanceStackTraceWithThreadDumpImpl(DebugProbesImpl.kt)\n" + + "\tat kotlinx.coroutines.debug.internal.DebugProbesImpl.dumpCoroutinesSynchronized(DebugProbesImpl.kt)\n" + + "\tat kotlinx.coroutines.debug.internal.DebugProbesImpl.dumpCoroutines(DebugProbesImpl.kt)\n" + + "\tat kotlinx.coroutines.debug.DebugProbes.dumpCoroutines(DebugProbes.kt)\n" + + "\tat kotlinx.coroutines.debug.StacktraceUtilsKt.verifyDump(StacktraceUtils.kt)\n" + + "\tat kotlinx.coroutines.debug.StacktraceUtilsKt.verifyDump\$default(StacktraceUtils.kt)\n" + + "\tat kotlinx.coroutines.debug.RunningThreadStackMergeTest\$testRunBlocking\$1.invokeSuspend(RunningThreadStackMergeTest.kt)" + ) + } + + + private suspend fun actualSuspensionPoint() { + nestedSuspensionPoint() + assertTrue(true) + } + + private suspend fun nestedSuspensionPoint() { + yield() + assertTrue(true) + } + + @Test // IDEA-specific debugger API test + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") + fun testActiveThread() = runBlocking { + launchCoroutine() + awaitCoroutineStarted() + val info = DebugProbesImpl.dumpDebuggerInfo().find { it.state == "RUNNING" } + assertNotNull(info) + assertNotNull(info.lastObservedThreadName) + coroutineBlocker.await() + } +} diff --git a/kotlinx-coroutines-debug/test/SanitizedProbesTest.kt b/kotlinx-coroutines-debug/test/SanitizedProbesTest.kt new file mode 100644 index 0000000000..b14ac7c566 --- /dev/null +++ b/kotlinx-coroutines-debug/test/SanitizedProbesTest.kt @@ -0,0 +1,145 @@ +@file:Suppress("PackageDirectoryMismatch") +package definitely.not.kotlinx.coroutines + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* +import kotlinx.coroutines.selects.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class SanitizedProbesTest : DebugTestBase() { + @Before + override fun setUp() { + super.setUp() + DebugProbes.sanitizeStackTraces = true + DebugProbes.enableCreationStackTraces = true + } + + @Test + fun testRecoveredStackTrace() = runTest { + val deferred = createDeferred() + val traces = listOf( + "java.util.concurrent.ExecutionException\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$createDeferredNested\$1.invokeSuspend(SanitizedProbesTest.kt:97)\n" + + "\tat _COROUTINE._BOUNDARY._(CoroutineDebugging.kt)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.oneMoreNestedMethod(SanitizedProbesTest.kt:67)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.nestedMethod(SanitizedProbesTest.kt:61)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$testRecoveredStackTrace\$1.invokeSuspend(SanitizedProbesTest.kt:50)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + + "\tat kotlinx.coroutines.testing.TestBase.runTest\$default(TestBase.kt:141)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.testRecoveredStackTrace(SanitizedProbesTest.kt:33)", + "Caused by: java.util.concurrent.ExecutionException\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$createDeferredNested\$1.invokeSuspend(SanitizedProbesTest.kt:57)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)\n" + ) + nestedMethod(deferred, traces) + deferred.join() + } + + @Test + fun testCoroutinesDump() = runTest { + val deferred = createActiveDeferred() + yield() + verifyDump( + "Coroutine \"coroutine#3\":BlockingCoroutine{Active}@7d68ef40, state: RUNNING\n" + + "\tat java.lang.Thread.getStackTrace(Thread.java)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + + "\tat kotlinx.coroutines.testing.TestBase.runTest\$default(TestBase.kt:141)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.testCoroutinesDump(SanitizedProbesTest.kt:56)", + + "Coroutine \"coroutine#4\":DeferredCoroutine{Active}@75c072cb, state: SUSPENDED\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$createActiveDeferred\$1.invokeSuspend(SanitizedProbesTest.kt:63)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:23)\n" + + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.async\$default(Builders.common.kt)\n" + + "\tat kotlinx.coroutines.BuildersKt.async\$default(Unknown Source)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.createActiveDeferred(SanitizedProbesTest.kt:62)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.access\$createActiveDeferred(SanitizedProbesTest.kt:16)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$testCoroutinesDump\$1.invokeSuspend(SanitizedProbesTest.kt:57)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)\n" + + "\tat kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:237)\n" + + "\tat kotlinx.coroutines.testing.TestBase.runTest\$default(TestBase.kt:141)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.testCoroutinesDump(SanitizedProbesTest.kt:56)" + ) + deferred.cancelAndJoin() + } + + @Test + fun testSelectBuilder() = runTest { + val selector = launchSelector() + expect(1) + yield() + expect(3) + verifyDump("Coroutine \"coroutine#1\":BlockingCoroutine{Active}@35fc6dc4, state: RUNNING\n" + + "\tat java.lang.Thread.getStackTrace(Thread.java:1552)\n" + // Skip the rest + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)", + + "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@1b68b9a4, state: SUSPENDED\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$launchSelector\$1\$1\$1.invokeSuspend(SanitizedProbesTest.kt)\n" + + "\tat _COROUTINE._CREATION._(CoroutineDebugging.kt)\n" + + "\tat kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:116)\n" + + "\tat kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:25)\n" + + "\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.launch\$default(Builders.common.kt)\n" + + "\tat kotlinx.coroutines.BuildersKt.launch\$default(Unknown Source)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.launchSelector(SanitizedProbesTest.kt:100)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.access\$launchSelector(SanitizedProbesTest.kt:16)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest\$testSelectBuilder\$1.invokeSuspend(SanitizedProbesTest.kt:89)\n" + + "\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)\n" + + "\tat kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:233)\n" + + "\tat kotlinx.coroutines.testing.TestBase.runTest\$default(TestBase.kt:154)\n" + + "\tat definitely.not.kotlinx.coroutines.SanitizedProbesTest.testSelectBuilder(SanitizedProbesTest.kt:88)") + finish(4) + selector.cancelAndJoin() + } + + private fun CoroutineScope.launchSelector(): Job { + val job = CompletableDeferred(Unit) + return launch { + select { + job.onJoin { + expect(2) + delay(Long.MAX_VALUE) + 1 + } + } + } + } + + private fun CoroutineScope.createActiveDeferred(): Deferred<*> = async { + suspendingMethod() + assertTrue(true) + } + + private suspend fun suspendingMethod() { + delay(Long.MAX_VALUE) + } + + private fun CoroutineScope.createDeferred(): Deferred<*> = createDeferredNested() + + private fun CoroutineScope.createDeferredNested(): Deferred<*> = async(NonCancellable) { + throw ExecutionException(null) + } + + private suspend fun nestedMethod(deferred: Deferred<*>, traces: List) { + oneMoreNestedMethod(deferred, traces) + assertTrue(true) // Prevent tail-call optimization + } + + private suspend fun oneMoreNestedMethod(deferred: Deferred<*>, traces: List) { + try { + deferred.await() + expectUnreached() + } catch (e: ExecutionException) { + verifyStackTrace(e, traces) + } + } +} diff --git a/kotlinx-coroutines-debug/test/ScopedBuildersTest.kt b/kotlinx-coroutines-debug/test/ScopedBuildersTest.kt new file mode 100644 index 0000000000..a1eb331f5d --- /dev/null +++ b/kotlinx-coroutines-debug/test/ScopedBuildersTest.kt @@ -0,0 +1,42 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import kotlin.coroutines.* + +class ScopedBuildersTest : DebugTestBase() { + + @Test + fun testNestedScopes() = runBlocking { + val job = launch { doInScope() } + yield() + yield() + verifyDump( + "Coroutine \"coroutine#1\":BlockingCoroutine{Active}@16612a51, state: RUNNING", + + "Coroutine \"coroutine#2\":StandaloneCoroutine{Active}@6b53e23f, state: SUSPENDED\n" + + "\tat kotlinx.coroutines.debug.ScopedBuildersTest\$doWithContext\$2.invokeSuspend(ScopedBuildersTest.kt:49)\n" + + "\tat kotlinx.coroutines.debug.ScopedBuildersTest.doWithContext(ScopedBuildersTest.kt:47)\n" + + "\tat kotlinx.coroutines.debug.ScopedBuildersTest\$doInScope\$2.invokeSuspend(ScopedBuildersTest.kt:41)\n" + + "\tat kotlinx.coroutines.debug.ScopedBuildersTest\$testNestedScopes\$1\$job\$1.invokeSuspend(ScopedBuildersTest.kt:30)" + ) + job.cancelAndJoin() + finish(4) + } + + private suspend fun doInScope() = coroutineScope { + expect(1) + doWithContext() + expectUnreached() + } + + private suspend fun doWithContext() { + expect(2) + withContext(wrapperDispatcher(coroutineContext)) { + expect(3) + delay(Long.MAX_VALUE) + } + expectUnreached() + } +} diff --git a/kotlinx-coroutines-debug/test/StacktraceUtils.kt b/kotlinx-coroutines-debug/test/StacktraceUtils.kt new file mode 100644 index 0000000000..90ce38907e --- /dev/null +++ b/kotlinx-coroutines-debug/test/StacktraceUtils.kt @@ -0,0 +1,226 @@ +package kotlinx.coroutines.debug + +import java.io.* +import kotlin.test.* + +public fun String.trimStackTrace(): String = + trimIndent() + // Remove source line + .replace(Regex(":[0-9]+"), "") + // Remove coroutine id + .replace(Regex("#[0-9]+"), "") + // Remove trace prefix: "java.base@11.0.16.1/java.lang.Thread.sleep" => "java.lang.Thread.sleep" + .replace(Regex("(?<=\tat )[^\n]*/"), "") + .replace(Regex("\t"), "") + .replace("sun.misc.Unsafe.", "jdk.internal.misc.Unsafe.") // JDK8->JDK11 + +public fun verifyStackTrace(e: Throwable, traces: List) { + val stacktrace = toStackTrace(e) + val trimmedStackTrace = stacktrace.trimStackTrace() + traces.forEach { + assertTrue( + trimmedStackTrace.contains(it.trimStackTrace()), + "\nExpected trace element:\n$it\n\nActual stacktrace:\n$stacktrace" + ) + } + + val causes = stacktrace.count("Caused by") + assertNotEquals(0, causes) + assertEquals(causes, traces.map { it.count("Caused by") }.sum()) +} + +public fun toStackTrace(t: Throwable): String { + val sw = StringWriter() + t.printStackTrace(PrintWriter(sw)) + return sw.toString() +} + +public fun String.count(substring: String): Int = split(substring).size - 1 + +public fun verifyDump(vararg traces: String, ignoredCoroutine: String? = null, finally: () -> Unit) { + try { + verifyDump(*traces, ignoredCoroutine = ignoredCoroutine) + } finally { + finally() + } +} + +/** Clean the stacktraces from artifacts of BlockHound instrumentation + * + * BlockHound works by switching a native call by a class generated with ByteBuddy, which, if the blocking + * call is allowed in this context, in turn calls the real native call that is now available under a + * different name. + * + * The traces thus undergo the following two changes when the execution is instrumented: + * - The original native call is replaced with a non-native one with the same FQN, and + * - An additional native call is placed on top of the stack, with the original name that also has + * `$$BlockHound$$_` prepended at the last component. + */ +private fun cleanBlockHoundTraces(frames: List): List { + val result = mutableListOf() + val blockHoundSubstr = "\$\$BlockHound\$\$_" + var i = 0 + while (i < frames.size) { + result.add(frames[i].replace(blockHoundSubstr, "")) + if (frames[i].contains(blockHoundSubstr)) { + i += 1 + } + i += 1 + } + return result +} + +/** + * Removes all frames that contain "java.util.concurrent" in it. + * + * We do leverage Java's locks for proper rendezvous and to fix the coroutine stack's state, + * but this API doesn't have (nor expected to) stable stacktrace, so we are filtering all such + * frames out. + * + * See https://github.com/Kotlin/kotlinx.coroutines/issues/3700 for the example of failure + */ +private fun removeJavaUtilConcurrentTraces(frames: List): List = + frames.filter { !it.contains("java.util.concurrent") } + +private data class CoroutineDump( + val header: CoroutineDumpHeader, + val coroutineStackTrace: List, + val threadStackTrace: List, + val originDump: String, + val originHeader: String, +) { + companion object { + private val COROUTINE_CREATION_FRAME_REGEX = + "at _COROUTINE\\._CREATION\\._\\(.*\\)".toRegex() + + fun parse(dump: String, traceCleaner: ((List) -> List)? = null): CoroutineDump { + val lines = dump + .trimStackTrace() + .split("\n") + val header = CoroutineDumpHeader.parse(lines[0]) + val traceLines = lines.slice(1 until lines.size) + val cleanedTraceLines = if (traceCleaner != null) { + traceCleaner(traceLines) + } else { + traceLines + } + val coroutineStackTrace = mutableListOf() + val threadStackTrace = mutableListOf() + var trace = coroutineStackTrace + for (line in cleanedTraceLines) { + if (line.isEmpty()) { + continue + } + if (line.matches(COROUTINE_CREATION_FRAME_REGEX)) { + require(trace !== threadStackTrace) { + "Found more than one coroutine creation frame" + } + trace = threadStackTrace + continue + } + trace.add(line) + } + return CoroutineDump(header, coroutineStackTrace, threadStackTrace, dump, lines[0]) + } + } + + fun verify(expected: CoroutineDump) { + assertEquals( + expected.header, header, + "Coroutine stacktrace headers are not matched:\n\t- ${expected.originHeader}\n\t+ ${originHeader}\n" + ) + verifyStackTrace("coroutine stack", coroutineStackTrace, expected.coroutineStackTrace) + verifyStackTrace("thread stack", threadStackTrace, expected.threadStackTrace) + } + + private fun verifyStackTrace(traceName: String, actualStackTrace: List, expectedStackTrace: List) { + // It is possible there are more stack frames in a dump than we check + for ((ix, expectedLine) in expectedStackTrace.withIndex()) { + val actualLine = actualStackTrace[ix] + assertEquals( + expectedLine, actualLine, + "Following lines from $traceName are not matched:\n\t- ${expectedLine}\n\t+ ${actualLine}\nActual dump:\n$originDump\n\n" + ) + } + } +} + +private data class CoroutineDumpHeader( + val name: String?, + val className: String, + val state: String, +) { + companion object { + /** + * Parses following strings: + * + * - Coroutine "coroutine#10":DeferredCoroutine{Active}@66d87651, state: RUNNING + * - Coroutine DeferredCoroutine{Active}@66d87651, state: RUNNING + * + * into: + * + * - `CoroutineDumpHeader(name = "coroutine", className = "DeferredCoroutine", state = "RUNNING")` + * - `CoroutineDumpHeader(name = null, className = "DeferredCoroutine", state = "RUNNING")` + */ + fun parse(header: String): CoroutineDumpHeader { + val (identFull, stateFull) = header.split(", ", limit = 2) + val nameAndClassName = identFull.removePrefix("Coroutine ").split('@', limit = 2)[0] + val (name, className) = nameAndClassName.split(':', limit = 2).let { parts -> + val (quotedName, classNameWithState) = if (parts.size == 1) { + null to parts[0] + } else { + parts[0] to parts[1] + } + val name = quotedName?.removeSurrounding("\"")?.split('#', limit = 2)?.get(0) + val className = classNameWithState.replace("\\{.*\\}".toRegex(), "") + name to className + } + val state = stateFull.removePrefix("state: ") + return CoroutineDumpHeader(name, className, state) + } + } +} + +public fun verifyDump(vararg expectedTraces: String, ignoredCoroutine: String? = null) { + val baos = ByteArrayOutputStream() + DebugProbes.dumpCoroutines(PrintStream(baos)) + val wholeDump = baos.toString() + val traces = wholeDump.split("\n\n") + assertTrue(traces[0].startsWith("Coroutines dump")) + + val dumps = traces + // Drop "Coroutine dump" line + .drop(1) + // Parse dumps and filter out ignored coroutines + .mapNotNull { trace -> + val dump = CoroutineDump.parse(trace, { + removeJavaUtilConcurrentTraces(cleanBlockHoundTraces(it)) + }) + if (dump.header.className == ignoredCoroutine) { + null + } else { + dump + } + } + + assertEquals(expectedTraces.size, dumps.size) + dumps.zip(expectedTraces.map { CoroutineDump.parse(it, ::removeJavaUtilConcurrentTraces) }) + .forEach { (dump, expectedDump) -> + dump.verify(expectedDump) + } +} + +public fun String.trimPackage() = replace("kotlinx.coroutines.debug.", "") + +public fun verifyPartialDump(createdCoroutinesCount: Int, vararg frames: String) { + val baos = ByteArrayOutputStream() + DebugProbes.dumpCoroutines(PrintStream(baos)) + val dump = baos.toString() + val trace = dump.split("\n\n") + val matches = frames.all { frame -> + trace.any { tr -> tr.contains(frame) } + } + + assertEquals(createdCoroutinesCount, DebugProbes.dumpCoroutinesInfo().size) + assertTrue(matches) +} diff --git a/kotlinx-coroutines-debug/test/StandardBuildersDebugTest.kt b/kotlinx-coroutines-debug/test/StandardBuildersDebugTest.kt new file mode 100644 index 0000000000..1a3076ebdf --- /dev/null +++ b/kotlinx-coroutines-debug/test/StandardBuildersDebugTest.kt @@ -0,0 +1,48 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import org.junit.Test +import kotlin.test.* + +class StandardBuildersDebugTest : DebugTestBase() { + + @Test + fun testBuildersAreMissingFromDumpByDefault() = runTest { + val (b1, b2) = createBuilders() + + val coroutines = DebugProbes.dumpCoroutinesInfo() + assertEquals(1, coroutines.size) + assertTrue { b1.hasNext() && b2.hasNext() } // Don't let GC collect our coroutines until the test is complete + } + + @Test + fun testBuildersCanBeEnabled() = runTest { + try { + DebugProbes.ignoreCoroutinesWithEmptyContext = false + val (b1, b2) = createBuilders() + val coroutines = DebugProbes.dumpCoroutinesInfo() + assertEquals(3, coroutines.size) + assertTrue { b1.hasNext() && b2.hasNext() } // Don't let GC collect our coroutines until the test is complete + } finally { + DebugProbes.ignoreCoroutinesWithEmptyContext = true + } + } + + private fun createBuilders(): Pair, Iterator> { + val fromSequence = sequence { + while (true) { + yield(1) + } + }.iterator() + + val fromIterator = iterator { + while (true) { + yield(1) + } + } + // Start coroutines + fromIterator.hasNext() + fromSequence.hasNext() + return fromSequence to fromIterator + } +} diff --git a/kotlinx-coroutines-debug/test/StartModeProbesTest.kt b/kotlinx-coroutines-debug/test/StartModeProbesTest.kt new file mode 100644 index 0000000000..d2e25523ca --- /dev/null +++ b/kotlinx-coroutines-debug/test/StartModeProbesTest.kt @@ -0,0 +1,151 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.junit.Test +import kotlin.test.* + +class StartModeProbesTest : DebugTestBase() { + + @Test + fun testUndispatched() = runTest { + expect(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + undispatchedSleeping() + assertTrue(true) + } + + yield() + expect(3) + verifyPartialDump(2, "StartModeProbesTest.undispatchedSleeping") + job.cancelAndJoin() + verifyPartialDump(1, "StartModeProbesTest\$testUndispatched") + finish(4) + } + + private suspend fun undispatchedSleeping() { + delay(Long.MAX_VALUE) + assertTrue(true) + } + + @Test + fun testWithTimeoutWithUndispatched() = runTest { + expect(1) + val job = launchUndispatched() + + yield() + expect(3) + verifyPartialDump( + 2, + "StartModeProbesTest\$launchUndispatched\$1.invokeSuspend", + "StartModeProbesTest.withTimeoutHelper", + "StartModeProbesTest\$withTimeoutHelper\$2.invokeSuspend" + ) + job.cancelAndJoin() + verifyPartialDump(1, "StartModeProbesTest\$testWithTimeoutWithUndispatched") + finish(4) + } + + private fun CoroutineScope.launchUndispatched(): Job { + return launch(start = CoroutineStart.UNDISPATCHED) { + withTimeoutHelper() + assertTrue(true) + } + } + + private suspend fun withTimeoutHelper() { + withTimeout(Long.MAX_VALUE) { + expect(2) + delay(Long.MAX_VALUE) + } + + assertTrue(true) + } + + @Test + fun testWithTimeout() = runTest { + withTimeout(Long.MAX_VALUE) { + testActiveDump( + false, + "StartModeProbesTest\$testWithTimeout\$1.invokeSuspend", + "state: RUNNING" + ) + } + } + + @Test + fun testWithTimeoutAfterYield() = runTest { + withTimeout(Long.MAX_VALUE) { + testActiveDump( + true, + "StartModeProbesTest\$testWithTimeoutAfterYield\$1.invokeSuspend", + "StartModeProbesTest\$testWithTimeoutAfterYield\$1\$1.invokeSuspend", + "StartModeProbesTest.testActiveDump", + "state: RUNNING" + ) + } + } + + private suspend fun testActiveDump(shouldYield: Boolean, vararg expectedFrames: String) { + if (shouldYield) yield() + verifyPartialDump(1, *expectedFrames) + assertTrue(true) + } + + @Test + fun testWithTailCall() = runTest { + expect(1) + val job = tailCallMethod() + yield() + expect(3) + verifyPartialDump(2, "StartModeProbesTest\$launchFromTailCall\$2") + job.cancelAndJoin() + verifyPartialDump(1, "StartModeProbesTest\$testWithTailCall") + finish(4) + } + + private suspend fun CoroutineScope.tailCallMethod(): Job = launchFromTailCall() + private suspend fun CoroutineScope.launchFromTailCall(): Job = launch { + expect(2) + delay(Long.MAX_VALUE) + } + + @Test + fun testCoroutineScope() = runTest { + expect(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + runScope() + } + + yield() + expect(3) + verifyPartialDump( + 2, + "StartModeProbesTest\$runScope\$2.invokeSuspend", + "StartModeProbesTest\$testCoroutineScope\$1\$job\$1.invokeSuspend") + job.cancelAndJoin() + finish(4) + } + + private suspend fun runScope() { + coroutineScope { + expect(2) + delay(Long.MAX_VALUE) + } + } + + @Test + fun testLazy() = runTest({ it is CancellationException }) { + launch(start = CoroutineStart.LAZY) { } + actor(start = CoroutineStart.LAZY) { } + @Suppress("DEPRECATION_ERROR") + broadcast(start = CoroutineStart.LAZY) { } + async(start = CoroutineStart.LAZY) { 1 } + verifyPartialDump(5, "BlockingCoroutine", + "LazyStandaloneCoroutine", "LazyActorCoroutine", + "LazyBroadcastCoroutine", "LazyDeferredCoroutine") + cancel() + } +} diff --git a/kotlinx-coroutines-debug/test/TestRuleExample.kt b/kotlinx-coroutines-debug/test/TestRuleExample.kt new file mode 100644 index 0000000000..df5473e084 --- /dev/null +++ b/kotlinx-coroutines-debug/test/TestRuleExample.kt @@ -0,0 +1,38 @@ +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.junit4.* +import org.junit.* + +@Ignore // do not run it on CI +class TestRuleExample { + + @JvmField + @Rule + public val timeout = CoroutinesTimeout.seconds(1) + + private suspend fun someFunctionDeepInTheStack() { + withContext(Dispatchers.IO) { + delay(Long.MAX_VALUE) + println("This line is never executed") + } + + println("This line is never executed as well") + } + + @Test + fun hangingTest() = runBlocking { + val job = launch { + someFunctionDeepInTheStack() + } + + println("Doing some work...") + job.join() + } + + @Test + fun successfulTest() = runBlocking { + launch { + delay(10) + }.join() + } + +} diff --git a/kotlinx-coroutines-debug/test/ToStringTest.kt b/kotlinx-coroutines-debug/test/ToStringTest.kt new file mode 100644 index 0000000000..ff476a7f9a --- /dev/null +++ b/kotlinx-coroutines-debug/test/ToStringTest.kt @@ -0,0 +1,139 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import org.junit.* +import org.junit.Test +import java.io.* +import kotlin.coroutines.* +import kotlin.test.* + +class ToStringTest : DebugTestBase() { + + private suspend fun CoroutineScope.launchNestedScopes(): Job { + return launch { + expect(1) + coroutineScope { + expect(2) + launchDelayed() + + supervisorScope { + expect(3) + launchDelayed() + } + } + } + } + + private fun CoroutineScope.launchDelayed(): Job { + return launch { + delay(Long.MAX_VALUE) + } + } + + @Test + fun testPrintHierarchyWithScopes() = runBlocking { + val tab = '\t' + val expectedString = """ + "coroutine":StandaloneCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchNestedScopes$2$1.invokeSuspend(ToStringTest.kt) + $tab"coroutine":StandaloneCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchDelayed$1.invokeSuspend(ToStringTest.kt) + $tab"coroutine":StandaloneCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchDelayed$1.invokeSuspend(ToStringTest.kt) + """.trimIndent() + + val job = launchNestedScopes() + try { + repeat(5) { yield() } + val expected = expectedString.trimStackTrace().trimPackage() + expect(4) + assertEquals(expected, DebugProbes.jobToString(job).trimEnd().trimStackTrace().trimPackage()) + assertEquals(expected, DebugProbes.scopeToString(CoroutineScope(job)).trimEnd().trimStackTrace().trimPackage()) + } finally { + finish(5) + job.cancelAndJoin() + } + } + + @Test + fun testCompletingHierarchy() = runBlocking { + val tab = '\t' + val expectedString = """ + "coroutine#2":StandaloneCoroutine{Completing} + $tab"foo#3":DeferredCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchHierarchy${'$'}1${'$'}1.invokeSuspend(ToStringTest.kt:30) + $tab"coroutine#4":ActorCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchHierarchy${'$'}1${'$'}2${'$'}1.invokeSuspend(ToStringTest.kt:40) + $tab$tab"coroutine#5":StandaloneCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchHierarchy${'$'}1${'$'}2${'$'}job$1.invokeSuspend(ToStringTest.kt:37) + """.trimIndent() + + checkHierarchy(isCompleting = true, expectedString = expectedString) + } + + @Test + fun testActiveHierarchy() = runBlocking { + val tab = '\t' + val expectedString = """ + "coroutine#2":StandaloneCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchHierarchy${'$'}1.invokeSuspend(ToStringTest.kt:94) + $tab"foo#3":DeferredCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchHierarchy${'$'}1${'$'}1.invokeSuspend(ToStringTest.kt:30) + $tab"coroutine#4":ActorCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchHierarchy${'$'}1${'$'}2${'$'}1.invokeSuspend(ToStringTest.kt:40) + $tab$tab"coroutine#5":StandaloneCoroutine{Active}, continuation is SUSPENDED at line ToStringTest${'$'}launchHierarchy${'$'}1${'$'}2${'$'}job$1.invokeSuspend(ToStringTest.kt:37) + """.trimIndent() + checkHierarchy(isCompleting = false, expectedString = expectedString) + } + + private suspend fun CoroutineScope.checkHierarchy(isCompleting: Boolean, expectedString: String) { + val root = launchHierarchy(isCompleting) + repeat(4) { yield() } + val expected = expectedString.trimStackTrace().trimPackage() + expect(6) + assertEquals(expected, DebugProbes.jobToString(root).trimEnd().trimStackTrace().trimPackage()) + assertEquals(expected, DebugProbes.scopeToString(CoroutineScope(root)).trimEnd().trimStackTrace().trimPackage()) + assertEquals(expected, printToString { DebugProbes.printScope(CoroutineScope(root), it) }.trimEnd().trimStackTrace().trimPackage()) + assertEquals(expected, printToString { DebugProbes.printJob(root, it) }.trimEnd().trimStackTrace().trimPackage()) + + root.cancelAndJoin() + finish(7) + } + + private fun CoroutineScope.launchHierarchy(isCompleting: Boolean): Job { + return launch { + expect(1) + async(CoroutineName("foo")) { + expect(2) + delay(Long.MAX_VALUE) + } + + actor { + expect(3) + val job = launch { + expect(4) + delay(Long.MAX_VALUE) + } + + withContext(wrapperDispatcher(coroutineContext)) { + expect(5) + job.join() + } + } + + if (!isCompleting) { + delay(Long.MAX_VALUE) + } + } + } + + private fun wrapperDispatcher(context: CoroutineContext): CoroutineContext { + val dispatcher = context[ContinuationInterceptor] as CoroutineDispatcher + return object : CoroutineDispatcher() { + override fun dispatch(context: CoroutineContext, block: Runnable) { + dispatcher.dispatch(context, block) + } + } + } + + private inline fun printToString(block: (PrintStream) -> Unit): String { + val baos = ByteArrayOutputStream() + val ps = PrintStream(baos) + block(ps) + ps.close() + return baos.toString() + } +} diff --git a/kotlinx-coroutines-debug/test/WithContextUndispatchedTest.kt b/kotlinx-coroutines-debug/test/WithContextUndispatchedTest.kt new file mode 100644 index 0000000000..47274d865a --- /dev/null +++ b/kotlinx-coroutines-debug/test/WithContextUndispatchedTest.kt @@ -0,0 +1,65 @@ +package kotlinx.coroutines.debug + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.junit.* + +// Test four our internal optimization "withContextUndispatched" +class WithContextUndispatchedTest : DebugTestBase() { + + @Test + fun testZip() = runTest { + val f1 = flowOf("a") + val f2 = flow { + nestedEmit() + yield() + } + f1.zip(f2) { i, j -> i + j }.collect { + bar(false) + } + } + + private suspend fun FlowCollector.nestedEmit() { + emit(1) + emit(2) + } + + @Test + fun testUndispatchedFlowOn() = runTest { + val flow = flowOf(1, 2, 3).flowOn(CoroutineName("...")) + flow.collect { + bar(true) + } + } + + @Test + fun testUndispatchedFlowOnWithNestedCaller() = runTest { + val flow = flow { + nestedEmit() + }.flowOn(CoroutineName("...")) + flow.collect { + bar(true) + } + } + + private suspend fun bar(forFlowOn: Boolean) { + yield() + if (forFlowOn) { + verifyFlowOn() + } else { + verifyZip() + } + yield() + } + + private suspend fun verifyFlowOn() { + yield() // suspend + verifyPartialDump(1, "verifyFlowOn", "bar") + } + + private suspend fun verifyZip() { + yield() // suspend + verifyPartialDump(2, "verifyZip", "bar", "nestedEmit") + } +} diff --git a/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutDisabledTracesTest.kt b/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutDisabledTracesTest.kt new file mode 100644 index 0000000000..8a24313926 --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutDisabledTracesTest.kt @@ -0,0 +1,44 @@ +package kotlinx.coroutines.debug.junit4 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.runners.model.* + +class CoroutinesTimeoutDisabledTracesTest : TestBase(disableOutCheck = true) { + + @Rule + @JvmField + public val validation = TestFailureValidation( + 500, true, false, + TestResultSpec( + "hangingTest", expectedOutParts = listOf( + "Coroutines dump", + "Test hangingTest timed out after 500 milliseconds", + "BlockingCoroutine{Active}", + "at kotlinx.coroutines.debug.junit4.CoroutinesTimeoutDisabledTracesTest.hangForever", + "at kotlinx.coroutines.debug.junit4.CoroutinesTimeoutDisabledTracesTest.waitForHangJob" + ), + notExpectedOutParts = listOf("_COROUTINE._CREATION._"), + error = TestTimedOutException::class.java + ) + ) + + private val job = GlobalScope.launch(Dispatchers.Unconfined) { hangForever() } + + private suspend fun hangForever() { + suspendCancellableCoroutine { } + expectUnreached() + } + + @Test + fun hangingTest() = runBlocking { + waitForHangJob() + expectUnreached() + } + + private suspend fun waitForHangJob() { + job.join() + expectUnreached() + } +} diff --git a/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutEagerTest.kt b/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutEagerTest.kt new file mode 100644 index 0000000000..458d0eed8b --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutEagerTest.kt @@ -0,0 +1,43 @@ +package kotlinx.coroutines.debug.junit4 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.runners.model.* + +class CoroutinesTimeoutEagerTest : TestBase(disableOutCheck = true) { + + @Rule + @JvmField + public val validation = TestFailureValidation( + 500, true, true, + TestResultSpec( + "hangingTest", expectedOutParts = listOf( + "Coroutines dump", + "Test hangingTest timed out after 500 milliseconds", + "BlockingCoroutine{Active}", + "runBlocking", + "at kotlinx.coroutines.debug.junit4.CoroutinesTimeoutEagerTest.hangForever", + "at kotlinx.coroutines.debug.junit4.CoroutinesTimeoutEagerTest.waitForHangJob"), + error = TestTimedOutException::class.java) + ) + + private val job = GlobalScope.launch(Dispatchers.Unconfined) { hangForever() } + + private suspend fun hangForever() { + suspendCancellableCoroutine { } + expectUnreached() + } + + @Test + fun hangingTest() = runBlocking { + waitForHangJob() + expectUnreached() + } + + private suspend fun waitForHangJob() { + job.join() + expectUnreached() + } + +} diff --git a/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutTest.kt b/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutTest.kt new file mode 100644 index 0000000000..9a429d5b3d --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit4/CoroutinesTimeoutTest.kt @@ -0,0 +1,52 @@ +package kotlinx.coroutines.debug.junit4 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.runners.model.* + +class CoroutinesTimeoutTest : TestBase(disableOutCheck = true) { + + @Rule + @JvmField + public val validation = TestFailureValidation( + 1000, false, true, + TestResultSpec("throwingTest", error = RuntimeException::class.java), + TestResultSpec("successfulTest"), + TestResultSpec( + "hangingTest", expectedOutParts = listOf( + "Coroutines dump", + "Test hangingTest timed out after 1 seconds", + "BlockingCoroutine{Active}", + "runBlocking", + "at kotlinx.coroutines.debug.junit4.CoroutinesTimeoutTest.suspendForever", + "at kotlinx.coroutines.debug.junit4.CoroutinesTimeoutTest\$hangingTest\$1.invokeSuspend"), + notExpectedOutParts = listOf("delay", "throwingTest"), + error = TestTimedOutException::class.java) + ) + + @Test + fun hangingTest() = runBlocking { + suspendForever() + expectUnreached() + } + + private suspend fun suspendForever() { + delay(Long.MAX_VALUE) + expectUnreached() + } + + @Test + fun throwingTest() = runBlocking { + throw RuntimeException() + } + + @Test + fun successfulTest() = runBlocking { + val job = launch { + yield() + } + + job.join() + } +} diff --git a/kotlinx-coroutines-debug/test/junit4/TestFailureValidation.kt b/kotlinx-coroutines-debug/test/junit4/TestFailureValidation.kt new file mode 100644 index 0000000000..5bbb84c788 --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit4/TestFailureValidation.kt @@ -0,0 +1,105 @@ +package kotlinx.coroutines.debug.junit4 + +import kotlinx.coroutines.debug.* +import org.junit.rules.* +import org.junit.runner.* +import org.junit.runners.model.* +import java.io.* +import kotlin.test.* + +internal fun TestFailureValidation( + timeoutMs: Long, + cancelOnTimeout: Boolean, + creationStackTraces: Boolean, + vararg specs: TestResultSpec +): RuleChain = + RuleChain + .outerRule(TestFailureValidation(specs.associateBy { it.testName })) + .around( + CoroutinesTimeout( + timeoutMs, + cancelOnTimeout, + creationStackTraces + ) + ) + +/** + * Rule that captures test result, serr and sout and validates it against provided [testsSpec] + */ +internal class TestFailureValidation(private val testsSpec: Map) : TestRule { + + companion object { + init { + DebugProbes.sanitizeStackTraces = false + } + } + override fun apply(base: Statement, description: Description): Statement { + return TestFailureStatement(base, description) + } + + inner class TestFailureStatement(private val test: Statement, private val description: Description) : Statement() { + private lateinit var sout: PrintStream + private lateinit var serr: PrintStream + private val capturedOut = ByteArrayOutputStream() + + override fun evaluate() { + try { + replaceOut() + test.evaluate() + } catch (e: Throwable) { + validateFailure(e) + return + } finally { + resetOut() + } + + validateSuccess() // To avoid falling into catch + } + + private fun validateSuccess() { + val spec = testsSpec[description.methodName] ?: error("Test spec not found: ${description.methodName}") + require(spec.error == null) { "Expected exception of type ${spec.error}, but test successfully passed" } + + val captured = capturedOut.toString() + assertFalse(captured.contains("Coroutines dump")) + assertTrue(captured.isEmpty(), captured) + } + + private fun validateFailure(e: Throwable) { + val spec = testsSpec[description.methodName] ?: error("Test spec not found: ${description.methodName}") + if (spec.error == null || !spec.error.isInstance(e)) { + throw IllegalStateException("Unexpected failure, expected ${spec.error}, had ${e::class}", e) + } + + if (e !is TestTimedOutException) return + + val captured = capturedOut.toString() + assertTrue(captured.contains("Coroutines dump")) + for (part in spec.expectedOutParts) { + assertTrue(captured.contains(part), "Expected $part to be part of the\n$captured") + } + + for (part in spec.notExpectedOutParts) { + assertFalse(captured.contains(part), "Expected $part not to be part of the\n$captured") + } + } + + private fun replaceOut() { + sout = System.out + serr = System.err + + System.setOut(PrintStream(capturedOut)) + System.setErr(PrintStream(capturedOut)) + } + + private fun resetOut() { + System.setOut(sout) + System.setErr(serr) + } + } +} + +data class TestResultSpec( + val testName: String, val expectedOutParts: List = listOf(), + val notExpectedOutParts: List = listOf(), val error: Class? = null +) diff --git a/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutExtensionTest.kt b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutExtensionTest.kt new file mode 100644 index 0000000000..12298152ce --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutExtensionTest.kt @@ -0,0 +1,117 @@ +package kotlinx.coroutines.debug.junit5 + +import kotlinx.coroutines.* +import org.junit.jupiter.api.Test +import org.junit.jupiter.api.extension.* +import org.junit.jupiter.api.parallel.* + +class CoroutinesTimeoutExtensionTest { + + /** + * Tests that disabling coroutine creation stacktraces in [CoroutinesTimeoutExtension] does lead to them not being + * created. + * + * Adapted from [CoroutinesTimeoutDisabledTracesTest], an identical test for the JUnit4 rule. + * + * This test class is not intended to be run manually. Instead, use [CoroutinesTimeoutTest] as the entry point. + */ + class DisabledStackTracesTest { + @JvmField + @RegisterExtension + internal val timeout = CoroutinesTimeoutExtension(500, true, false) + + private val job = GlobalScope.launch(Dispatchers.Unconfined) { hangForever() } + + private suspend fun hangForever() { + suspendCancellableCoroutine { } + expectUnreached() + } + + @Test + fun hangingTest() = runBlocking { + waitForHangJob() + expectUnreached() + } + + private suspend fun waitForHangJob() { + job.join() + expectUnreached() + } + } + + /** + * Tests that [CoroutinesTimeoutExtension] is installed eagerly and detects the coroutines that were launched before + * any test events start happening. + * + * Adapted from [CoroutinesTimeoutEagerTest], an identical test for the JUnit4 rule. + * + * This test class is not intended to be run manually. Instead, use [CoroutinesTimeoutTest] as the entry point. + */ + class EagerTest { + + @JvmField + @RegisterExtension + internal val timeout = CoroutinesTimeoutExtension(500) + + private val job = GlobalScope.launch(Dispatchers.Unconfined) { hangForever() } + + private suspend fun hangForever() { + suspendCancellableCoroutine { } + expectUnreached() + } + + @Test + fun hangingTest() = runBlocking { + waitForHangJob() + expectUnreached() + } + + private suspend fun waitForHangJob() { + job.join() + expectUnreached() + } + } + + /** + * Tests that [CoroutinesTimeoutExtension] performs sensibly in some simple scenarios. + * + * Adapted from [CoroutinesTimeoutTest], an identical test for the JUnit4 rule. + * + * This test class is not intended to be run manually. Instead, use [CoroutinesTimeoutTest] as the entry point. + */ + class SimpleTest { + + @JvmField + @RegisterExtension + internal val timeout = CoroutinesTimeoutExtension(1000, false, true) + + @Test + fun hangingTest() = runBlocking { + suspendForever() + expectUnreached() + } + + private suspend fun suspendForever() { + delay(Long.MAX_VALUE) + expectUnreached() + } + + @Test + fun throwingTest() = runBlocking { + throw RuntimeException() + } + + @Test + fun successfulTest() = runBlocking { + val job = launch { + yield() + } + + job.join() + } + } +} + +private fun expectUnreached(): Nothing { + error("Should not be reached") +} \ No newline at end of file diff --git a/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutInheritanceTest.kt b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutInheritanceTest.kt new file mode 100644 index 0000000000..4aa90bb74f --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutInheritanceTest.kt @@ -0,0 +1,57 @@ +package kotlinx.coroutines.debug.junit5 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.jupiter.api.* + +/** + * Tests that [CoroutinesTimeout] is inherited. + * + * This test class is not intended to be run manually. Instead, use [CoroutinesTimeoutTest] as the entry point. + */ +class CoroutinesTimeoutInheritanceTest { + + @CoroutinesTimeout(100) + open class Base + + @TestMethodOrder(MethodOrderer.OrderAnnotation::class) + class InheritedWithNoTimeout: Base() { + + @Test + @Order(1) + fun usesBaseClassTimeout() = runBlocking { + delay(1000) + } + + @CoroutinesTimeout(300) + @Test + @Order(2) + fun methodOverridesBaseClassTimeoutWithGreaterTimeout() = runBlocking { + delay(200) + } + + @CoroutinesTimeout(10) + @Test + @Order(3) + fun methodOverridesBaseClassTimeoutWithLesserTimeout() = runBlocking { + delay(50) + } + + } + + @CoroutinesTimeout(300) + class InheritedWithGreaterTimeout : TestBase() { + + @Test + fun classOverridesBaseClassTimeout1() = runBlocking { + delay(200) + } + + @Test + fun classOverridesBaseClassTimeout2() = runBlocking { + delay(400) + } + + } + +} \ No newline at end of file diff --git a/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutMethodTest.kt b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutMethodTest.kt new file mode 100644 index 0000000000..02bcafcd6b --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutMethodTest.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.debug.junit5 + +import kotlinx.coroutines.* +import org.junit.jupiter.api.* + +/** + * Tests usage of [CoroutinesTimeout] on classes and test methods when only methods are annotated. + * + * This test class is not intended to be run manually. Instead, use [CoroutinesTimeoutTest] as the entry point. + */ +@TestMethodOrder(MethodOrderer.OrderAnnotation::class) +class CoroutinesTimeoutMethodTest { + + @Test + @Order(1) + fun noClassTimeout() { + runBlocking { + delay(150) + } + } + + @CoroutinesTimeout(100) + @Test + @Order(2) + fun usesMethodTimeoutWithNoClassTimeout() { + runBlocking { + delay(1000) + } + } + + @CoroutinesTimeout(1000) + @Test + @Order(3) + fun fitsInMethodTimeout() { + runBlocking { + delay(10) + } + } + +} diff --git a/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutNestedTest.kt b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutNestedTest.kt new file mode 100644 index 0000000000..afb51698ed --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutNestedTest.kt @@ -0,0 +1,25 @@ +package kotlinx.coroutines.debug.junit5 + +import kotlinx.coroutines.* +import org.junit.jupiter.api.* + +/** + * This test checks that nested classes correctly recognize the [CoroutinesTimeout] annotation. + * + * This test class is not intended to be run manually. Instead, use [CoroutinesTimeoutTest] as the entry point. + */ +@CoroutinesTimeout(200) +class CoroutinesTimeoutNestedTest { + @Nested + inner class NestedInInherited { + @Test + fun usesOuterClassTimeout() = runBlocking { + delay(1000) + } + + @Test + fun fitsInOuterClassTimeout() = runBlocking { + delay(10) + } + } +} diff --git a/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutSimpleTest.kt b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutSimpleTest.kt new file mode 100644 index 0000000000..01487b76a2 --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutSimpleTest.kt @@ -0,0 +1,57 @@ +package kotlinx.coroutines.debug.junit5 + +import kotlinx.coroutines.* +import org.junit.jupiter.api.* + +/** + * Tests the basic usage of [CoroutinesTimeout] on classes and test methods. + * + * This test class is not intended to be run manually. Instead, use [CoroutinesTimeoutTest] as the entry point. + */ +@TestMethodOrder(MethodOrderer.OrderAnnotation::class) +@CoroutinesTimeout(100) +class CoroutinesTimeoutSimpleTest { + + @Test + @Order(1) + fun usesClassTimeout1() { + runBlocking { + delay(150) + } + } + + @CoroutinesTimeout(1000) + @Test + @Order(2) + fun ignoresClassTimeout() { + runBlocking { + delay(150) + } + } + + @CoroutinesTimeout(200) + @Test + @Order(3) + fun usesMethodTimeout() { + runBlocking { + delay(300) + } + } + + @Test + @Order(4) + fun fitsInClassTimeout() { + runBlocking { + delay(50) + } + } + + @Test + @Order(5) + fun usesClassTimeout2() { + runBlocking { + delay(150) + } + } + +} diff --git a/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutTest.kt b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutTest.kt new file mode 100644 index 0000000000..e504a3c315 --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit5/CoroutinesTimeoutTest.kt @@ -0,0 +1,166 @@ +package kotlinx.coroutines.debug.junit5 + +import org.assertj.core.api.* +import org.junit.Ignore +import org.junit.Assert.* +import org.junit.Test +import org.junit.platform.engine.* +import org.junit.platform.engine.discovery.DiscoverySelectors.* +import org.junit.platform.testkit.engine.* +import org.junit.platform.testkit.engine.EventConditions.* +import java.io.* + +// note that these tests are run using JUnit4 in order not to mix the testing systems. +class CoroutinesTimeoutTest { + + // This test is ignored because it just checks an example. + @Test + @Ignore + fun testRegisterExtensionExample() { + val capturedOut = ByteArrayOutputStream() + eventsForSelector(selectClass(RegisterExtensionExample::class.java), capturedOut) + .testTimedOut("testThatHangs", 5000) + } + + @Test + fun testCoroutinesTimeoutSimple() { + val capturedOut = ByteArrayOutputStream() + eventsForSelector(selectClass(CoroutinesTimeoutSimpleTest::class.java), capturedOut) + .testFinishedSuccessfully("ignoresClassTimeout") + .testFinishedSuccessfully("fitsInClassTimeout") + .testTimedOut("usesClassTimeout1", 100) + .testTimedOut("usesMethodTimeout", 200) + .testTimedOut("usesClassTimeout2", 100) + assertEquals(capturedOut.toString(), 3, countDumps(capturedOut)) + } + + @Test + fun testCoroutinesTimeoutMethod() { + val capturedOut = ByteArrayOutputStream() + eventsForSelector(selectClass(CoroutinesTimeoutMethodTest::class.java), capturedOut) + .testFinishedSuccessfully("fitsInMethodTimeout") + .testFinishedSuccessfully("noClassTimeout") + .testTimedOut("usesMethodTimeoutWithNoClassTimeout", 100) + assertEquals(capturedOut.toString(), 1, countDumps(capturedOut)) + } + + @Test + fun testCoroutinesTimeoutNested() { + val capturedOut = ByteArrayOutputStream() + eventsForSelector(selectClass(CoroutinesTimeoutNestedTest::class.java), capturedOut) + .testFinishedSuccessfully("fitsInOuterClassTimeout") + .testTimedOut("usesOuterClassTimeout", 200) + assertEquals(capturedOut.toString(), 1, countDumps(capturedOut)) + } + + @Test + fun testCoroutinesTimeoutInheritanceWithNoTimeoutInDerived() { + val capturedOut = ByteArrayOutputStream() + eventsForSelector(selectClass(CoroutinesTimeoutInheritanceTest.InheritedWithNoTimeout::class.java), capturedOut) + .testFinishedSuccessfully("methodOverridesBaseClassTimeoutWithGreaterTimeout") + .testTimedOut("usesBaseClassTimeout", 100) + .testTimedOut("methodOverridesBaseClassTimeoutWithLesserTimeout", 10) + assertEquals(capturedOut.toString(), 2, countDumps(capturedOut)) + } + + @Test + fun testCoroutinesTimeoutInheritanceWithGreaterTimeoutInDerived() { + val capturedOut = ByteArrayOutputStream() + eventsForSelector( + selectClass(CoroutinesTimeoutInheritanceTest.InheritedWithGreaterTimeout::class.java), + capturedOut + ) + .testFinishedSuccessfully("classOverridesBaseClassTimeout1") + .testTimedOut("classOverridesBaseClassTimeout2", 300) + assertEquals(capturedOut.toString(), 1, countDumps(capturedOut)) + } + + /* Currently there's no ability to replicate [TestFailureValidation] as is for JUnit5: + https://github.com/junit-team/junit5/issues/506. So, the test mechanism is more ad-hoc. */ + + @Test + fun testCoroutinesTimeoutExtensionDisabledTraces() { + val capturedOut = ByteArrayOutputStream() + eventsForSelector(selectClass(CoroutinesTimeoutExtensionTest.DisabledStackTracesTest::class.java), capturedOut) + .testTimedOut("hangingTest", 500) + assertEquals(false, capturedOut.toString().contains("Coroutine creation stacktrace")) + assertEquals(capturedOut.toString(), 1, countDumps(capturedOut)) + } + + @Test + fun testCoroutinesTimeoutExtensionEager() { + val capturedOut = ByteArrayOutputStream() + eventsForSelector(selectClass(CoroutinesTimeoutExtensionTest.EagerTest::class.java), capturedOut) + .testTimedOut("hangingTest", 500) + for (expectedPart in listOf("hangForever", "waitForHangJob", "BlockingCoroutine{Active}")) { + assertEquals(expectedPart, true, capturedOut.toString().contains(expectedPart)) + } + assertEquals(capturedOut.toString(), 1, countDumps(capturedOut)) + } + + @Test + fun testCoroutinesTimeoutExtensionSimple() { + val capturedOut = ByteArrayOutputStream() + eventsForSelector(selectClass(CoroutinesTimeoutExtensionTest.SimpleTest::class.java), capturedOut) + .testFinishedSuccessfully("successfulTest") + .testTimedOut("hangingTest", 1000) + .haveExactly(1, event( + test("throwingTest"), + finishedWithFailure(Condition({ it is RuntimeException}, "is RuntimeException")) + )) + for (expectedPart in listOf("suspendForever", "invokeSuspend", "BlockingCoroutine{Active}")) { + assertEquals(expectedPart, true, capturedOut.toString().contains(expectedPart)) + } + for (nonExpectedPart in listOf("delay", "throwingTest")) { + assertEquals(nonExpectedPart, false, capturedOut.toString().contains(nonExpectedPart)) + } + assertEquals(capturedOut.toString(), 1, countDumps(capturedOut)) + } +} + +private fun eventsForSelector(selector: DiscoverySelector, capturedOut: OutputStream): ListAssert { + val systemOut: PrintStream = System.out + val systemErr: PrintStream = System.err + return try { + System.setOut(PrintStream(capturedOut)) + System.setErr(PrintStream(capturedOut)) + EngineTestKit.engine("junit-jupiter") + .selectors(selector) + .execute() + .testEvents() + .assertThatEvents() + } finally { + System.setOut(systemOut) + System.setErr(systemErr) + } +} + +private fun ListAssert.testFinishedSuccessfully(testName: String): ListAssert = + haveExactly(1, event( + test(testName), + finishedSuccessfully() + )) + +private fun ListAssert.testTimedOut(testName: String, after: Long): ListAssert = + haveExactly(1, event( + test(testName), + finishedWithFailure(Condition({ it is CoroutinesTimeoutException && it.timeoutMs == after }, + "is CoroutinesTimeoutException($after)")) + )) + +/** Counts the number of occurrences of "Coroutines dump" in [capturedOut] */ +private fun countDumps(capturedOut: ByteArrayOutputStream): Int { + var result = 0 + val outStr = capturedOut.toString() + val header = "Coroutines dump" + var i = 0 + while (i < outStr.length - header.length) { + if (outStr.substring(i, i + header.length) == header) { + result += 1 + i += header.length + } else { + i += 1 + } + } + return result +} \ No newline at end of file diff --git a/kotlinx-coroutines-debug/test/junit5/RegisterExtensionExample.kt b/kotlinx-coroutines-debug/test/junit5/RegisterExtensionExample.kt new file mode 100644 index 0000000000..10411b7144 --- /dev/null +++ b/kotlinx-coroutines-debug/test/junit5/RegisterExtensionExample.kt @@ -0,0 +1,16 @@ +package kotlinx.coroutines.debug.junit5 + +import kotlinx.coroutines.* +import org.junit.jupiter.api.* +import org.junit.jupiter.api.extension.* + +class RegisterExtensionExample { + @JvmField + @RegisterExtension + internal val timeout = CoroutinesTimeoutExtension.seconds(5) + + @Test + fun testThatHangs() = runBlocking { + delay(Long.MAX_VALUE) // somewhere deep in the stack + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-jdk8/README.md b/kotlinx-coroutines-jdk8/README.md deleted file mode 100644 index 988459a973..0000000000 --- a/kotlinx-coroutines-jdk8/README.md +++ /dev/null @@ -1,33 +0,0 @@ -# Module kotlinx-coroutines-jdk8 - -Additional libraries for JDK8 (or Android API level 24). - -Coroutine builders: - -| **Name** | **Result** | **Scope** | **Description** -| -------- | ---------- | ---------- | --------------- -| [future] | [CompletableFuture][java.util.concurrent.CompletableFuture] | [CoroutineScope] | Returns a single value with the future result - -Extension functions: - -| **Name** | **Description** -| -------- | --------------- -| [CompletableFuture.await][java.util.concurrent.CompletableFuture.await] | Awaits for completion of the future -| [Deferred.asCompletableFuture][kotlinx.coroutines.experimental.Deferred.asCompletableFuture] | Converts a deferred value to the future - -# Package kotlinx.coroutines.experimental.future - -Additional libraries for JDK8 [CompletableFuture][java.util.concurrent.CompletableFuture]. - - - - -[CoroutineScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-scope/index.html - - - -[future]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.experimental.future/future.html -[java.util.concurrent.CompletableFuture]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.experimental.future/java.util.concurrent.-completable-future/index.html -[java.util.concurrent.CompletableFuture.await]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.experimental.future/java.util.concurrent.-completable-future/await.html -[kotlinx.coroutines.experimental.Deferred.asCompletableFuture]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-jdk8/kotlinx.coroutines.experimental.future/kotlinx.coroutines.experimental.-deferred/as-completable-future.html - diff --git a/kotlinx-coroutines-jdk8/pom.xml b/kotlinx-coroutines-jdk8/pom.xml deleted file mode 100644 index 87d9c9e6b9..0000000000 --- a/kotlinx-coroutines-jdk8/pom.xml +++ /dev/null @@ -1,45 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - - - kotlinx-coroutines-jdk8 - jar - - - src/main/kotlin - src/test/kotlin - - - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - compile - - - - diff --git a/kotlinx-coroutines-jdk8/src/main/kotlin/kotlinx/coroutines/experimental/future/Future.kt b/kotlinx-coroutines-jdk8/src/main/kotlin/kotlinx/coroutines/experimental/future/Future.kt deleted file mode 100644 index d95a456209..0000000000 --- a/kotlinx-coroutines-jdk8/src/main/kotlin/kotlinx/coroutines/experimental/future/Future.kt +++ /dev/null @@ -1,110 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.future - -import kotlinx.coroutines.experimental.* -import java.util.concurrent.CompletableFuture -import kotlin.coroutines.experimental.Continuation -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Starts new coroutine and returns its results an an implementation of [CompletableFuture]. - * This coroutine builder uses [CommonPool] context by default and is conceptually similar to [CompletableFuture.supplyAsync]. - * - * The running coroutine is cancelled when the resulting future is cancelled or otherwise completed. - * If the [context] for the new coroutine is explicitly specified, then it must include [CoroutineDispatcher] element. - * See [CoroutineDispatcher] for the standard [context] implementations that are provided by `kotlinx.coroutines`. - * The specified context is added to the context of the parent running coroutine (if any) inside which this function - * is invoked. The [Job] of the resulting coroutine is a child of the job of the parent coroutine (if any). - * - * See [newCoroutineContext] for a description of debugging facilities that are available for newly created coroutine. - */ -public fun future(context: CoroutineContext = CommonPool, block: suspend () -> T): CompletableFuture { - val newContext = newCoroutineContext(CommonPool + context) - val job = Job(newContext[Job]) - val future = CompletableFutureCoroutine(newContext + job) - job.cancelFutureOnCompletion(future) - future.whenComplete { _, exception -> job.cancel(exception) } - block.startCoroutine(future) - return future -} - - -/** - * Converts this deferred value to the instance of [CompletableFuture]. - * The deferred value is cancelled when the resulting future is cancelled or otherwise completed. - * @suppress: **Deprecated**: Renamed to [asCompletableFuture] - */ -@Deprecated("Renamed to `asCompletableFuture`", - replaceWith = ReplaceWith("asCompletableFuture()")) -public fun Deferred.toCompletableFuture(): CompletableFuture = asCompletableFuture() - -/** - * Converts this deferred value to the instance of [CompletableFuture]. - * The deferred value is cancelled when the resulting future is cancelled or otherwise completed. - */ -public fun Deferred.asCompletableFuture(): CompletableFuture { - val future = CompletableFuture() - future.whenComplete { _, exception -> cancel(exception) } - invokeOnCompletion { - try { - future.complete(getCompleted()) - } catch (exception: Exception) { - future.completeExceptionally(exception) - } - } - return future -} - -/** - * Awaits for completion of the future without blocking a thread. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException] . - */ -public suspend fun CompletableFuture.await(): T { - if (isDone) { - // then only way to get unwrapped exception from the CompletableFuture... - var result: T? = null - var exception: Throwable? = null - whenComplete { r, e -> - result = r - exception = e - } - if (exception != null) throw exception!! - return result as T - } - return suspendCancellableCoroutine { cont: CancellableContinuation -> - val completionFuture = whenComplete { result, exception -> - if (exception == null) // the future has been completed normally - cont.resume(result) - else // the future has completed with an exception - cont.resumeWithException(exception) - } - cont.cancelFutureOnCompletion(completionFuture) - Unit - } -} - -private class CompletableFutureCoroutine( - override val context: CoroutineContext -) : CompletableFuture(), Continuation { - override fun resume(value: T) { complete(value) } - override fun resumeWithException(exception: Throwable) { completeExceptionally(exception) } -} diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/CancelFuture-example.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/examples/CancelFuture-example.kt deleted file mode 100644 index d80fb893d5..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/CancelFuture-example.kt +++ /dev/null @@ -1,42 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import kotlinx.coroutines.experimental.delay -import kotlinx.coroutines.experimental.future.future - -fun main(args: Array) { - val f = future { - try { - log("Started f") - delay(500) - log("Slept 500 ms #1") - delay(500) - log("Slept 500 ms #2") - delay(500) - log("Slept 500 ms #3") - delay(500) - log("Slept 500 ms #4") - delay(500) - log("Slept 500 ms #5") - } catch(e: Exception) { - log("Aborting because of $e") - } - } - Thread.sleep(1200) - f.cancel(false) -} diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/ExplicitJob-example.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/examples/ExplicitJob-example.kt deleted file mode 100644 index fabf947371..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/ExplicitJob-example.kt +++ /dev/null @@ -1,49 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.delay -import kotlinx.coroutines.experimental.future.future -import java.util.concurrent.CancellationException - -fun main(args: Array) { - val job = Job() - log("Starting futures f && g") - val f = future(job) { - log("Started f") - delay(500) - log("f should not execute this line") - } - val g = future(job) { - log("Started g") - try { - delay(500) - } finally { - log("g is executing finally!") - } - log("g should not execute this line") - } - log("Started futures f && g... will not wait -- cancel them!!!") - job.cancel(CancellationException("I don't want it")) - check(f.isCancelled) - check(g.isCancelled) - log("f result = ${Try { f.get() }}") - log("g result = ${Try { g.get() }}") - Thread.sleep(1000L) - log("Nothing executed!") -} diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/ToFuture-example.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/examples/ToFuture-example.kt deleted file mode 100644 index 974e1d0b31..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/ToFuture-example.kt +++ /dev/null @@ -1,35 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.async -import kotlinx.coroutines.experimental.delay -import kotlinx.coroutines.experimental.future.asCompletableFuture -import java.util.concurrent.TimeUnit - -fun main(args: Array) { - log("Started") - val deferred = async(CommonPool) { - log("Busy...") - delay(1, TimeUnit.SECONDS) - log("Done...") - 42 - } - val future = deferred.asCompletableFuture() - log("Got ${future.get()}") -} \ No newline at end of file diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/Try.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/examples/Try.kt deleted file mode 100644 index 807a88453a..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/Try.kt +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -public class Try private constructor(private val _value: Any?) { - private class Fail(val exception: Throwable) { - override fun toString(): String = "Failure[$exception]" - } - - public companion object { - public operator fun invoke(block: () -> T): Try = - try { Success(block()) } catch(e: Throwable) { Failure(e) } - public fun Success(value: T) = Try(value) - public fun Failure(exception: Throwable) = Try(Fail(exception)) - } - - @Suppress("UNCHECKED_CAST") - public val value: T get() = if (_value is Fail) throw _value.exception else _value as T - - public val exception: Throwable? get() = (_value as? Fail)?.exception - - override fun toString(): String = _value.toString() -} diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/log.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/examples/log.kt deleted file mode 100644 index a875334b28..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/log.kt +++ /dev/null @@ -1,22 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import java.text.SimpleDateFormat -import java.util.* - -fun log(msg: String) = println("${SimpleDateFormat("yyyyMMdd-HHmmss.sss").format(Date())} [${Thread.currentThread().name}] $msg") \ No newline at end of file diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/simple-example-1.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/examples/simple-example-1.kt deleted file mode 100644 index b3e0dcc8d0..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/simple-example-1.kt +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import kotlinx.coroutines.experimental.future.await -import kotlinx.coroutines.experimental.runBlocking -import java.util.concurrent.CompletableFuture - -fun main(args: Array) { - // Let's assume that we have a future coming from some 3rd party API - val future: CompletableFuture = CompletableFuture.supplyAsync { - Thread.sleep(1000L) // imitate some long-running computation, actually - 42 - } - // now let's launch a coroutine and await for this future inside it - runBlocking { - println("We can do something else, while we are waiting for future...") - println("We've got ${future.await()} from the future!") - } -} \ No newline at end of file diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/simple-example-2.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/examples/simple-example-2.kt deleted file mode 100644 index 533ab6ac2f..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/simple-example-2.kt +++ /dev/null @@ -1,35 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import kotlinx.coroutines.experimental.delay -import kotlinx.coroutines.experimental.future.future -import java.util.concurrent.CompletableFuture -import java.util.concurrent.TimeUnit - -// this function returns a CompletableFuture using Kotlin coroutines -fun supplyTheAnswerAsync(): CompletableFuture = future { - println("We might be doing some asynchronous IO here or something else...") - delay(1, TimeUnit.SECONDS) // just do a non-blocking delay - 42 // The answer! -} - -fun main(args: Array) { - // We can use `supplyTheAnswerAsync` just like any other future-supplier function - val future = supplyTheAnswerAsync() - println("The answer is ${future.get()}") -} \ No newline at end of file diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/simple-example-3.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/examples/simple-example-3.kt deleted file mode 100644 index 734f304c20..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/simple-example-3.kt +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import kotlinx.coroutines.experimental.future.await -import kotlinx.coroutines.experimental.future.future -import java.util.concurrent.CompletableFuture - -fun main(args: Array) { - // this example shows how easy it is to perform multiple async operations with coroutines - val future = future { - (1..5).map { // loops are no problem at all - startLongAsyncOperation(it).await() // suspend while the long method is running - }.joinToString("\n") - } - println("We have a long-running computation in background, let's wait for its result...") - println(future.get()) -} - -fun startLongAsyncOperation(num: Int): CompletableFuture = - CompletableFuture.supplyAsync { - Thread.sleep(1000L) // imitate some long-running computation, actually - "$num" // and return a number converted to string - } diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/withTimeout-example.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/examples/withTimeout-example.kt deleted file mode 100644 index fb0d4ddb8c..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/examples/withTimeout-example.kt +++ /dev/null @@ -1,54 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import kotlinx.coroutines.experimental.CancellationException -import kotlinx.coroutines.experimental.delay -import kotlinx.coroutines.experimental.future.await -import kotlinx.coroutines.experimental.future.future -import kotlinx.coroutines.experimental.withTimeout - -fun main(args: Array) { - fun slow(s: String) = future { - delay(500L) - s - } - - val f = future { - log("Started f") - val a = slow("A").await() - log("a = $a") - withTimeout(1000L) { - val b = slow("B").await() - log("b = $b") - } - try { - withTimeout(750L) { - val c = slow("C").await() - log("c = $c") - val d = slow("D").await() - log("d = $d") - } - } catch (ex: CancellationException) { - log("timed out with $ex") - } - val e = slow("E").await() - log("e = $e") - "done" - } - log("f.get() = ${f.get()}") -} diff --git a/kotlinx-coroutines-jdk8/src/test/kotlin/kotlinx/coroutines/experimental/future/FutureTest.kt b/kotlinx-coroutines-jdk8/src/test/kotlin/kotlinx/coroutines/experimental/future/FutureTest.kt deleted file mode 100644 index ee2c0598d2..0000000000 --- a/kotlinx-coroutines-jdk8/src/test/kotlin/kotlinx/coroutines/experimental/future/FutureTest.kt +++ /dev/null @@ -1,140 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.future - -import kotlinx.coroutines.experimental.CoroutineDispatcher -import org.junit.Test -import java.util.concurrent.CompletableFuture -import java.util.concurrent.ExecutionException -import java.util.concurrent.atomic.AtomicInteger -import kotlin.coroutines.experimental.CoroutineContext -import org.junit.Assert.* - -class FutureTest { - @Test - fun testSimple() { - val future = future { - CompletableFuture.supplyAsync { - "O" - }.await() + "K" - } - - assertEquals("OK", future.get()) - } - - @Test - fun testWaitForCompletion() { - val toAwait = CompletableFuture() - val future = future { - toAwait.await() + "K" - } - - assertFalse(future.isDone) - toAwait.complete("O") - - assertEquals("OK", future.get()) - } - - @Test - fun testDoneFutureCompletedExceptionally() { - val toAwait = CompletableFuture() - toAwait.completeExceptionally(RuntimeException("O")) - val future = future { - try { - toAwait.await() - } catch (e: RuntimeException) { - e.message!! - } + "K" - } - assertEquals("OK", future.get()) - } - - @Test - fun testAwaitedFutureCompletedExceptionally() { - val toAwait = CompletableFuture() - val future = future { - try { - toAwait.await() - } catch (e: RuntimeException) { - e.message!! - } + "K" - } - - assertFalse(future.isDone) - toAwait.completeExceptionally(RuntimeException("O")) - - assertEquals("OK", future.get()) - } - - @Test - fun testExceptionInsideCoroutine() { - val future = future { - if (CompletableFuture.supplyAsync { true }.await()) { - throw IllegalStateException("OK") - } - CompletableFuture.supplyAsync { "fail" }.await() - } - - try { - future.get() - fail("'get' should've throw an exception") - } catch (e: ExecutionException) { - assertTrue(e.cause is IllegalStateException) - assertEquals("OK", e.cause!!.message) - } - } - - @Test - fun testContinuationWrapped() { - val depth = AtomicInteger() - - val future = future(wrapContinuation { - depth.andIncrement - it() - depth.andDecrement - }) { - assertEquals("Part before first suspension must be wrapped", 1, depth.get()) - - val result = - CompletableFuture.supplyAsync { - while (depth.get() > 0) ; - assertEquals("Part inside suspension point should not be wrapped", 0, depth.get()) - "OK" - }.await() - - assertEquals("Part after first suspension should be wrapped", 1, depth.get()) - - CompletableFuture.supplyAsync { - while (depth.get() > 0) ; - assertEquals("Part inside suspension point should not be wrapped", 0, depth.get()) - "ignored" - }.await() - - result - } - - assertEquals("OK", future.get()) - } - - private fun wrapContinuation(wrapper: (() -> Unit) -> Unit): CoroutineDispatcher = object : CoroutineDispatcher() { - override fun dispatch(context: CoroutineContext, block: Runnable) { - wrapper { - block.run() - } - } - } -} diff --git a/kotlinx-coroutines-nio/README.md b/kotlinx-coroutines-nio/README.md deleted file mode 100644 index 9af37a8376..0000000000 --- a/kotlinx-coroutines-nio/README.md +++ /dev/null @@ -1,11 +0,0 @@ -# Module kotlinx-coroutines-nio - -Extensions for asynchronous IO on JDK7+. - -# Package kotlinx.coroutines.experimental.nio - -Extensions for asynchronous IO on JDK7+. - -* `AsynchronousFileChannel` extensions `aLock`, `aRead`, and `aWrite`. -* `AsynchronousServerSocketChannel` extension `aAccept`. -* `AsynchronousSocketChannel` extensions `aConnect`, `aRead`, and `aWrite`. diff --git a/kotlinx-coroutines-nio/pom.xml b/kotlinx-coroutines-nio/pom.xml deleted file mode 100644 index 78b8146571..0000000000 --- a/kotlinx-coroutines-nio/pom.xml +++ /dev/null @@ -1,51 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - - - kotlinx-coroutines-nio - jar - - - src/main/kotlin - src/test/kotlin - - - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - compile - - - org.apache.commons - commons-io - 1.3.2 - test - - - - diff --git a/kotlinx-coroutines-nio/src/main/kotlin/kotlinx/coroutines/experimental/nio/Nio.kt b/kotlinx-coroutines-nio/src/main/kotlin/kotlinx/coroutines/experimental/nio/Nio.kt deleted file mode 100644 index 45d8d18d93..0000000000 --- a/kotlinx-coroutines-nio/src/main/kotlin/kotlinx/coroutines/experimental/nio/Nio.kt +++ /dev/null @@ -1,178 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.nio - -import kotlinx.coroutines.experimental.CancellableContinuation -import kotlinx.coroutines.experimental.CancellationException -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.suspendCancellableCoroutine -import java.net.SocketAddress -import java.nio.ByteBuffer -import java.nio.channels.* -import java.util.concurrent.TimeUnit - -/** - * Performs [AsynchronousFileChannel.lock] without blocking a thread and resumes when asynchronous operation completes. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * *closes the underlying channel* and immediately resumes with [CancellationException]. - */ -suspend fun AsynchronousFileChannel.aLock() = suspendCancellableCoroutine { cont -> - lock(cont, asyncIOHandler()) - closeOnCancel(cont) -} - -/** - * Performs [AsynchronousFileChannel.lock] without blocking a thread and resumes when asynchronous operation completes. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * *closes the underlying channel* and immediately resumes with [CancellationException]. - */ -suspend fun AsynchronousFileChannel.aLock( - position: Long, - size: Long, - shared: Boolean -) = suspendCancellableCoroutine { cont -> - lock(position, size, shared, cont, asyncIOHandler()) - closeOnCancel(cont) -} - -/** - * Performs [AsynchronousFileChannel.read] without blocking a thread and resumes when asynchronous operation completes. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * *closes the underlying channel* and immediately resumes with [CancellationException]. - */ -suspend fun AsynchronousFileChannel.aRead( - buf: ByteBuffer, - position: Long -) = suspendCancellableCoroutine { cont -> - read(buf, position, cont, asyncIOHandler()) - closeOnCancel(cont) -} - -/** - * Performs [AsynchronousFileChannel.write] without blocking a thread and resumes when asynchronous operation completes. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * *closes the underlying channel* and immediately resumes with [CancellationException]. - */ -suspend fun AsynchronousFileChannel.aWrite( - buf: ByteBuffer, - position: Long -) = suspendCancellableCoroutine { cont -> - write(buf, position, cont, asyncIOHandler()) - closeOnCancel(cont) -} - -/** - * Performs [AsynchronousServerSocketChannel.accept] without blocking a thread and resumes when asynchronous operation completes. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * *closes the underlying channel* and immediately resumes with [CancellationException]. - */ -suspend fun AsynchronousServerSocketChannel.aAccept() = suspendCancellableCoroutine { cont -> - accept(cont, asyncIOHandler()) - closeOnCancel(cont) -} - -/** - * Performs [AsynchronousSocketChannel.connect] without blocking a thread and resumes when asynchronous operation completes. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * *closes the underlying channel* and immediately resumes with [CancellationException]. - */ -suspend fun AsynchronousSocketChannel.aConnect( - socketAddress: SocketAddress -) = suspendCancellableCoroutine { cont -> - connect(socketAddress, cont, AsyncVoidIOHandler) - closeOnCancel(cont) -} - -/** - * Performs [AsynchronousSocketChannel.read] without blocking a thread and resumes when asynchronous operation completes. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * *closes the underlying channel* and immediately resumes with [CancellationException]. - */ -suspend fun AsynchronousSocketChannel.aRead( - buf: ByteBuffer, - timeout: Long = 0L, - timeUnit: TimeUnit = TimeUnit.MILLISECONDS -) = suspendCancellableCoroutine { cont -> - read(buf, timeout, timeUnit, cont, asyncIOHandler()) - closeOnCancel(cont) -} - -/** - * Performs [AsynchronousSocketChannel.write] without blocking a thread and resumes when asynchronous operation completes. - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is suspended, this function - * *closes the underlying channel* and immediately resumes with [CancellationException]. - */ -suspend fun AsynchronousSocketChannel.aWrite( - buf: ByteBuffer, - timeout: Long = 0L, - timeUnit: TimeUnit = TimeUnit.MILLISECONDS -) = suspendCancellableCoroutine { cont -> - write(buf, timeout, timeUnit, cont, asyncIOHandler()) - closeOnCancel(cont) -} - -// ---------------- private details ---------------- - -private fun Channel.closeOnCancel(cont: CancellableContinuation<*>) { - cont.invokeOnCompletion { - if (cont.isCancelled) - try { - close() - } catch (ex: Throwable) { - // Specification says that it is Ok to call it any time, but reality is different, - // so we have just to ignore exception - } - } -} - -@Suppress("UNCHECKED_CAST") -private fun asyncIOHandler(): CompletionHandler> = - AsyncIOHandlerAny as CompletionHandler> - -private object AsyncIOHandlerAny : CompletionHandler> { - override fun completed(result: Any, cont: CancellableContinuation) { - cont.resume(result) - } - - override fun failed(ex: Throwable, cont: CancellableContinuation) { - // just return if already cancelled and got an expected exception for that case - if (ex is AsynchronousCloseException && cont.isCancelled) return - cont.resumeWithException(ex) - } -} - -private object AsyncVoidIOHandler : CompletionHandler> { - override fun completed(result: Void?, cont: CancellableContinuation) { - cont.resume(Unit) - } - - override fun failed(ex: Throwable, cont: CancellableContinuation) { - // just return if already cancelled and got an expected exception for that case - if (ex is AsynchronousCloseException && cont.isCancelled) return - cont.resumeWithException(ex) - } -} - - diff --git a/kotlinx-coroutines-nio/src/test/kotlin/examples/echo-example.kt b/kotlinx-coroutines-nio/src/test/kotlin/examples/echo-example.kt deleted file mode 100644 index 49f6499e9f..0000000000 --- a/kotlinx-coroutines-nio/src/test/kotlin/examples/echo-example.kt +++ /dev/null @@ -1,74 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.nio.aAccept -import kotlinx.coroutines.experimental.nio.aRead -import kotlinx.coroutines.experimental.nio.aWrite -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.withTimeout -import java.net.InetSocketAddress -import java.nio.ByteBuffer -import java.nio.channels.AsynchronousServerSocketChannel -import java.nio.channels.AsynchronousSocketChannel - -val PORT = 12345 -val CLIENT_READ_TIMEOUT = 5000L // 5 sec -val CLIENT_WRITE_TIMEOUT = 1000L // 1 sec -val BUFFER_SIZE = 1024 - -fun main(args: Array) = runBlocking { - val serverChannel = - AsynchronousServerSocketChannel - .open() - .bind(InetSocketAddress(PORT)) - log("Listening on port $PORT") - // loop and accept connections forever - while (true) { - val client = serverChannel.aAccept() - val address = try { - val ia = client.remoteAddress as InetSocketAddress - "${ia.address.hostAddress}:${ia.port}" - } catch (ex: Throwable) { - log("Accepted client connection but failed to get its address because of $ex") - continue /* accept next connection */ - } - log("Accepted client connection from $address") - // just start a new coroutine for each client connection - launch(context) { - try { - handleClient(client) - log("Client connection from $address has terminated normally") - } catch (ex: Throwable) { - log("Client connection from $address has terminated because of $ex") - } - } - } -} - -suspend fun handleClient(client: AsynchronousSocketChannel) { - val buffer = ByteBuffer.allocate(BUFFER_SIZE) - while (true) { - val bytes = withTimeout(CLIENT_READ_TIMEOUT) { client.aRead(buffer) } - if (bytes < 0) break - buffer.flip() - withTimeout(CLIENT_WRITE_TIMEOUT) { client.aWrite(buffer) } - buffer.clear() - } -} - diff --git a/kotlinx-coroutines-nio/src/test/kotlin/examples/log.kt b/kotlinx-coroutines-nio/src/test/kotlin/examples/log.kt deleted file mode 100644 index a875334b28..0000000000 --- a/kotlinx-coroutines-nio/src/test/kotlin/examples/log.kt +++ /dev/null @@ -1,22 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package examples - -import java.text.SimpleDateFormat -import java.util.* - -fun log(msg: String) = println("${SimpleDateFormat("yyyyMMdd-HHmmss.sss").format(Date())} [${Thread.currentThread().name}] $msg") \ No newline at end of file diff --git a/kotlinx-coroutines-nio/src/test/kotlin/kotlinx/coroutines/experimental/nio/AsyncIOTest.kt b/kotlinx-coroutines-nio/src/test/kotlin/kotlinx/coroutines/experimental/nio/AsyncIOTest.kt deleted file mode 100644 index fdc00da216..0000000000 --- a/kotlinx-coroutines-nio/src/test/kotlin/kotlinx/coroutines/experimental/nio/AsyncIOTest.kt +++ /dev/null @@ -1,115 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.nio - -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import org.apache.commons.io.FileUtils -import org.junit.Rule -import org.junit.Test -import org.junit.rules.TemporaryFolder -import java.net.InetSocketAddress -import java.nio.ByteBuffer -import java.nio.channels.AsynchronousFileChannel -import java.nio.channels.AsynchronousServerSocketChannel -import java.nio.channels.AsynchronousSocketChannel -import java.nio.file.StandardOpenOption -import org.junit.Assert.* - -class AsyncIOTest { - @Rule - @JvmField - val tmpDir = TemporaryFolder() - - @Test - fun testFileChannels() { - val inputFile = tmpDir.newFile() - val outputFile = tmpDir.newFile() - - FileUtils.writeStringToFile( - inputFile, - (1..100000).map(Int::toString).joinToString("")) - - val input = AsynchronousFileChannel.open(inputFile.toPath()) - val output = - AsynchronousFileChannel.open( - outputFile.toPath(), - StandardOpenOption.CREATE, StandardOpenOption.WRITE) - val buf = ByteBuffer.allocate(1024) - - runBlocking { - var totalBytesRead = 0L - var totalBytesWritten = 0L - while (totalBytesRead < input.size()) { - while (buf.hasRemaining() && totalBytesRead < input.size()) { - // async read - totalBytesRead += input.aRead(buf, totalBytesRead) - } - - buf.flip() - - while (buf.hasRemaining()) { - // async write - totalBytesWritten += output.aWrite(buf, totalBytesWritten) - } - - buf.clear() - } - } - - assertTrue(FileUtils.contentEquals(inputFile, outputFile)) - } - - @Test - fun testNetworkChannels() = runBlocking { - val serverChannel = - AsynchronousServerSocketChannel - .open() - .bind(InetSocketAddress(0)) - - val serverPort = (serverChannel.localAddress as InetSocketAddress).port - - val c1 = launch(context) { - val client = serverChannel.aAccept() - val buffer = ByteBuffer.allocate(2) - client.aRead(buffer) - buffer.flip() - assertEquals("OK", Charsets.UTF_8.decode(buffer).toString()) - - client.aWrite(Charsets.UTF_8.encode("123")) - client.close() - } - - val c2 = launch(context) { - val connection = - AsynchronousSocketChannel.open() - // async calls - connection.aConnect(InetSocketAddress("127.0.0.1", serverPort)) - connection.aWrite(Charsets.UTF_8.encode("OK")) - - val buffer = ByteBuffer.allocate(3) - - // async call - connection.aRead(buffer) - buffer.flip() - assertEquals("123", Charsets.UTF_8.decode(buffer).toString()) - } - - c1.join() - c2.join() - } -} diff --git a/kotlinx-coroutines-test/MIGRATION.md b/kotlinx-coroutines-test/MIGRATION.md new file mode 100644 index 0000000000..0e9e6092be --- /dev/null +++ b/kotlinx-coroutines-test/MIGRATION.md @@ -0,0 +1,447 @@ +# Migration to the new kotlinx-coroutines-test API + +In version 1.6.0, the API of the test module changed significantly. +This is a guide for gradually adapting the existing test code to the new API. +This guide is written step-by-step; the idea is to separate the migration into several sets of small changes. + +## Remove custom UncaughtExceptionCaptor, DelayController, and TestCoroutineScope implementations + +We couldn't find any code that defined new implementations of these interfaces, so they are deprecated. It's likely that +you don't need to do anything for this section. + +### UncaughtExceptionCaptor + +If the code base has an `UncaughtExceptionCaptor`, its special behavior as opposed to just `CoroutineExceptionHandler` +was that, at the end of `runBlockingTest` or `cleanupTestCoroutines` (or both), its `cleanupTestCoroutines` procedure +was called. + +We currently don't provide a replacement for this. +However, `runTest` follows structured concurrency better than `runBlockingTest` did, so exceptions from child coroutines +are propagated structurally, which makes uncaught exception handlers less useful. + +If you have a use case for this, please tell us about it at the issue tracker. +Meanwhile, it should be possible to use a custom exception captor, which should only implement +`CoroutineExceptionHandler` now, like this: + +```kotlin +@Test +fun testFoo() = runTest { + val customCaptor = MyUncaughtExceptionCaptor() + launch(customCaptor) { + // ... + } + advanceUntilIdle() + customCaptor.cleanupTestCoroutines() +} +``` + +### DelayController + +We don't provide a way to define custom dispatching strategies that support virtual time. +That said, we significantly enhanced this mechanism: +* Using multiple test dispatchers simultaneously is supported. + For the dispatchers to have a shared knowledge of the virtual time, either the same `TestCoroutineScheduler` should be + passed to each of them, or all of them should be constructed after `Dispatchers.setMain` is called with some test + dispatcher. +* Both a simple `StandardTestDispatcher` that is always paused, and unconfined `UnconfinedTestDispatcher` are provided. + +If you have a use case for `DelayController` that's not covered by what we provide, please tell us about it in the issue +tracker. + +### TestCoroutineScope + +This scope couldn't be meaningfully used in tandem with `runBlockingTest`: according to the definition of +`TestCoroutineScope.runBlockingTest`, only the scope's `coroutineContext` is used. +So, there could be two reasons for defining a custom implementation: + +* Avoiding the restrictions on placed `coroutineContext` in the `TestCoroutineScope` constructor function. + These restrictions consisted of requirements for `CoroutineExceptionHandler` being an `UncaughtExceptionCaptor`, and + `ContinuationInterceptor` being a `DelayController`, so it is also possible to fulfill these restrictions by defining + conforming instances. In this case, follow the instructions about replacing them. +* Using without `runBlockingTest`. In this case, you don't even need to implement `TestCoroutineScope`: nothing else + accepts a `TestCoroutineScope` specifically as an argument. + +## Remove usages of TestCoroutineExceptionHandler and TestCoroutineScope.uncaughtExceptions + +It is already illegal to use a `TestCoroutineScope` without performing `cleanupTestCoroutines`, so the valid uses of +`TestCoroutineExceptionHandler` include: + +* Accessing `uncaughtExceptions` in the middle of the test to make sure that there weren't any uncaught exceptions + *yet*. + If there are any, they will be thrown by the cleanup procedure anyway. + We don't support this use case, given how comparatively rare it is, but it can be handled in the same way as the + following one. +* Accessing `uncaughtExceptions` when the uncaught exceptions are actually expected. + In this case, `cleanupTestCoroutines` will fail with an exception that is being caught later. + It would be better in this case to use a custom `CoroutineExceptionHandler` so that actual problems that could be + found by the cleanup procedure are not superseded by the exceptions that are expected. + An example is shown below. + +```kotlin +val exceptions = mutableListOf() +val customCaptor = CoroutineExceptionHandler { ctx, throwable -> + exceptions.add(throwable) // add proper synchronization if the test is multithreaded +} + +@Test +fun testFoo() = runTest { + launch(customCaptor) { + // ... + } + advanceUntilIdle() + // check the list of the caught exceptions +} +``` + +## Auto-replace TestCoroutineScope constructor function with createTestCoroutineScope + +This should not break anything, as `TestCoroutineScope` is now defined in terms of `createTestCoroutineScope`. +If it does break something, it means that you already supplied a `TestCoroutineScheduler` to some scope; in this case, +also pass this scheduler as the argument to the dispatcher. + +## Replace usages of pauseDispatcher and resumeDispatcher with a StandardTestDispatcher + +* In places where `pauseDispatcher` in its block form is called, replace it with a call to + `withContext(StandardTestDispatcher(testScheduler))` + (`testScheduler` is available as a field of `TestCoroutineScope`, + or `scheduler` is available as a field of `TestCoroutineDispatcher`), + followed by `advanceUntilIdle()`. + This is not an automatic replacement, as there can be tricky situations where the test dispatcher is already paused + when `pauseDispatcher { X }` is called. In such cases, simply replace `pauseDispatcher { X }` with `X`. +* Often, `pauseDispatcher()` in a non-block form is used at the start of the test. + Then, attempt to remove `TestCoroutineDispatcher` from the arguments to `createTestCoroutineScope`, + if a standalone `TestCoroutineScope` or the `scope.runBlockingTest` form is used, + or pass a `StandardTestDispatcher` as an argument to `runBlockingTest`. + This will lead to the test using a `StandardTestDispatcher`, which does not allow pausing and resuming, + instead of the deprecated `TestCoroutineDispatcher`. +* Sometimes, `pauseDispatcher()` and `resumeDispatcher()` are employed used throughout the test. + In this case, attempt to wrap everything until the next `resumeDispatcher()` in + a `withContext(StandardTestDispatcher(testScheduler))` block, or try using some other combinations of + `StandardTestDispatcher` (where dispatches are needed) and `UnconfinedTestDispatcher` (where it isn't important where + execution happens). + +## Replace advanceTimeBy(n) with advanceTimeBy(n); runCurrent() + +For `TestCoroutineScope` and `DelayController`, the `advanceTimeBy` method is deprecated. +It is not deprecated for `TestCoroutineScheduler` and `TestScope`, but has a different meaning: it does not run the +tasks scheduled *at* `currentTime + n`. + +There is an automatic replacement for this deprecation, which produces correct but inelegant code. + +Alternatively, you can wait until replacing `TestCoroutineScope` with `TestScope`: it's possible that you will not +encounter this edge case. + +## Replace runBlockingTest with runTest(UnconfinedTestDispatcher()) + +This is a major change, affecting many things, and can be done in parallel with replacing `TestCoroutineScope` with +`TestScope`. + +Significant differences of `runTest` from `runBlockingTest` are each given a section below. + +### It works properly with other dispatchers and asynchronous completions. + +No action on your part is required, other than replacing `runBlocking` with `runTest` as well. + +### It uses StandardTestDispatcher by default, not TestCoroutineDispatcher. + +By now, calls to `pauseDispatcher` and `resumeDispatcher` should be purged from the code base, so only the unpaused +variant of `TestCoroutineDispatcher` should be used. +This version of the dispatcher has the property of eagerly entering `launch` and `async` blocks: +code until the first suspension is executed without dispatching. + +There are two common ways in which this property is useful. + +#### TestCoroutineDispatcher for the top-level coroutine + +Some tests that rely on `launch` and `async` blocks being entered immediately have a form similar to this: +```kotlin +runTest(TestCoroutineDispatcher()) { + launch { + updateSomething() + } + checkThatSomethingWasUpdated() + launch { + updateSomethingElse() + } + checkThatSomethingElseWasUpdated() +} +``` + +If the `TestCoroutineDispatcher()` is simply removed, `StandardTestDispatcher()` will be used, which will cause +the test to fail. + +In these cases, `UnconfinedTestDispatcher()` should be used. +We ensured that, when run with an `UnconfinedTestDispatcher`, `runTest` also eagerly enters `launch` and `async` +blocks. + +Note though that *this only works at the top level*: if a child coroutine also called `launch` or `async`, we don't provide +any guarantees about their dispatching order. + +#### TestCoroutineDispatcher for testing intermediate emissions + +Some code tests `StateFlow` or channels in a manner similar to this: + +```kotlin +@Test +fun testAllEmissions() = runTest(TestCoroutineDispatcher()) { + val values = mutableListOf() + val stateFlow = MutableStateFlow(0) + val job = launch { + stateFlow.collect { + values.add(it) + } + } + stateFlow.value = 1 + stateFlow.value = 2 + stateFlow.value = 3 + job.cancel() + // each assignment will immediately resume the collecting child coroutine, + // so no values will be skipped. + assertEquals(listOf(0, 1, 2, 3), values) +} +``` + +Such code will fail when `TestCoroutineDispatcher()` is not used: not every emission will be listed. +In this particular case, none will be listed at all. + +The reason for this is that setting `stateFlow.value` (as is sending to a channel, as are some other things) wakes up +the coroutine waiting for the new value, but *typically* does not immediately run the collecting code, instead simply +dispatching it. +The exceptions are the coroutines running in dispatchers that don't (always) go through a dispatch, +`Dispatchers.Unconfined`, `Dispatchers.Main.immediate`, `UnconfinedTestDispatcher`, or `TestCoroutineDispatcher` in +the unpaused state. + +Therefore, a solution is to launch the collection in an unconfined dispatcher: + +```kotlin +@Test +fun testAllEmissions() = runTest { + val values = mutableListOf() + val stateFlow = MutableStateFlow(0) + val job = launch(UnconfinedTestDispatcher(testScheduler)) { // <------ + stateFlow.collect { + values.add(it) + } + } + stateFlow.value = 1 + stateFlow.value = 2 + stateFlow.value = 3 + job.cancel() + // each assignment will immediately resume the collecting child coroutine, + // so no values will be skipped. + assertEquals(listOf(0, 1, 2, 3), values) +} +``` + +Note that `testScheduler` is passed so that the unconfined dispatcher is linked to `runTest`. +Also, note that `UnconfinedTestDispatcher` is not passed to `runTest`. +This is due to the fact that, *inside* the `UnconfinedTestDispatcher`, there are no execution order guarantees, +so it would not be guaranteed that setting `stateFlow.value` would immediately run the collecting code +(though in this case, it does). + +#### Other considerations + +Using `UnconfinedTestDispatcher` as an argument to `runTest` will probably lead to the test being executed as it +did, but it's still possible that the test relies on the specific dispatching order of `TestCoroutineDispatcher`, +so it will need to be tweaked. + +If some code is expected to have run at some point, but it hasn't, use `runCurrent` to force the tasks scheduled +at this moment of time to run. +For example, the `StateFlow` example above can also be forced to succeed by doing this: + +```kotlin +@Test +fun testAllEmissions() = runTest { + val values = mutableListOf() + val stateFlow = MutableStateFlow(0) + val job = launch { + stateFlow.collect { + values.add(it) + } + } + runCurrent() + stateFlow.value = 1 + runCurrent() + stateFlow.value = 2 + runCurrent() + stateFlow.value = 3 + runCurrent() + job.cancel() + // each assignment will immediately resume the collecting child coroutine, + // so no values will be skipped. + assertEquals(listOf(0, 1, 2, 3), values) +} +``` + +Be wary though of this approach: using `runCurrent`, `advanceTimeBy`, or `advanceUntilIdle` is, essentially, +simulating some particular execution order, which is not guaranteed to happen in production code. +For example, using `UnconfinedTestDispatcher` to fix this test reflects how, in production code, one could use +`Dispatchers.Unconfined` to observe all emitted values without conflation, but the `runCurrent()` approach only +states that the behavior would be observed if a dispatch were to happen at some chosen points. +It is, therefore, recommended to structure tests in a way that does not rely on a particular interleaving, unless +that is the intention. + +### The job hierarchy is completely different. + +- Structured concurrency is used, with the scope provided as the receiver of `runTest` actually being the scope of the + created coroutine. +- Not `SupervisorJob` but a normal `Job` is used for the `TestCoroutineScope`. +- The job passed as an argument is used as a parent job. + +Most tests should not be affected by this. In case your test is, try explicitly launching a child coroutine with a +`SupervisorJob`; this should make the job hierarchy resemble what it used to be. + +```kotlin +@Test +fun testFoo() = runTest { + val deferred = async(SupervisorJob()) { + // test code + } + advanceUntilIdle() + deferred.getCompletionExceptionOrNull()?.let { + throw it + } +} +``` + +### Only a single call to runTest is permitted per test. + +In order to work on JS, only a single call to `runTest` must happen during one test, and its result must be returned +immediately: + +```kotlin +@Test +fun testFoo(): TestResult { + // arbitrary code here + return runTest { + // ... + } +} +``` + +When used only on the JVM, `runTest` will work when called repeatedly, but this is not supported. +Please only call `runTest` once per test, and if for some reason you can't, please tell us about in on the issue +tracker. + +### It uses TestScope, not TestCoroutineScope, by default. + +There is a `runTestWithLegacyScope` method that allows migrating from `runBlockingTest` to `runTest` before migrating +from `TestCoroutineScope` to `TestScope`, if exactly the `TestCoroutineScope` needs to be passed somewhere else and +`TestScope` will not suffice. + +## Replace TestCoroutineScope.cleanupTestCoroutines with runTest + +Likely can be done together with the next step. + +Remove all calls to `TestCoroutineScope.cleanupTestCoroutines` from the code base. +Instead, as the last step of each test, do `return scope.runTest`; if possible, the whole test body should go inside +the `runTest` block. + +The cleanup procedure in `runTest` will not check that the virtual time doesn't advance during cleanup. +If a test must check that no other delays are remaining after it has finished, the following form may help: +```kotlin +runTest { + testBody() + val timeAfterTest = currentTime() + advanceUntilIdle() // run the remaining tasks + assertEquals(timeAfterTest, currentTime()) // will fail if there were tasks scheduled at a later moment +} +``` +Note that this will report time advancement even if the job scheduled at a later point was cancelled. + +It may be the case that `cleanupTestCoroutines` must be executed after de-initialization in `@AfterTest`, which happens +outside the test itself. +In this case, we propose that you write a wrapper of the form: + +```kotlin +fun runTestAndCleanup(body: TestScope.() -> Unit) = runTest { + try { + body() + } finally { + // the usual cleanup procedures that used to happen before `cleanupTestCoroutines` + } +} +``` + +## Replace runBlockingTest with runBlockingTestOnTestScope, createTestCoroutineScope with TestScope + +Also, replace `runTestWithLegacyScope` with just `runTest`. +All of this can be done in parallel with replacing `runBlockingTest` with `runTest`. + +This step should remove all uses of `TestCoroutineScope`, explicit or implicit. + +Replacing `runTestWithLegacyScope` and `runBlockingTest` with `runTest` and `runBlockingTestOnTestScope` should be +straightforward if there is no more code left that requires passing exactly `TestCoroutineScope` to it. +Some tests may fail because `TestCoroutineScope.cleanupTestCoroutines` and the cleanup procedure in `runTest` +handle cancelled tasks differently: if there are *cancelled* jobs pending at the moment of +`TestCoroutineScope.cleanupTestCoroutines`, they are ignored, whereas `runTest` will report them. + +Of all the methods supported by `TestCoroutineScope`, only `cleanupTestCoroutines` is not provided on `TestScope`, +and its usages should have been removed during the previous step. + +## Replace runBlocking with runTest + +Now that `runTest` works properly with asynchronous completions, `runBlocking` is only occasionally useful. +As is, most uses of `runBlocking` in tests come from the need to interact with dispatchers that execute on other +threads, like `Dispatchers.IO` or `Dispatchers.Default`. + +## Replace TestCoroutineDispatcher with UnconfinedTestDispatcher and StandardTestDispatcher + +`TestCoroutineDispatcher` is a dispatcher with two modes: +* ("unpaused") Almost (but not quite) unconfined, with the ability to eagerly enter `launch` and `async` blocks. +* ("paused") Behaving like a `StandardTestDispatcher`. + +In one of the earlier steps, we replaced `pauseDispatcher` with `StandardTestDispatcher` usage, and replaced the +implicit `TestCoroutineScope` dispatcher in `runBlockingTest` with `UnconfinedTestDispatcher` during migration to +`runTest`. + +Now, the rest of the usages should be replaced with whichever dispatcher is most appropriate. + +## Simplify code by removing unneeded entities + +Likely, now some code has the form + +```kotlin +val dispatcher = StandardTestDispatcher() +val scope = TestScope(dispatcher) + +@BeforeTest +fun setUp() { + Dispatchers.setMain(dispatcher) +} + +@AfterTest +fun tearDown() { + Dispatchers.resetMain() +} + +@Test +fun testFoo() = scope.runTest { + // ... +} +``` + +The point of this pattern is to ensure that the test runs with the same `TestCoroutineScheduler` as the one used for +`Dispatchers.Main`. + +However, now this can be simplified to just + +```kotlin +@BeforeTest +fun setUp() { + Dispatchers.setMain(StandardTestDispatcher()) +} + +@AfterTest +fun tearDown() { + Dispatchers.resetMain() +} + +@Test +fun testFoo() = runTest { + // ... +} +``` + +The reason this works is that all entities that depend on `TestCoroutineScheduler` will attempt to acquire one from +the current `Dispatchers.Main`. diff --git a/kotlinx-coroutines-test/README.md b/kotlinx-coroutines-test/README.md new file mode 100644 index 0000000000..fa64a9e731 --- /dev/null +++ b/kotlinx-coroutines-test/README.md @@ -0,0 +1,452 @@ +# Module kotlinx-coroutines-test + +Test utilities for `kotlinx.coroutines`. + +## Overview + +This package provides utilities for efficiently testing coroutines. + +| Name | Description | +| ---- | ----------- | +| [runTest] | Runs the test code, automatically skipping delays and handling uncaught exceptions. | +| [TestCoroutineScheduler] | The shared source of virtual time, used for controlling execution order and skipping delays. | +| [TestScope] | A [CoroutineScope] that integrates with [runTest], providing access to [TestCoroutineScheduler]. | +| [TestDispatcher] | A [CoroutineDispatcher] whose delays are controlled by a [TestCoroutineScheduler]. | +| [Dispatchers.setMain] | Mocks the main dispatcher using the provided one. If mocked with a [TestDispatcher], its [TestCoroutineScheduler] is used everywhere by default. | + +Provided [TestDispatcher] implementations: + +| Name | Description | +| ---- | ----------- | +| [StandardTestDispatcher] | A simple dispatcher with no special behavior other than being linked to a [TestCoroutineScheduler]. | +| [UnconfinedTestDispatcher] | A dispatcher that behaves like [Dispatchers.Unconfined]. | + +## Using in your project + +Add `kotlinx-coroutines-test` to your project test dependencies: +``` +dependencies { + testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.10.2' +} +``` + +**Do not** depend on this project in your main sources, all utilities here are intended and designed to be used only from tests. + +## Dispatchers.Main Delegation + +`Dispatchers.setMain` will override the `Main` dispatcher in test scenarios. +This is helpful when one wants to execute a test in situations where the platform `Main` dispatcher is not available, +or to replace `Dispatchers.Main` with a testing dispatcher. + +On the JVM, +the [`ServiceLoader`](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) mechanism is responsible +for overwriting [Dispatchers.Main] with a testable implementation, which by default will delegate its calls to the real +`Main` dispatcher, if any. + +The `Main` implementation can be overridden using [Dispatchers.setMain][setMain] method with any [CoroutineDispatcher] +implementation, e.g.: + +```kotlin + +class SomeTest { + + private val mainThreadSurrogate = newSingleThreadContext("UI thread") + + @Before + fun setUp() { + Dispatchers.setMain(mainThreadSurrogate) + } + + @After + fun tearDown() { + Dispatchers.resetMain() // reset the main dispatcher to the original Main dispatcher + mainThreadSurrogate.close() + } + + @Test + fun testSomeUI() = runBlocking { + launch(Dispatchers.Main) { // Will be launched in the mainThreadSurrogate dispatcher + // ... + } + } +} +``` + +Calling `setMain` or `resetMain` immediately changes the `Main` dispatcher globally. + +If `Main` is overridden with a [TestDispatcher], then its [TestCoroutineScheduler] is used when new [TestDispatcher] or +[TestScope] instances are created without [TestCoroutineScheduler] being passed as an argument. + +## runTest + +[runTest] is the way to test code that involves coroutines. `suspend` functions can be called inside it. + +**IMPORTANT: in order to work with on Kotlin/JS, the result of `runTest` must be immediately `return`-ed from each test.** +The typical invocation of [runTest] thus looks like this: + +```kotlin +@Test +fun testFoo() = runTest { + // code under test +} +``` + +In more advanced scenarios, it's possible instead to use the following form: +```kotlin +@Test +fun testFoo(): TestResult { + // initialize some test state + return runTest { + // code under test + } +} +``` + +[runTest] is similar to running the code with `runBlocking` on Kotlin/JVM and Kotlin/Native, or launching a new promise +on Kotlin/JS. The main differences are the following: + +* **The calls to `delay` are automatically skipped**, preserving the relative execution order of the tasks. This way, + it's possible to make tests finish more-or-less immediately. +* **The execution times out after 60 seconds**, cancelling the test coroutine to prevent tests from hanging forever + and eating up the CI resources. +* **Controlling the virtual time**: in case just skipping delays is not sufficient, it's possible to more carefully + guide the execution, advancing the virtual time by a duration, draining the queue of the awaiting tasks, or running + the tasks scheduled at the present moment. +* **Handling uncaught exceptions** spawned in the child coroutines by throwing them at the end of the test. +* **Waiting for asynchronous callbacks**. + Sometimes, especially when working with third-party code, it's impossible to mock all the dispatchers in use. + [runTest] will handle the situations where some code runs in dispatchers not integrated with the test module. + +## Timeout + +Test automatically time out after 60 seconds. For example, this test will fail with a timeout exception: + +```kotlin +@Test +fun testHanging() = runTest { + CompletableDeferred().await() // will hang forever +} +``` + +In case the test is expected to take longer than 60 seconds, the timeout can be increased by passing the `timeout` +parameter: + +```kotlin +@Test +fun testTakingALongTime() = runTest(timeout = 30.seconds) { + val result = withContext(Dispatchers.Default) { + delay(20.seconds) // this delay is not in the test dispatcher and will not be skipped + 3 + } + assertEquals(3, result) +} +``` + +## Delay-skipping + +To test regular suspend functions, which may have a delay, just run them inside the [runTest] block. + +```kotlin +@Test +fun testFoo() = runTest { // a coroutine with an extra test control + val actual = foo() + // ... +} + +suspend fun foo() { + delay(1_000) // when run in `runTest`, will finish immediately instead of delaying + // ... +} +``` + +## launch and async + +The coroutine dispatcher used for tests is single-threaded, meaning that the child coroutines of the [runTest] block +will run on the thread that started the test, and will never run in parallel. + +If several coroutines are waiting to be executed next, the one scheduled after the smallest delay will be chosen. +The virtual time will automatically advance to the point of its resumption. + +```kotlin +@Test +fun testWithMultipleDelays() = runTest { + launch { + delay(1_000) + println("1. $currentTime") // 1000 + delay(200) + println("2. $currentTime") // 1200 + delay(2_000) + println("4. $currentTime") // 3200 + } + val deferred = async { + delay(3_000) + println("3. $currentTime") // 3000 + delay(500) + println("5. $currentTime") // 3500 + } + deferred.await() +} +``` + +## Controlling the virtual time + +Inside [runTest], the execution is scheduled by [TestCoroutineScheduler], which is a virtual time scheduler. +The scheduler has several special methods that allow controlling the virtual time: +* `currentTime` gets the current virtual time. +* `runCurrent()` runs the tasks that are scheduled at this point of virtual time. +* `advanceUntilIdle()` runs all enqueued tasks until there are no more. +* `advanceTimeBy(timeDelta)` runs the enqueued tasks until the current virtual time advances by `timeDelta`. +* `timeSource` returns a `TimeSource` that uses the virtual time. + +```kotlin +@Test +fun testFoo() = runTest { + launch { + val workDuration = testScheduler.timeSource.measureTime { + println(1) // executes during runCurrent() + delay(1_000) // suspends until time is advanced by at least 1_000 + println(2) // executes during advanceTimeBy(2_000) + delay(500) // suspends until the time is advanced by another 500 ms + println(3) // also executes during advanceTimeBy(2_000) + delay(5_000) // will suspend by another 4_500 ms + println(4) // executes during advanceUntilIdle() + } + assertEquals(6500.milliseconds, workDuration) // the work took 6_500 ms of virtual time + } + // the child coroutine has not run yet + testScheduler.runCurrent() + // the child coroutine has called println(1), and is suspended on delay(1_000) + testScheduler.advanceTimeBy(2.seconds) // progress time, this will cause two calls to `delay` to resume + // the child coroutine has called println(2) and println(3) and suspends for another 4_500 virtual milliseconds + testScheduler.advanceUntilIdle() // will run the child coroutine to completion + assertEquals(6500, currentTime) // the child coroutine finished at virtual time of 6_500 milliseconds +} +``` + +## Using multiple test dispatchers + +The virtual time is controlled by an entity called the [TestCoroutineScheduler], which behaves as the shared source of +virtual time. + +Several dispatchers can be created that use the same [TestCoroutineScheduler], in which case they will share their +knowledge of the virtual time. + +To access the scheduler used for this test, use the [TestScope.testScheduler] property. + +```kotlin +@Test +fun testWithMultipleDispatchers() = runTest { + val scheduler = testScheduler // the scheduler used for this test + val dispatcher1 = StandardTestDispatcher(scheduler, name = "IO dispatcher") + val dispatcher2 = StandardTestDispatcher(scheduler, name = "Background dispatcher") + launch(dispatcher1) { + delay(1_000) + println("1. $currentTime") // 1000 + delay(200) + println("2. $currentTime") // 1200 + delay(2_000) + println("4. $currentTime") // 3200 + } + val deferred = async(dispatcher2) { + delay(3_000) + println("3. $currentTime") // 3000 + delay(500) + println("5. $currentTime") // 3500 + } + deferred.await() + } +``` + +**Note: if [Dispatchers.Main] is replaced by a [TestDispatcher], [runTest] will automatically use its scheduler. +This is done so that there is no need to go through the ceremony of passing the correct scheduler to [runTest].** + +## Accessing the test coroutine scope + +Structured concurrency ties coroutines to scopes in which they are launched. +[TestScope] is a special coroutine scope designed for testing coroutines, and a new one is automatically created +for [runTest] and used as the receiver for the test body. + +However, it can be convenient to access a `CoroutineScope` before the test has started, for example, to perform mocking +of some +parts of the system in `@BeforeTest` via dependency injection. +In these cases, it is possible to manually create [TestScope], the scope for the test coroutines, in advance, +before the test begins. + +[TestScope] on its own does not automatically run the code launched in it. +In addition, it is stateful in order to keep track of executing coroutines and uncaught exceptions. +Therefore, it is important to ensure that [TestScope.runTest] is called eventually. + +```kotlin +val scope = TestScope() + +@BeforeTest +fun setUp() { + Dispatchers.setMain(StandardTestDispatcher(scope.testScheduler)) + TestSubject.setScope(scope) +} + +@AfterTest +fun tearDown() { + Dispatchers.resetMain() + TestSubject.resetScope() +} + +@Test +fun testSubject() = scope.runTest { + // the receiver here is `testScope` +} +``` + +## Running background work + +Sometimes, the fact that [runTest] waits for all the coroutines to finish is undesired. +For example, the system under test may need to receive data from coroutines that always run in the background. +Emulating such coroutines by launching them from the test body is not sufficient, because [runTest] will wait for them +to finish, which they never typically do. + +For these cases, there is a special coroutine scope: [TestScope.backgroundScope]. +Coroutines launched in it will be cancelled at the end of the test. + +```kotlin +@Test +fun testExampleBackgroundJob() = runTest { + val channel = Channel() + backgroundScope.launch { + var i = 0 + while (true) { + channel.send(i++) + } + } + repeat(100) { + assertEquals(it, channel.receive()) + } +} +``` + +## Eagerly entering launch and async blocks + +Some tests only test functionality and don't particularly care about the precise order in which coroutines are +dispatched. +In these cases, it can be cumbersome to always call [runCurrent] or [yield] to observe the effects of the coroutines +after they are launched. + +If [runTest] executes with an [UnconfinedTestDispatcher], the child coroutines launched at the top level are entered +*eagerly*, that is, they don't go through a dispatch until the first suspension. + +```kotlin +@Test +fun testEagerlyEnteringChildCoroutines() = runTest(UnconfinedTestDispatcher()) { + var entered = false + val deferred = CompletableDeferred() + var completed = false + launch { + entered = true + deferred.await() + completed = true + } + assertTrue(entered) // `entered = true` already executed. + assertFalse(completed) // however, the child coroutine then suspended, so it is enqueued. + deferred.complete(Unit) // resume the coroutine. + assertTrue(completed) // now the child coroutine is immediately completed. +} +``` + +If this behavior is desirable, but some parts of the test still require accurate dispatching, for example, to ensure +that the code executes on the correct thread, then simply `launch` a new coroutine with the [StandardTestDispatcher]. + +```kotlin +@Test +fun testEagerlyEnteringSomeChildCoroutines() = runTest(UnconfinedTestDispatcher()) { + var entered1 = false + launch { + entered1 = true + } + assertTrue(entered1) // `entered1 = true` already executed + + var entered2 = false + launch(StandardTestDispatcher(testScheduler)) { + // this block and every coroutine launched inside it will explicitly go through the needed dispatches + entered2 = true + } + assertFalse(entered2) + runCurrent() // need to explicitly run the dispatched continuation + assertTrue(entered2) +} +``` + +### Using withTimeout inside runTest + +Timeouts are also susceptible to time control, so the code below will immediately finish. + +```kotlin +@Test +fun testFooWithTimeout() = runTest { + assertFailsWith { + withTimeout(1_000) { + delay(999) + delay(2) + println("this won't be reached") + } + } +} +``` + +## Virtual time support with other dispatchers + +Calls to `withContext(Dispatchers.IO)`, `withContext(Dispatchers.Default)` ,and `withContext(Dispatchers.Main)` are +common in coroutines-based code bases. Unfortunately, just executing code in a test will not lead to these dispatchers +using the virtual time source, so delays will not be skipped in them. + +```kotlin +suspend fun veryExpensiveFunction() = withContext(Dispatchers.Default) { + delay(1_000) + 1 +} + +fun testExpensiveFunction() = runTest { + val result = veryExpensiveFunction() // will take a whole real-time second to execute + // the virtual time at this point is still 0 +} +``` + +Tests should, when possible, replace these dispatchers with a [TestDispatcher] if the `withContext` calls `delay` in the +function under test. For example, `veryExpensiveFunction` above should allow mocking with a [TestDispatcher] using +either dependency injection, a service locator, or a default parameter, if it is to be used with virtual time. + +### Status of the API + +Many parts of the API is experimental, and it is may change before migrating out of experimental (while it is marked as +[`@ExperimentalCoroutinesApi`][ExperimentalCoroutinesApi]). +Changes during experimental may have deprecation applied when possible, but it is not +advised to use the API in stable code before it leaves experimental due to possible breaking changes. + +If you have any suggestions for improvements to this experimental API please share them on the +[issue tracker](https://github.com/Kotlin/kotlinx.coroutines/issues). + + + + +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[CoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/index.html +[Dispatchers.Unconfined]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-unconfined.html +[Dispatchers.Main]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-main.html +[yield]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/yield.html +[ExperimentalCoroutinesApi]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-experimental-coroutines-api/index.html + + + + +[runTest]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/run-test.html +[TestCoroutineScheduler]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-test-coroutine-scheduler/index.html +[TestScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-test-scope/index.html +[TestDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-test-dispatcher/index.html +[Dispatchers.setMain]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/set-main.html +[StandardTestDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-standard-test-dispatcher.html +[UnconfinedTestDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-unconfined-test-dispatcher.html +[setMain]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/set-main.html +[TestScope.testScheduler]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-test-scope/test-scheduler.html +[TestScope.runTest]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/run-test.html +[TestScope.backgroundScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/-test-scope/background-scope.html +[runCurrent]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/run-current.html + + diff --git a/kotlinx-coroutines-test/api/kotlinx-coroutines-test.api b/kotlinx-coroutines-test/api/kotlinx-coroutines-test.api new file mode 100644 index 0000000000..77cc854cd6 --- /dev/null +++ b/kotlinx-coroutines-test/api/kotlinx-coroutines-test.api @@ -0,0 +1,103 @@ +public final class kotlinx/coroutines/test/TestBuildersKt { + public static final fun runBlockingTest (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)V + public static final fun runBlockingTest (Lkotlinx/coroutines/test/TestCoroutineDispatcher;Lkotlin/jvm/functions/Function2;)V + public static final fun runBlockingTest (Lkotlinx/coroutines/test/TestCoroutineScope;Lkotlin/jvm/functions/Function2;)V + public static final fun runBlockingTest (Lkotlinx/coroutines/test/TestScope;Lkotlin/jvm/functions/Function2;)V + public static synthetic fun runBlockingTest$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static final fun runBlockingTestOnTestScope (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)V + public static synthetic fun runBlockingTestOnTestScope$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static final fun runTest (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;)V + public static final fun runTest (Lkotlinx/coroutines/test/TestCoroutineScope;JLkotlin/jvm/functions/Function2;)V + public static final fun runTest (Lkotlinx/coroutines/test/TestScope;JLkotlin/jvm/functions/Function2;)V + public static synthetic fun runTest$default (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static synthetic fun runTest$default (Lkotlinx/coroutines/test/TestCoroutineScope;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static final synthetic fun runTest$default (Lkotlinx/coroutines/test/TestScope;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static final fun runTest-8Mi8wO0 (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;)V + public static final fun runTest-8Mi8wO0 (Lkotlinx/coroutines/test/TestScope;JLkotlin/jvm/functions/Function2;)V + public static synthetic fun runTest-8Mi8wO0$default (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static synthetic fun runTest-8Mi8wO0$default (Lkotlinx/coroutines/test/TestScope;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V + public static final fun runTestWithLegacyScope (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;)V + public static synthetic fun runTestWithLegacyScope$default (Lkotlin/coroutines/CoroutineContext;JLkotlin/jvm/functions/Function2;ILjava/lang/Object;)V +} + +public final class kotlinx/coroutines/test/TestCoroutineDispatcher : kotlinx/coroutines/test/TestDispatcher, kotlinx/coroutines/Delay { + public fun ()V + public fun (Lkotlinx/coroutines/test/TestCoroutineScheduler;)V + public synthetic fun (Lkotlinx/coroutines/test/TestCoroutineScheduler;ILkotlin/jvm/internal/DefaultConstructorMarker;)V + public final fun advanceUntilIdle ()J + public final fun cleanupTestCoroutines ()V + public fun dispatch (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Runnable;)V + public fun dispatchYield (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Runnable;)V + public final fun getCurrentTime ()J + public fun getScheduler ()Lkotlinx/coroutines/test/TestCoroutineScheduler; + public final fun runCurrent ()V + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/test/TestCoroutineDispatchersKt { + public static final fun StandardTestDispatcher (Lkotlinx/coroutines/test/TestCoroutineScheduler;Ljava/lang/String;)Lkotlinx/coroutines/test/TestDispatcher; + public static synthetic fun StandardTestDispatcher$default (Lkotlinx/coroutines/test/TestCoroutineScheduler;Ljava/lang/String;ILjava/lang/Object;)Lkotlinx/coroutines/test/TestDispatcher; + public static final fun UnconfinedTestDispatcher (Lkotlinx/coroutines/test/TestCoroutineScheduler;Ljava/lang/String;)Lkotlinx/coroutines/test/TestDispatcher; + public static synthetic fun UnconfinedTestDispatcher$default (Lkotlinx/coroutines/test/TestCoroutineScheduler;Ljava/lang/String;ILjava/lang/Object;)Lkotlinx/coroutines/test/TestDispatcher; +} + +public final class kotlinx/coroutines/test/TestCoroutineScheduler : kotlin/coroutines/AbstractCoroutineContextElement, kotlin/coroutines/CoroutineContext$Element { + public static final field Key Lkotlinx/coroutines/test/TestCoroutineScheduler$Key; + public fun ()V + public final fun advanceTimeBy (J)V + public final fun advanceTimeBy-LRDsOJo (J)V + public final fun advanceUntilIdle ()V + public final fun getCurrentTime ()J + public final fun getTimeSource ()Lkotlin/time/TimeSource$WithComparableMarks; + public final fun runCurrent ()V +} + +public final class kotlinx/coroutines/test/TestCoroutineScheduler$Key : kotlin/coroutines/CoroutineContext$Key { +} + +public abstract interface class kotlinx/coroutines/test/TestCoroutineScope : kotlinx/coroutines/CoroutineScope { + public abstract fun cleanupTestCoroutines ()V + public abstract fun getTestScheduler ()Lkotlinx/coroutines/test/TestCoroutineScheduler; +} + +public final class kotlinx/coroutines/test/TestCoroutineScopeKt { + public static final fun TestCoroutineScope (Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/test/TestCoroutineScope; + public static synthetic fun TestCoroutineScope$default (Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lkotlinx/coroutines/test/TestCoroutineScope; + public static final fun advanceUntilIdle (Lkotlinx/coroutines/test/TestCoroutineScope;)V + public static final fun createTestCoroutineScope (Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/test/TestCoroutineScope; + public static synthetic fun createTestCoroutineScope$default (Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lkotlinx/coroutines/test/TestCoroutineScope; + public static final fun getCurrentTime (Lkotlinx/coroutines/test/TestCoroutineScope;)J + public static final fun runCurrent (Lkotlinx/coroutines/test/TestCoroutineScope;)V +} + +public abstract class kotlinx/coroutines/test/TestDispatcher : kotlinx/coroutines/CoroutineDispatcher, kotlinx/coroutines/Delay, kotlinx/coroutines/DelayWithTimeoutDiagnostics { + public fun delay (JLkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun getScheduler ()Lkotlinx/coroutines/test/TestCoroutineScheduler; + public fun invokeOnTimeout (JLjava/lang/Runnable;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/DisposableHandle; + public fun scheduleResumeAfterDelay (JLkotlinx/coroutines/CancellableContinuation;)V + public synthetic fun timeoutMessage-LRDsOJo (J)Ljava/lang/String; +} + +public final class kotlinx/coroutines/test/TestDispatchers { + public static final fun resetMain (Lkotlinx/coroutines/Dispatchers;)V + public static final fun setMain (Lkotlinx/coroutines/Dispatchers;Lkotlinx/coroutines/CoroutineDispatcher;)V +} + +public abstract interface class kotlinx/coroutines/test/TestScope : kotlinx/coroutines/CoroutineScope { + public abstract fun getBackgroundScope ()Lkotlinx/coroutines/CoroutineScope; + public abstract fun getTestScheduler ()Lkotlinx/coroutines/test/TestCoroutineScheduler; +} + +public final class kotlinx/coroutines/test/TestScopeKt { + public static final fun TestScope (Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/test/TestScope; + public static synthetic fun TestScope$default (Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lkotlinx/coroutines/test/TestScope; + public static final fun advanceTimeBy (Lkotlinx/coroutines/test/TestScope;J)V + public static final fun advanceTimeBy-HG0u8IE (Lkotlinx/coroutines/test/TestScope;J)V + public static final fun advanceUntilIdle (Lkotlinx/coroutines/test/TestScope;)V + public static final fun getCatchNonTestRelatedExceptions ()Z + public static final fun getCurrentTime (Lkotlinx/coroutines/test/TestScope;)J + public static final fun getTestTimeSource (Lkotlinx/coroutines/test/TestScope;)Lkotlin/time/TimeSource$WithComparableMarks; + public static final fun runCurrent (Lkotlinx/coroutines/test/TestScope;)V + public static final fun setCatchNonTestRelatedExceptions (Z)V +} + diff --git a/kotlinx-coroutines-test/api/kotlinx-coroutines-test.klib.api b/kotlinx-coroutines-test/api/kotlinx-coroutines-test.klib.api new file mode 100644 index 0000000000..38dfad99c6 --- /dev/null +++ b/kotlinx-coroutines-test/api/kotlinx-coroutines-test.klib.api @@ -0,0 +1,107 @@ +// Klib ABI Dump +// Targets: [androidNativeArm32, androidNativeArm64, androidNativeX64, androidNativeX86, iosArm64, iosSimulatorArm64, iosX64, js, linuxArm64, linuxX64, macosArm64, macosX64, mingwX64, tvosArm64, tvosSimulatorArm64, tvosX64, wasmJs, wasmWasi, watchosArm32, watchosArm64, watchosDeviceArm64, watchosSimulatorArm64, watchosX64] +// Alias: native => [androidNativeArm32, androidNativeArm64, androidNativeX64, androidNativeX86, iosArm64, iosSimulatorArm64, iosX64, linuxArm64, linuxX64, macosArm64, macosX64, mingwX64, tvosArm64, tvosSimulatorArm64, tvosX64, watchosArm32, watchosArm64, watchosDeviceArm64, watchosSimulatorArm64, watchosX64] +// Rendering settings: +// - Signature version: 2 +// - Show manifest properties: true +// - Show declarations: true + +// Library unique name: +sealed interface kotlinx.coroutines.test/TestScope : kotlinx.coroutines/CoroutineScope { // kotlinx.coroutines.test/TestScope|null[0] + abstract val backgroundScope // kotlinx.coroutines.test/TestScope.backgroundScope|{}backgroundScope[0] + abstract fun (): kotlinx.coroutines/CoroutineScope // kotlinx.coroutines.test/TestScope.backgroundScope.|(){}[0] + abstract val testScheduler // kotlinx.coroutines.test/TestScope.testScheduler|{}testScheduler[0] + abstract fun (): kotlinx.coroutines.test/TestCoroutineScheduler // kotlinx.coroutines.test/TestScope.testScheduler.|(){}[0] +} + +abstract class kotlinx.coroutines.test/TestDispatcher : kotlinx.coroutines/CoroutineDispatcher, kotlinx.coroutines/Delay, kotlinx.coroutines/DelayWithTimeoutDiagnostics { // kotlinx.coroutines.test/TestDispatcher|null[0] + abstract val scheduler // kotlinx.coroutines.test/TestDispatcher.scheduler|{}scheduler[0] + abstract fun (): kotlinx.coroutines.test/TestCoroutineScheduler // kotlinx.coroutines.test/TestDispatcher.scheduler.|(){}[0] + + open fun invokeOnTimeout(kotlin/Long, kotlinx.coroutines/Runnable, kotlin.coroutines/CoroutineContext): kotlinx.coroutines/DisposableHandle // kotlinx.coroutines.test/TestDispatcher.invokeOnTimeout|invokeOnTimeout(kotlin.Long;kotlinx.coroutines.Runnable;kotlin.coroutines.CoroutineContext){}[0] + open fun scheduleResumeAfterDelay(kotlin/Long, kotlinx.coroutines/CancellableContinuation) // kotlinx.coroutines.test/TestDispatcher.scheduleResumeAfterDelay|scheduleResumeAfterDelay(kotlin.Long;kotlinx.coroutines.CancellableContinuation){}[0] + open fun timeoutMessage(kotlin.time/Duration): kotlin/String // kotlinx.coroutines.test/TestDispatcher.timeoutMessage|timeoutMessage(kotlin.time.Duration){}[0] +} + +final class kotlinx.coroutines.test/TestCoroutineScheduler : kotlin.coroutines/AbstractCoroutineContextElement, kotlin.coroutines/CoroutineContext.Element { // kotlinx.coroutines.test/TestCoroutineScheduler|null[0] + constructor () // kotlinx.coroutines.test/TestCoroutineScheduler.|(){}[0] + + final val timeSource // kotlinx.coroutines.test/TestCoroutineScheduler.timeSource|{}timeSource[0] + final fun (): kotlin.time/TimeSource.WithComparableMarks // kotlinx.coroutines.test/TestCoroutineScheduler.timeSource.|(){}[0] + + final var currentTime // kotlinx.coroutines.test/TestCoroutineScheduler.currentTime|{}currentTime[0] + final fun (): kotlin/Long // kotlinx.coroutines.test/TestCoroutineScheduler.currentTime.|(){}[0] + + final fun advanceTimeBy(kotlin.time/Duration) // kotlinx.coroutines.test/TestCoroutineScheduler.advanceTimeBy|advanceTimeBy(kotlin.time.Duration){}[0] + final fun advanceTimeBy(kotlin/Long) // kotlinx.coroutines.test/TestCoroutineScheduler.advanceTimeBy|advanceTimeBy(kotlin.Long){}[0] + final fun advanceUntilIdle() // kotlinx.coroutines.test/TestCoroutineScheduler.advanceUntilIdle|advanceUntilIdle(){}[0] + final fun runCurrent() // kotlinx.coroutines.test/TestCoroutineScheduler.runCurrent|runCurrent(){}[0] + + final object Key : kotlin.coroutines/CoroutineContext.Key // kotlinx.coroutines.test/TestCoroutineScheduler.Key|null[0] +} + +final val kotlinx.coroutines.test/currentTime // kotlinx.coroutines.test/currentTime|@kotlinx.coroutines.test.TestScope{}currentTime[0] + final fun (kotlinx.coroutines.test/TestScope).(): kotlin/Long // kotlinx.coroutines.test/currentTime.|@kotlinx.coroutines.test.TestScope(){}[0] +final val kotlinx.coroutines.test/testTimeSource // kotlinx.coroutines.test/testTimeSource|@kotlinx.coroutines.test.TestScope{}testTimeSource[0] + final fun (kotlinx.coroutines.test/TestScope).(): kotlin.time/TimeSource.WithComparableMarks // kotlinx.coroutines.test/testTimeSource.|@kotlinx.coroutines.test.TestScope(){}[0] + +final var kotlinx.coroutines.test/catchNonTestRelatedExceptions // kotlinx.coroutines.test/catchNonTestRelatedExceptions|{}catchNonTestRelatedExceptions[0] + final fun (): kotlin/Boolean // kotlinx.coroutines.test/catchNonTestRelatedExceptions.|(){}[0] + final fun (kotlin/Boolean) // kotlinx.coroutines.test/catchNonTestRelatedExceptions.|(kotlin.Boolean){}[0] + +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/advanceTimeBy(kotlin.time/Duration) // kotlinx.coroutines.test/advanceTimeBy|advanceTimeBy@kotlinx.coroutines.test.TestScope(kotlin.time.Duration){}[0] +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/advanceTimeBy(kotlin/Long) // kotlinx.coroutines.test/advanceTimeBy|advanceTimeBy@kotlinx.coroutines.test.TestScope(kotlin.Long){}[0] +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/advanceUntilIdle() // kotlinx.coroutines.test/advanceUntilIdle|advanceUntilIdle@kotlinx.coroutines.test.TestScope(){}[0] +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/runCurrent() // kotlinx.coroutines.test/runCurrent|runCurrent@kotlinx.coroutines.test.TestScope(){}[0] +final fun (kotlinx.coroutines/Dispatchers).kotlinx.coroutines.test/resetMain() // kotlinx.coroutines.test/resetMain|resetMain@kotlinx.coroutines.Dispatchers(){}[0] +final fun (kotlinx.coroutines/Dispatchers).kotlinx.coroutines.test/setMain(kotlinx.coroutines/CoroutineDispatcher) // kotlinx.coroutines.test/setMain|setMain@kotlinx.coroutines.Dispatchers(kotlinx.coroutines.CoroutineDispatcher){}[0] +final fun kotlinx.coroutines.test/StandardTestDispatcher(kotlinx.coroutines.test/TestCoroutineScheduler? = ..., kotlin/String? = ...): kotlinx.coroutines.test/TestDispatcher // kotlinx.coroutines.test/StandardTestDispatcher|StandardTestDispatcher(kotlinx.coroutines.test.TestCoroutineScheduler?;kotlin.String?){}[0] +final fun kotlinx.coroutines.test/TestScope(kotlin.coroutines/CoroutineContext = ...): kotlinx.coroutines.test/TestScope // kotlinx.coroutines.test/TestScope|TestScope(kotlin.coroutines.CoroutineContext){}[0] +final fun kotlinx.coroutines.test/UnconfinedTestDispatcher(kotlinx.coroutines.test/TestCoroutineScheduler? = ..., kotlin/String? = ...): kotlinx.coroutines.test/TestDispatcher // kotlinx.coroutines.test/UnconfinedTestDispatcher|UnconfinedTestDispatcher(kotlinx.coroutines.test.TestCoroutineScheduler?;kotlin.String?){}[0] + +// Targets: [native, wasmWasi] +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/runTest(kotlin.time/Duration = ..., kotlin.coroutines/SuspendFunction1) // kotlinx.coroutines.test/runTest|runTest@kotlinx.coroutines.test.TestScope(kotlin.time.Duration;kotlin.coroutines.SuspendFunction1){}[0] + +// Targets: [native, wasmWasi] +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/runTest(kotlin/Long, kotlin.coroutines/SuspendFunction1) // kotlinx.coroutines.test/runTest|runTest@kotlinx.coroutines.test.TestScope(kotlin.Long;kotlin.coroutines.SuspendFunction1){}[0] + +// Targets: [native, wasmWasi] +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/runTestLegacy(kotlin/Long, kotlin.coroutines/SuspendFunction1, kotlin/Int, kotlin/Any?) // kotlinx.coroutines.test/runTestLegacy|runTestLegacy@kotlinx.coroutines.test.TestScope(kotlin.Long;kotlin.coroutines.SuspendFunction1;kotlin.Int;kotlin.Any?){}[0] + +// Targets: [native, wasmWasi] +final fun kotlinx.coroutines.test/runTest(kotlin.coroutines/CoroutineContext = ..., kotlin.time/Duration = ..., kotlin.coroutines/SuspendFunction1) // kotlinx.coroutines.test/runTest|runTest(kotlin.coroutines.CoroutineContext;kotlin.time.Duration;kotlin.coroutines.SuspendFunction1){}[0] + +// Targets: [native, wasmWasi] +final fun kotlinx.coroutines.test/runTest(kotlin.coroutines/CoroutineContext = ..., kotlin/Long, kotlin.coroutines/SuspendFunction1) // kotlinx.coroutines.test/runTest|runTest(kotlin.coroutines.CoroutineContext;kotlin.Long;kotlin.coroutines.SuspendFunction1){}[0] + +// Targets: [js, wasmJs] +final class kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting { // kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting|null[0] + constructor () // kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting.|(){}[0] + + // Targets: [js] + final fun then(kotlin/Function1): kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting // kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting.then|then(kotlin.Function1){}[0] + + // Targets: [js] + final fun then(kotlin/Function1, kotlin/Function1): kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting // kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting.then|then(kotlin.Function1;kotlin.Function1){}[0] + + // Targets: [wasmJs] + final fun then(kotlin/Function1): kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting // kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting.then|then(kotlin.Function1){}[0] + + // Targets: [wasmJs] + final fun then(kotlin/Function1, kotlin/Function1): kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting // kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting.then|then(kotlin.Function1;kotlin.Function1){}[0] +} + +// Targets: [js, wasmJs] +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/runTest(kotlin.time/Duration = ..., kotlin.coroutines/SuspendFunction1): kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting // kotlinx.coroutines.test/runTest|runTest@kotlinx.coroutines.test.TestScope(kotlin.time.Duration;kotlin.coroutines.SuspendFunction1){}[0] + +// Targets: [js, wasmJs] +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/runTest(kotlin/Long, kotlin.coroutines/SuspendFunction1): kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting // kotlinx.coroutines.test/runTest|runTest@kotlinx.coroutines.test.TestScope(kotlin.Long;kotlin.coroutines.SuspendFunction1){}[0] + +// Targets: [js, wasmJs] +final fun (kotlinx.coroutines.test/TestScope).kotlinx.coroutines.test/runTestLegacy(kotlin/Long, kotlin.coroutines/SuspendFunction1, kotlin/Int, kotlin/Any?): kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting // kotlinx.coroutines.test/runTestLegacy|runTestLegacy@kotlinx.coroutines.test.TestScope(kotlin.Long;kotlin.coroutines.SuspendFunction1;kotlin.Int;kotlin.Any?){}[0] + +// Targets: [js, wasmJs] +final fun kotlinx.coroutines.test/runTest(kotlin.coroutines/CoroutineContext = ..., kotlin.time/Duration = ..., kotlin.coroutines/SuspendFunction1): kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting // kotlinx.coroutines.test/runTest|runTest(kotlin.coroutines.CoroutineContext;kotlin.time.Duration;kotlin.coroutines.SuspendFunction1){}[0] + +// Targets: [js, wasmJs] +final fun kotlinx.coroutines.test/runTest(kotlin.coroutines/CoroutineContext = ..., kotlin/Long, kotlin.coroutines/SuspendFunction1): kotlinx.coroutines.test.internal/JsPromiseInterfaceForTesting // kotlinx.coroutines.test/runTest|runTest(kotlin.coroutines.CoroutineContext;kotlin.Long;kotlin.coroutines.SuspendFunction1){}[0] diff --git a/kotlinx-coroutines-test/build.gradle.kts b/kotlinx-coroutines-test/build.gradle.kts new file mode 100644 index 0000000000..5e29e6ae75 --- /dev/null +++ b/kotlinx-coroutines-test/build.gradle.kts @@ -0,0 +1,23 @@ +import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl + +kotlin { + sourceSets { + jvmTest { + dependencies { + implementation(project(":kotlinx-coroutines-debug")) + } + } + } + + @OptIn(ExperimentalWasmDsl::class) + wasmJs { + nodejs { + testTask { + filter.apply { + // https://youtrack.jetbrains.com/issue/KT-61888 + excludeTest("TestDispatchersTest", "testMainMocking") + } + } + } + } +} diff --git a/kotlinx-coroutines-test/common/src/TestBuilders.kt b/kotlinx-coroutines-test/common/src/TestBuilders.kt new file mode 100644 index 0000000000..57ae3e1150 --- /dev/null +++ b/kotlinx-coroutines-test/common/src/TestBuilders.kt @@ -0,0 +1,614 @@ +@file:JvmName("TestBuildersKt") +@file:JvmMultifileClass + +package kotlinx.coroutines.test + +import kotlinx.atomicfu.atomic +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import kotlin.coroutines.* +import kotlin.jvm.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.seconds + +/** + * A test result. + * + * - On JVM and Native, this resolves to [Unit], representing the fact that tests are run in a blocking manner on these + * platforms: a call to a function returning a [TestResult] will simply execute the test inside it. + * - On JS, this is a `Promise`, which reflects the fact that the test-running function does not wait for a test to + * finish. The JS test frameworks typically support returning `Promise` from a test and will correctly handle it. + * + * Because of the behavior on JS, extra care must be taken when writing multiplatform tests to avoid losing test errors: + * - Don't do anything after running the functions returning a [TestResult]. On JS, this code will execute *before* the + * test finishes. + * - As a corollary, don't run functions returning a [TestResult] more than once per test. The only valid thing to do + * with a [TestResult] is to immediately `return` it from a test. + * - Don't nest functions returning a [TestResult]. + */ +public expect class TestResult + +/** + * Executes [testBody] as a test in a new coroutine, returning [TestResult]. + * + * On JVM and Native, this function behaves similarly to `runBlocking`, with the difference that the code that it runs + * will skip delays. This allows to use [delay] in tests without causing them to take more time than necessary. + * On JS, this function creates a `Promise` that executes the test body with the delay-skipping behavior. + * + * ``` + * @Test + * fun exampleTest() = runTest { + * val deferred = async { + * delay(1.seconds) + * async { + * delay(1.seconds) + * }.await() + * } + * + * deferred.await() // result available immediately + * } + * ``` + * + * The platform difference entails that, in order to use this function correctly in common code, one must always + * immediately return the produced [TestResult] from the test method, without doing anything else afterwards. See + * [TestResult] for details on this. + * + * The test is run on a single thread, unless other [CoroutineDispatcher] are used for child coroutines. + * Because of this, child coroutines are not executed in parallel to the test body. + * In order for the spawned-off asynchronous code to actually be executed, one must either [yield] or suspend the + * test body some other way, or use commands that control scheduling (see [TestCoroutineScheduler]). + * + * ``` + * @Test + * fun exampleWaitingForAsyncTasks1() = runTest { + * // 1 + * val job = launch { + * // 3 + * } + * // 2 + * job.join() // the main test coroutine suspends here, so the child is executed + * // 4 + * } + * + * @Test + * fun exampleWaitingForAsyncTasks2() = runTest { + * // 1 + * launch { + * // 3 + * } + * // 2 + * testScheduler.advanceUntilIdle() // runs the tasks until their queue is empty + * // 4 + * } + * ``` + * + * ### Task scheduling + * + * Delay skipping is achieved by using virtual time. + * If [Dispatchers.Main] is set to a [TestDispatcher] via [Dispatchers.setMain] before the test, + * then its [TestCoroutineScheduler] is used; + * otherwise, a new one is automatically created (or taken from [context] in some way) and can be used to control + * the virtual time, advancing it, running the tasks scheduled at a specific time etc. + * The scheduler can be accessed via [TestScope.testScheduler]. + * + * Delays in code that runs inside dispatchers that don't use a [TestCoroutineScheduler] don't get skipped: + * ``` + * @Test + * fun exampleTest() = runTest { + * val elapsed = TimeSource.Monotonic.measureTime { + * val deferred = async { + * delay(1.seconds) // will be skipped + * withContext(Dispatchers.Default) { + * delay(5.seconds) // Dispatchers.Default doesn't know about TestCoroutineScheduler + * } + * } + * deferred.await() + * } + * println(elapsed) // about five seconds + * } + * ``` + * + * ### Failures + * + * #### Test body failures + * + * If the created coroutine completes with an exception, then this exception will be thrown at the end of the test. + * + * #### Timing out + * + * There's a built-in timeout of 60 seconds for the test body. If the test body doesn't complete within this time, + * then the test fails with an [AssertionError]. The timeout can be changed for each test separately by setting the + * [timeout] parameter. + * + * Additionally, setting the `kotlinx.coroutines.test.default_timeout` system property on the + * JVM to any string that can be parsed using [Duration.parse] (like `1m`, `30s` or `1500ms`) will change the default + * timeout to that value for all tests whose [timeout] is not set explicitly; setting it to anything else will throw an + * exception every time [runTest] is invoked. + * + * On timeout, the test body is cancelled so that the test finishes. If the code inside the test body does not + * respond to cancellation, the timeout will not be able to make the test execution stop. + * In that case, the test will hang despite the attempt to terminate it. + * + * On the JVM, if `DebugProbes` from the `kotlinx-coroutines-debug` module are installed, the current dump of the + * coroutines' stack is printed to the console on timeout before the test body is cancelled. + * + * #### Reported exceptions + * + * Unhandled exceptions will be thrown at the end of the test. + * If uncaught exceptions happen after the test finishes, they are propagated in a platform-specific manner: + * see [handleCoroutineException] for details. + * If the test coroutine completes with an exception, the unhandled exceptions are suppressed by it. + * + * #### Uncompleted coroutines + * + * Otherwise, the test will hang until all the coroutines launched inside [testBody] complete. + * This may be an issue when there are some coroutines that are not supposed to complete, like infinite loops that + * perform some background work and are supposed to outlive the test. + * In that case, [TestScope.backgroundScope] can be used to launch such coroutines. + * They will be cancelled automatically when the test finishes. + * + * ### Configuration + * + * [context] can be used to affect the environment of the code under test. Beside just being passed to the coroutine + * scope created for the test, [context] also can be used to change how the test is executed. + * See the [TestScope] constructor function documentation for details. + * + * @throws IllegalArgumentException if the [context] is invalid. See the [TestScope] constructor docs for details. + */ +public fun runTest( + context: CoroutineContext = EmptyCoroutineContext, + timeout: Duration = DEFAULT_TIMEOUT.getOrThrow(), + testBody: suspend TestScope.() -> Unit +): TestResult { + check(context[RunningInRunTest] == null) { + "Calls to `runTest` can't be nested. Please read the docs on `TestResult` for details." + } + return TestScope(context + RunningInRunTest).runTest(timeout, testBody) +} + +/** + * Executes [testBody] as a test in a new coroutine, returning [TestResult]. + * + * On JVM and Native, this function behaves similarly to `runBlocking`, with the difference that the code that it runs + * will skip delays. This allows to use [delay] in without causing the tests to take more time than necessary. + * On JS, this function creates a `Promise` that executes the test body with the delay-skipping behavior. + * + * ``` + * @Test + * fun exampleTest() = runTest { + * val deferred = async { + * delay(1.seconds) + * async { + * delay(1.seconds) + * }.await() + * } + * + * deferred.await() // result available immediately + * } + * ``` + * + * The platform difference entails that, in order to use this function correctly in common code, one must always + * immediately return the produced [TestResult] from the test method, without doing anything else afterwards. See + * [TestResult] for details on this. + * + * The test is run in a single thread, unless other [CoroutineDispatcher] are used for child coroutines. + * Because of this, child coroutines are not executed in parallel to the test body. + * In order for the spawned-off asynchronous code to actually be executed, one must either [yield] or suspend the + * test body some other way, or use commands that control scheduling (see [TestCoroutineScheduler]). + * + * ``` + * @Test + * fun exampleWaitingForAsyncTasks1() = runTest { + * // 1 + * val job = launch { + * // 3 + * } + * // 2 + * job.join() // the main test coroutine suspends here, so the child is executed + * // 4 + * } + * + * @Test + * fun exampleWaitingForAsyncTasks2() = runTest { + * // 1 + * launch { + * // 3 + * } + * // 2 + * advanceUntilIdle() // runs the tasks until their queue is empty + * // 4 + * } + * ``` + * + * ### Task scheduling + * + * Delay-skipping is achieved by using virtual time. + * If [Dispatchers.Main] is set to a [TestDispatcher] via [Dispatchers.setMain] before the test, + * then its [TestCoroutineScheduler] is used; + * otherwise, a new one is automatically created (or taken from [context] in some way) and can be used to control + * the virtual time, advancing it, running the tasks scheduled at a specific time etc. + * Some convenience methods are available on [TestScope] to control the scheduler. + * + * Delays in code that runs inside dispatchers that don't use a [TestCoroutineScheduler] don't get skipped: + * ``` + * @Test + * fun exampleTest() = runTest { + * val elapsed = TimeSource.Monotonic.measureTime { + * val deferred = async { + * delay(1.seconds) // will be skipped + * withContext(Dispatchers.Default) { + * delay(5.seconds) // Dispatchers.Default doesn't know about TestCoroutineScheduler + * } + * } + * deferred.await() + * } + * println(elapsed) // about five seconds + * } + * ``` + * + * ### Failures + * + * #### Test body failures + * + * If the created coroutine completes with an exception, then this exception will be thrown at the end of the test. + * + * #### Reported exceptions + * + * Unhandled exceptions will be thrown at the end of the test. + * If the uncaught exceptions happen after the test finishes, the error is propagated in a platform-specific manner. + * If the test coroutine completes with an exception, the unhandled exceptions are suppressed by it. + * + * #### Uncompleted coroutines + * + * This method requires that, after the test coroutine has completed, all the other coroutines launched inside + * [testBody] also complete, or are cancelled. + * Otherwise, the test will be failed (which, on JVM and Native, means that [runTest] itself will throw + * [AssertionError], whereas on JS, the `Promise` will fail with it). + * + * In the general case, if there are active jobs, it's impossible to detect if they are going to complete eventually due + * to the asynchronous nature of coroutines. In order to prevent tests hanging in this scenario, [runTest] will wait + * for [dispatchTimeoutMs] from the moment when [TestCoroutineScheduler] becomes + * idle before throwing [AssertionError]. If some dispatcher linked to [TestCoroutineScheduler] receives a + * task during that time, the timer gets reset. + * + * ### Configuration + * + * [context] can be used to affect the environment of the code under test. Beside just being passed to the coroutine + * scope created for the test, [context] also can be used to change how the test is executed. + * See the [TestScope] constructor function documentation for details. + * + * @throws IllegalArgumentException if the [context] is invalid. See the [TestScope] constructor docs for details. + */ +@Deprecated( + "Define a total timeout for the whole test instead of using dispatchTimeoutMs. " + + "Warning: the proposed replacement is not identical as it uses 'dispatchTimeoutMs' as the timeout for the whole test!", + ReplaceWith("runTest(context, timeout = dispatchTimeoutMs.milliseconds, testBody)", + "kotlin.time.Duration.Companion.milliseconds"), + DeprecationLevel.WARNING +) // Warning since 1.7.0, was experimental in 1.6.x +public fun runTest( + context: CoroutineContext = EmptyCoroutineContext, + dispatchTimeoutMs: Long, + testBody: suspend TestScope.() -> Unit +): TestResult { + if (context[RunningInRunTest] != null) + throw IllegalStateException("Calls to `runTest` can't be nested. Please read the docs on `TestResult` for details.") + @Suppress("DEPRECATION") + return TestScope(context + RunningInRunTest).runTest(dispatchTimeoutMs = dispatchTimeoutMs, testBody) +} + +/** + * Performs [runTest] on an existing [TestScope]. See the documentation for [runTest] for details. + */ +public fun TestScope.runTest( + timeout: Duration = DEFAULT_TIMEOUT.getOrThrow(), + testBody: suspend TestScope.() -> Unit +): TestResult = asSpecificImplementation().let { scope -> + scope.enter() + createTestResult { + val testBodyFinished = AtomicBoolean(false) + /** TODO: moving this [AbstractCoroutine.start] call outside [createTestResult] fails on JS. */ + scope.start(CoroutineStart.UNDISPATCHED, scope) { + /* we're using `UNDISPATCHED` to avoid the event loop, but we do want to set up the timeout machinery + before any code executes, so we have to park here. */ + yield() + try { + testBody() + } finally { + testBodyFinished.value = true + } + } + var timeoutError: Throwable? = null + var cancellationException: CancellationException? = null + val workRunner = launch(CoroutineName("kotlinx.coroutines.test runner")) { + while (true) { + val executedSomething = testScheduler.tryRunNextTaskUnless { !isActive } + if (executedSomething) { + /** yield to check for cancellation. On JS, we can't use [ensureActive] here, as the cancellation + * procedure needs a chance to run concurrently. */ + yield() + } else { + // waiting for the next task to be scheduled, or for the test runner to be cancelled + testScheduler.receiveDispatchEvent() + } + } + } + try { + withTimeout(timeout) { + coroutineContext.job.invokeOnCompletion(onCancelling = true) { exception -> + if (exception is TimeoutCancellationException) { + dumpCoroutines() + val activeChildren = scope.children.filter(Job::isActive).toList() + val message = "After waiting for $timeout, " + when { + testBodyFinished.value && activeChildren.isNotEmpty() -> + "there were active child jobs: $activeChildren. " + + "Use `TestScope.backgroundScope` " + + "to launch the coroutines that need to be cancelled when the test body finishes" + testBodyFinished.value -> + "the test completed, but only after the timeout" + else -> + "the test body did not run to completion" + } + timeoutError = UncompletedCoroutinesError(message) + cancellationException = CancellationException("The test timed out") + (scope as Job).cancel(cancellationException!!) + } + } + scope.join() + workRunner.cancelAndJoin() + } + } catch (_: TimeoutCancellationException) { + scope.join() + val completion = scope.getCompletionExceptionOrNull() + if (completion != null && completion !== cancellationException) { + timeoutError!!.addSuppressed(completion) + } + workRunner.cancelAndJoin() + } finally { + backgroundScope.cancel() + testScheduler.advanceUntilIdleOr { false } + val uncaughtExceptions = scope.leave() + throwAll(timeoutError ?: scope.getCompletionExceptionOrNull(), uncaughtExceptions) + } + } +} + +/** + * Performs [runTest] on an existing [TestScope]. + * + * In the general case, if there are active jobs, it's impossible to detect if they are going to complete eventually due + * to the asynchronous nature of coroutines. In order to prevent tests hanging in this scenario, [runTest] will wait + * for [dispatchTimeoutMs] from the moment when [TestCoroutineScheduler] becomes + * idle before throwing [AssertionError]. If some dispatcher linked to [TestCoroutineScheduler] receives a + * task during that time, the timer gets reset. + */ +@Deprecated( + "Define a total timeout for the whole test instead of using dispatchTimeoutMs. " + + "Warning: the proposed replacement is not identical as it uses 'dispatchTimeoutMs' as the timeout for the whole test!", + ReplaceWith("this.runTest(timeout = dispatchTimeoutMs.milliseconds, testBody)", + "kotlin.time.Duration.Companion.milliseconds"), + DeprecationLevel.WARNING +) // Warning since 1.7.0, was experimental in 1.6.x +public fun TestScope.runTest( + dispatchTimeoutMs: Long, + testBody: suspend TestScope.() -> Unit +): TestResult = asSpecificImplementation().let { + it.enter() + @Suppress("DEPRECATION") + createTestResult { + runTestCoroutineLegacy(it, dispatchTimeoutMs.milliseconds, TestScopeImpl::tryGetCompletionCause, testBody) { + backgroundScope.cancel() + testScheduler.advanceUntilIdleOr { false } + it.legacyLeave() + } + } +} + +/** + * Runs [testProcedure], creating a [TestResult]. + */ +internal expect fun createTestResult(testProcedure: suspend CoroutineScope.() -> Unit): TestResult + +/** A coroutine context element indicating that the coroutine is running inside `runTest`. */ +internal object RunningInRunTest : CoroutineContext.Key, CoroutineContext.Element { + override val key: CoroutineContext.Key<*> + get() = this + + override fun toString(): String = "RunningInRunTest" +} + +/** The default timeout to use when waiting for asynchronous completions of the coroutines managed by + * a [TestCoroutineScheduler]. */ +internal const val DEFAULT_DISPATCH_TIMEOUT_MS = 60_000L + +/** + * The default timeout to use when running a test. + * + * It's not just a [Duration] but a [Result] so that every access to [runTest] + * throws the same clear exception if parsing the environment variable failed. + * Otherwise, the parsing error would only be thrown in one tests, while the + * other ones would get an incomprehensible `NoClassDefFoundError`. + */ +private val DEFAULT_TIMEOUT: Result = runCatching { + systemProperty("kotlinx.coroutines.test.default_timeout", Duration::parse, 60.seconds) +} + +/** + * Run the [body][testBody] of the [test coroutine][coroutine], waiting for asynchronous completions for at most + * [dispatchTimeout] and performing the [cleanup] procedure at the end. + * + * [tryGetCompletionCause] is the [JobSupport.completionCause], which is passed explicitly because it is protected. + * + * The [cleanup] procedure may either throw [UncompletedCoroutinesError] to denote that child coroutines were leaked, or + * return a list of uncaught exceptions that should be reported at the end of the test. + */ +@Deprecated("Used for support of legacy behavior") +internal suspend fun > CoroutineScope.runTestCoroutineLegacy( + coroutine: T, + dispatchTimeout: Duration, + tryGetCompletionCause: T.() -> Throwable?, + testBody: suspend T.() -> Unit, + cleanup: () -> List, +) { + val scheduler = coroutine.coroutineContext[TestCoroutineScheduler]!! + /** TODO: moving this [AbstractCoroutine.start] call outside [createTestResult] fails on JS. */ + coroutine.start(CoroutineStart.UNDISPATCHED, coroutine) { + testBody() + } + /** + * This is the legacy behavior, kept for now for compatibility only. + * + * The general procedure here is as follows: + * 1. Try running the work that the scheduler knows about, both background and foreground. + * + * 2. Wait until we run out of foreground work to do. This could mean one of the following: + * - The main coroutine is already completed. This is checked separately; then we leave the procedure. + * - It's switched to another dispatcher that doesn't know about the [TestCoroutineScheduler]. + * - Generally, it's waiting for something external (like a network request, or just an arbitrary callback). + * - The test simply hanged. + * - The main coroutine is waiting for some background work. + * + * 3. We await progress from things that are not the code under test: + * the background work that the scheduler knows about, the external callbacks, + * the work on dispatchers not linked to the scheduler, etc. + * + * When we observe that the code under test can proceed, we go to step 1 again. + * If there is no activity for [dispatchTimeoutMs] milliseconds, we consider the test to have hanged. + * + * The background work is not running on a dedicated thread. + * Instead, the test thread itself is used, by spawning a separate coroutine. + */ + var completed = false + while (!completed) { + scheduler.advanceUntilIdle() + if (coroutine.isCompleted) { + /* don't even enter `withTimeout`; this allows to use a timeout of zero to check that there are no + non-trivial dispatches. */ + completed = true + continue + } + // in case progress depends on some background work, we need to keep spinning it. + val backgroundWorkRunner = launch(CoroutineName("background work runner")) { + while (true) { + val executedSomething = scheduler.tryRunNextTaskUnless { !isActive } + if (executedSomething) { + // yield so that the `select` below has a chance to finish successfully or time out + yield() + } else { + // no more tasks, we should suspend until there are some more. + // this doesn't interfere with the `select` below, because different channels are used. + scheduler.receiveDispatchEvent() + } + } + } + try { + select { + coroutine.onJoin { + // observe that someone completed the test coroutine and leave without waiting for the timeout + completed = true + } + scheduler.onDispatchEventForeground { + // we received knowledge that `scheduler` observed a dispatch event, so we reset the timeout + } + onTimeout(dispatchTimeout) { + throw handleTimeout(coroutine, dispatchTimeout, tryGetCompletionCause, cleanup) + } + } + } finally { + backgroundWorkRunner.cancelAndJoin() + } + } + coroutine.getCompletionExceptionOrNull()?.let { exception -> + val exceptions = try { + cleanup() + } catch (e: UncompletedCoroutinesError) { + // it's normal that some jobs are not completed if the test body has failed, won't clutter the output + emptyList() + } + throwAll(exception, exceptions) + } + throwAll(null, cleanup()) +} + +/** + * Invoked on timeout in [runTest]. Just builds a nice [UncompletedCoroutinesError] and returns it. + */ +private inline fun > handleTimeout( + coroutine: T, + dispatchTimeout: Duration, + tryGetCompletionCause: T.() -> Throwable?, + cleanup: () -> List, +): AssertionError { + val uncaughtExceptions = try { + cleanup() + } catch (e: UncompletedCoroutinesError) { + // we expect these and will instead throw a more informative exception. + emptyList() + } + val activeChildren = coroutine.children.filter { it.isActive }.toList() + val completionCause = if (coroutine.isCancelled) coroutine.tryGetCompletionCause() else null + var message = "After waiting for $dispatchTimeout" + if (completionCause == null) + message += ", the test coroutine is not completing" + if (activeChildren.isNotEmpty()) + message += ", there were active child jobs: $activeChildren" + if (completionCause != null && activeChildren.isEmpty()) { + message += if (coroutine.isCompleted) + ", the test coroutine completed" + else + ", the test coroutine was not completed" + } + val error = UncompletedCoroutinesError(message) + completionCause?.let { cause -> error.addSuppressed(cause) } + uncaughtExceptions.forEach { error.addSuppressed(it) } + return error +} + +internal fun throwAll(head: Throwable?, other: List) { + if (head != null) { + other.forEach { head.addSuppressed(it) } + throw head + } else { + with(other) { + firstOrNull()?.apply { + drop(1).forEach { addSuppressed(it) } + throw this + } + } + } +} + +internal expect fun dumpCoroutines() + +private fun systemProperty( + name: String, + parse: (String) -> T, + default: T, +): T { + val value = systemPropertyImpl(name) ?: return default + return parse(value) +} + +internal expect fun systemPropertyImpl(name: String): String? + +@Deprecated( + "This is for binary compatibility with the `runTest` overload that existed at some point", + level = DeprecationLevel.HIDDEN +) +@JvmName("runTest\$default") +@Suppress("DEPRECATION", "UNUSED_PARAMETER") +public fun TestScope.runTestLegacy( + dispatchTimeoutMs: Long, + testBody: suspend TestScope.() -> Unit, + marker: Int, + unused2: Any?, +): TestResult = runTest(dispatchTimeoutMs = if (marker and 1 != 0) dispatchTimeoutMs else 60_000L, testBody) + +// Remove after https://youtrack.jetbrains.com/issue/KT-62423/ +private class AtomicBoolean(initial: Boolean) { + private val container = atomic(initial) + var value: Boolean + get() = container.value + set(value: Boolean) { container.value = value } +} diff --git a/kotlinx-coroutines-test/common/src/TestCoroutineDispatchers.kt b/kotlinx-coroutines-test/common/src/TestCoroutineDispatchers.kt new file mode 100644 index 0000000000..bf1b62a171 --- /dev/null +++ b/kotlinx-coroutines-test/common/src/TestCoroutineDispatchers.kt @@ -0,0 +1,154 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.test.internal.TestMainDispatcher +import kotlin.coroutines.* + +/** + * Creates an instance of an unconfined [TestDispatcher]. + * + * This dispatcher is similar to [Dispatchers.Unconfined]: the tasks that it executes are not confined to any particular + * thread and form an event loop; it's different in that it skips delays, as all [TestDispatcher]s do. + * + * Like [Dispatchers.Unconfined], this one does not provide guarantees about the execution order when several coroutines + * are queued in this dispatcher. However, we ensure that the [launch] and [async] blocks at the top level of [runTest] + * are entered eagerly. This allows launching child coroutines and not calling [runCurrent] for them to start executing. + * + * ``` + * @Test + * fun testEagerlyEnteringChildCoroutines() = runTest(UnconfinedTestDispatcher()) { + * var entered = false + * val deferred = CompletableDeferred() + * var completed = false + * launch { + * entered = true + * deferred.await() + * completed = true + * } + * assertTrue(entered) // `entered = true` already executed. + * assertFalse(completed) // however, the child coroutine then suspended, so it is enqueued. + * deferred.complete(Unit) // resume the coroutine. + * assertTrue(completed) // now the child coroutine is immediately completed. + * } + * ``` + * + * Using this [TestDispatcher] can greatly simplify writing tests where it's not important which thread is used when and + * in which order the queued coroutines are executed. + * Another typical use case for this dispatcher is launching child coroutines that are resumed immediately, without + * going through a dispatch; this can be helpful for testing [Channel] and [StateFlow] usages. + * + * ``` + * @Test + * fun testUnconfinedDispatcher() = runTest { + * val values = mutableListOf() + * val stateFlow = MutableStateFlow(0) + * val job = launch(UnconfinedTestDispatcher(testScheduler)) { + * stateFlow.collect { + * values.add(it) + * } + * } + * stateFlow.value = 1 + * stateFlow.value = 2 + * stateFlow.value = 3 + * job.cancel() + * // each assignment will immediately resume the collecting child coroutine, + * // so no values will be skipped. + * assertEquals(listOf(0, 1, 2, 3), values) + * } + * ``` + * + * Please be aware that, like [Dispatchers.Unconfined], this is a specific dispatcher with execution order + * guarantees that are unusual and not shared by most other dispatchers, so it can only be used reliably for testing + * functionality, not the specific order of actions. + * See [Dispatchers.Unconfined] for a discussion of the execution order guarantees. + * + * In order to support delay skipping, this dispatcher is linked to a [TestCoroutineScheduler], which is used to control + * the virtual time and can be shared among many test dispatchers. + * If no [scheduler] is passed as an argument, [Dispatchers.Main] is checked, and if it was mocked with a + * [TestDispatcher] via [Dispatchers.setMain], the [TestDispatcher.scheduler] of the mock dispatcher is used; if + * [Dispatchers.Main] is not mocked with a [TestDispatcher], a new [TestCoroutineScheduler] is created. + * + * Additionally, [name] can be set to distinguish each dispatcher instance when debugging. + * + * @see StandardTestDispatcher for a more predictable [TestDispatcher]. + */ +@ExperimentalCoroutinesApi +@Suppress("FunctionName") +public fun UnconfinedTestDispatcher( + scheduler: TestCoroutineScheduler? = null, + name: String? = null +): TestDispatcher = UnconfinedTestDispatcherImpl( + scheduler ?: TestMainDispatcher.currentTestScheduler ?: TestCoroutineScheduler(), name) + +private class UnconfinedTestDispatcherImpl( + override val scheduler: TestCoroutineScheduler, + private val name: String? = null +) : TestDispatcher() { + + override fun isDispatchNeeded(context: CoroutineContext): Boolean = false + + // do not remove the INVISIBLE_REFERENCE and INVISIBLE_SETTER suppressions: required in K2 + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE", "INVISIBLE_SETTER") + override fun dispatch(context: CoroutineContext, block: Runnable) { + checkSchedulerInContext(scheduler, context) + scheduler.sendDispatchEvent(context) + + /** copy-pasted from [kotlinx.coroutines.Unconfined.dispatch] */ + /** It can only be called by the [yield] function. See also code of [yield] function. */ + val yieldContext = context[YieldContext] + if (yieldContext !== null) { + // report to "yield" that it is an unconfined dispatcher and don't call "block.run()" + yieldContext.dispatcherWasUnconfined = true + return + } + throw UnsupportedOperationException( + "Function UnconfinedTestCoroutineDispatcher.dispatch can only be used by " + + "the yield function. If you wrap Unconfined dispatcher in your code, make sure you properly delegate " + + "isDispatchNeeded and dispatch calls." + ) + } + + override fun toString(): String = "${name ?: "UnconfinedTestDispatcher"}[scheduler=$scheduler]" +} + +/** + * Creates an instance of a [TestDispatcher] whose tasks are run inside calls to the [scheduler]. + * + * This [TestDispatcher] instance does not itself execute any of the tasks. Instead, it always sends them to its + * [scheduler], which can then be accessed via [TestCoroutineScheduler.runCurrent], + * [TestCoroutineScheduler.advanceUntilIdle], or [TestCoroutineScheduler.advanceTimeBy], which will then execute these + * tasks in a blocking manner. + * + * In practice, this means that [launch] or [async] blocks will not be entered immediately (unless they are + * parameterized with [CoroutineStart.UNDISPATCHED]), and one should either call [TestCoroutineScheduler.runCurrent] to + * run these pending tasks, which will block until there are no more tasks scheduled at this point in time, or, when + * inside [runTest], call [yield] to yield the (only) thread used by [runTest] to the newly-launched coroutines. + * + * If no [scheduler] is passed as an argument, [Dispatchers.Main] is checked, and if it was mocked with a + * [TestDispatcher] via [Dispatchers.setMain], the [TestDispatcher.scheduler] of the mock dispatcher is used; if + * [Dispatchers.Main] is not mocked with a [TestDispatcher], a new [TestCoroutineScheduler] is created. + * + * One can additionally pass a [name] in order to more easily distinguish this dispatcher during debugging. + * + * @see UnconfinedTestDispatcher for a dispatcher that is not confined to any particular thread. + */ +@Suppress("FunctionName") +public fun StandardTestDispatcher( + scheduler: TestCoroutineScheduler? = null, + name: String? = null +): TestDispatcher = StandardTestDispatcherImpl( + scheduler ?: TestMainDispatcher.currentTestScheduler ?: TestCoroutineScheduler(), name) + +private class StandardTestDispatcherImpl( + override val scheduler: TestCoroutineScheduler = TestCoroutineScheduler(), + private val name: String? = null +) : TestDispatcher() { + + override fun dispatch(context: CoroutineContext, block: Runnable) { + scheduler.registerEvent(this, 0, block, context) { false } + } + + override fun toString(): String = "${name ?: "StandardTestDispatcher"}[scheduler=$scheduler]" +} diff --git a/kotlinx-coroutines-test/common/src/TestCoroutineScheduler.kt b/kotlinx-coroutines-test/common/src/TestCoroutineScheduler.kt new file mode 100644 index 0000000000..8c70fa8e05 --- /dev/null +++ b/kotlinx-coroutines-test/common/src/TestCoroutineScheduler.kt @@ -0,0 +1,276 @@ +package kotlinx.coroutines.test + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.channels.Channel.Factory.CONFLATED +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlin.coroutines.* +import kotlin.jvm.* +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds + +/** + * This is a scheduler for coroutines used in tests, providing the delay-skipping behavior. + * + * [Test dispatchers][TestDispatcher] are parameterized with a scheduler. Several dispatchers can share the + * same scheduler, in which case their knowledge about the virtual time will be synchronized. When the dispatchers + * require scheduling an event at a later point in time, they notify the scheduler, which will establish the order of + * the tasks. + * + * The scheduler can be queried to advance the time (via [advanceTimeBy]), run all the scheduled tasks advancing the + * virtual time as needed (via [advanceUntilIdle]), or run the tasks that are scheduled to run as soon as possible but + * haven't yet been dispatched (via [runCurrent]). + */ +public class TestCoroutineScheduler : AbstractCoroutineContextElement(TestCoroutineScheduler), + CoroutineContext.Element { + + /** @suppress */ + public companion object Key : CoroutineContext.Key + + /** This heap stores the knowledge about which dispatchers are interested in which moments of virtual time. */ + // TODO: all the synchronization is done via a separate lock, so a non-thread-safe priority queue can be used. + private val events = ThreadSafeHeap>() + + /** Establishes that [currentTime] can't exceed the time of the earliest event in [events]. */ + private val lock = SynchronizedObject() + + /** This counter establishes some order on the events that happen at the same virtual time. */ + private val count = atomic(0L) + + /** The current virtual time in milliseconds. */ + @ExperimentalCoroutinesApi + public var currentTime: Long = 0 + get() = synchronized(lock) { field } + private set + + /** A channel for notifying about the fact that a foreground work dispatch recently happened. */ + private val dispatchEventsForeground: Channel = Channel(CONFLATED) + + /** A channel for notifying about the fact that a dispatch recently happened. */ + private val dispatchEvents: Channel = Channel(CONFLATED) + + /** + * Registers a request for the scheduler to notify [dispatcher] at a virtual moment [timeDeltaMillis] milliseconds + * later via [TestDispatcher.processEvent], which will be called with the provided [marker] object. + * + * Returns the handler which can be used to cancel the registration. + */ + internal fun registerEvent( + dispatcher: TestDispatcher, + timeDeltaMillis: Long, + marker: T, + context: CoroutineContext, + isCancelled: (T) -> Boolean + ): DisposableHandle { + require(timeDeltaMillis >= 0) { "Attempted scheduling an event earlier in time (with the time delta $timeDeltaMillis)" } + checkSchedulerInContext(this, context) + val count = count.getAndIncrement() + val isForeground = context[BackgroundWork] === null + return synchronized(lock) { + val time = addClamping(currentTime, timeDeltaMillis) + val event = TestDispatchEvent(dispatcher, count, time, marker as Any, isForeground) { isCancelled(marker) } + events.addLast(event) + /** can't be moved above: otherwise, [onDispatchEventForeground] or [onDispatchEvent] could consume the + * token sent here before there's actually anything in the event queue. */ + sendDispatchEvent(context) + DisposableHandle { + synchronized(lock) { + events.remove(event) + } + } + } + } + + /** + * Runs the next enqueued task, advancing the virtual time to the time of its scheduled awakening, + * unless [condition] holds. + */ + internal fun tryRunNextTaskUnless(condition: () -> Boolean): Boolean { + val event = synchronized(lock) { + if (condition()) return false + val event = events.removeFirstOrNull() ?: return false + if (currentTime > event.time) + currentTimeAheadOfEvents() + currentTime = event.time + event + } + event.dispatcher.processEvent(event.marker) + return true + } + + /** + * Runs the enqueued tasks in the specified order, advancing the virtual time as needed until there are no more + * tasks associated with the dispatchers linked to this scheduler. + * + * A breaking change from `TestCoroutineDispatcher.advanceTimeBy` is that it no longer returns the total number of + * milliseconds by which the execution of this method has advanced the virtual time. If you want to recreate that + * functionality, query [currentTime] before and after the execution to achieve the same result. + */ + public fun advanceUntilIdle(): Unit = advanceUntilIdleOr { events.none(TestDispatchEvent<*>::isForeground) } + + /** + * [condition]: guaranteed to be invoked under the lock. + */ + internal fun advanceUntilIdleOr(condition: () -> Boolean) { + while (true) { + if (!tryRunNextTaskUnless(condition)) + return + } + } + + /** + * Runs the tasks that are scheduled to execute at this moment of virtual time. + */ + public fun runCurrent() { + val timeMark = synchronized(lock) { currentTime } + while (true) { + val event = synchronized(lock) { + events.removeFirstIf { it.time <= timeMark } ?: return + } + event.dispatcher.processEvent(event.marker) + } + } + + /** + * Moves the virtual clock of this dispatcher forward by [the specified amount][delayTimeMillis], running the + * scheduled tasks in the meantime. + * + * Breaking changes from [TestCoroutineDispatcher.advanceTimeBy]: + * - Intentionally doesn't return a `Long` value, as its use cases are unclear. We may restore it in the future; + * please describe your use cases at [the issue tracker](https://github.com/Kotlin/kotlinx.coroutines/issues/). + * For now, it's possible to query [currentTime] before and after execution of this method, to the same effect. + * - It doesn't run the tasks that are scheduled at exactly [currentTime] + [delayTimeMillis]. For example, + * advancing the time by one millisecond used to run the tasks at the current millisecond *and* the next + * millisecond, but now will stop just before executing any task starting at the next millisecond. + * - Overflowing the target time used to lead to nothing being done, but will now run the tasks scheduled at up to + * (but not including) [Long.MAX_VALUE]. + * + * @throws IllegalArgumentException if passed a negative [delay][delayTimeMillis]. + */ + @ExperimentalCoroutinesApi + public fun advanceTimeBy(delayTimeMillis: Long): Unit = advanceTimeBy(delayTimeMillis.milliseconds) + + /** + * Moves the virtual clock of this dispatcher forward by [the specified amount][delayTime], running the + * scheduled tasks in the meantime. + * + * @throws IllegalArgumentException if passed a negative [delay][delayTime]. + */ + public fun advanceTimeBy(delayTime: Duration) { + require(!delayTime.isNegative()) { "Can not advance time by a negative delay: $delayTime" } + val startingTime = currentTime + val targetTime = addClamping(startingTime, delayTime.inWholeMilliseconds) + while (true) { + val event = synchronized(lock) { + val timeMark = currentTime + val event = events.removeFirstIf { targetTime > it.time } + when { + event == null -> { + currentTime = targetTime + return + } + timeMark > event.time -> currentTimeAheadOfEvents() + else -> { + currentTime = event.time + event + } + } + } + event.dispatcher.processEvent(event.marker) + } + } + + /** + * Checks that the only tasks remaining in the scheduler are cancelled. + */ + internal fun isIdle(strict: Boolean = true): Boolean = + synchronized(lock) { + if (strict) events.isEmpty else events.none { !it.isCancelled() } + } + + /** + * Notifies this scheduler about a dispatch event. + * + * [context] is the context in which the task will be dispatched. + */ + internal fun sendDispatchEvent(context: CoroutineContext) { + dispatchEvents.trySend(Unit) + if (context[BackgroundWork] !== BackgroundWork) + dispatchEventsForeground.trySend(Unit) + } + + /** + * Waits for a notification about a dispatch event. + */ + internal suspend fun receiveDispatchEvent() = dispatchEvents.receive() + + /** + * Consumes the knowledge that a dispatch event happened recently. + */ + internal val onDispatchEvent: SelectClause1 get() = dispatchEvents.onReceive + + /** + * Consumes the knowledge that a foreground work dispatch event happened recently. + */ + internal val onDispatchEventForeground: SelectClause1 get() = dispatchEventsForeground.onReceive + + /** + * Returns the [TimeSource] representation of the virtual time of this scheduler. + */ + public val timeSource: TimeSource.WithComparableMarks = object : AbstractLongTimeSource(DurationUnit.MILLISECONDS) { + override fun read(): Long = currentTime + } +} + +// Some error-throwing functions for pretty stack traces +private fun currentTimeAheadOfEvents(): Nothing = invalidSchedulerState() + +private fun invalidSchedulerState(): Nothing = + throw IllegalStateException("The test scheduler entered an invalid state. Please report this at https://github.com/Kotlin/kotlinx.coroutines/issues.") + +/** [ThreadSafeHeap] node representing a scheduled task, ordered by the planned execution time. */ +private class TestDispatchEvent( + @JvmField val dispatcher: TestDispatcher, + private val count: Long, + @JvmField val time: Long, + @JvmField val marker: T, + @JvmField val isForeground: Boolean, + // TODO: remove once the deprecated API is gone + @JvmField val isCancelled: () -> Boolean +) : Comparable>, ThreadSafeHeapNode { + override var heap: ThreadSafeHeap<*>? = null + override var index: Int = 0 + + override fun compareTo(other: TestDispatchEvent<*>) = + compareValuesBy(this, other, TestDispatchEvent<*>::time, TestDispatchEvent<*>::count) + + override fun toString() = "TestDispatchEvent(time=$time, dispatcher=$dispatcher${if (isForeground) "" else ", background"})" +} + +// works with positive `a`, `b` +private fun addClamping(a: Long, b: Long): Long = (a + b).let { if (it >= 0) it else Long.MAX_VALUE } + +internal fun checkSchedulerInContext(scheduler: TestCoroutineScheduler, context: CoroutineContext) { + context[TestCoroutineScheduler]?.let { + check(it === scheduler) { + "Detected use of different schedulers. If you need to use several test coroutine dispatchers, " + + "create one `TestCoroutineScheduler` and pass it to each of them." + } + } +} + +/** + * A coroutine context key denoting that the work is to be executed in the background. + * @see [TestScope.backgroundScope] + */ +internal object BackgroundWork : CoroutineContext.Key, CoroutineContext.Element { + override val key: CoroutineContext.Key<*> + get() = this + + override fun toString(): String = "BackgroundWork" +} + +private fun ThreadSafeHeap.none(predicate: (T) -> Boolean) where T: ThreadSafeHeapNode, T: Comparable = + find(predicate) == null diff --git a/kotlinx-coroutines-test/common/src/TestDispatcher.kt b/kotlinx-coroutines-test/common/src/TestDispatcher.kt new file mode 100644 index 0000000000..a4427a1a6f --- /dev/null +++ b/kotlinx-coroutines-test/common/src/TestDispatcher.kt @@ -0,0 +1,64 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.jvm.* +import kotlin.time.* + +/** + * A test dispatcher that can interface with a [TestCoroutineScheduler]. + * + * The available implementations are: + * - [StandardTestDispatcher] is a dispatcher that places new tasks into a queue. + * - [UnconfinedTestDispatcher] is a dispatcher that behaves like [Dispatchers.Unconfined] while allowing to control + * the virtual time. + */ +@Suppress("INVISIBLE_REFERENCE") +public abstract class TestDispatcher internal constructor() : CoroutineDispatcher(), Delay, DelayWithTimeoutDiagnostics { + /** The scheduler that this dispatcher is linked to. */ + public abstract val scheduler: TestCoroutineScheduler + + /** Notifies the dispatcher that it should process a single event marked with [marker] happening at time [time]. */ + internal fun processEvent(marker: Any) { + check(marker is Runnable) + marker.run() + } + + /** @suppress */ + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val timedRunnable = CancellableContinuationRunnable(continuation, this) + val handle = scheduler.registerEvent( + this, + timeMillis, + timedRunnable, + continuation.context, + ::cancellableRunnableIsCancelled + ) + continuation.disposeOnCancellation(handle) + } + + /** @suppress */ + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + scheduler.registerEvent(this, timeMillis, block, context) { false } + + /** @suppress */ + @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") + @Deprecated("Is only needed internally", level = DeprecationLevel.HIDDEN) + public override fun timeoutMessage(timeout: Duration): String = + "Timed out after $timeout of _virtual_ (kotlinx.coroutines.test) time. " + + "To use the real time, wrap 'withTimeout' in 'withContext(Dispatchers.Default.limitedParallelism(1))'" +} + +/** + * This class exists to allow cleanup code to avoid throwing for cancelled continuations scheduled + * in the future. + */ +private class CancellableContinuationRunnable( + @JvmField val continuation: CancellableContinuation, + private val dispatcher: CoroutineDispatcher +) : Runnable { + override fun run() = with(dispatcher) { with(continuation) { resumeUndispatched(Unit) } } +} + +private fun cancellableRunnableIsCancelled(runnable: CancellableContinuationRunnable): Boolean = + !runnable.continuation.isActive diff --git a/kotlinx-coroutines-test/common/src/TestDispatchers.kt b/kotlinx-coroutines-test/common/src/TestDispatchers.kt new file mode 100644 index 0000000000..a9e4524507 --- /dev/null +++ b/kotlinx-coroutines-test/common/src/TestDispatchers.kt @@ -0,0 +1,35 @@ +@file:JvmName("TestDispatchers") + +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.test.internal.* +import kotlin.jvm.* + +/** + * Sets the given [dispatcher] as an underlying dispatcher of [Dispatchers.Main]. + * All subsequent usages of [Dispatchers.Main] will use the given [dispatcher] under the hood. + * + * Using [TestDispatcher] as an argument has special behavior: subsequently-called [runTest], as well as + * [TestScope] and test dispatcher constructors, will use the [TestCoroutineScheduler] of the provided dispatcher. + * + * It is unsafe to call this method if alive coroutines launched in [Dispatchers.Main] exist. + */ +@ExperimentalCoroutinesApi +public fun Dispatchers.setMain(dispatcher: CoroutineDispatcher) { + require(dispatcher !is TestMainDispatcher) { "Dispatchers.setMain(Dispatchers.Main) is prohibited, probably Dispatchers.resetMain() should be used instead" } + getTestMainDispatcher().setDispatcher(dispatcher) +} + +/** + * Resets state of the [Dispatchers.Main] to the original main dispatcher. + * + * For example, in Android, the Main thread dispatcher will be set as [Dispatchers.Main]. + * This method undoes a dependency injection performed for tests, and so should be used in tear down (`@After`) methods. + * + * It is unsafe to call this method if alive coroutines launched in [Dispatchers.Main] exist. + */ +@ExperimentalCoroutinesApi +public fun Dispatchers.resetMain() { + getTestMainDispatcher().resetDispatcher() +} diff --git a/kotlinx-coroutines-test/common/src/TestScope.kt b/kotlinx-coroutines-test/common/src/TestScope.kt new file mode 100644 index 0000000000..66532450b4 --- /dev/null +++ b/kotlinx-coroutines-test/common/src/TestScope.kt @@ -0,0 +1,329 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.test.internal.* +import kotlin.coroutines.* +import kotlin.time.* + +/** + * A coroutine scope that for launching test coroutines. + * + * The scope provides the following functionality: + * - The [coroutineContext] includes a [coroutine dispatcher][TestDispatcher] that supports delay-skipping, using + * a [TestCoroutineScheduler] for orchestrating the virtual time. + * This scheduler is also available via the [testScheduler] property, and some helper extension + * methods are defined to more conveniently interact with it: see [TestScope.currentTime], [TestScope.runCurrent], + * [TestScope.advanceTimeBy], and [TestScope.advanceUntilIdle]. + * - When inside [runTest], uncaught exceptions from the child coroutines of this scope will be reported at the end of + * the test. + * It is invalid for child coroutines to throw uncaught exceptions when outside the call to [TestScope.runTest]: + * the only guarantee in this case is the best effort to deliver the exception. + * + * The usual way to access a [TestScope] is to call [runTest], but it can also be constructed manually, in order to + * use it to initialize the components that participate in the test. + * + * #### Differences from the deprecated [TestCoroutineScope] + * + * - This doesn't provide an equivalent of [TestCoroutineScope.cleanupTestCoroutines], and so can't be used as a + * standalone mechanism for writing tests: it does require that [runTest] is eventually called. + * The reason for this is that a proper cleanup procedure that supports using non-test dispatchers and arbitrary + * coroutine suspensions would be equivalent to [runTest], but would also be more error-prone, due to the potential + * for forgetting to perform the cleanup. + * - [TestCoroutineScope.advanceTimeBy] also calls [TestCoroutineScheduler.runCurrent] after advancing the virtual time. + * - No support for dispatcher pausing, like [DelayController] allows. [TestCoroutineDispatcher], which supported + * pausing, is deprecated; now, instead of pausing a dispatcher, one can use [withContext] to run a dispatcher that's + * paused by default, like [StandardTestDispatcher]. + * - No access to the list of unhandled exceptions. + */ +public sealed interface TestScope : CoroutineScope { + /** + * The delay-skipping scheduler used by the test dispatchers running the code in this scope. + */ + public val testScheduler: TestCoroutineScheduler + + /** + * A scope for background work. + * + * This scope is automatically cancelled when the test finishes. + * The coroutines in this scope are run as usual when using [advanceTimeBy] and [runCurrent]. + * [advanceUntilIdle], on the other hand, will stop advancing the virtual time once only the coroutines in this + * scope are left unprocessed. + * + * Failures in coroutines in this scope do not terminate the test. + * Instead, they are reported at the end of the test. + * Likewise, failure in the [TestScope] itself will not affect its [backgroundScope], + * because there's no parent-child relationship between them. + * + * A typical use case for this scope is to launch tasks that would outlive the tested code in + * the production environment. + * + * In this example, the coroutine that continuously sends new elements to the channel will get + * cancelled: + * ``` + * @Test + * fun testExampleBackgroundJob() = runTest { + * val channel = Channel() + * backgroundScope.launch { + * var i = 0 + * while (true) { + * channel.send(i++) + * } + * } + * repeat(100) { + * assertEquals(it, channel.receive()) + * } + * } + * ``` + */ + public val backgroundScope: CoroutineScope +} + +/** + * The current virtual time on [testScheduler][TestScope.testScheduler]. + * @see TestCoroutineScheduler.currentTime + */ +@ExperimentalCoroutinesApi +public val TestScope.currentTime: Long + get() = testScheduler.currentTime + +/** + * Advances the [testScheduler][TestScope.testScheduler] to the point where there are no tasks remaining. + * @see TestCoroutineScheduler.advanceUntilIdle + */ +@ExperimentalCoroutinesApi +public fun TestScope.advanceUntilIdle(): Unit = testScheduler.advanceUntilIdle() + +/** + * Run any tasks that are pending at the current virtual time, according to + * the [testScheduler][TestScope.testScheduler]. + * + * @see TestCoroutineScheduler.runCurrent + */ +@ExperimentalCoroutinesApi +public fun TestScope.runCurrent(): Unit = testScheduler.runCurrent() + +/** + * Moves the virtual clock of this dispatcher forward by [the specified amount][delayTimeMillis], running the + * scheduled tasks in the meantime. + * + * In contrast with `TestCoroutineScope.advanceTimeBy`, this function does not run the tasks scheduled at the moment + * [currentTime] + [delayTimeMillis]. + * + * @throws IllegalStateException if passed a negative [delay][delayTimeMillis]. + * @see TestCoroutineScheduler.advanceTimeBy + */ +@ExperimentalCoroutinesApi +public fun TestScope.advanceTimeBy(delayTimeMillis: Long): Unit = testScheduler.advanceTimeBy(delayTimeMillis) + +/** + * Moves the virtual clock of this dispatcher forward by [the specified amount][delayTime], running the + * scheduled tasks in the meantime. + * + * @throws IllegalStateException if passed a negative [delay][delayTime]. + * @see TestCoroutineScheduler.advanceTimeBy + */ +@ExperimentalCoroutinesApi +public fun TestScope.advanceTimeBy(delayTime: Duration): Unit = testScheduler.advanceTimeBy(delayTime) + +/** + * The [test scheduler][TestScope.testScheduler] as a [TimeSource]. + * @see TestCoroutineScheduler.timeSource + */ +@ExperimentalCoroutinesApi +public val TestScope.testTimeSource: TimeSource.WithComparableMarks get() = testScheduler.timeSource + +/** + * Creates a [TestScope]. + * + * It ensures that all the test module machinery is properly initialized. + * - If [context] doesn't provide a [TestCoroutineScheduler] for orchestrating the virtual time used for delay-skipping, + * a new one is created, unless either + * - a [TestDispatcher] is provided, in which case [TestDispatcher.scheduler] is used; + * - at the moment of the creation of the scope, [Dispatchers.Main] is delegated to a [TestDispatcher], in which case + * its [TestCoroutineScheduler] is used. + * - If [context] doesn't have a [TestDispatcher], a [StandardTestDispatcher] is created. + * - A [CoroutineExceptionHandler] is created that makes [TestCoroutineScope.cleanupTestCoroutines] throw if there were + * any uncaught exceptions, or forwards the exceptions further in a platform-specific manner if the cleanup was + * already performed when an exception happened. Passing a [CoroutineExceptionHandler] is illegal, unless it's an + * [UncaughtExceptionCaptor], in which case the behavior is preserved for the time being for backward compatibility. + * If you need to have a specific [CoroutineExceptionHandler], please pass it to [launch] on an already-created + * [TestCoroutineScope] and share your use case at + * [our issue tracker](https://github.com/Kotlin/kotlinx.coroutines/issues). + * - If [context] provides a [Job], that job is used as a parent for the new scope. + * + * @throws IllegalArgumentException if [context] has both [TestCoroutineScheduler] and a [TestDispatcher] linked to a + * different scheduler. + * @throws IllegalArgumentException if [context] has a [ContinuationInterceptor] that is not a [TestDispatcher]. + * @throws IllegalArgumentException if [context] has an [CoroutineExceptionHandler] that is not an + * [UncaughtExceptionCaptor]. + */ +@Suppress("FunctionName") +public fun TestScope(context: CoroutineContext = EmptyCoroutineContext): TestScope { + val ctxWithDispatcher = context.withDelaySkipping() + var scope: TestScopeImpl? = null + val exceptionHandler = when (ctxWithDispatcher[CoroutineExceptionHandler]) { + null -> CoroutineExceptionHandler { _, exception -> + scope!!.reportException(exception) + } + else -> throw IllegalArgumentException( + "A CoroutineExceptionHandler was passed to TestScope. " + + "Please pass it as an argument to a `launch` or `async` block on an already-created scope " + + "if uncaught exceptions require special treatment." + ) + } + return TestScopeImpl(ctxWithDispatcher + exceptionHandler).also { scope = it } +} + +/** + * Adds a [TestDispatcher] and a [TestCoroutineScheduler] to the context if there aren't any already. + * + * @throws IllegalArgumentException if both a [TestCoroutineScheduler] and a [TestDispatcher] are passed. + * @throws IllegalArgumentException if a [ContinuationInterceptor] is passed that is not a [TestDispatcher]. + */ +internal fun CoroutineContext.withDelaySkipping(): CoroutineContext { + val dispatcher: TestDispatcher = when (val dispatcher = get(ContinuationInterceptor)) { + is TestDispatcher -> { + val ctxScheduler = get(TestCoroutineScheduler) + if (ctxScheduler != null) { + require(dispatcher.scheduler === ctxScheduler) { + "Both a TestCoroutineScheduler $ctxScheduler and TestDispatcher $dispatcher linked to " + + "another scheduler were passed." + } + } + dispatcher + } + null -> StandardTestDispatcher(get(TestCoroutineScheduler)) + else -> throw IllegalArgumentException("Dispatcher must implement TestDispatcher: $dispatcher") + } + return this + dispatcher + dispatcher.scheduler +} + +internal class TestScopeImpl(context: CoroutineContext) : + AbstractCoroutine(context, initParentJob = true, active = true), TestScope { + + override val testScheduler get() = context[TestCoroutineScheduler]!! + + private var entered = false + private var finished = false + private val uncaughtExceptions = mutableListOf() + private val lock = SynchronizedObject() + + override val backgroundScope: CoroutineScope = + CoroutineScope(coroutineContext + BackgroundWork + ReportingSupervisorJob { + if (it !is CancellationException) reportException(it) + }) + + /** Called upon entry to [runTest]. Will throw if called more than once. */ + fun enter() { + val exceptions = synchronized(lock) { + if (entered) + throw IllegalStateException("Only a single call to `runTest` can be performed during one test.") + entered = true + check(!finished) + /** the order is important: [reportException] is only guaranteed not to throw if [entered] is `true` but + * [finished] is `false`. + * However, we also want [uncaughtExceptions] to be queried after the callback is registered, + * because the exception collector will be able to report the exceptions that arrived before this test but + * after the previous one, and learning about such exceptions as soon is possible is nice. */ + @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + run { ensurePlatformExceptionHandlerLoaded(ExceptionCollector) } + if (catchNonTestRelatedExceptions) { + ExceptionCollector.addOnExceptionCallback(lock, this::reportException) + } + uncaughtExceptions + } + if (exceptions.isNotEmpty()) { + ExceptionCollector.removeOnExceptionCallback(lock) + throw UncaughtExceptionsBeforeTest().apply { + for (e in exceptions) + addSuppressed(e) + } + } + } + + /** Called at the end of the test. May only be called once. Returns the list of caught unhandled exceptions. */ + fun leave(): List = synchronized(lock) { + check(entered && !finished) + /** After [finished] becomes `true`, it is no longer valid to have [reportException] as the callback. */ + ExceptionCollector.removeOnExceptionCallback(lock) + finished = true + uncaughtExceptions + } + + /** Called at the end of the test. May only be called once. */ + fun legacyLeave(): List { + val exceptions = synchronized(lock) { + check(entered && !finished) + /** After [finished] becomes `true`, it is no longer valid to have [reportException] as the callback. */ + ExceptionCollector.removeOnExceptionCallback(lock) + finished = true + uncaughtExceptions + } + val activeJobs = children.filter { it.isActive }.toList() // only non-empty if used with `runBlockingTest` + if (exceptions.isEmpty()) { + if (activeJobs.isNotEmpty()) + throw UncompletedCoroutinesError( + "Active jobs found during the tear-down. " + + "Ensure that all coroutines are completed or cancelled by your test. " + + "The active jobs: $activeJobs" + ) + if (!testScheduler.isIdle()) + throw UncompletedCoroutinesError( + "Unfinished coroutines found during the tear-down. " + + "Ensure that all coroutines are completed or cancelled by your test." + ) + } + return exceptions + } + + /** Stores an exception to report after [runTest], or rethrows it if not inside [runTest]. */ + fun reportException(throwable: Throwable) { + synchronized(lock) { + if (finished) { + throw throwable + } else { + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + for (existingThrowable in uncaughtExceptions) { + // avoid reporting exceptions that already were reported. + if (unwrap(throwable) == unwrap(existingThrowable)) + return + } + uncaughtExceptions.add(throwable) + if (!entered) + throw UncaughtExceptionsBeforeTest().apply { addSuppressed(throwable) } + } + } + } + + /** Throws an exception if the coroutine is not completing. */ + fun tryGetCompletionCause(): Throwable? = completionCause + + override fun toString(): String = + "TestScope[" + (if (finished) "test ended" else if (entered) "test started" else "test not started") + "]" +} + +/** Use the knowledge that any [TestScope] that we receive is necessarily a [TestScopeImpl]. */ +internal fun TestScope.asSpecificImplementation(): TestScopeImpl = when (this) { + is TestScopeImpl -> this +} + +internal class UncaughtExceptionsBeforeTest : IllegalStateException( + "There were uncaught exceptions before the test started. Please avoid this," + + " as such exceptions are also reported in a platform-dependent manner so that they are not lost." +) + +/** + * Thrown when a test has completed and there are tasks that are not completed or cancelled. + */ +@ExperimentalCoroutinesApi +internal class UncompletedCoroutinesError(message: String) : AssertionError(message) + +/** + * A flag that controls whether [TestScope] should attempt to catch arbitrary exceptions flying through the system. + * If it is enabled, then any exception that is not caught by the user code will be reported as a test failure. + * By default, it is enabled, but some tests may want to disable it to test the behavior of the system when they have + * their own exception handling procedures. + */ +@PublishedApi +internal var catchNonTestRelatedExceptions: Boolean = true diff --git a/kotlinx-coroutines-test/common/src/internal/ExceptionCollector.kt b/kotlinx-coroutines-test/common/src/internal/ExceptionCollector.kt new file mode 100644 index 0000000000..67341ddaf9 --- /dev/null +++ b/kotlinx-coroutines-test/common/src/internal/ExceptionCollector.kt @@ -0,0 +1,96 @@ +package kotlinx.coroutines.test.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +/** + * If [addOnExceptionCallback] is called, the provided callback will be evaluated each time + * [handleCoroutineException] is executed and can't find a [CoroutineExceptionHandler] to + * process the exception. + * + * When a callback is registered once, even if it's later removed, the system starts to assume that + * other callbacks will eventually be registered, and so collects the exceptions. + * Once a new callback is registered, the collected exceptions are used with it. + * + * The callbacks in this object are the last resort before relying on platform-dependent + * ways to report uncaught exceptions from coroutines. + */ +internal object ExceptionCollector : AbstractCoroutineContextElement(CoroutineExceptionHandler), CoroutineExceptionHandler { + private val lock = SynchronizedObject() + private var enabled = false + private val unprocessedExceptions = mutableListOf() + private val callbacks = mutableMapOf Unit>() + + /** + * Registers [callback] to be executed when an uncaught exception happens. + * [owner] is a key by which to distinguish different callbacks. + */ + fun addOnExceptionCallback(owner: Any, callback: (Throwable) -> Unit) = synchronized(lock) { + enabled = true // never becomes `false` again + val previousValue = callbacks.put(owner, callback) + check(previousValue === null) + // try to process the exceptions using the newly-registered callback + unprocessedExceptions.forEach { reportException(it) } + unprocessedExceptions.clear() + } + + /** + * Unregisters the callback associated with [owner]. + */ + fun removeOnExceptionCallback(owner: Any) = synchronized(lock) { + if (enabled) { + val existingValue = callbacks.remove(owner) + check(existingValue !== null) + } + } + + /** + * Tries to handle the exception by propagating it to an interested consumer. + * Returns `true` if the exception does not need further processing. + * + * Doesn't throw. + */ + fun handleException(exception: Throwable): Boolean = synchronized(lock) { + if (!enabled) return false + if (reportException(exception)) return true + /** we don't return the result of the `add` function because we don't have a guarantee + * that a callback will eventually appear and collect the unprocessed exceptions, so + * we can't consider [exception] to be properly handled. */ + unprocessedExceptions.add(exception) + return false + } + + /** + * Try to report [exception] to the existing callbacks. + */ + private fun reportException(exception: Throwable): Boolean { + var executedACallback = false + for (callback in callbacks.values) { + callback(exception) + executedACallback = true + /** We don't leave the function here because we want to fan-out the exceptions to every interested consumer, + * it's not enough to have the exception processed by one of them. + * The reason is, it's less big of a deal to observe multiple concurrent reports of bad behavior than not + * to observe the report in the exact callback that is connected to that bad behavior. */ + } + return executedACallback + } + + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + override fun handleException(context: CoroutineContext, exception: Throwable) { + if (handleException(exception)) { + throw ExceptionSuccessfullyProcessed + } + } + + override fun equals(other: Any?): Boolean = other is ExceptionCollector || other is ExceptionCollectorAsService +} + +/** + * A workaround for being unable to treat an object as a `ServiceLoader` service. + */ +internal class ExceptionCollectorAsService: CoroutineExceptionHandler by ExceptionCollector { + override fun equals(other: Any?): Boolean = other is ExceptionCollectorAsService || other is ExceptionCollector + override fun hashCode(): Int = ExceptionCollector.hashCode() +} diff --git a/kotlinx-coroutines-test/common/src/internal/ReportingSupervisorJob.kt b/kotlinx-coroutines-test/common/src/internal/ReportingSupervisorJob.kt new file mode 100644 index 0000000000..782ef71f28 --- /dev/null +++ b/kotlinx-coroutines-test/common/src/internal/ReportingSupervisorJob.kt @@ -0,0 +1,22 @@ +package kotlinx.coroutines.test.internal + +import kotlinx.coroutines.* + +/** + * A variant of [SupervisorJob] that additionally notifies about child failures via a callback. + */ +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +internal class ReportingSupervisorJob( + parent: Job? = null, + val onChildCancellation: (Throwable) -> Unit +) : JobImpl(parent) { + override fun childCancelled(cause: Throwable): Boolean = + try { + onChildCancellation(cause) + } catch (e: Throwable) { + cause.addSuppressed(e) + /* the coroutine context does not matter here, because we're only interested in reporting this exception + to the platform-specific global handler, not to a [CoroutineExceptionHandler] of any sort. */ + handleCoroutineException(this, cause) + }.let { false } +} diff --git a/kotlinx-coroutines-test/common/src/internal/TestMainDispatcher.kt b/kotlinx-coroutines-test/common/src/internal/TestMainDispatcher.kt new file mode 100644 index 0000000000..f9ab265c8d --- /dev/null +++ b/kotlinx-coroutines-test/common/src/internal/TestMainDispatcher.kt @@ -0,0 +1,100 @@ +package kotlinx.coroutines.test.internal + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.test.* +import kotlin.coroutines.* + +/** + * The testable main dispatcher used by kotlinx-coroutines-test. + * It is a [MainCoroutineDispatcher] that delegates all actions to a settable delegate. + */ +internal class TestMainDispatcher(createInnerMain: () -> CoroutineDispatcher): + MainCoroutineDispatcher(), + Delay +{ + internal constructor(delegate: CoroutineDispatcher): this({ delegate }) + + private val mainDispatcher by lazy(createInnerMain) + private var delegate = NonConcurrentlyModifiable(null, "Dispatchers.Main") + + private val dispatcher + get() = delegate.value ?: mainDispatcher + + private val delay + get() = dispatcher as? Delay ?: defaultDelay + + override val immediate: MainCoroutineDispatcher + get() = (dispatcher as? MainCoroutineDispatcher)?.immediate ?: this + + override fun dispatch(context: CoroutineContext, block: Runnable) = dispatcher.dispatch(context, block) + + override fun isDispatchNeeded(context: CoroutineContext): Boolean = dispatcher.isDispatchNeeded(context) + + override fun dispatchYield(context: CoroutineContext, block: Runnable) = dispatcher.dispatchYield(context, block) + + fun setDispatcher(dispatcher: CoroutineDispatcher) { + delegate.value = dispatcher + } + + fun resetDispatcher() { + delegate.value = null + } + + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) = + delay.scheduleResumeAfterDelay(timeMillis, continuation) + + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + delay.invokeOnTimeout(timeMillis, block, context) + + companion object { + internal val currentTestDispatcher + get() = (Dispatchers.Main as? TestMainDispatcher)?.delegate?.value as? TestDispatcher + + internal val currentTestScheduler + get() = currentTestDispatcher?.scheduler + } + + /** + * A wrapper around a value that attempts to throw when writing happens concurrently with reading. + * + * The read operations never throw. Instead, the failures detected inside them will be remembered and thrown on the + * next modification. + */ + private class NonConcurrentlyModifiable(initialValue: T, private val name: String) { + private val reader: AtomicRef = atomic(null) // last reader to attempt access + private val readers = atomic(0) // number of concurrent readers + private val writer: AtomicRef = atomic(null) // writer currently performing value modification + private val exceptionWhenReading: AtomicRef = atomic(null) // exception from reading + private val _value = atomic(initialValue) // the backing field for the value + + private fun concurrentWW(location: Throwable) = IllegalStateException("$name is modified concurrently", location) + private fun concurrentRW(location: Throwable) = IllegalStateException("$name is used concurrently with setting it", location) + + var value: T + get() { + reader.value = Throwable("reader location") + readers.incrementAndGet() + writer.value?.let { exceptionWhenReading.value = concurrentRW(it) } + val result = _value.value + readers.decrementAndGet() + return result + } + set(value) { + exceptionWhenReading.getAndSet(null)?.let { throw it } + if (readers.value != 0) reader.value?.let { throw concurrentRW(it) } + val writerLocation = Throwable("other writer location") + writer.getAndSet(writerLocation)?.let { throw concurrentWW(it) } + _value.value = value + writer.compareAndSet(writerLocation, null) + if (readers.value != 0) reader.value?.let { throw concurrentRW(it) } + } + } +} + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 +private val defaultDelay + inline get() = DefaultDelay + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 +internal expect fun Dispatchers.getTestMainDispatcher(): TestMainDispatcher diff --git a/kotlinx-coroutines-test/common/test/Helpers.kt b/kotlinx-coroutines-test/common/test/Helpers.kt new file mode 100644 index 0000000000..8679739dc8 --- /dev/null +++ b/kotlinx-coroutines-test/common/test/Helpers.kt @@ -0,0 +1,31 @@ +package kotlinx.coroutines.test + +/** + * Runs [test], and then invokes [block], passing to it the lambda that functionally behaves + * the same way [test] does. + */ +fun testResultMap(block: (() -> Unit) -> Unit, test: () -> TestResult): TestResult = testResultChain( + block = test, + after = { + block { it.getOrThrow() } + createTestResult { } + } +) + +/** + * Chains together [block] and [after], passing the result of [block] to [after]. + */ +expect fun testResultChain(block: () -> TestResult, after: (Result) -> TestResult): TestResult + +fun testResultChain(vararg chained: (Result) -> TestResult, initialResult: Result = Result.success(Unit)): TestResult = + if (chained.isEmpty()) { + createTestResult { + initialResult.getOrThrow() + } + } else { + testResultChain(block = { + chained[0](initialResult) + }) { + testResultChain(*chained.drop(1).toTypedArray(), initialResult = it) + } + } diff --git a/kotlinx-coroutines-test/common/test/RunTestTest.kt b/kotlinx-coroutines-test/common/test/RunTestTest.kt new file mode 100644 index 0000000000..a595299121 --- /dev/null +++ b/kotlinx-coroutines-test/common/test/RunTestTest.kt @@ -0,0 +1,490 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* +import kotlin.test.assertFailsWith +import kotlin.time.* +import kotlin.time.Duration.Companion.milliseconds + +class RunTestTest { + + /** Tests that [withContext] that sends work to other threads works in [runTest]. */ + @Test + fun testWithContextDispatching() = runTest { + var counter = 0 + withContext(Dispatchers.Default) { + counter += 1 + } + assertEquals(counter, 1) + } + + /** Tests that joining [GlobalScope.launch] works in [runTest]. */ + @Test + fun testJoiningForkedJob() = runTest { + var counter = 0 + val job = GlobalScope.launch { + counter += 1 + } + job.join() + assertEquals(counter, 1) + } + + /** Tests [suspendCoroutine] not failing [runTest]. */ + @Test + fun testSuspendCoroutine() = runTest { + val answer = suspendCoroutine { + it.resume(42) + } + assertEquals(42, answer) + } + + /** Tests that [runTest] attempts to detect it being run inside another [runTest] and failing in such scenarios. */ + @Test + fun testNestedRunTestForbidden() = runTest { + assertFailsWith { + runTest { } + } + } + + /** Tests that even the dispatch timeout of `0` is fine if all the dispatches go through the same scheduler. */ + @Test + fun testRunTestWithZeroDispatchTimeoutWithControlledDispatches() = runTest(dispatchTimeoutMs = 0) { + // below is some arbitrary concurrent code where all dispatches go through the same scheduler. + launch { + delay(2000) + } + val deferred = async { + val job = launch(StandardTestDispatcher(testScheduler)) { + launch { + delay(500) + } + delay(1000) + } + job.join() + } + deferred.await() + } + + /** Tests that too low of a dispatch timeout causes crashes. */ + @Test + fun testRunTestWithSmallDispatchTimeout() = testResultMap({ fn -> + try { + fn() + fail("shouldn't be reached") + } catch (e: Throwable) { + assertIs(e) + } + }) { + runTest(dispatchTimeoutMs = 100) { + withContext(Dispatchers.Default) { + delay(10000) + 3 + } + fail("shouldn't be reached") + } + } + + /** + * Tests that [runTest] times out after the specified time. + */ + @Test + fun testRunTestWithSmallTimeout() = testResultMap({ fn -> + try { + fn() + fail("shouldn't be reached") + } catch (e: Throwable) { + assertIs(e) + } + }) { + runTest(timeout = 100.milliseconds) { + withContext(Dispatchers.Default) { + delay(10000) + 3 + } + fail("shouldn't be reached") + } + } + + /** Tests that [runTest] times out after the specified time, even if the test framework always knows the test is + * still doing something. */ + @Test + fun testRunTestWithSmallTimeoutAndManyDispatches() = testResultMap({ fn -> + try { + fn() + fail("shouldn't be reached") + } catch (e: Throwable) { + assertIs(e) + } + }) { + runTest(timeout = 100.milliseconds) { + while (true) { + withContext(Dispatchers.Default) { + delay(10) + 3 + } + } + } + } + + /** Tests that, on timeout, the names of the active coroutines are listed, + * whereas the names of the completed ones are not. */ + @Test + @NoJs + @NoNative + @NoWasmWasi + @NoWasmJs + fun testListingActiveCoroutinesOnTimeout(): TestResult { + val name1 = "GoodUniqueName" + val name2 = "BadUniqueName" + return testResultMap({ + try { + it() + fail("unreached") + } catch (e: UncompletedCoroutinesError) { + assertContains(e.message ?: "", name1) + assertFalse((e.message ?: "").contains(name2)) + } + }) { + runTest(dispatchTimeoutMs = 10) { + launch(CoroutineName(name1)) { + CompletableDeferred().await() + } + launch(CoroutineName(name2)) { + } + } + } + } + + /** Tests that the [UncompletedCoroutinesError] suppresses an exception with which the coroutine is completing. */ + @Test + fun testFailureWithPendingCoroutine() = testResultMap({ + try { + it() + fail("unreached") + } catch (e: UncompletedCoroutinesError) { + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + val suppressed = unwrap(e).suppressedExceptions + assertEquals(1, suppressed.size, "$suppressed") + assertIs(suppressed[0]).also { + assertEquals("A", it.message) + } + } + }) { + runTest(timeout = 10.milliseconds) { + launch(start = CoroutineStart.UNDISPATCHED) { + withContext(NonCancellable + Dispatchers.Default) { + delay(100.milliseconds) + } + } + throw TestException("A") + } + } + + /** Tests that real delays can be accounted for with a large enough dispatch timeout. */ + @Test + fun testRunTestWithLargeDispatchTimeout() = runTest(dispatchTimeoutMs = 5000) { + withContext(Dispatchers.Default) { + delay(50) + } + } + + /** Tests that delays can be accounted for with a large enough timeout. */ + @Test + fun testRunTestWithLargeTimeout() = runTest(timeout = 5000.milliseconds) { + withContext(Dispatchers.Default) { + delay(50) + } + } + + /** Tests uncaught exceptions being suppressed by the dispatch timeout error. */ + @Test + fun testRunTestTimingOutAndThrowing() = testResultMap({ fn -> + try { + fn() + fail("unreached") + } catch (e: UncompletedCoroutinesError) { + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + val suppressed = unwrap(e).suppressedExceptions + assertEquals(1, suppressed.size, "$suppressed") + assertIs(suppressed[0]).also { + assertEquals("A", it.message) + } + } + }) { + runTest(timeout = 100.milliseconds) { + coroutineContext[CoroutineExceptionHandler]!!.handleException(coroutineContext, TestException("A")) + withContext(Dispatchers.Default) { + delay(10000) + 3 + } + fail("shouldn't be reached") + } + } + + /** Tests that passing invalid contexts to [runTest] causes it to fail (on JS, without forking). */ + @Test + fun testRunTestWithIllegalContext() { + for (ctx in TestScopeTest.invalidContexts) { + assertFailsWith { + runTest(ctx) { } + } + } + } + + /** Tests that throwing exceptions in [runTest] fails the test with them. */ + @Test + fun testThrowingInRunTestBody() = testResultMap({ + assertFailsWith { it() } + }) { + runTest { + throw RuntimeException() + } + } + + /** Tests that throwing exceptions in pending tasks [runTest] fails the test with them. */ + @Test + fun testThrowingInRunTestPendingTask() = testResultMap({ + assertFailsWith { it() } + }) { + runTest { + launch { + delay(SLOW) + throw RuntimeException() + } + } + } + + @Test + fun reproducer2405() = runTest { + val dispatcher = StandardTestDispatcher(testScheduler) + var collectedError = false + withContext(dispatcher) { + flow { emit(1) } + .combine( + flow { throw IllegalArgumentException() } + ) { int, string -> int.toString() + string } + .catch { emit("error") } + .collect { + assertEquals("error", it) + collectedError = true + } + } + assertTrue(collectedError) + } + + /** Tests that, once the test body has thrown, the child coroutines are cancelled. */ + @Test + fun testChildrenCancellationOnTestBodyFailure(): TestResult { + var job: Job? = null + return testResultMap({ + assertFailsWith { it() } + assertTrue(job!!.isCancelled) + }) { + runTest { + job = launch { + while (true) { + delay(1000) + } + } + throw AssertionError() + } + } + } + + /** Tests that [runTest] reports [TimeoutCancellationException]. */ + @Test + fun testTimeout() = testResultMap({ + assertFailsWith { it() } + }) { + runTest { + withTimeout(50) { + launch { + delay(1000) + } + } + } + } + + /** Checks that [runTest] throws the root cause and not [JobCancellationException] when a child coroutine throws. */ + @Test + fun testRunTestThrowsRootCause() = testResultMap({ + assertFailsWith { it() } + }) { + runTest { + launch { + throw TestException() + } + } + } + + /** Tests that [runTest] completes its job. */ + @Test + fun testCompletesOwnJob(): TestResult { + var handlerCalled = false + return testResultMap({ + it() + assertTrue(handlerCalled) + }) { + runTest { + coroutineContext.job.invokeOnCompletion { + handlerCalled = true + } + } + } + } + + /** Tests that [runTest] doesn't complete the job that was passed to it as an argument. */ + @Test + fun testDoesNotCompleteGivenJob(): TestResult { + var handlerCalled = false + val job = Job() + job.invokeOnCompletion { + handlerCalled = true + } + return testResultMap({ + it() + assertFalse(handlerCalled) + assertEquals(0, job.children.filter { it.isActive }.count()) + }) { + runTest(job) { + assertTrue(coroutineContext.job in job.children) + } + } + } + + /** Tests that, when the test body fails, the reported exceptions are suppressed. */ + @Test + fun testSuppressedExceptions() = testResultMap({ + try { + it() + fail("should not be reached") + } catch (e: TestException) { + assertEquals("w", e.message) + val suppressed = e.suppressedExceptions + + (e.suppressedExceptions.firstOrNull()?.suppressedExceptions ?: emptyList()) + assertEquals(3, suppressed.size) + assertEquals("x", suppressed[0].message) + assertEquals("y", suppressed[1].message) + assertEquals("z", suppressed[2].message) + } + }) { + runTest { + launch(SupervisorJob()) { throw TestException("x") } + launch(SupervisorJob()) { throw TestException("y") } + launch(SupervisorJob()) { throw TestException("z") } + throw TestException("w") + } + } + + /** Tests that [TestScope.runTest] does not inherit the exception handler and works. */ + @Test + fun testScopeRunTestExceptionHandler(): TestResult { + val scope = TestScope() + return testResultMap({ + try { + it() + fail("should not be reached") + } catch (e: TestException) { + // expected + } + }) { + scope.runTest { + launch(SupervisorJob()) { throw TestException("x") } + } + } + } + + /** + * Tests that if the main coroutine is completed without a dispatch, [runTest] will not consider this to be + * inactivity. + * + * The test will hang if this is not the case. + */ + @Test + fun testCoroutineCompletingWithoutDispatch() = runTest(timeout = Duration.INFINITE) { + launch(Dispatchers.Default) { delay(100) } + } + + /** + * Tests that [runTest] cleans up the exception handler even if it threw on initialization. + * + * This test must be run manually, because it writes garbage to the log. + * + * The JVM-only source set contains a test equivalent to this one that isn't ignored. + */ + @Test + @Ignore + fun testExceptionCaptorCleanedUpOnPreliminaryExit(): TestResult = testResultChain({ + // step 1: installing the exception handler + println("step 1") + runTest { } + }, { + it.getOrThrow() + // step 2: throwing an uncaught exception to be caught by the exception-handling system + println("step 2") + createTestResult { + launch(NonCancellable) { throw TestException("A") } + } + }, { + it.getOrThrow() + // step 3: trying to run a test should immediately fail, even before entering the test body + println("step 3") + try { + runTest { + fail("unreached") + } + fail("unreached") + } catch (e: UncaughtExceptionsBeforeTest) { + val cause = e.suppressedExceptions.single() + assertIs(cause) + assertEquals("A", cause.message) + } + // step 4: trying to run a test again should not fail with an exception + println("step 4") + runTest { + } + }, { + it.getOrThrow() + // step 5: throwing an uncaught exception to be caught by the exception-handling system, again + println("step 5") + createTestResult { + launch(NonCancellable) { throw TestException("B") } + } + }, { + it.getOrThrow() + // step 6: trying to run a test should immediately fail, again + println("step 6") + try { + runTest { + fail("unreached") + } + fail("unreached") + } catch (e: Exception) { + val cause = e.suppressedExceptions.single() + assertIs(cause) + assertEquals("B", cause.message) + } + // step 7: trying to run a test again should not fail with an exception, again + println("step 7") + runTest { + } + }) + + @Test + fun testCancellingTestScope() = testResultMap({ + try { + it() + fail("unreached") + } catch (e: CancellationException) { + // expected + } + }) { + runTest { + cancel(CancellationException("Oh no", TestException())) + } + } +} diff --git a/kotlinx-coroutines-test/common/test/StandardTestDispatcherTest.kt b/kotlinx-coroutines-test/common/test/StandardTestDispatcherTest.kt new file mode 100644 index 0000000000..c0334c898e --- /dev/null +++ b/kotlinx-coroutines-test/common/test/StandardTestDispatcherTest.kt @@ -0,0 +1,75 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.test.* + +class StandardTestDispatcherTest: OrderedExecutionTestBase() { + + private val scope = TestScope(StandardTestDispatcher()) + + @BeforeTest + fun init() { + scope.asSpecificImplementation().enter() + } + + @AfterTest + fun cleanup() { + scope.runCurrent() + assertEquals(listOf(), scope.asSpecificImplementation().legacyLeave()) + } + + /** Tests that the [StandardTestDispatcher] follows an execution order similar to `runBlocking`. */ + @Test + fun testFlowsNotSkippingValues() = scope.launch { + // https://github.com/Kotlin/kotlinx.coroutines/issues/1626#issuecomment-554632852 + val list = flowOf(1).onStart { emit(0) } + .combine(flowOf("A")) { int, str -> "$str$int" } + .toList() + assertEquals(list, listOf("A0", "A1")) + }.void() + + /** Tests that each [launch] gets dispatched. */ + @Test + fun testLaunchDispatched() = scope.launch { + expect(1) + launch { + expect(3) + } + finish(2) + }.void() + + /** Tests that dispatching is done in a predictable order and [yield] puts this task at the end of the queue. */ + @Test + fun testYield() = scope.launch { + expect(1) + scope.launch { + expect(3) + yield() + expect(6) + } + scope.launch { + expect(4) + yield() + finish(7) + } + expect(2) + yield() + expect(5) + }.void() + + /** Tests that the [TestCoroutineScheduler] used for [Dispatchers.Main] gets used by default. */ + @Test + fun testSchedulerReuse() { + val dispatcher1 = StandardTestDispatcher() + Dispatchers.setMain(dispatcher1) + try { + val dispatcher2 = StandardTestDispatcher() + assertSame(dispatcher1.scheduler, dispatcher2.scheduler) + } finally { + Dispatchers.resetMain() + } + } + +} diff --git a/kotlinx-coroutines-test/common/test/TestCoroutineSchedulerTest.kt b/kotlinx-coroutines-test/common/test/TestCoroutineSchedulerTest.kt new file mode 100644 index 0000000000..a7dd8c623f --- /dev/null +++ b/kotlinx-coroutines-test/common/test/TestCoroutineSchedulerTest.kt @@ -0,0 +1,337 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import kotlin.test.* +import kotlin.test.assertFailsWith +import kotlin.time.* +import kotlin.time.Duration.Companion.seconds +import kotlin.time.Duration.Companion.milliseconds + +class TestCoroutineSchedulerTest { + /** Tests that `TestCoroutineScheduler` attempts to detect if there are several instances of it. */ + @Test + fun testContextElement() = runTest { + assertFailsWith { + withContext(StandardTestDispatcher()) { + } + } + } + + /** Tests that, as opposed to [DelayController.advanceTimeBy] or [TestCoroutineScope.advanceTimeBy], + * [TestCoroutineScheduler.advanceTimeBy] doesn't run the tasks scheduled at the target moment. */ + @Test + fun testAdvanceTimeByDoesNotRunCurrent() = runTest { + var entered = false + launch { + delay(15) + entered = true + } + testScheduler.advanceTimeBy(15.milliseconds) + assertFalse(entered) + testScheduler.runCurrent() + assertTrue(entered) + } + + /** Tests that [TestCoroutineScheduler.advanceTimeBy] doesn't accept negative delays. */ + @Test + fun testAdvanceTimeByWithNegativeDelay() { + val scheduler = TestCoroutineScheduler() + assertFailsWith { + scheduler.advanceTimeBy((-1).milliseconds) + } + } + + /** Tests that if [TestCoroutineScheduler.advanceTimeBy] encounters an arithmetic overflow, all the tasks scheduled + * until the moment [Long.MAX_VALUE] get run. */ + @Test + fun testAdvanceTimeByEnormousDelays() = forTestDispatchers { + assertRunsFast { + with (TestScope(it)) { + launch { + val initialDelay = 10L + delay(initialDelay) + assertEquals(initialDelay, currentTime) + var enteredInfinity = false + launch { + delay(Long.MAX_VALUE - 1) // delay(Long.MAX_VALUE) does nothing + assertEquals(Long.MAX_VALUE, currentTime) + enteredInfinity = true + } + var enteredNearInfinity = false + launch { + delay(Long.MAX_VALUE - initialDelay - 1) + assertEquals(Long.MAX_VALUE - 1, currentTime) + enteredNearInfinity = true + } + testScheduler.advanceTimeBy(Duration.INFINITE) + assertFalse(enteredInfinity) + assertTrue(enteredNearInfinity) + assertEquals(Long.MAX_VALUE, currentTime) + testScheduler.runCurrent() + assertTrue(enteredInfinity) + } + testScheduler.advanceUntilIdle() + } + } + } + + /** Tests the basic functionality of [TestCoroutineScheduler.advanceTimeBy]. */ + @Test + fun testAdvanceTimeBy() = runTest { + assertRunsFast { + var stage = 1 + launch { + delay(1_000) + assertEquals(1_000, currentTime) + stage = 2 + delay(500) + assertEquals(1_500, currentTime) + stage = 3 + delay(501) + assertEquals(2_001, currentTime) + stage = 4 + } + assertEquals(1, stage) + assertEquals(0, currentTime) + advanceTimeBy(2.seconds) + assertEquals(3, stage) + assertEquals(2_000, currentTime) + advanceTimeBy(2.milliseconds) + assertEquals(4, stage) + assertEquals(2_002, currentTime) + } + } + + /** Tests the basic functionality of [TestCoroutineScheduler.runCurrent]. */ + @Test + fun testRunCurrent() = runTest { + var stage = 0 + launch { + delay(1) + ++stage + delay(1) + stage += 10 + } + launch { + delay(1) + ++stage + delay(1) + stage += 10 + } + testScheduler.advanceTimeBy(1.milliseconds) + assertEquals(0, stage) + runCurrent() + assertEquals(2, stage) + testScheduler.advanceTimeBy(1.milliseconds) + assertEquals(2, stage) + runCurrent() + assertEquals(22, stage) + } + + /** Tests that [TestCoroutineScheduler.runCurrent] will not run new tasks after the current time has advanced. */ + @Test + fun testRunCurrentNotDrainingQueue() = forTestDispatchers { + assertRunsFast { + val scheduler = it.scheduler + val scope = TestScope(it) + var stage = 1 + scope.launch { + delay(SLOW) + launch { + delay(SLOW) + stage = 3 + } + scheduler.advanceTimeBy(SLOW.milliseconds) + stage = 2 + } + scheduler.advanceTimeBy(SLOW.milliseconds) + assertEquals(1, stage) + scheduler.runCurrent() + assertEquals(2, stage) + scheduler.runCurrent() + assertEquals(3, stage) + } + } + + /** Tests that [TestCoroutineScheduler.advanceUntilIdle] doesn't hang when itself running in a scheduler task. */ + @Test + fun testNestedAdvanceUntilIdle() = forTestDispatchers { + assertRunsFast { + val scheduler = it.scheduler + val scope = TestScope(it) + var executed = false + scope.launch { + launch { + delay(SLOW) + executed = true + } + scheduler.advanceUntilIdle() + } + scheduler.advanceUntilIdle() + assertTrue(executed) + } + } + + /** Tests [yield] scheduling tasks for future execution and not executing immediately. */ + @Test + fun testYield() = forTestDispatchers { + val scope = TestScope(it) + var stage = 0 + scope.launch { + yield() + assertEquals(1, stage) + stage = 2 + } + scope.launch { + yield() + assertEquals(2, stage) + stage = 3 + } + assertEquals(0, stage) + stage = 1 + scope.runCurrent() + } + + /** Tests that dispatching the delayed tasks is ordered by their waking times. */ + @Test + fun testDelaysPriority() = forTestDispatchers { + val scope = TestScope(it) + var lastMeasurement = 0L + fun checkTime(time: Long) { + assertTrue(lastMeasurement < time) + assertEquals(time, scope.currentTime) + lastMeasurement = scope.currentTime + } + scope.launch { + launch { + delay(100) + checkTime(100) + val deferred = async { + delay(70) + checkTime(170) + } + delay(1) + checkTime(101) + deferred.await() + delay(1) + checkTime(171) + } + launch { + delay(200) + checkTime(200) + } + launch { + delay(150) + checkTime(150) + delay(22) + checkTime(172) + } + delay(201) + } + scope.advanceUntilIdle() + checkTime(201) + } + + private fun TestScope.checkTimeout( + timesOut: Boolean, timeoutMillis: Long = SLOW, block: suspend () -> Unit + ) = assertRunsFast { + var caughtException = false + asSpecificImplementation().enter() + launch { + try { + withTimeout(timeoutMillis) { + block() + } + } catch (e: TimeoutCancellationException) { + caughtException = true + } + } + advanceUntilIdle() + throwAll(null, asSpecificImplementation().legacyLeave()) + if (timesOut) + assertTrue(caughtException) + else + assertFalse(caughtException) + } + + /** Tests that timeouts get triggered. */ + @Test + fun testSmallTimeouts() = forTestDispatchers { + val scope = TestScope(it) + scope.checkTimeout(true) { + val half = SLOW / 2 + delay(half) + delay(SLOW - half) + } + } + + /** Tests that timeouts don't get triggered if the code finishes in time. */ + @Test + fun testLargeTimeouts() = forTestDispatchers { + val scope = TestScope(it) + scope.checkTimeout(false) { + val half = SLOW / 2 + delay(half) + delay(SLOW - half - 1) + } + } + + /** Tests that timeouts get triggered if the code fails to finish in time asynchronously. */ + @Test + fun testSmallAsynchronousTimeouts() = forTestDispatchers { + val scope = TestScope(it) + val deferred = CompletableDeferred() + scope.launch { + val half = SLOW / 2 + delay(half) + delay(SLOW - half) + deferred.complete(Unit) + } + scope.checkTimeout(true) { + deferred.await() + } + } + + /** Tests that timeouts don't get triggered if the code finishes in time, even if it does so asynchronously. */ + @Test + fun testLargeAsynchronousTimeouts() = forTestDispatchers { + val scope = TestScope(it) + val deferred = CompletableDeferred() + scope.launch { + val half = SLOW / 2 + delay(half) + delay(SLOW - half - 1) + deferred.complete(Unit) + } + scope.checkTimeout(false) { + deferred.await() + } + } + + @Test + fun testAdvanceTimeSource() = runTest { + val expected = 1.seconds + val before = testTimeSource.markNow() + val actual = testTimeSource.measureTime { + delay(expected) + } + assertEquals(expected, actual) + val after = testTimeSource.markNow() + assertTrue(before < after) + assertEquals(expected, after - before) + } + + private fun forTestDispatchers(block: (TestDispatcher) -> Unit): Unit = + @Suppress("DEPRECATION") + listOf( + StandardTestDispatcher(), + UnconfinedTestDispatcher() + ).forEach { + try { + block(it) + } catch (e: Throwable) { + throw RuntimeException("Test failed for dispatcher $it", e) + } + } +} diff --git a/kotlinx-coroutines-test/common/test/TestDispatchersTest.kt b/kotlinx-coroutines-test/common/test/TestDispatchersTest.kt new file mode 100644 index 0000000000..614c46ae31 --- /dev/null +++ b/kotlinx-coroutines-test/common/test/TestDispatchersTest.kt @@ -0,0 +1,95 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.test.internal.* +import kotlin.coroutines.* +import kotlin.test.* + +class TestDispatchersTest: OrderedExecutionTestBase() { + + @BeforeTest + fun setUp() { + Dispatchers.setMain(StandardTestDispatcher()) + } + + @AfterTest + fun tearDown() { + Dispatchers.resetMain() + } + + /** Tests that asynchronous execution of tests does not happen concurrently with [AfterTest]. */ + @Test + fun testMainMocking() = runTest { + val mainAtStart = TestMainDispatcher.currentTestDispatcher + assertNotNull(mainAtStart) + withContext(Dispatchers.Main) { + delay(10) + } + withContext(Dispatchers.Default) { + delay(10) + } + withContext(Dispatchers.Main) { + delay(10) + } + assertSame(mainAtStart, TestMainDispatcher.currentTestDispatcher) + } + + /** Tests that the mocked [Dispatchers.Main] correctly forwards [Delay] methods. */ + @Test + fun testMockedMainImplementsDelay() = runTest { + val main = Dispatchers.Main + withContext(main) { + delay(10) + } + withContext(Dispatchers.Default) { + delay(10) + } + withContext(main) { + delay(10) + } + } + + /** Tests that [Distpachers.setMain] fails when called with [Dispatchers.Main]. */ + @Test + fun testSelfSet() { + assertFailsWith { Dispatchers.setMain(Dispatchers.Main) } + } + + @Test + fun testImmediateDispatcher() = runTest { + Dispatchers.setMain(ImmediateDispatcher()) + expect(1) + withContext(Dispatchers.Main) { + expect(3) + } + + Dispatchers.setMain(RegularDispatcher()) + withContext(Dispatchers.Main) { + expect(6) + } + + finish(7) + } + + private inner class ImmediateDispatcher : CoroutineDispatcher() { + override fun isDispatchNeeded(context: CoroutineContext): Boolean { + expect(2) + return false + } + + override fun dispatch(context: CoroutineContext, block: Runnable) = throw RuntimeException("Shouldn't be reached") + } + + private inner class RegularDispatcher : CoroutineDispatcher() { + override fun isDispatchNeeded(context: CoroutineContext): Boolean { + expect(4) + return true + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + expect(5) + block.run() + } + } +} diff --git a/kotlinx-coroutines-test/common/test/TestScopeTest.kt b/kotlinx-coroutines-test/common/test/TestScopeTest.kt new file mode 100644 index 0000000000..ca98c6d214 --- /dev/null +++ b/kotlinx-coroutines-test/common/test/TestScopeTest.kt @@ -0,0 +1,557 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* +import kotlin.test.assertFailsWith +import kotlin.time.Duration.Companion.milliseconds + +class TestScopeTest { + /** Tests failing to create a [TestScope] with incorrect contexts. */ + @Test + fun testCreateThrowsOnInvalidArguments() { + for (ctx in invalidContexts) { + assertFailsWith { + TestScope(ctx) + } + } + } + + /** Tests that a newly-created [TestScope] provides the correct scheduler. */ + @Test + fun testCreateProvidesScheduler() { + // Creates a new scheduler. + run { + val scope = TestScope() + assertNotNull(scope.coroutineContext[TestCoroutineScheduler]) + } + // Reuses the scheduler that the dispatcher is linked to. + run { + val dispatcher = StandardTestDispatcher() + val scope = TestScope(dispatcher) + assertSame(dispatcher.scheduler, scope.coroutineContext[TestCoroutineScheduler]) + } + // Uses the scheduler passed to it. + run { + val scheduler = TestCoroutineScheduler() + val scope = TestScope(scheduler) + assertSame(scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertSame(scheduler, (scope.coroutineContext[ContinuationInterceptor] as TestDispatcher).scheduler) + } + // Doesn't touch the passed dispatcher and the scheduler if they match. + run { + val scheduler = TestCoroutineScheduler() + val dispatcher = StandardTestDispatcher(scheduler) + val scope = TestScope(scheduler + dispatcher) + assertSame(scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertSame(dispatcher, scope.coroutineContext[ContinuationInterceptor]) + } + } + + /** Part of [testCreateProvidesScheduler], disabled for Native */ + @Test + fun testCreateReusesScheduler() { + // Reuses the scheduler of `Dispatchers.Main` + run { + val scheduler = TestCoroutineScheduler() + val mainDispatcher = StandardTestDispatcher(scheduler) + Dispatchers.setMain(mainDispatcher) + try { + val scope = TestScope() + assertSame(scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertNotSame(mainDispatcher, scope.coroutineContext[ContinuationInterceptor]) + } finally { + Dispatchers.resetMain() + } + } + // Does not reuse the scheduler of `Dispatchers.Main` if one is explicitly passed + run { + val mainDispatcher = StandardTestDispatcher() + Dispatchers.setMain(mainDispatcher) + try { + val scheduler = TestCoroutineScheduler() + val scope = TestScope(scheduler) + assertSame(scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertNotSame(mainDispatcher.scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertNotSame(mainDispatcher, scope.coroutineContext[ContinuationInterceptor]) + } finally { + Dispatchers.resetMain() + } + } + } + + /** Tests that the cleanup procedure throws if there were uncompleted delays by the end. */ + @Test + fun testPresentDelaysThrowing() { + val scope = TestScope() + var result = false + scope.launch { + delay(5) + result = true + } + assertFalse(result) + scope.asSpecificImplementation().enter() + assertFailsWith { scope.asSpecificImplementation().legacyLeave() } + assertFalse(result) + } + + /** Tests that the cleanup procedure throws if there were active jobs by the end. */ + @Test + fun testActiveJobsThrowing() { + val scope = TestScope() + var result = false + val deferred = CompletableDeferred() + scope.launch { + deferred.await() + result = true + } + assertFalse(result) + scope.asSpecificImplementation().enter() + assertFailsWith { scope.asSpecificImplementation().legacyLeave() } + assertFalse(result) + } + + /** Tests that the cleanup procedure throws even if it detects that the job is already cancelled. */ + @Test + fun testCancelledDelaysThrowing() { + val scope = TestScope() + var result = false + val deferred = CompletableDeferred() + val job = scope.launch { + deferred.await() + result = true + } + job.cancel() + assertFalse(result) + scope.asSpecificImplementation().enter() + assertFailsWith { scope.asSpecificImplementation().legacyLeave() } + assertFalse(result) + } + + /** Tests that uncaught exceptions are thrown at the cleanup. */ + @Test + fun testGetsCancelledOnChildFailure(): TestResult { + val scope = TestScope() + val exception = TestException("test") + scope.launch { + throw exception + } + return testResultMap({ + try { + it() + fail("should not reach") + } catch (e: TestException) { + // expected + } + }) { + scope.runTest { + } + } + } + + /** Tests that, when reporting several exceptions, the first one is thrown, with the rest suppressed. */ + @Test + fun testSuppressedExceptions() { + TestScope().apply { + asSpecificImplementation().enter() + launch(SupervisorJob()) { throw TestException("x") } + launch(SupervisorJob()) { throw TestException("y") } + launch(SupervisorJob()) { throw TestException("z") } + runCurrent() + val e = asSpecificImplementation().legacyLeave() + assertEquals(3, e.size) + assertEquals("x", e[0].message) + assertEquals("y", e[1].message) + assertEquals("z", e[2].message) + } + } + + /** Tests that the background work is being run at all. */ + @Test + fun testBackgroundWorkBeingRun(): TestResult = runTest { + var i = 0 + var j = 0 + backgroundScope.launch { + ++i + } + backgroundScope.launch { + delay(10) + ++j + } + assertEquals(0, i) + assertEquals(0, j) + delay(1) + assertEquals(1, i) + assertEquals(0, j) + delay(10) + assertEquals(1, i) + assertEquals(1, j) + } + + /** + * Tests that the background work gets cancelled after the test body finishes. + */ + @Test + fun testBackgroundWorkCancelled(): TestResult { + var cancelled = false + return testResultMap({ + it() + assertTrue(cancelled) + }) { + runTest { + var i = 0 + backgroundScope.launch { + try { + while (isActive) { + ++i + yield() + } + } catch (e: CancellationException) { + cancelled = true + } + } + repeat(5) { + assertEquals(i, it) + yield() + } + } + } + } + + /** Tests the interactions between the time-control commands and the background work. */ + @Test + fun testBackgroundWorkTimeControl(): TestResult = runTest { + var i = 0 + var j = 0 + backgroundScope.launch { + while (true) { + ++i + delay(100) + } + } + backgroundScope.launch { + while (true) { + ++j + delay(50) + } + } + advanceUntilIdle() // should do nothing, as only background work is left. + assertEquals(0, i) + assertEquals(0, j) + val job = launch { + delay(1) + // the background work scheduled for earlier gets executed before the normal work scheduled for later does + assertEquals(1, i) + assertEquals(1, j) + } + job.join() + advanceTimeBy(199.milliseconds) // should work the same for the background tasks + assertEquals(2, i) + assertEquals(4, j) + advanceUntilIdle() // once again, should do nothing + assertEquals(2, i) + assertEquals(4, j) + runCurrent() // should behave the same way as for the normal work + assertEquals(3, i) + assertEquals(5, j) + launch { + delay(1001) + assertEquals(13, i) + assertEquals(25, j) + } + advanceUntilIdle() // should execute the normal work, and with that, the background one, too + } + + /** + * Tests that an error in a background coroutine does not cancel the test, but is reported at the end. + */ + @Test + fun testBackgroundWorkErrorReporting(): TestResult { + var testFinished = false + val exception = RuntimeException("x") + return testResultMap({ + try { + it() + fail("unreached") + } catch (e: Throwable) { + assertSame(e, exception) + assertTrue(testFinished) + } + }) { + runTest { + backgroundScope.launch { + throw exception + } + delay(1000) + testFinished = true + } + } + } + + /** + * Tests that the background work gets to finish what it's doing after the test is completed. + */ + @Test + fun testBackgroundWorkFinalizing(): TestResult { + var taskEnded = 0 + val nTasks = 10 + return testResultMap({ + try { + it() + fail("unreached") + } catch (e: TestException) { + assertEquals(2, e.suppressedExceptions.size) + assertEquals(nTasks, taskEnded) + } + }) { + runTest { + repeat(nTasks) { + backgroundScope.launch { + try { + while (true) { + delay(1) + } + } finally { + ++taskEnded + if (taskEnded <= 2) + throw TestException() + } + } + } + delay(100) + throw TestException() + } + } + } + + /** + * Tests using [Flow.stateIn] as a background job. + */ + @Test + fun testExampleBackgroundJob1() = runTest { + val myFlow = flow { + var i = 0 + while (true) { + emit(++i) + delay(1) + } + } + val stateFlow = myFlow.stateIn(backgroundScope, SharingStarted.Eagerly, 0) + var j = 0 + repeat(100) { + assertEquals(j++, stateFlow.value) + delay(1) + } + } + + /** + * A test from the documentation of [TestScope.backgroundScope]. + */ + @Test + fun testExampleBackgroundJob2() = runTest { + val channel = Channel() + backgroundScope.launch { + var i = 0 + while (true) { + channel.send(i++) + } + } + repeat(100) { + assertEquals(it, channel.receive()) + } + } + + /** + * Tests that the test will timeout due to idleness even if some background tasks are running. + */ + @Test + fun testBackgroundWorkNotPreventingTimeout(): TestResult = testResultMap({ + try { + it() + fail("unreached") + } catch (_: UncompletedCoroutinesError) { + + } + }) { + runTest(timeout = 100.milliseconds) { + backgroundScope.launch { + while (true) { + yield() + } + } + backgroundScope.launch { + while (true) { + delay(1) + } + } + val deferred = CompletableDeferred() + deferred.await() + } + + } + + /** + * Tests that the background work will not prevent the test from timing out even in some cases + * when the unconfined dispatcher is used. + */ + @Test + fun testUnconfinedBackgroundWorkNotPreventingTimeout(): TestResult = testResultMap({ + try { + it() + fail("unreached") + } catch (_: UncompletedCoroutinesError) { + + } + }) { + runTest(UnconfinedTestDispatcher(), timeout = 100.milliseconds) { + /** + * Having a coroutine like this will still cause the test to hang: + backgroundScope.launch { + while (true) { + yield() + } + } + * The reason is that even the initial [advanceUntilIdle] will never return in this case. + */ + backgroundScope.launch { + while (true) { + delay(1) + } + } + val deferred = CompletableDeferred() + deferred.await() + } + } + + /** + * Tests that even the exceptions in the background scope that don't typically get reported and need to be queried + * (like failures in [async]) will still surface in some simple scenarios. + */ + @Test + fun testAsyncFailureInBackgroundReported() = testResultMap({ + try { + it() + fail("unreached") + } catch (e: TestException) { + assertEquals("z", e.message) + assertEquals(setOf("x", "y"), e.suppressedExceptions.map { it.message }.toSet()) + } + }) { + runTest { + backgroundScope.async { + throw TestException("x") + } + backgroundScope.produce { + throw TestException("y") + } + delay(1) + throw TestException("z") + } + } + + /** + * Tests that, if an exception reaches the [TestScope] exception reporting mechanism via several + * channels, it will only be reported once. + */ + @Test + fun testNoDuplicateExceptions() = testResultMap({ + try { + it() + fail("unreached") + } catch (e: TestException) { + assertEquals("y", e.message) + assertEquals(listOf("x"), e.suppressedExceptions.map { it.message }) + } + }) { + runTest { + backgroundScope.launch { + throw TestException("x") + } + delay(1) + throw TestException("y") + } + } + + /** + * Tests that [TestScope.withTimeout] notifies the programmer about using the virtual time. + */ + @Test + fun testTimingOutWithVirtualTimeMessage() = runTest { + try { + withTimeout(1_000_000) { + Channel().receive() + } + } catch (e: TimeoutCancellationException) { + assertContains(e.message!!, "virtual") + } + } + + /* + * Tests that the [TestScope] exception reporting mechanism will report the exceptions that happen between + * different tests. + * + * This test must be run manually, because such exceptions still go through the global exception handler + * (as there's no guarantee that another test will happen), and the global exception handler will + * log the exceptions or, on Native, crash the test suite. + * + * The JVM-only source set contains a test equivalent to this one that isn't ignored. + */ + @Test + @Ignore + fun testReportingStrayUncaughtExceptionsBetweenTests() { + val thrown = TestException("x") + testResultChain({ + // register a handler for uncaught exceptions + runTest { } + }, { + GlobalScope.launch(start = CoroutineStart.UNDISPATCHED) { + throw thrown + } + runTest { + fail("unreached") + } + }, { + // this `runTest` will not report the exception + runTest { + when (val exception = it.exceptionOrNull()) { + is UncaughtExceptionsBeforeTest -> { + assertEquals(1, exception.suppressedExceptions.size) + assertSame(exception.suppressedExceptions[0], thrown) + } + else -> fail("unexpected exception: $exception") + } + } + }) + } + + /** + * Tests that the uncaught exceptions that happen during the test are reported. + */ + @Test + fun testReportingStrayUncaughtExceptionsDuringTest(): TestResult { + val thrown = TestException("x") + return testResultChain({ _ -> + runTest { + val job = launch(Dispatchers.Default + NonCancellable) { + throw thrown + } + job.join() + } + }, { + runTest { + assertEquals(thrown, it.exceptionOrNull()) + } + }) + } + + companion object { + internal val invalidContexts = listOf( + Dispatchers.Default, // not a [TestDispatcher] + CoroutineExceptionHandler { _, _ -> }, // exception handlers can't be overridden + StandardTestDispatcher() + TestCoroutineScheduler(), // the dispatcher is not linked to the scheduler + ) + } +} diff --git a/kotlinx-coroutines-test/common/test/UnconfinedTestDispatcherTest.kt b/kotlinx-coroutines-test/common/test/UnconfinedTestDispatcherTest.kt new file mode 100644 index 0000000000..ddcdc4a548 --- /dev/null +++ b/kotlinx-coroutines-test/common/test/UnconfinedTestDispatcherTest.kt @@ -0,0 +1,163 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlin.test.* + +class UnconfinedTestDispatcherTest { + + @Test + fun reproducer1742() { + class ObservableValue(initial: T) { + var value: T = initial + private set + + private val listeners = mutableListOf<(T) -> Unit>() + + fun set(value: T) { + this.value = value + listeners.forEach { it(value) } + } + + fun addListener(listener: (T) -> Unit) { + listeners.add(listener) + } + + fun removeListener(listener: (T) -> Unit) { + listeners.remove(listener) + } + } + + fun ObservableValue.observe(): Flow = + callbackFlow { + val listener = { value: T -> + if (!isClosedForSend) { + trySend(value) + } + } + addListener(listener) + listener(value) + awaitClose { removeListener(listener) } + } + + val intProvider = ObservableValue(0) + val stringProvider = ObservableValue("") + var data = Pair(0, "") + val scope = CoroutineScope(UnconfinedTestDispatcher()) + scope.launch { + combine( + intProvider.observe(), + stringProvider.observe() + ) { intValue, stringValue -> Pair(intValue, stringValue) } + .collect { pair -> + data = pair + } + } + + intProvider.set(1) + stringProvider.set("3") + intProvider.set(2) + intProvider.set(3) + + scope.cancel() + assertEquals(Pair(3, "3"), data) + } + + @Test + fun reproducer2082() = runTest { + val subject1 = MutableStateFlow(1) + val subject2 = MutableStateFlow("a") + val values = mutableListOf>() + + val job = launch(UnconfinedTestDispatcher(testScheduler)) { + combine(subject1, subject2) { intVal, strVal -> intVal to strVal } + .collect { + delay(10000) + values += it + } + } + + subject1.value = 2 + delay(10000) + subject2.value = "b" + delay(10000) + + subject1.value = 3 + delay(10000) + subject2.value = "c" + delay(10000) + delay(10000) + delay(1) + + job.cancel() + + assertEquals(listOf(Pair(1, "a"), Pair(2, "a"), Pair(2, "b"), Pair(3, "b"), Pair(3, "c")), values) + } + + @Test + fun reproducer2405() = createTestResult { + val dispatcher = UnconfinedTestDispatcher() + var collectedError = false + withContext(dispatcher) { + flow { emit(1) } + .combine( + flow { throw IllegalArgumentException() } + ) { int, string -> int.toString() + string } + .catch { emit("error") } + .collect { + assertEquals("error", it) + collectedError = true + } + } + assertTrue(collectedError) + } + + /** An example from the [UnconfinedTestDispatcher] documentation. */ + @Test + fun testUnconfinedDispatcher() = runTest { + val values = mutableListOf() + val stateFlow = MutableStateFlow(0) + val job = launch(UnconfinedTestDispatcher(testScheduler)) { + stateFlow.collect { + values.add(it) + } + } + stateFlow.value = 1 + stateFlow.value = 2 + stateFlow.value = 3 + job.cancel() + assertEquals(listOf(0, 1, 2, 3), values) + } + + /** Tests that child coroutines are eagerly entered. */ + @Test + fun testEagerlyEnteringChildCoroutines() = runTest(UnconfinedTestDispatcher()) { + var entered = false + val deferred = CompletableDeferred() + var completed = false + launch { + entered = true + deferred.await() + completed = true + } + assertTrue(entered) // `entered = true` already executed. + assertFalse(completed) // however, the child coroutine then suspended, so it is enqueued. + deferred.complete(Unit) // resume the coroutine. + assertTrue(completed) // now the child coroutine is immediately completed. + } + + /** Tests that the [TestCoroutineScheduler] used for [Dispatchers.Main] gets used by default. */ + @Test + fun testSchedulerReuse() { + val dispatcher1 = StandardTestDispatcher() + Dispatchers.setMain(dispatcher1) + try { + val dispatcher2 = UnconfinedTestDispatcher() + assertSame(dispatcher1.scheduler, dispatcher2.scheduler) + } finally { + Dispatchers.resetMain() + } + } + +} diff --git a/kotlinx-coroutines-test/js/src/TestBuilders.kt b/kotlinx-coroutines-test/js/src/TestBuilders.kt new file mode 100644 index 0000000000..e5265615e0 --- /dev/null +++ b/kotlinx-coroutines-test/js/src/TestBuilders.kt @@ -0,0 +1,17 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.test.internal.* + +public actual typealias TestResult = JsPromiseInterfaceForTesting + + +@Suppress("CAST_NEVER_SUCCEEDS") +internal actual fun createTestResult(testProcedure: suspend CoroutineScope.() -> Unit): TestResult = + GlobalScope.promise { + testProcedure() + } as JsPromiseInterfaceForTesting + +internal actual fun dumpCoroutines() { } + +internal actual fun systemPropertyImpl(name: String): String? = null diff --git a/kotlinx-coroutines-test/js/src/internal/JsPromiseInterfaceForTesting.kt b/kotlinx-coroutines-test/js/src/internal/JsPromiseInterfaceForTesting.kt new file mode 100644 index 0000000000..753c51dfaa --- /dev/null +++ b/kotlinx-coroutines-test/js/src/internal/JsPromiseInterfaceForTesting.kt @@ -0,0 +1,19 @@ +package kotlinx.coroutines.test.internal + +/* This is a declaration of JS's `Promise`. We need to keep it a separate class, because +`actual typealias TestResult = Promise` fails: you can't instantiate an `expect class` with a typealias to +a parametric class. So, we make a non-parametric class just for this. */ +/** + * @suppress + */ +@JsName("Promise") +public external class JsPromiseInterfaceForTesting { + /** + * @suppress + */ + public fun then(onFulfilled: ((Unit) -> Unit), onRejected: ((Throwable) -> Unit)): JsPromiseInterfaceForTesting + /** + * @suppress + */ + public fun then(onFulfilled: ((Unit) -> Unit)): JsPromiseInterfaceForTesting +} \ No newline at end of file diff --git a/kotlinx-coroutines-test/js/src/internal/TestMainDispatcher.kt b/kotlinx-coroutines-test/js/src/internal/TestMainDispatcher.kt new file mode 100644 index 0000000000..5fca8947c7 --- /dev/null +++ b/kotlinx-coroutines-test/js/src/internal/TestMainDispatcher.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines.test.internal +import kotlinx.coroutines.* + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 +internal actual fun Dispatchers.getTestMainDispatcher(): TestMainDispatcher = + when (val mainDispatcher = Main) { + is TestMainDispatcher -> mainDispatcher + else -> TestMainDispatcher(mainDispatcher).also { injectMain(it) } + } diff --git a/kotlinx-coroutines-test/js/test/Helpers.kt b/kotlinx-coroutines-test/js/test/Helpers.kt new file mode 100644 index 0000000000..9464f8c70a --- /dev/null +++ b/kotlinx-coroutines-test/js/test/Helpers.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines.test + +actual fun testResultChain(block: () -> TestResult, after: (Result) -> TestResult): TestResult = + block().then( + { + after(Result.success(Unit)) + }, { + after(Result.failure(it)) + }) diff --git a/kotlinx-coroutines-test/js/test/PromiseTest.kt b/kotlinx-coroutines-test/js/test/PromiseTest.kt new file mode 100644 index 0000000000..8774e4aba4 --- /dev/null +++ b/kotlinx-coroutines-test/js/test/PromiseTest.kt @@ -0,0 +1,18 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlin.test.* + +class PromiseTest { + @Test + fun testCompletionFromPromise() = runTest { + var promiseEntered = false + val p = promise { + delay(1) + promiseEntered = true + } + delay(2) + p.await() + assertTrue(promiseEntered) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-test/jvm/resources/META-INF/proguard/coroutines.pro b/kotlinx-coroutines-test/jvm/resources/META-INF/proguard/coroutines.pro new file mode 100644 index 0000000000..1fdfb78711 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/resources/META-INF/proguard/coroutines.pro @@ -0,0 +1,9 @@ +# ServiceLoader support +-keepnames class kotlinx.coroutines.test.internal.TestMainDispatcherFactory {} +-keepnames class kotlinx.coroutines.android.AndroidExceptionPreHandler {} +-keepnames class kotlinx.coroutines.android.AndroidDispatcherFactory {} + +# Most of volatile fields are updated with AFU and should not be mangled +-keepclassmembers class kotlinx.coroutines.** { + volatile ; +} diff --git a/kotlinx-coroutines-test/jvm/resources/META-INF/services/kotlinx.coroutines.CoroutineExceptionHandler b/kotlinx-coroutines-test/jvm/resources/META-INF/services/kotlinx.coroutines.CoroutineExceptionHandler new file mode 100644 index 0000000000..c9aaec2e60 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/resources/META-INF/services/kotlinx.coroutines.CoroutineExceptionHandler @@ -0,0 +1 @@ +kotlinx.coroutines.test.internal.ExceptionCollectorAsService diff --git a/kotlinx-coroutines-test/jvm/resources/META-INF/services/kotlinx.coroutines.internal.MainDispatcherFactory b/kotlinx-coroutines-test/jvm/resources/META-INF/services/kotlinx.coroutines.internal.MainDispatcherFactory new file mode 100644 index 0000000000..0ec0c9d5fc --- /dev/null +++ b/kotlinx-coroutines-test/jvm/resources/META-INF/services/kotlinx.coroutines.internal.MainDispatcherFactory @@ -0,0 +1 @@ +kotlinx.coroutines.test.internal.TestMainDispatcherFactory diff --git a/kotlinx-coroutines-test/jvm/src/TestBuildersJvm.kt b/kotlinx-coroutines-test/jvm/src/TestBuildersJvm.kt new file mode 100644 index 0000000000..dc7e0988bd --- /dev/null +++ b/kotlinx-coroutines-test/jvm/src/TestBuildersJvm.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.internal.* + +@Suppress("ACTUAL_WITHOUT_EXPECT") +public actual typealias TestResult = Unit + +internal actual fun createTestResult(testProcedure: suspend CoroutineScope.() -> Unit) { + runBlocking { + testProcedure() + } +} + +internal actual fun systemPropertyImpl(name: String): String? = + try { + System.getProperty(name) + } catch (e: SecurityException) { + null + } + +internal actual fun dumpCoroutines() { + @Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + if (DebugProbesImpl.isInstalled) { + DebugProbesImpl.install() + try { + DebugProbesImpl.dumpCoroutines(System.err) + System.err.flush() + } finally { + DebugProbesImpl.uninstall() + } + } +} diff --git a/kotlinx-coroutines-test/jvm/src/internal/TestMainDispatcherJvm.kt b/kotlinx-coroutines-test/jvm/src/internal/TestMainDispatcherJvm.kt new file mode 100644 index 0000000000..9626c7a87a --- /dev/null +++ b/kotlinx-coroutines-test/jvm/src/internal/TestMainDispatcherJvm.kt @@ -0,0 +1,52 @@ +package kotlinx.coroutines.test.internal + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* + +internal class TestMainDispatcherFactory : MainDispatcherFactory { + + override fun createDispatcher(allFactories: List): MainCoroutineDispatcher { + val otherFactories = allFactories.filter { it !== this } + val secondBestFactory = otherFactories.maxByOrNull { it.loadPriority } ?: MissingMainCoroutineDispatcherFactory + /* Do not immediately create the alternative dispatcher, as with `SUPPORT_MISSING` set to `false`, + it will throw an exception. Instead, create it lazily. */ + return TestMainDispatcher({ + val dispatcher = try { + secondBestFactory.tryCreateDispatcher(otherFactories) + } catch (e: Throwable) { + reportMissingMainCoroutineDispatcher(e) + } + if (dispatcher.isMissing()) { + reportMissingMainCoroutineDispatcher(runCatching { + // attempt to dispatch something to the missing dispatcher to trigger the exception. + dispatcher.dispatch(dispatcher, Runnable { }) + }.exceptionOrNull()) // can not be null, but it does not matter. + } else { + dispatcher + } + }) + } + + /** + * [Int.MAX_VALUE] -- test dispatcher always wins no matter what factories are present in the classpath. + * By default, all actions are delegated to the second-priority dispatcher, so that it won't be the issue. + */ + override val loadPriority: Int + get() = Int.MAX_VALUE +} + +internal actual fun Dispatchers.getTestMainDispatcher(): TestMainDispatcher { + val mainDispatcher = Main + require(mainDispatcher is TestMainDispatcher) { "TestMainDispatcher is not set as main dispatcher, have $mainDispatcher instead." } + return mainDispatcher +} + +private fun reportMissingMainCoroutineDispatcher(e: Throwable? = null): Nothing { + throw IllegalStateException( + "Dispatchers.Main was accessed when the platform dispatcher was absent " + + "and the test dispatcher was unset. Please make sure that Dispatchers.setMain() is called " + + "before accessing Dispatchers.Main and that Dispatchers.Main is not accessed after " + + "Dispatchers.resetMain().", + e + ) +} diff --git a/kotlinx-coroutines-test/jvm/src/migration/TestBuildersDeprecated.kt b/kotlinx-coroutines-test/jvm/src/migration/TestBuildersDeprecated.kt new file mode 100644 index 0000000000..53d963a9f7 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/src/migration/TestBuildersDeprecated.kt @@ -0,0 +1,227 @@ +@file:Suppress("DEPRECATION", "DEPRECATION_ERROR") +@file:JvmName("TestBuildersKt") +@file:JvmMultifileClass + +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlin.coroutines.* +import kotlin.jvm.* +import kotlin.time.Duration.Companion.milliseconds + +/** + * Executes a [testBody] inside an immediate execution dispatcher. + * + * This method is deprecated in favor of [runTest]. Please see the + * [migration guide](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md) + * for an instruction on how to update the code for the new API. + * + * This is similar to [runBlocking] but it will immediately progress past delays and into [launch] and [async] blocks. + * You can use this to write tests that execute in the presence of calls to [delay] without causing your test to take + * extra time. + * + * ``` + * @Test + * fun exampleTest() = runBlockingTest { + * val deferred = async { + * delay(1_000) + * async { + * delay(1_000) + * }.await() + * } + * + * deferred.await() // result available immediately + * } + * + * ``` + * + * This method requires that all coroutines launched inside [testBody] complete, or are cancelled, as part of the test + * conditions. + * + * Unhandled exceptions thrown by coroutines in the test will be re-thrown at the end of the test. + * + * @throws AssertionError If the [testBody] does not complete (or cancel) all coroutines that it launches + * (including coroutines suspended on join/await). + * + * @param context additional context elements. If [context] contains [CoroutineDispatcher] or [CoroutineExceptionHandler], + * then they must implement [DelayController] and [TestCoroutineExceptionHandler] respectively. + * @param testBody The code of the unit-test. + */ +@Deprecated( + "Use `runTest` instead to support completing from other dispatchers. " + + "Please see the migration guide for details: " + + "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", + level = DeprecationLevel.ERROR +) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public fun runBlockingTest( + context: CoroutineContext = EmptyCoroutineContext, + testBody: suspend TestCoroutineScope.() -> Unit +) { + val scope = createTestCoroutineScope(TestCoroutineDispatcher() + SupervisorJob() + context) + val scheduler = scope.testScheduler + val deferred = scope.async { + scope.testBody() + } + scheduler.advanceUntilIdle() + deferred.getCompletionExceptionOrNull()?.let { + throw it + } + scope.cleanupTestCoroutines() +} + +/** + * A version of [runBlockingTest] that works with [TestScope]. + */ +@Deprecated("Use `runTest` instead to support completing from other dispatchers.", level = DeprecationLevel.ERROR) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public fun runBlockingTestOnTestScope( + context: CoroutineContext = EmptyCoroutineContext, + testBody: suspend TestScope.() -> Unit +) { + val completeContext = TestCoroutineDispatcher() + SupervisorJob() + context + val startJobs = completeContext.activeJobs() + val scope = TestScope(completeContext).asSpecificImplementation() + scope.enter() + scope.start(CoroutineStart.UNDISPATCHED, scope) { + scope.testBody() + } + scope.testScheduler.advanceUntilIdle() + val throwable = try { + scope.getCompletionExceptionOrNull() + } catch (e: IllegalStateException) { + null // the deferred was not completed yet; `scope.legacyLeave()` should complain then about unfinished jobs + } + scope.backgroundScope.cancel() + scope.testScheduler.advanceUntilIdleOr { false } + throwable?.let { + val exceptions = try { + scope.legacyLeave() + } catch (e: UncompletedCoroutinesError) { + listOf() + } + throwAll(it, exceptions) + return + } + throwAll(null, scope.legacyLeave()) + val jobs = completeContext.activeJobs() - startJobs + if (jobs.isNotEmpty()) + throw UncompletedCoroutinesError("Some jobs were not completed at the end of the test: $jobs") +} + +/** + * Convenience method for calling [runBlockingTest] on an existing [TestCoroutineScope]. + * + * This method is deprecated in favor of [runTest], whereas [TestCoroutineScope] is deprecated in favor of [TestScope]. + * Please see the + * [migration guide](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md) + * for an instruction on how to update the code for the new API. + */ +@Deprecated( + "Use `runTest` instead to support completing from other dispatchers. " + + "Please see the migration guide for details: " + + "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", + level = DeprecationLevel.ERROR +) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public fun TestCoroutineScope.runBlockingTest(block: suspend TestCoroutineScope.() -> Unit): Unit = + runBlockingTest(coroutineContext, block) + +/** + * Convenience method for calling [runBlockingTestOnTestScope] on an existing [TestScope]. + */ +@Deprecated("Use `runTest` instead to support completing from other dispatchers.", level = DeprecationLevel.ERROR) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public fun TestScope.runBlockingTest(block: suspend TestScope.() -> Unit): Unit = + runBlockingTestOnTestScope(coroutineContext, block) + +/** + * Convenience method for calling [runBlockingTest] on an existing [TestCoroutineDispatcher]. + * + * This method is deprecated in favor of [runTest], whereas [TestCoroutineScope] is deprecated in favor of [TestScope]. + * Please see the + * [migration guide](https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md) + * for an instruction on how to update the code for the new API. + */ +@Deprecated( + "Use `runTest` instead to support completing from other dispatchers. " + + "Please see the migration guide for details: " + + "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", + level = DeprecationLevel.ERROR +) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public fun TestCoroutineDispatcher.runBlockingTest(block: suspend TestCoroutineScope.() -> Unit): Unit = + runBlockingTest(this, block) + +/** + * This is an overload of [runTest] that works with [TestCoroutineScope]. + */ +@ExperimentalCoroutinesApi +@Deprecated("Use `runTest` instead.", level = DeprecationLevel.ERROR) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public fun runTestWithLegacyScope( + context: CoroutineContext = EmptyCoroutineContext, + dispatchTimeoutMs: Long = DEFAULT_DISPATCH_TIMEOUT_MS, + testBody: suspend TestCoroutineScope.() -> Unit +) { + if (context[RunningInRunTest] != null) + throw IllegalStateException("Calls to `runTest` can't be nested. Please read the docs on `TestResult` for details.") + val testScope = TestBodyCoroutine(createTestCoroutineScope(context + RunningInRunTest)) + return createTestResult { + runTestCoroutineLegacy( + testScope, + dispatchTimeoutMs.milliseconds, + TestBodyCoroutine::tryGetCompletionCause, + testBody + ) { + try { + testScope.cleanup() + emptyList() + } catch (e: UncompletedCoroutinesError) { + throw e + } catch (e: Throwable) { + listOf(e) + } + } + } +} + +/** + * Runs a test in a [TestCoroutineScope] based on this one. + * + * Calls [runTest] using a coroutine context from this [TestCoroutineScope]. The [TestCoroutineScope] used to run the + * [block] will be different from this one, but will use its [Job] as a parent. + * + * Since this function returns [TestResult], in order to work correctly on the JS, its result must be returned + * immediately from the test body. See the docs for [TestResult] for details. + */ +@ExperimentalCoroutinesApi +@Deprecated("Use `TestScope.runTest` instead.", level = DeprecationLevel.ERROR) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public fun TestCoroutineScope.runTest( + dispatchTimeoutMs: Long = DEFAULT_DISPATCH_TIMEOUT_MS, + block: suspend TestCoroutineScope.() -> Unit +): TestResult = runTestWithLegacyScope(coroutineContext, dispatchTimeoutMs, block) + +private class TestBodyCoroutine( + private val testScope: TestCoroutineScope, +) : AbstractCoroutine(testScope.coroutineContext, initParentJob = true, active = true), TestCoroutineScope { + + override val testScheduler get() = testScope.testScheduler + + @Deprecated( + "This deprecation is to prevent accidentally calling `cleanupTestCoroutines` in our own code.", + ReplaceWith("this.cleanup()"), + DeprecationLevel.ERROR + ) + override fun cleanupTestCoroutines() = + throw UnsupportedOperationException( + "Calling `cleanupTestCoroutines` inside `runTest` is prohibited: " + + "it will be called at the end of the test in any case." + ) + + fun cleanup() = testScope.cleanupTestCoroutines() + + /** Throws an exception if the coroutine is not completing. */ + fun tryGetCompletionCause(): Throwable? = completionCause +} diff --git a/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineDispatcher.kt b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineDispatcher.kt new file mode 100644 index 0000000000..585f77b7b9 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineDispatcher.kt @@ -0,0 +1,69 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * @suppress + */ +@Deprecated("The execution order of `TestCoroutineDispatcher` can be confusing, and the mechanism of " + + "pausing is typically misunderstood. Please use `StandardTestDispatcher` or `UnconfinedTestDispatcher` instead.", + level = DeprecationLevel.ERROR) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public class TestCoroutineDispatcher(public override val scheduler: TestCoroutineScheduler = TestCoroutineScheduler()): + TestDispatcher(), Delay +{ + private var dispatchImmediately = true + set(value) { + field = value + if (value) { + // there may already be tasks from setup code we need to run + scheduler.advanceUntilIdle() + } + } + + /** @suppress */ + override fun dispatch(context: CoroutineContext, block: Runnable) { + checkSchedulerInContext(scheduler, context) + if (dispatchImmediately) { + scheduler.sendDispatchEvent(context) + block.run() + } else { + post(block, context) + } + } + + /** @suppress */ + override fun dispatchYield(context: CoroutineContext, block: Runnable) { + checkSchedulerInContext(scheduler, context) + post(block, context) + } + + /** @suppress */ + override fun toString(): String = "TestCoroutineDispatcher[scheduler=$scheduler]" + + private fun post(block: Runnable, context: CoroutineContext) = + scheduler.registerEvent(this, 0, block, context) { false } + + public val currentTime: Long + get() = scheduler.currentTime + + public fun advanceUntilIdle(): Long { + val oldTime = scheduler.currentTime + scheduler.advanceUntilIdle() + return scheduler.currentTime - oldTime + } + + public fun runCurrent(): Unit = scheduler.runCurrent() + + public fun cleanupTestCoroutines() { + // process any pending cancellations or completions, but don't advance time + scheduler.runCurrent() + if (!scheduler.isIdle(strict = false)) { + throw UncompletedCoroutinesError( + "Unfinished coroutines during tear-down. Ensure all coroutines are" + + " completed or cancelled by your test." + ) + } + } +} diff --git a/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineExceptionHandler.kt b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineExceptionHandler.kt new file mode 100644 index 0000000000..4956b245e2 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineExceptionHandler.kt @@ -0,0 +1,35 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +internal class TestCoroutineExceptionHandler : + AbstractCoroutineContextElement(CoroutineExceptionHandler), CoroutineExceptionHandler { + private val _exceptions = mutableListOf() + private val _lock = SynchronizedObject() + private var _coroutinesCleanedUp = false + + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + override fun handleException(context: CoroutineContext, exception: Throwable) { + synchronized(_lock) { + if (_coroutinesCleanedUp) { + handleUncaughtCoroutineException(context, exception) + } + _exceptions += exception + } + } + + val uncaughtExceptions: List + get() = synchronized(_lock) { _exceptions.toList() } + + fun cleanupTestCoroutines() { + synchronized(_lock) { + _coroutinesCleanedUp = true + val exception = _exceptions.firstOrNull() ?: return + // log the rest + _exceptions.drop(1).forEach { it.printStackTrace() } + throw exception + } + } +} diff --git a/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineScope.kt b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineScope.kt new file mode 100644 index 0000000000..91d29a480a --- /dev/null +++ b/kotlinx-coroutines-test/jvm/src/migration/TestCoroutineScope.kt @@ -0,0 +1,199 @@ +@file:Suppress("DEPRECATION_ERROR", "DEPRECATION") + +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +/** + * @suppress + */ +@ExperimentalCoroutinesApi +@Deprecated("Use `TestScope` in combination with `runTest` instead." + + "Please see the migration guide for details: " + + "https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md", + level = DeprecationLevel.ERROR) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public interface TestCoroutineScope : CoroutineScope { + /** + * @suppress + */ + @ExperimentalCoroutinesApi + @Deprecated( + "Please call `runTest`, which automatically performs the cleanup, instead of using this function.", + level = DeprecationLevel.ERROR + ) + // Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later + public fun cleanupTestCoroutines() + + /** + * The delay-skipping scheduler used by the test dispatchers running the code in this scope. + */ + @ExperimentalCoroutinesApi + public val testScheduler: TestCoroutineScheduler +} + +private class TestCoroutineScopeImpl( + override val coroutineContext: CoroutineContext +) : TestCoroutineScope { + private val lock = SynchronizedObject() + private var exceptions = mutableListOf() + private var cleanedUp = false + + /** + * Reports an exception so that it is thrown on [cleanupTestCoroutines]. + * + * If several exceptions are reported, only the first one will be thrown, and the other ones will be suppressed by + * it. + * + * Returns `false` if [cleanupTestCoroutines] was already called. + */ + fun reportException(throwable: Throwable): Boolean = + synchronized(lock) { + if (cleanedUp) { + false + } else { + exceptions.add(throwable) + true + } + } + + override val testScheduler: TestCoroutineScheduler + get() = coroutineContext[TestCoroutineScheduler]!! + + /** These jobs existed before the coroutine scope was used, so it's alright if they don't get cancelled. */ + private val initialJobs = coroutineContext.activeJobs() + + @Deprecated("Please call `runTest`, which automatically performs the cleanup, instead of using this function.") + override fun cleanupTestCoroutines() { + val delayController = coroutineContext.delayController + val hasUnfinishedJobs = if (delayController != null) { + try { + delayController.cleanupTestCoroutines() + false + } catch (_: UncompletedCoroutinesError) { + true + } + } else { + testScheduler.runCurrent() + !testScheduler.isIdle(strict = false) + } + (coroutineContext[CoroutineExceptionHandler] as? TestCoroutineExceptionHandler)?.cleanupTestCoroutines() + synchronized(lock) { + if (cleanedUp) + throw IllegalStateException("Attempting to clean up a test coroutine scope more than once.") + cleanedUp = true + } + exceptions.firstOrNull()?.let { toThrow -> + exceptions.drop(1).forEach { toThrow.addSuppressed(it) } + throw toThrow + } + if (hasUnfinishedJobs) + throw UncompletedCoroutinesError( + "Unfinished coroutines during teardown. Ensure all coroutines are" + + " completed or cancelled by your test." + ) + val jobs = coroutineContext.activeJobs() + if ((jobs - initialJobs).isNotEmpty()) + throw UncompletedCoroutinesError("Test finished with active jobs: $jobs") + } +} + +internal fun CoroutineContext.activeJobs(): Set { + return checkNotNull(this[Job]).children.filter { it.isActive }.toSet() +} + +/** + * @suppress + */ +@Deprecated( + "This constructs a `TestCoroutineScope` with a deprecated `CoroutineDispatcher` by default. " + + "Please use `createTestCoroutineScope` instead.", + ReplaceWith( + "createTestCoroutineScope(TestCoroutineDispatcher() + TestCoroutineExceptionHandler() + context)", + "kotlin.coroutines.EmptyCoroutineContext" + ), + level = DeprecationLevel.ERROR +) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public fun TestCoroutineScope(context: CoroutineContext = EmptyCoroutineContext): TestCoroutineScope { + val scheduler = context[TestCoroutineScheduler] ?: TestCoroutineScheduler() + return createTestCoroutineScope(TestCoroutineDispatcher(scheduler) + TestCoroutineExceptionHandler() + context) +} + +/** + * @suppress + */ +@ExperimentalCoroutinesApi +@Deprecated( + "This function was introduced in order to help migrate from TestCoroutineScope to TestScope. " + + "Please use TestScope() construction instead, or just runTest(), without creating a scope.", + level = DeprecationLevel.ERROR +) +// Since 1.6.0, kept as warning in 1.7.0, ERROR in 1.9.0 and removed as experimental later +public fun createTestCoroutineScope(context: CoroutineContext = EmptyCoroutineContext): TestCoroutineScope { + val ctxWithDispatcher = context.withDelaySkipping() + var scope: TestCoroutineScopeImpl? = null + val ownExceptionHandler = + object : AbstractCoroutineContextElement(CoroutineExceptionHandler), TestCoroutineScopeExceptionHandler { + override fun handleException(context: CoroutineContext, exception: Throwable) { + if (!scope!!.reportException(exception)) + throw exception // let this exception crash everything + } + } + val exceptionHandler = when (val exceptionHandler = ctxWithDispatcher[CoroutineExceptionHandler]) { + is TestCoroutineExceptionHandler -> exceptionHandler + null -> ownExceptionHandler + is TestCoroutineScopeExceptionHandler -> ownExceptionHandler + else -> throw IllegalArgumentException( + "A CoroutineExceptionHandler was passed to TestCoroutineScope. " + + "Please pass it as an argument to a `launch` or `async` block on an already-created scope " + + "if uncaught exceptions require special treatment." + ) + } + val job: Job = ctxWithDispatcher[Job] ?: Job() + return TestCoroutineScopeImpl(ctxWithDispatcher + exceptionHandler + job).also { + scope = it + } +} + +/** A marker that shows that this [CoroutineExceptionHandler] was created for [TestCoroutineScope]. With this, + * constructing a new [TestCoroutineScope] with the [CoroutineScope.coroutineContext] of an existing one will override + * the exception handler, instead of failing. */ +private interface TestCoroutineScopeExceptionHandler : CoroutineExceptionHandler + +private inline val CoroutineContext.delayController: TestCoroutineDispatcher? + get() { + val handler = this[ContinuationInterceptor] + return handler as? TestCoroutineDispatcher + } + + +/** + * The current virtual time on [testScheduler][TestCoroutineScope.testScheduler]. + * @see TestCoroutineScheduler.currentTime + */ +@ExperimentalCoroutinesApi +public val TestCoroutineScope.currentTime: Long + get() = coroutineContext.delayController?.currentTime ?: testScheduler.currentTime + +/** + * Advances the [testScheduler][TestCoroutineScope.testScheduler] to the point where there are no tasks remaining. + * @see TestCoroutineScheduler.advanceUntilIdle + */ +@ExperimentalCoroutinesApi +public fun TestCoroutineScope.advanceUntilIdle() { + coroutineContext.delayController?.advanceUntilIdle() ?: testScheduler.advanceUntilIdle() +} + +/** + * Run any tasks that are pending at the current virtual time, according to + * the [testScheduler][TestCoroutineScope.testScheduler]. + * + * @see TestCoroutineScheduler.runCurrent + */ +@ExperimentalCoroutinesApi +public fun TestCoroutineScope.runCurrent() { + coroutineContext.delayController?.runCurrent() ?: testScheduler.runCurrent() +} diff --git a/kotlinx-coroutines-test/jvm/src/module-info.java b/kotlinx-coroutines-test/jvm/src/module-info.java new file mode 100644 index 0000000000..9846263c6c --- /dev/null +++ b/kotlinx-coroutines-test/jvm/src/module-info.java @@ -0,0 +1,15 @@ +import kotlinx.coroutines.CoroutineExceptionHandler; +import kotlinx.coroutines.internal.MainDispatcherFactory; +import kotlinx.coroutines.test.internal.ExceptionCollectorAsService; +import kotlinx.coroutines.test.internal.TestMainDispatcherFactory; + +module kotlinx.coroutines.test { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.atomicfu; + + exports kotlinx.coroutines.test; + + provides MainDispatcherFactory with TestMainDispatcherFactory; + provides CoroutineExceptionHandler with ExceptionCollectorAsService; +} diff --git a/kotlinx-coroutines-test/jvm/test/DumpOnTimeoutTest.kt b/kotlinx-coroutines-test/jvm/test/DumpOnTimeoutTest.kt new file mode 100644 index 0000000000..36f73c2d39 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/DumpOnTimeoutTest.kt @@ -0,0 +1,44 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.debug.* +import org.junit.Test +import java.io.* +import kotlin.test.* +import kotlin.time.Duration.Companion.milliseconds + +class DumpOnTimeoutTest { + /** + * Tests that the dump on timeout contains the correct stacktrace. + */ + @Test + fun testDumpOnTimeout() { + val oldErr = System.err + val baos = ByteArrayOutputStream() + try { + System.setErr(PrintStream(baos, true)) + DebugProbes.withDebugProbes { + try { + runTest(timeout = 100.milliseconds) { + uniquelyNamedFunction() + } + throw IllegalStateException("unreachable") + } catch (e: UncompletedCoroutinesError) { + // do nothing + } + } + baos.toString().let { + assertTrue(it.contains("uniquelyNamedFunction"), "Actual trace:\n$it") + } + } finally { + System.setErr(oldErr) + } + } + + fun CoroutineScope.uniquelyNamedFunction() { + while (true) { + ensureActive() + Thread.sleep(10) + } + } +} diff --git a/kotlinx-coroutines-test/jvm/test/HelpersJvm.kt b/kotlinx-coroutines-test/jvm/test/HelpersJvm.kt new file mode 100644 index 0000000000..ed3afa39e4 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/HelpersJvm.kt @@ -0,0 +1,10 @@ +package kotlinx.coroutines.test + +actual fun testResultChain(block: () -> TestResult, after: (Result) -> TestResult): TestResult { + try { + block() + after(Result.success(Unit)) + } catch (e: Throwable) { + after(Result.failure(e)) + } +} diff --git a/kotlinx-coroutines-test/jvm/test/MemoryLeakTest.kt b/kotlinx-coroutines-test/jvm/test/MemoryLeakTest.kt new file mode 100644 index 0000000000..218d8aaa67 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/MemoryLeakTest.kt @@ -0,0 +1,21 @@ +import kotlinx.coroutines.* +import kotlinx.coroutines.test.* +import kotlinx.coroutines.testing.* +import kotlin.test.* + +class MemoryLeakTest { + + @Test + fun testCancellationLeakInTestCoroutineScheduler() = runTest { + val leakingObject = Any() + val job = launch(start = CoroutineStart.UNDISPATCHED) { + delay(1) + // This code is needed to hold a reference to `leakingObject` until the job itself is weakly reachable. + leakingObject.hashCode() + } + job.cancel() + FieldWalker.assertReachableCount(1, testScheduler) { it === leakingObject } + runCurrent() + FieldWalker.assertReachableCount(0, testScheduler) { it === leakingObject } + } +} diff --git a/kotlinx-coroutines-test/jvm/test/MultithreadingTest.kt b/kotlinx-coroutines-test/jvm/test/MultithreadingTest.kt new file mode 100644 index 0000000000..3df302e754 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/MultithreadingTest.kt @@ -0,0 +1,119 @@ +import kotlinx.coroutines.* +import kotlinx.coroutines.test.* +import kotlin.concurrent.* +import kotlin.coroutines.* +import kotlin.test.* + +@Suppress("DEPRECATION", "DEPRECATION_ERROR") +class MultithreadingTest { + + @Test + fun incorrectlyCalledRunBlocking_doesNotHaveSameInterceptor() = runBlockingTest { + // this code is an error as a production test, please do not use this as an example + + // this test exists to document this error condition, if it's possible to make this code work please update + val outerInterceptor = coroutineContext[ContinuationInterceptor] + // runBlocking always requires an argument to pass the context in tests + runBlocking { + assertNotSame(coroutineContext[ContinuationInterceptor], outerInterceptor) + } + } + + @Test + fun testSingleThreadExecutor() = runBlocking { + val mainThread = Thread.currentThread() + Dispatchers.setMain(Dispatchers.Unconfined) + newSingleThreadContext("testSingleThread").use { threadPool -> + withContext(Dispatchers.Main) { + assertSame(mainThread, Thread.currentThread()) + } + + Dispatchers.setMain(threadPool) + withContext(Dispatchers.Main) { + assertNotSame(mainThread, Thread.currentThread()) + } + assertSame(mainThread, Thread.currentThread()) + + withContext(Dispatchers.Main.immediate) { + assertNotSame(mainThread, Thread.currentThread()) + } + assertSame(mainThread, Thread.currentThread()) + + Dispatchers.setMain(Dispatchers.Unconfined) + withContext(Dispatchers.Main.immediate) { + assertSame(mainThread, Thread.currentThread()) + } + assertSame(mainThread, Thread.currentThread()) + } + } + + @Test + fun whenDispatchCalled_runsOnCurrentThread() { + val currentThread = Thread.currentThread() + val subject = TestCoroutineDispatcher() + val scope = TestCoroutineScope(subject) + + val deferred = scope.async(Dispatchers.Default) { + withContext(subject) { + assertNotSame(currentThread, Thread.currentThread()) + 3 + } + } + + runBlocking { + // just to ensure the above code terminates + assertEquals(3, deferred.await()) + } + } + + @Test + fun whenAllDispatchersMocked_runsOnSameThread() { + val currentThread = Thread.currentThread() + val subject = TestCoroutineDispatcher() + val scope = TestCoroutineScope(subject) + + val deferred = scope.async(subject) { + withContext(subject) { + assertSame(currentThread, Thread.currentThread()) + 3 + } + } + + runBlocking { + // just to ensure the above code terminates + assertEquals(3, deferred.await()) + } + } + + /** Tests that resuming the coroutine of [runTest] asynchronously in reasonable time succeeds. */ + @Test + fun testResumingFromAnotherThread() = runTest { + suspendCancellableCoroutine { cont -> + thread { + Thread.sleep(10) + cont.resume(Unit) + } + } + } + + /** Tests that [StandardTestDispatcher] is not executed in-place but confined to the thread in which the + * virtual time control happens. */ + @Test + fun testStandardTestDispatcherIsConfined(): Unit = runBlocking { + val scheduler = TestCoroutineScheduler() + val initialThread = Thread.currentThread() + val job = launch(StandardTestDispatcher(scheduler)) { + assertEquals(initialThread, Thread.currentThread()) + withContext(Dispatchers.IO) { + val ioThread = Thread.currentThread() + assertNotSame(initialThread, ioThread) + } + assertEquals(initialThread, Thread.currentThread()) + } + scheduler.advanceUntilIdle() + while (job.isActive) { + scheduler.receiveDispatchEvent() + scheduler.advanceUntilIdle() + } + } +} diff --git a/kotlinx-coroutines-test/jvm/test/RunTestStressTest.kt b/kotlinx-coroutines-test/jvm/test/RunTestStressTest.kt new file mode 100644 index 0000000000..d6469c5400 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/RunTestStressTest.kt @@ -0,0 +1,24 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import kotlin.concurrent.* +import kotlin.coroutines.* +import kotlin.test.* + +class RunTestStressTest { + /** Tests that notifications about asynchronous resumptions aren't lost. */ + @Test + fun testRunTestActivityNotificationsRace() { + val n = 1_000 * stressTestMultiplier + for (i in 0 until n) { + runTest { + suspendCancellableCoroutine { cont -> + thread { + cont.resume(Unit) + } + } + } + } + } +} diff --git a/kotlinx-coroutines-test/jvm/test/UncaughtExceptionsTest.kt b/kotlinx-coroutines-test/jvm/test/UncaughtExceptionsTest.kt new file mode 100644 index 0000000000..57893b8683 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/UncaughtExceptionsTest.kt @@ -0,0 +1,49 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import org.junit.Test +import kotlin.test.* + +/** + * Tests that check the behavior of the test framework when there are stray uncaught exceptions. + * These tests are JVM-only because only the JVM allows to set a global uncaught exception handler and validate the + * behavior without checking the test logs. + * Nevertheless, each test here has a corresponding test in the common source set that can be run manually. + */ +class UncaughtExceptionsTest { + + val oldExceptionHandler = Thread.getDefaultUncaughtExceptionHandler() + val uncaughtExceptions = mutableListOf() + + @BeforeTest + fun setUp() { + Thread.setDefaultUncaughtExceptionHandler { thread, throwable -> + uncaughtExceptions.add(throwable) + } + } + + @AfterTest + fun tearDown() { + Thread.setDefaultUncaughtExceptionHandler(oldExceptionHandler) + } + + @Test + fun testReportingStrayUncaughtExceptionsBetweenTests() { + TestScopeTest().testReportingStrayUncaughtExceptionsBetweenTests() + assertEquals(1, uncaughtExceptions.size, "Expected 1 uncaught exception, but got $uncaughtExceptions") + val exception = assertIs(uncaughtExceptions.single()) + assertEquals("x", exception.message) + } + + @Test + fun testExceptionCaptorCleanedUpOnPreliminaryExit() { + RunTestTest().testExceptionCaptorCleanedUpOnPreliminaryExit() + assertEquals(2, uncaughtExceptions.size, "Expected 2 uncaught exceptions, but got $uncaughtExceptions") + for (exception in uncaughtExceptions) { + assertIs(exception) + } + assertEquals("A", uncaughtExceptions[0].message) + assertEquals("B", uncaughtExceptions[1].message) + } +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/RunBlockingTestOnTestScopeTest.kt b/kotlinx-coroutines-test/jvm/test/migration/RunBlockingTestOnTestScopeTest.kt new file mode 100644 index 0000000000..0e8f3f772a --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/migration/RunBlockingTestOnTestScopeTest.kt @@ -0,0 +1,371 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.testing.* +import kotlin.test.* +import kotlin.test.assertFailsWith + +/** Copy of [RunTestTest], but for [runBlockingTestOnTestScope], where applicable. */ +@Suppress("DEPRECATION", "DEPRECATION_ERROR") +class RunBlockingTestOnTestScopeTest { + + @Test + fun testRunTestWithIllegalContext() { + for (ctx in TestScopeTest.invalidContexts) { + assertFailsWith { + runBlockingTestOnTestScope(ctx) { } + } + } + } + + @Test + fun testThrowingInRunTestBody() { + assertFailsWith { + runBlockingTestOnTestScope { + throw RuntimeException() + } + } + } + + @Test + fun testThrowingInRunTestPendingTask() { + assertFailsWith { + runBlockingTestOnTestScope { + launch { + delay(SLOW) + throw RuntimeException() + } + } + } + } + + @Test + fun reproducer2405() = runBlockingTestOnTestScope { + val dispatcher = StandardTestDispatcher(testScheduler) + var collectedError = false + withContext(dispatcher) { + flow { emit(1) } + .combine( + flow { throw IllegalArgumentException() } + ) { int, string -> int.toString() + string } + .catch { emit("error") } + .collect { + assertEquals("error", it) + collectedError = true + } + } + assertTrue(collectedError) + } + + @Test + fun testChildrenCancellationOnTestBodyFailure() { + var job: Job? = null + assertFailsWith { + runBlockingTestOnTestScope { + job = launch { + while (true) { + delay(1000) + } + } + throw AssertionError() + } + } + assertTrue(job!!.isCancelled) + } + + @Test + fun testTimeout() { + assertFailsWith { + runBlockingTestOnTestScope { + withTimeout(50) { + launch { + delay(1000) + } + } + } + } + } + + @Test + fun testRunTestThrowsRootCause() { + assertFailsWith { + runBlockingTestOnTestScope { + launch { + throw TestException() + } + } + } + } + + @Test + fun testCompletesOwnJob() { + var handlerCalled = false + runBlockingTestOnTestScope { + coroutineContext.job.invokeOnCompletion { + handlerCalled = true + } + } + assertTrue(handlerCalled) + } + + @Test + fun testDoesNotCompleteGivenJob() { + var handlerCalled = false + val job = Job() + job.invokeOnCompletion { + handlerCalled = true + } + runBlockingTestOnTestScope(job) { + assertTrue(coroutineContext.job in job.children) + } + assertFalse(handlerCalled) + assertEquals(0, job.children.filter { it.isActive }.count()) + } + + @Test + fun testSuppressedExceptions() { + try { + runBlockingTestOnTestScope { + launch(SupervisorJob()) { throw TestException("x") } + launch(SupervisorJob()) { throw TestException("y") } + launch(SupervisorJob()) { throw TestException("z") } + throw TestException("w") + } + fail("should not be reached") + } catch (e: TestException) { + assertEquals("w", e.message) + val suppressed = e.suppressedExceptions + + (e.suppressedExceptions.firstOrNull()?.suppressedExceptions ?: emptyList()) + assertEquals(3, suppressed.size) + assertEquals("x", suppressed[0].message) + assertEquals("y", suppressed[1].message) + assertEquals("z", suppressed[2].message) + } + } + + @Test + fun testScopeRunTestExceptionHandler(): TestResult { + val scope = TestCoroutineScope() + return testResultMap({ + try { + it() + fail("should not be reached") + } catch (e: TestException) { + // expected + } + }) { + scope.runTest { + launch(SupervisorJob()) { throw TestException("x") } + } + } + } + + @Test + fun testBackgroundWorkBeingRun() = runBlockingTestOnTestScope { + var i = 0 + var j = 0 + backgroundScope.launch { + yield() + ++i + } + backgroundScope.launch { + yield() + delay(10) + ++j + } + assertEquals(0, i) + assertEquals(0, j) + delay(1) + assertEquals(1, i) + assertEquals(0, j) + delay(10) + assertEquals(1, i) + assertEquals(1, j) + } + + @Test + fun testBackgroundWorkCancelled() { + var cancelled = false + runBlockingTestOnTestScope { + var i = 0 + backgroundScope.launch { + yield() + try { + while (isActive) { + ++i + yield() + } + } catch (e: CancellationException) { + cancelled = true + } + } + repeat(5) { + assertEquals(i, it) + yield() + } + } + assertTrue(cancelled) + } + + @Test + fun testBackgroundWorkTimeControl(): TestResult = runBlockingTestOnTestScope { + var i = 0 + var j = 0 + backgroundScope.launch { + yield() + while (true) { + ++i + delay(100) + } + } + backgroundScope.launch { + yield() + while (true) { + ++j + delay(50) + } + } + advanceUntilIdle() // should do nothing, as only background work is left. + assertEquals(0, i) + assertEquals(0, j) + val job = launch { + delay(1) + // the background work scheduled for earlier gets executed before the normal work scheduled for later does + assertEquals(1, i) + assertEquals(1, j) + } + job.join() + advanceTimeBy(199) // should work the same for the background tasks + assertEquals(2, i) + assertEquals(4, j) + advanceUntilIdle() // once again, should do nothing + assertEquals(2, i) + assertEquals(4, j) + runCurrent() // should behave the same way as for the normal work + assertEquals(3, i) + assertEquals(5, j) + launch { + delay(1001) + assertEquals(13, i) + assertEquals(25, j) + } + advanceUntilIdle() // should execute the normal work, and with that, the background one, too + } + + @Test + fun testBackgroundWorkErrorReporting() { + var testFinished = false + val exception = RuntimeException("x") + try { + runBlockingTestOnTestScope { + backgroundScope.launch { + throw exception + } + delay(1000) + testFinished = true + } + fail("unreached") + } catch (e: Throwable) { + assertSame(e, exception) + assertTrue(testFinished) + } + } + + @Test + fun testBackgroundWorkFinalizing() { + var taskEnded = 0 + val nTasks = 10 + try { + runBlockingTestOnTestScope { + repeat(nTasks) { + backgroundScope.launch { + try { + while (true) { + delay(1) + } + } finally { + ++taskEnded + if (taskEnded <= 2) + throw TestException() + } + } + } + delay(100) + throw TestException() + } + fail("unreached") + } catch (e: TestException) { + assertEquals(2, e.suppressedExceptions.size) + assertEquals(nTasks, taskEnded) + } + } + + @Test + fun testExampleBackgroundJob1() = runBlockingTestOnTestScope { + val myFlow = flow { + yield() + var i = 0 + while (true) { + emit(++i) + delay(1) + } + } + val stateFlow = myFlow.stateIn(backgroundScope, SharingStarted.Eagerly, 0) + var j = 0 + repeat(100) { + assertEquals(j++, stateFlow.value) + delay(1) + } + } + + @Test + fun testExampleBackgroundJob2() = runBlockingTestOnTestScope { + val channel = Channel() + backgroundScope.launch { + var i = 0 + while (true) { + channel.send(i++) + } + } + repeat(100) { + assertEquals(it, channel.receive()) + } + } + + @Test + fun testAsyncFailureInBackgroundReported() = + try { + runBlockingTestOnTestScope { + backgroundScope.async { + throw TestException("x") + } + backgroundScope.produce { + throw TestException("y") + } + delay(1) + throw TestException("z") + } + fail("unreached") + } catch (e: TestException) { + assertEquals("z", e.message) + assertEquals(setOf("x", "y"), e.suppressedExceptions.map { it.message }.toSet()) + } + + @Test + fun testNoDuplicateExceptions() = + try { + runBlockingTestOnTestScope { + backgroundScope.launch { + throw TestException("x") + } + delay(1) + throw TestException("y") + } + fail("unreached") + } catch (e: TestException) { + assertEquals("y", e.message) + assertEquals(listOf("x"), e.suppressedExceptions.map { it.message }) + } +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/RunTestLegacyScopeTest.kt b/kotlinx-coroutines-test/jvm/test/migration/RunTestLegacyScopeTest.kt new file mode 100644 index 0000000000..379abdcc17 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/migration/RunTestLegacyScopeTest.kt @@ -0,0 +1,273 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* +import kotlin.test.assertFailsWith + +/** Copy of [RunTestTest], but for [TestCoroutineScope] */ +@Suppress("DEPRECATION", "DEPRECATION_ERROR") +class RunTestLegacyScopeTest { + + @Test + fun testWithContextDispatching() = runTestWithLegacyScope { + var counter = 0 + withContext(Dispatchers.Default) { + counter += 1 + } + assertEquals(counter, 1) + } + + @Test + fun testJoiningForkedJob() = runTestWithLegacyScope { + var counter = 0 + val job = GlobalScope.launch { + counter += 1 + } + job.join() + assertEquals(counter, 1) + } + + @Test + fun testSuspendCoroutine() = runTestWithLegacyScope { + val answer = suspendCoroutine { + it.resume(42) + } + assertEquals(42, answer) + } + + @Test + fun testNestedRunTestForbidden() = runTestWithLegacyScope { + assertFailsWith { + runTest { } + } + } + + @Test + fun testRunTestWithZeroTimeoutWithControlledDispatches() = runTestWithLegacyScope(dispatchTimeoutMs = 0) { + // below is some arbitrary concurrent code where all dispatches go through the same scheduler. + launch { + delay(2000) + } + val deferred = async { + val job = launch(StandardTestDispatcher(testScheduler)) { + launch { + delay(500) + } + delay(1000) + } + job.join() + } + deferred.await() + } + + @Test + fun testRunTestWithSmallTimeout() = testResultMap({ fn -> + assertFailsWith { fn() } + }) { + runTestWithLegacyScope(dispatchTimeoutMs = 100) { + withContext(Dispatchers.Default) { + delay(10000) + 3 + } + fail("shouldn't be reached") + } + } + + @Test + fun testRunTestWithLargeTimeout() = runTestWithLegacyScope(dispatchTimeoutMs = 5000) { + withContext(Dispatchers.Default) { + delay(50) + } + } + + @Test + fun testRunTestTimingOutAndThrowing() = testResultMap({ fn -> + try { + fn() + fail("unreached") + } catch (e: UncompletedCoroutinesError) { + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + val suppressed = unwrap(e).suppressedExceptions + assertEquals(1, suppressed.size) + assertIs(suppressed[0]).also { + assertEquals("A", it.message) + } + } + }) { + runTestWithLegacyScope(dispatchTimeoutMs = 1) { + coroutineContext[CoroutineExceptionHandler]!!.handleException(coroutineContext, TestException("A")) + withContext(Dispatchers.Default) { + delay(10000) + 3 + } + fail("shouldn't be reached") + } + } + + @Test + fun testRunTestWithIllegalContext() { + for (ctx in TestScopeTest.invalidContexts) { + assertFailsWith { + runTestWithLegacyScope(ctx) { } + } + } + } + + @Test + fun testThrowingInRunTestBody() = testResultMap({ + assertFailsWith { it() } + }) { + runTestWithLegacyScope { + throw RuntimeException() + } + } + + @Test + fun testThrowingInRunTestPendingTask() = testResultMap({ + assertFailsWith { it() } + }) { + runTestWithLegacyScope { + launch { + delay(SLOW) + throw RuntimeException() + } + } + } + + @Test + fun reproducer2405() = runTestWithLegacyScope { + val dispatcher = StandardTestDispatcher(testScheduler) + var collectedError = false + withContext(dispatcher) { + flow { emit(1) } + .combine( + flow { throw IllegalArgumentException() } + ) { int, string -> int.toString() + string } + .catch { emit("error") } + .collect { + assertEquals("error", it) + collectedError = true + } + } + assertTrue(collectedError) + } + + @Test + fun testChildrenCancellationOnTestBodyFailure(): TestResult { + var job: Job? = null + return testResultMap({ + assertFailsWith { it() } + assertTrue(job!!.isCancelled) + }) { + runTestWithLegacyScope { + job = launch { + while (true) { + delay(1000) + } + } + throw AssertionError() + } + } + } + + @Test + fun testTimeout() = testResultMap({ + assertFailsWith { it() } + }) { + runTestWithLegacyScope { + withTimeout(50) { + launch { + delay(1000) + } + } + } + } + + @Test + fun testRunTestThrowsRootCause() = testResultMap({ + assertFailsWith { it() } + }) { + runTestWithLegacyScope { + launch { + throw TestException() + } + } + } + + @Test + fun testCompletesOwnJob(): TestResult { + var handlerCalled = false + return testResultMap({ + it() + assertTrue(handlerCalled) + }) { + runTestWithLegacyScope { + coroutineContext.job.invokeOnCompletion { + handlerCalled = true + } + } + } + } + + @Test + fun testDoesNotCompleteGivenJob(): TestResult { + var handlerCalled = false + val job = Job() + job.invokeOnCompletion { + handlerCalled = true + } + return testResultMap({ + it() + assertFalse(handlerCalled) + assertEquals(0, job.children.filter { it.isActive }.count()) + }) { + runTestWithLegacyScope(job) { + assertTrue(coroutineContext.job in job.children) + } + } + } + + @Test + fun testSuppressedExceptions() = testResultMap({ + try { + it() + fail("should not be reached") + } catch (e: TestException) { + assertEquals("w", e.message) + val suppressed = e.suppressedExceptions + + (e.suppressedExceptions.firstOrNull()?.suppressedExceptions ?: emptyList()) + assertEquals(3, suppressed.size) + assertEquals("x", suppressed[0].message) + assertEquals("y", suppressed[1].message) + assertEquals("z", suppressed[2].message) + } + }) { + runTestWithLegacyScope { + launch(SupervisorJob()) { throw TestException("x") } + launch(SupervisorJob()) { throw TestException("y") } + launch(SupervisorJob()) { throw TestException("z") } + throw TestException("w") + } + } + + @Test + fun testScopeRunTestExceptionHandler(): TestResult { + val scope = TestCoroutineScope() + return testResultMap({ + try { + it() + fail("should not be reached") + } catch (e: TestException) { + // expected + } + }) { + scope.runTest { + launch(SupervisorJob()) { throw TestException("x") } + } + } + } +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestBuildersTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestBuildersTest.kt new file mode 100644 index 0000000000..6973f19575 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/migration/TestBuildersTest.kt @@ -0,0 +1,103 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* + +@Suppress("DEPRECATION", "DEPRECATION_ERROR") +class TestBuildersTest { + + @Test + fun scopeRunBlocking_passesDispatcher() { + val scope = TestCoroutineScope() + scope.runBlockingTest { + assertSame(scope.coroutineContext[ContinuationInterceptor], coroutineContext[ContinuationInterceptor]) + } + } + + @Test + fun dispatcherRunBlocking_passesDispatcher() { + val dispatcher = TestCoroutineDispatcher() + dispatcher.runBlockingTest { + assertSame(dispatcher, coroutineContext[ContinuationInterceptor]) + } + } + + @Test + fun scopeRunBlocking_advancesPreviousDelay() { + val scope = TestCoroutineScope() + val deferred = scope.async { + delay(SLOW) + 3 + } + + scope.runBlockingTest { + assertRunsFast { + assertEquals(3, deferred.await()) + } + } + } + + @Test + fun dispatcherRunBlocking_advancesPreviousDelay() { + val dispatcher = TestCoroutineDispatcher() + val scope = CoroutineScope(dispatcher) + val deferred = scope.async { + delay(SLOW) + 3 + } + + dispatcher.runBlockingTest { + assertRunsFast { + assertEquals(3, deferred.await()) + } + } + } + + @Test + fun scopeRunBlocking_disablesImmediatelyOnExit() { + val scope = TestCoroutineScope() + scope.runBlockingTest { + assertRunsFast { + delay(SLOW) + } + } + + val deferred = scope.async { + delay(SLOW) + 3 + } + scope.runCurrent() + assertTrue(deferred.isActive) + + scope.advanceUntilIdle() + assertEquals(3, deferred.getCompleted()) + } + + @Test + fun whenInRunBlocking_runBlockingTest_nestsProperly() { + // this is not a supported use case, but it is possible so ensure it works + + val scope = TestCoroutineScope() + var calls = 0 + + scope.runBlockingTest { + delay(1_000) + calls++ + runBlockingTest { + val job = launch { + delay(1_000) + calls++ + } + assertTrue(job.isActive) + advanceUntilIdle() + assertFalse(job.isActive) + calls++ + } + ++calls + } + + assertEquals(4, calls) + } +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherTest.kt new file mode 100644 index 0000000000..eee5eea7e5 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineDispatcherTest.kt @@ -0,0 +1,36 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlin.test.* + +@Suppress("DEPRECATION", "DEPRECATION_ERROR") +class TestCoroutineDispatcherTest { + + @Test + fun whenDispatcherResumed_doesAutoProgressCurrent() { + val subject = TestCoroutineDispatcher() + val scope = CoroutineScope(subject) + var executed = 0 + scope.launch { + executed++ + } + + assertEquals(1, executed) + } + + @Test + fun whenDispatcherResumed_doesNotAutoProgressTime() { + val subject = TestCoroutineDispatcher() + val scope = CoroutineScope(subject) + var executed = 0 + scope.launch { + delay(1_000) + executed++ + } + + assertEquals(0, executed) + subject.advanceUntilIdle() + assertEquals(1, executed) + } + +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineScopeTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineScopeTest.kt new file mode 100644 index 0000000000..d275556c3d --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/migration/TestCoroutineScopeTest.kt @@ -0,0 +1,216 @@ +@file:Suppress("DEPRECATION", "DEPRECATION_ERROR") + +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import kotlin.coroutines.* +import kotlin.test.* +import kotlin.test.assertFailsWith + +class TestCoroutineScopeTest { + /** Tests failing to create a [TestCoroutineScope] with incorrect contexts. */ + @Test + fun testCreateThrowsOnInvalidArguments() { + for (ctx in invalidContexts) { + assertFailsWith { + createTestCoroutineScope(ctx) + } + } + } + + /** Tests that a newly-created [TestCoroutineScope] provides the correct scheduler. */ + @Test + fun testCreateProvidesScheduler() { + // Creates a new scheduler. + run { + val scope = createTestCoroutineScope() + assertNotNull(scope.coroutineContext[TestCoroutineScheduler]) + } + // Reuses the scheduler that the dispatcher is linked to. + run { + val dispatcher = StandardTestDispatcher() + val scope = createTestCoroutineScope(dispatcher) + assertSame(dispatcher.scheduler, scope.coroutineContext[TestCoroutineScheduler]) + } + // Uses the scheduler passed to it. + run { + val scheduler = TestCoroutineScheduler() + val scope = createTestCoroutineScope(scheduler) + assertSame(scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertSame(scheduler, (scope.coroutineContext[ContinuationInterceptor] as TestDispatcher).scheduler) + } + // Doesn't touch the passed dispatcher and the scheduler if they match. + run { + val scheduler = TestCoroutineScheduler() + val dispatcher = StandardTestDispatcher(scheduler) + val scope = createTestCoroutineScope(scheduler + dispatcher) + assertSame(scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertSame(dispatcher, scope.coroutineContext[ContinuationInterceptor]) + } + // Reuses the scheduler of `Dispatchers.Main` + run { + val scheduler = TestCoroutineScheduler() + val mainDispatcher = StandardTestDispatcher(scheduler) + Dispatchers.setMain(mainDispatcher) + try { + val scope = createTestCoroutineScope() + assertSame(scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertNotSame(mainDispatcher, scope.coroutineContext[ContinuationInterceptor]) + } finally { + Dispatchers.resetMain() + } + } + // Does not reuse the scheduler of `Dispatchers.Main` if one is explicitly passed + run { + val mainDispatcher = StandardTestDispatcher() + Dispatchers.setMain(mainDispatcher) + try { + val scheduler = TestCoroutineScheduler() + val scope = createTestCoroutineScope(scheduler) + assertSame(scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertNotSame(mainDispatcher.scheduler, scope.coroutineContext[TestCoroutineScheduler]) + assertNotSame(mainDispatcher, scope.coroutineContext[ContinuationInterceptor]) + } finally { + Dispatchers.resetMain() + } + } + } + + /** Tests that the cleanup procedure throws if there were uncompleted delays by the end. */ + @Test + fun testPresentDelaysThrowing() { + val scope = createTestCoroutineScope() + var result = false + scope.launch { + delay(5) + result = true + } + assertFalse(result) + assertFailsWith { scope.cleanupTestCoroutines() } + assertFalse(result) + } + + /** Tests that the cleanup procedure throws if there were active jobs by the end. */ + @Test + fun testActiveJobsThrowing() { + val scope = createTestCoroutineScope() + var result = false + val deferred = CompletableDeferred() + scope.launch { + deferred.await() + result = true + } + assertFalse(result) + assertFailsWith { scope.cleanupTestCoroutines() } + assertFalse(result) + } + + /** Tests that the cleanup procedure doesn't throw if it detects that the job is already cancelled. */ + @Test + fun testCancelledDelaysNotThrowing() { + val scope = createTestCoroutineScope() + var result = false + val deferred = CompletableDeferred() + val job = scope.launch { + deferred.await() + result = true + } + job.cancel() + assertFalse(result) + scope.cleanupTestCoroutines() + assertFalse(result) + } + + /** Tests that uncaught exceptions are thrown at the cleanup. */ + @Test + fun testThrowsUncaughtExceptionsOnCleanup() { + val scope = createTestCoroutineScope() + val exception = TestException("test") + scope.launch { + throw exception + } + assertFailsWith { + scope.cleanupTestCoroutines() + } + } + + /** Tests that uncaught exceptions take priority over uncompleted jobs when throwing on cleanup. */ + @Test + fun testUncaughtExceptionsPrioritizedOnCleanup() { + val scope = createTestCoroutineScope() + val exception = TestException("test") + scope.launch { + throw exception + } + scope.launch { + delay(1000) + } + assertFailsWith { + scope.cleanupTestCoroutines() + } + } + + /** Tests that cleaning up twice is forbidden. */ + @Test + fun testClosingTwice() { + val scope = createTestCoroutineScope() + scope.cleanupTestCoroutines() + assertFailsWith { + scope.cleanupTestCoroutines() + } + } + + /** Tests that, when reporting several exceptions, the first one is thrown, with the rest suppressed. */ + @Test + fun testSuppressedExceptions() { + createTestCoroutineScope().apply { + launch(SupervisorJob()) { throw TestException("x") } + launch(SupervisorJob()) { throw TestException("y") } + launch(SupervisorJob()) { throw TestException("z") } + try { + cleanupTestCoroutines() + fail("should not be reached") + } catch (e: TestException) { + assertEquals("x", e.message) + assertEquals(2, e.suppressedExceptions.size) + assertEquals("y", e.suppressedExceptions[0].message) + assertEquals("z", e.suppressedExceptions[1].message) + } + } + } + + /** Tests that constructing a new [TestCoroutineScope] using another one's scope works and overrides the exception + * handler. */ + @Test + fun testCopyingContexts() { + val deferred = CompletableDeferred() + val scope1 = createTestCoroutineScope() + scope1.launch { deferred.await() } // a pending job in the outer scope + val scope2 = createTestCoroutineScope(scope1.coroutineContext) + val scope3 = createTestCoroutineScope(scope1.coroutineContext) + assertEquals( + scope1.coroutineContext.minusKey(CoroutineExceptionHandler), + scope2.coroutineContext.minusKey(CoroutineExceptionHandler)) + scope2.launch(SupervisorJob()) { throw TestException("x") } // will fail the cleanup of scope2 + try { + scope2.cleanupTestCoroutines() + fail("should not be reached") + } catch (e: TestException) { } + scope3.cleanupTestCoroutines() // the pending job in the outer scope will not cause this to fail + try { + scope1.cleanupTestCoroutines() + fail("should not be reached") + } catch (e: UncompletedCoroutinesError) { + // the pending job in the outer scope + } + } + + companion object { + internal val invalidContexts = listOf( + Dispatchers.Default, // not a [TestDispatcher] + CoroutineExceptionHandler { _, _ -> }, // not an [UncaughtExceptionCaptor] + StandardTestDispatcher() + TestCoroutineScheduler(), // the dispatcher is not linked to the scheduler + ) + } +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingOrderTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingOrderTest.kt new file mode 100644 index 0000000000..5ba1216909 --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingOrderTest.kt @@ -0,0 +1,77 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlin.test.* + +@Suppress("DEPRECATION", "DEPRECATION_ERROR") +class TestRunBlockingOrderTest: OrderedExecutionTestBase() { + + @Test + fun testLaunchImmediate() = runBlockingTest { + expect(1) + launch { + expect(2) + } + finish(3) + } + + @Test + fun testYield() = runBlockingTest { + expect(1) + launch { + expect(2) + yield() + finish(4) + } + expect(3) + } + + @Test + fun testLaunchWithDelayCompletes() = runBlockingTest { + expect(1) + launch { + delay(100) + finish(3) + } + expect(2) + } + + @Test + fun testLaunchDelayOrdered() = runBlockingTest { + expect(1) + launch { + delay(200) // long delay + finish(4) + } + launch { + delay(100) // shorter delay + expect(3) + } + expect(2) + } + + @Test + fun testVeryLongDelay() = runBlockingTest { + expect(1) + delay(100) // move time forward a bit some that naive time + delay gives an overflow + launch { + delay(Long.MAX_VALUE / 2) // very long delay + finish(4) + } + launch { + delay(100) // short delay + expect(3) + } + expect(2) + } + + @Test + fun testAdvanceUntilIdle_inRunBlocking() = runBlockingTest { + expect(1) + assertRunsFast { + advanceUntilIdle() // ensure this doesn't block forever + } + finish(2) + } +} diff --git a/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingTest.kt b/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingTest.kt new file mode 100644 index 0000000000..a11638695f --- /dev/null +++ b/kotlinx-coroutines-test/jvm/test/migration/TestRunBlockingTest.kt @@ -0,0 +1,395 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.* +import kotlin.test.* +import kotlin.test.assertFailsWith + +@Suppress("DEPRECATION", "DEPRECATION_ERROR") +class TestRunBlockingTest { + + @Test + fun delay_advancesTimeAutomatically() = runBlockingTest { + assertRunsFast { + delay(SLOW) + } + } + + @Test + fun callingSuspendWithDelay_advancesAutomatically() = runBlockingTest { + suspend fun withDelay(): Int { + delay(SLOW) + return 3 + } + + assertRunsFast { + assertEquals(3, withDelay()) + } + } + + @Test + fun launch_advancesAutomatically() = runBlockingTest { + val job = launch { + delay(SLOW) + } + assertRunsFast { + job.join() + assertTrue(job.isCompleted) + } + } + + @Test + fun async_advancesAutomatically() = runBlockingTest { + val deferred = async { + delay(SLOW) + 3 + } + + assertRunsFast { + assertEquals(3, deferred.await()) + } + } + + @Test + fun whenUsingTimeout_triggersWhenDelayed() { + assertFailsWith { + runBlockingTest { + assertRunsFast { + withTimeout(SLOW) { + delay(SLOW) + } + } + } + } + } + + @Test + fun whenUsingTimeout_doesNotTriggerWhenFast() = runBlockingTest { + assertRunsFast { + withTimeout(SLOW) { + delay(0) + } + } + } + + @Test + fun whenUsingTimeout_triggersWhenWaiting() { + assertFailsWith { + runBlockingTest { + val uncompleted = CompletableDeferred() + assertRunsFast { + withTimeout(SLOW) { + uncompleted.await() + } + } + } + } + } + + @Test + fun whenUsingTimeout_doesNotTriggerWhenComplete() = runBlockingTest { + val completed = CompletableDeferred() + assertRunsFast { + completed.complete(Unit) + withTimeout(SLOW) { + completed.await() + } + } + } + + @Test + fun testDelayInAsync_withAwait() = runBlockingTest { + assertRunsFast { + val deferred = async { + delay(SLOW) + 3 + } + assertEquals(3, deferred.await()) + } + } + + @Test + fun whenUsingTimeout_inAsync_triggersWhenDelayed() { + assertFailsWith { + runBlockingTest { + val deferred = async { + withTimeout(SLOW) { + delay(SLOW) + } + } + + assertRunsFast { + deferred.await() + } + } + } + } + + @Test + fun whenUsingTimeout_inAsync_doesNotTriggerWhenNotDelayed() = runBlockingTest { + val deferred = async { + withTimeout(SLOW) { + delay(0) + } + } + + assertRunsFast { + deferred.await() + } + } + + @Test + fun whenUsingTimeout_inLaunch_triggersWhenDelayed() { + assertFailsWith { + runBlockingTest { + val job = launch { + withTimeout(1) { + delay(SLOW + 1) + } + } + + assertRunsFast { + job.join() + throw job.getCancellationException() + } + } + } + } + + @Test + fun whenUsingTimeout_inLaunch_doesNotTriggerWhenNotDelayed() = runBlockingTest { + val job = launch { + withTimeout(SLOW) { + delay(0) + } + } + + assertRunsFast { + job.join() + assertTrue(job.isCompleted) + } + } + + @Test + fun throwingException_throws() { + assertFailsWith { + runBlockingTest { + assertRunsFast { + delay(SLOW) + throw IllegalArgumentException("Test") + } + } + } + } + + @Test + fun throwingException_inLaunch_throws() { + assertFailsWith { + runBlockingTest { + val job = launch { + delay(SLOW) + throw IllegalArgumentException("Test") + } + + assertRunsFast { + job.join() + throw job.getCancellationException().cause ?: AssertionError("expected exception") + } + } + } + } + + @Test + fun throwingException__inAsync_throws() { + assertFailsWith { + runBlockingTest { + val deferred: Deferred = async { + delay(SLOW) + throw IllegalArgumentException("Test") + } + + assertRunsFast { + deferred.await() + } + } + } + } + + @Test + fun callingLaunchFunction_executesLaunchBlockImmediately() = runBlockingTest { + assertRunsFast { + var executed = false + launch { + delay(SLOW) + executed = true + } + + delay(SLOW) + assertTrue(executed) + } + } + + @Test + fun callingAsyncFunction_executesAsyncBlockImmediately() = runBlockingTest { + assertRunsFast { + var executed = false + val deferred = async { + delay(SLOW) + executed = true + } + delay(SLOW) + + assertTrue(deferred.isCompleted) + assertTrue(executed) + } + } + + @Test + fun nestingBuilders_executesSecondLevelImmediately() = runBlockingTest { + assertRunsFast { + var levels = 0 + launch { + delay(SLOW) + levels++ + launch { + delay(SLOW) + levels++ + } + } + advanceUntilIdle() + + assertEquals(2, levels) + } + } + + @Test + fun testCancellationException() = runBlockingTest { + var actual: CancellationException? = null + val uncompleted = CompletableDeferred() + val job = launch { + actual = kotlin.runCatching { uncompleted.await() }.exceptionOrNull() as? CancellationException + } + + assertNull(actual) + job.cancel() + assertNotNull(actual) + } + + @Test + fun testCancellationException_notThrown() = runBlockingTest { + val uncompleted = CompletableDeferred() + val job = launch { + uncompleted.await() + } + + job.cancel() + job.join() + } + + @Test + fun whenACoroutineLeaks_errorIsThrown() { + assertFailsWith { + runBlockingTest { + val uncompleted = CompletableDeferred() + launch { + uncompleted.await() + } + } + } + } + + @Test + fun runBlockingTestBuilder_throwsOnBadDispatcher() { + assertFailsWith { + runBlockingTest(Dispatchers.Default) { + + } + } + } + + @Test + fun runBlockingTestBuilder_throwsOnBadHandler() { + assertFailsWith { + runBlockingTest(CoroutineExceptionHandler { _, _ -> }) { + + } + } + } + + @Test + fun testWithTestContextThrowingAnAssertionError() { + assertFailsWith { + runBlockingTest { + val expectedError = TestException("hello") + + launch { + throw expectedError + } + + // don't rethrow or handle the exception + } + } + } + + @Test + fun testExceptionHandlingWithLaunch() { + assertFailsWith { + runBlockingTest { + val expectedError = TestException("hello") + + launch { + throw expectedError + } + } + } + } + + @Test + fun testExceptions_notThrownImmediately() { + assertFailsWith { + runBlockingTest { + val expectedException = TestException("hello") + val result = runCatching { + launch { + throw expectedException + } + } + runCurrent() + assertEquals(true, result.isSuccess) + } + } + } + + + private val exceptionHandler = TestCoroutineExceptionHandler() + + @Test + fun testPartialContextOverride() = runBlockingTest(CoroutineName("named")) { + assertEquals(CoroutineName("named"), coroutineContext[CoroutineName]) + assertNotNull(coroutineContext[CoroutineExceptionHandler]) + assertNotSame(coroutineContext[CoroutineExceptionHandler], exceptionHandler) + } + + @Test + fun testPartialDispatcherOverride() { + assertFailsWith { + runBlockingTest(Dispatchers.Unconfined) { + fail("Unreached") + } + } + } + + @Test + fun testOverrideExceptionHandler() = runBlockingTest(exceptionHandler) { + assertSame(coroutineContext[CoroutineExceptionHandler], exceptionHandler) + } + + @Test + fun testOverrideExceptionHandlerError() { + assertFailsWith { + runBlockingTest(CoroutineExceptionHandler { _, _ -> }) { + fail("Unreached") + } + } + } +} diff --git a/kotlinx-coroutines-test/native/src/TestBuilders.kt b/kotlinx-coroutines-test/native/src/TestBuilders.kt new file mode 100644 index 0000000000..b7432c21ea --- /dev/null +++ b/kotlinx-coroutines-test/native/src/TestBuilders.kt @@ -0,0 +1,14 @@ +package kotlinx.coroutines.test +import kotlinx.coroutines.* + +public actual typealias TestResult = Unit + +internal actual fun createTestResult(testProcedure: suspend CoroutineScope.() -> Unit) { + runBlocking { + testProcedure() + } +} + +internal actual fun systemPropertyImpl(name: String): String? = null + +internal actual fun dumpCoroutines() { } diff --git a/kotlinx-coroutines-test/native/src/internal/TestMainDispatcher.kt b/kotlinx-coroutines-test/native/src/internal/TestMainDispatcher.kt new file mode 100644 index 0000000000..5fca8947c7 --- /dev/null +++ b/kotlinx-coroutines-test/native/src/internal/TestMainDispatcher.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines.test.internal +import kotlinx.coroutines.* + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 +internal actual fun Dispatchers.getTestMainDispatcher(): TestMainDispatcher = + when (val mainDispatcher = Main) { + is TestMainDispatcher -> mainDispatcher + else -> TestMainDispatcher(mainDispatcher).also { injectMain(it) } + } diff --git a/kotlinx-coroutines-test/native/test/Helpers.kt b/kotlinx-coroutines-test/native/test/Helpers.kt new file mode 100644 index 0000000000..ed3afa39e4 --- /dev/null +++ b/kotlinx-coroutines-test/native/test/Helpers.kt @@ -0,0 +1,10 @@ +package kotlinx.coroutines.test + +actual fun testResultChain(block: () -> TestResult, after: (Result) -> TestResult): TestResult { + try { + block() + after(Result.success(Unit)) + } catch (e: Throwable) { + after(Result.failure(e)) + } +} diff --git a/kotlinx-coroutines-test/npm/README.md b/kotlinx-coroutines-test/npm/README.md new file mode 100644 index 0000000000..4df4825da9 --- /dev/null +++ b/kotlinx-coroutines-test/npm/README.md @@ -0,0 +1,4 @@ +# kotlinx-coroutines-test + +Testing support for `kotlinx-coroutines` in +[Kotlin/JS](https://kotlinlang.org/docs/js-overview.html). diff --git a/kotlinx-coroutines-test/npm/package.json b/kotlinx-coroutines-test/npm/package.json new file mode 100644 index 0000000000..b59d92fe03 --- /dev/null +++ b/kotlinx-coroutines-test/npm/package.json @@ -0,0 +1,23 @@ +{ + "name": "kotlinx-coroutines-test", + "version" : "$version", + "description" : "Test utilities for kotlinx-coroutines", + "main" : "kotlinx-coroutines-test.js", + "author": "JetBrains", + "license": "Apache-2.0", + "homepage": "https://github.com/Kotlin/kotlinx.coroutines", + "bugs": { + "url": "https://github.com/Kotlin/kotlinx.coroutines/issues" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/Kotlin/kotlinx.coroutines.git" + }, + "keywords": [ + "Kotlin", + "async", + "coroutines", + "JetBrains", + "test" + ] +} diff --git a/kotlinx-coroutines-test/wasmJs/src/TestBuilders.kt b/kotlinx-coroutines-test/wasmJs/src/TestBuilders.kt new file mode 100644 index 0000000000..8409b0f2d0 --- /dev/null +++ b/kotlinx-coroutines-test/wasmJs/src/TestBuilders.kt @@ -0,0 +1,17 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.test.internal.* +import kotlin.js.* + +public actual typealias TestResult = JsPromiseInterfaceForTesting + +@Suppress("INFERRED_TYPE_VARIABLE_INTO_POSSIBLE_EMPTY_INTERSECTION") +internal actual fun createTestResult(testProcedure: suspend CoroutineScope.() -> Unit): TestResult = + GlobalScope.promise { + testProcedure() + }.unsafeCast() + +internal actual fun dumpCoroutines() { } + +internal actual fun systemPropertyImpl(name: String): String? = null diff --git a/kotlinx-coroutines-test/wasmJs/src/internal/JsPromiseInterfaceForTesting.kt b/kotlinx-coroutines-test/wasmJs/src/internal/JsPromiseInterfaceForTesting.kt new file mode 100644 index 0000000000..e6697db307 --- /dev/null +++ b/kotlinx-coroutines-test/wasmJs/src/internal/JsPromiseInterfaceForTesting.kt @@ -0,0 +1,19 @@ +package kotlinx.coroutines.test.internal + +/* This is a declaration of JS's `Promise`. We need to keep it a separate class, because +`actual typealias TestResult = Promise` fails: you can't instantiate an `expect class` with a typealias to +a parametric class. So, we make a non-parametric class just for this. */ +/** + * @suppress + */ +@JsName("Promise") +public external class JsPromiseInterfaceForTesting { + /** + * @suppress + */ + public fun then(onFulfilled: ((JsAny) -> Unit), onRejected: ((JsAny) -> Unit)): JsPromiseInterfaceForTesting + /** + * @suppress + */ + public fun then(onFulfilled: ((JsAny) -> Unit)): JsPromiseInterfaceForTesting +} \ No newline at end of file diff --git a/kotlinx-coroutines-test/wasmJs/src/internal/TestMainDispatcher.kt b/kotlinx-coroutines-test/wasmJs/src/internal/TestMainDispatcher.kt new file mode 100644 index 0000000000..be0b1686a5 --- /dev/null +++ b/kotlinx-coroutines-test/wasmJs/src/internal/TestMainDispatcher.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines.test.internal +import kotlinx.coroutines.* + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +internal actual fun Dispatchers.getTestMainDispatcher(): TestMainDispatcher = + when (val mainDispatcher = Main) { + is TestMainDispatcher -> mainDispatcher + else -> TestMainDispatcher(mainDispatcher).also { injectMain(it) } + } diff --git a/kotlinx-coroutines-test/wasmJs/test/Helpers.kt b/kotlinx-coroutines-test/wasmJs/test/Helpers.kt new file mode 100644 index 0000000000..a394c1f19f --- /dev/null +++ b/kotlinx-coroutines-test/wasmJs/test/Helpers.kt @@ -0,0 +1,11 @@ +package kotlinx.coroutines.test + +actual fun testResultChain(block: () -> TestResult, after: (Result) -> TestResult): TestResult = + block().then( + { + after(Result.success(Unit)) + null + }, { + after(Result.failure(it.toThrowableOrNull() ?: Throwable("Unexpected non-Kotlin exception $it"))) + null + }) diff --git a/kotlinx-coroutines-test/wasmJs/test/PromiseTest.kt b/kotlinx-coroutines-test/wasmJs/test/PromiseTest.kt new file mode 100644 index 0000000000..f55517154a --- /dev/null +++ b/kotlinx-coroutines-test/wasmJs/test/PromiseTest.kt @@ -0,0 +1,18 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlin.test.* + +class PromiseTest { + @Test + fun testCompletionFromPromise() = runTest { + var promiseEntered = false + val p = promise { + delay(1) + promiseEntered = true + } + delay(2) + p.await() + assertTrue(promiseEntered) + } +} \ No newline at end of file diff --git a/kotlinx-coroutines-test/wasmWasi/src/TestBuilders.kt b/kotlinx-coroutines-test/wasmWasi/src/TestBuilders.kt new file mode 100644 index 0000000000..7c407d3bf4 --- /dev/null +++ b/kotlinx-coroutines-test/wasmWasi/src/TestBuilders.kt @@ -0,0 +1,15 @@ +package kotlinx.coroutines.test + +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* + +@Suppress("ACTUAL_WITHOUT_EXPECT") +public actual typealias TestResult = Unit + +internal actual fun systemPropertyImpl(name: String): String? = null + +internal actual fun createTestResult(testProcedure: suspend CoroutineScope.() -> Unit) = + runTestCoroutine(EmptyCoroutineContext, testProcedure) + +internal actual fun dumpCoroutines() { } \ No newline at end of file diff --git a/kotlinx-coroutines-test/wasmWasi/src/internal/TestMainDispatcher.kt b/kotlinx-coroutines-test/wasmWasi/src/internal/TestMainDispatcher.kt new file mode 100644 index 0000000000..1419f672e9 --- /dev/null +++ b/kotlinx-coroutines-test/wasmWasi/src/internal/TestMainDispatcher.kt @@ -0,0 +1,9 @@ +package kotlinx.coroutines.test.internal +import kotlinx.coroutines.* + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +internal actual fun Dispatchers.getTestMainDispatcher(): TestMainDispatcher = + when (val mainDispatcher = Main) { + is TestMainDispatcher -> mainDispatcher + else -> TestMainDispatcher(mainDispatcher).also { injectMain(it) } + } \ No newline at end of file diff --git a/kotlinx-coroutines-test/wasmWasi/test/Helpers.kt b/kotlinx-coroutines-test/wasmWasi/test/Helpers.kt new file mode 100644 index 0000000000..ed3afa39e4 --- /dev/null +++ b/kotlinx-coroutines-test/wasmWasi/test/Helpers.kt @@ -0,0 +1,10 @@ +package kotlinx.coroutines.test + +actual fun testResultChain(block: () -> TestResult, after: (Result) -> TestResult): TestResult { + try { + block() + after(Result.success(Unit)) + } catch (e: Throwable) { + after(Result.failure(e)) + } +} diff --git a/license/LICENSE.txt b/license/LICENSE.txt deleted file mode 100644 index 0ac9844f93..0000000000 --- a/license/LICENSE.txt +++ /dev/null @@ -1,15 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ diff --git a/license/NOTICE.txt b/license/NOTICE.txt index bbf4ebde8c..fbb2395ad6 100644 --- a/license/NOTICE.txt +++ b/license/NOTICE.txt @@ -1,8 +1,8 @@ - ========================================================================= - == NOTICE file corresponding to the section 4 d of == - == the Apache License, Version 2.0, == - == in this case for the kotlix.coroutines library. == - ========================================================================= +========================================================================= +== NOTICE file corresponding to the section 4 d of == +== the Apache License, Version 2.0, == +== in this case for the kotlinx.coroutines library. == +========================================================================= - kotlinx.coroutines library. - Copyright 2016-2017 JetBrains s.r.o and respective authors and developers +kotlinx.coroutines library. +Copyright 2016-2025 JetBrains s.r.o and contributors diff --git a/license/third_party/minima_LICENSE.txt b/license/third_party/minima_LICENSE.txt deleted file mode 100644 index e8c3c2d56b..0000000000 --- a/license/third_party/minima_LICENSE.txt +++ /dev/null @@ -1,21 +0,0 @@ -The MIT License (MIT) - -Copyright (c) 2016 Parker Moore - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in -all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. diff --git a/pom.xml b/pom.xml deleted file mode 100644 index 45a224f20c..0000000000 --- a/pom.xml +++ /dev/null @@ -1,241 +0,0 @@ - - - - - - 4.0.0 - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - pom - - Coroutines support libraries for Kotlin 1.1 - - https://github.com/Kotlin/kotlinx.coroutines/ - - - - The Apache Software License, Version 2.0 - http://www.apache.org/licenses/LICENSE-2.0.txt - repo - A business-friendly OSS license - - - - - UTF-8 - 1.1.2 - 0.9.14 - 4.12 - 1.6 - 1.6 - - - - 3.0.2 - - - - - JetBrains - JetBrains Team - JetBrains - http://www.jetbrains.com - - - - - https://github.com/Kotlin/kotlinx.coroutines - scm:git:https://github.com/Kotlin/kotlinx.coroutines.git - scm:git:https://github.com/Kotlin/kotlinx.coroutines.git - - - - - bintray - bintray - http://dl.bintray.com/kotlin/kotlin-eap-1.1 - - - - - - bintray - bintray - http://dl.bintray.com/kotlin/kotlin-eap-1.1 - - - bintray-kotlin-dokka - bintray-kotlin-dokka - http://dl.bintray.com/kotlin/dokka - - - - - - bintray - https://api.bintray.com/maven/kotlin/kotlinx/kotlinx.coroutines - - - - - kotlinx-coroutines-core - kotlinx-coroutines-jdk8 - kotlinx-coroutines-nio - reactive/kotlinx-coroutines-reactive - reactive/kotlinx-coroutines-reactor - reactive/kotlinx-coroutines-rx1 - reactive/kotlinx-coroutines-rx2 - reactive/kotlinx-coroutines-rx-example - ui/kotlinx-coroutines-swing - ui/kotlinx-coroutines-javafx - ui/kotlinx-coroutines-android - site - benchmarks - - - - - org.jetbrains.kotlin - kotlin-stdlib - ${kotlin.version} - compile - - - junit - junit - ${junit.version} - test - - - - - - - maven-source-plugin - - - - - package - attach-sources - - jar-no-fork - - - - - - - org.jetbrains.kotlin - kotlin-maven-plugin - - - compile - compile - - compile - - - - test-compile - test-compile - - test-compile - - - - - - -Xcoroutines=enable - - - - - org.jetbrains.dokka - dokka-maven-plugin - ${dokka.version} - - - pre-site - - dokka - - - - - - - ${project.basedir}/src/main/kotlin - http://github.com/kotlin/kotlinx.coroutines/tree/master/${project.artifactId}/src/main/kotlin - #L - - - - ${project.basedir}/README.md - - kotlin-website - 8 - - - - maven-site-plugin - - true - - - - - - - - org.jetbrains.kotlin - kotlin-maven-plugin - ${kotlin.version} - - - org.apache.maven.plugins - maven-source-plugin - 2.4 - - - org.apache.maven.plugins - maven-release-plugin - 2.5.2 - - - org.apache.maven.plugins - maven-deploy-plugin - 2.8.2 - - - org.apache.maven.plugins - maven-surefire-plugin - 2.19.1 - - - org.apache.maven.plugins - maven-site-plugin - 3.3 - - - - - \ No newline at end of file diff --git a/reactive/README.md b/reactive/README.md index 186b15c19d..d7f234c769 100644 --- a/reactive/README.md +++ b/reactive/README.md @@ -1,10 +1,11 @@ # Coroutines for reactive streams -This directory contains modules with utilities for various reactive stream libraries: +This directory contains modules with utilities for various reactive stream libraries. +Module name below corresponds to the artifact name in Maven/Gradle. ## Modules -* [kotlinx-coroutines-reactive](kotlinx-coroutines-reactive) -- utilities for [Reactive Streams](http://www.reactive-streams.org) -* [kotlinx-coroutines-reactor](kotlinx-coroutines-reactor) -- utilities for [Reactor](https://projectreactor.io) -* [kotlinx-coroutines-rx1](kotlinx-coroutines-rx1) -- utilities for [RxJava 1.x](https://github.com/ReactiveX/RxJava/tree/1.x) -* [kotlinx-coroutines-rx2](kotlinx-coroutines-rx2) -- utilities for [RxJava 2.x](https://github.com/ReactiveX/RxJava) +* [kotlinx-coroutines-reactive](kotlinx-coroutines-reactive/README.md) -- utilities for [Reactive Streams](https://www.reactive-streams.org) +* [kotlinx-coroutines-reactor](kotlinx-coroutines-reactor/README.md) -- utilities for [Reactor](https://projectreactor.io) +* [kotlinx-coroutines-rx2](kotlinx-coroutines-rx2/README.md) -- utilities for [RxJava 2.x](https://github.com/ReactiveX/RxJava/tree/2.x) +* [kotlinx-coroutines-rx3](kotlinx-coroutines-rx3/README.md) -- utilities for [RxJava 3.x](https://github.com/ReactiveX/RxJava) diff --git a/reactive/coroutines-guide-reactive.md b/reactive/coroutines-guide-reactive.md deleted file mode 100644 index 3e49212941..0000000000 --- a/reactive/coroutines-guide-reactive.md +++ /dev/null @@ -1,1081 +0,0 @@ - - - - -# Guide to reactive streams with coroutines - -This guide explains key differences between Kotlin coroutines and reactive streams and shows -how they can be used together for greater good. Prior familiarity with basic coroutine concepts -that are covered in [Guide to kotlinx.coroutines](../coroutines-guide.md) is not required, -but is a big plus. If you are familiar with reactive streams, you may find this guide -a better introduction into the world of coroutines. - -There are several modules in `kotlinx.coroutines` project that are related to reactive streams: - -* [kotlinx-coroutines-reactive](kotlinx-coroutines-reactive) -- utilities for [Reactive Streams](http://www.reactive-streams.org) -* [kotlinx-coroutines-reactor](kotlinx-coroutines-reactor) -- utilities for [Reactor](https://projectreactor.io) -* [kotlinx-coroutines-rx1](kotlinx-coroutines-rx1) -- utilities for [RxJava 1.x](https://github.com/ReactiveX/RxJava/tree/1.x) -* [kotlinx-coroutines-rx2](kotlinx-coroutines-rx2) -- utilities for [RxJava 2.x](https://github.com/ReactiveX/RxJava) - -This guide is mostly based on [Reactive Streams](http://www.reactive-streams.org) specification and uses -its `Publisher` interface with some examples based on [RxJava 2.x](https://github.com/ReactiveX/RxJava), -which implements reactive streams specification. - -You are welcome to clone -[`kotlinx.coroutines` project](https://github.com/Kotlin/kotlinx.coroutines) -from GitHub to your workstation in order to -run all the presented examples. They are contained in -[reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide](kotlinx-coroutines-rx2/src/test/kotlin/guide) -directory of the project. - -## Table of contents - - - -* [Differences between reactive streams and channels](#differences-between-reactive-streams-and-channels) - * [Basics of iteration](#basics-of-iteration) - * [Subscription and cancellation](#subscription-and-cancellation) - * [Backpressure](#backpressure) - * [Rx Subject vs BroadcastChannel](#rx-subject-vs-broadcastchannel) -* [Operators](#operators) - * [Range](#range) - * [Fused filter-map hybrid](#fused-filter-map-hybrid) - * [Take until](#take-until) - * [Merge](#merge) -* [Coroutine context](#coroutine-context) - * [Threads with Rx](#threads-with-rx) - * [Threads with coroutines](#threads-with-coroutines) - * [Rx observeOn](#rx-observeon) - * [Coroutine context to rule them all](#coroutine-context-to-rule-them-all) - * [Unconfined context](#unconfined-context) - - - -## Differences between reactive streams and channels - -This section outlines key differences between reactive streams and coroutine-based channels. - -### Basics of iteration - -The [Channel] is somewhat similar concept to the following reactive stream classes: - -* Reactive stream [Publisher](https://github.com/reactive-streams/reactive-streams-jvm/blob/master/api/src/main/java/org/reactivestreams/Publisher.java); -* Rx Java 1.x [Observable](http://reactivex.io/RxJava/javadoc/rx/Observable.html); -* Rx Java 2.x [Flowable](http://reactivex.io/RxJava/2.x/javadoc/), which implements `Publisher`. - -They all describe an asynchronous stream of elements (aka items in Rx), either infinite or finite, -and all of them support backpressure. - -However, the `Channel` always represents a _hot_ stream of items, using Rx terminology. Elements are being sent -into the channel by producer coroutines and are received from it by consumer coroutines. -Every [receive][ReceiveChannel.receive] invocation consumes an element from the channel. -Let us illustrate it with the following example: - - - -```kotlin -fun main(args: Array) = runBlocking { - // create a channel that produces numbers from 1 to 3 with 200ms delays between them - val source = produce(context) { - println("Begin") // mark the beginning of this coroutine in output - for (x in 1..3) { - delay(200) // wait for 200ms - send(x) // send number x to the channel - } - } - // print elements from the source - println("Elements:") - source.consumeEach { // consume elements from it - println(it) - } - // print elements from the source AGAIN - println("Again:") - source.consumeEach { // consume elements from it - println(it) - } -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-01.kt) - -This code produces the following output: - -```text -Elements: -Begin -1 -2 -3 -Again: -``` - - - -Notice, how "Begin" line was printed just once, because [produce] _coroutine builder_, when it is executed, -launches one coroutine to produce a stream of elements. All the produced elements are consumed -with [ReceiveChannel.consumeEach][consumeEach] -extension function. There is no way to receive the elements from this -channel again. The channel is closed when the producer coroutine is over and the attempt to receive -from it again cannot receive anything. - -Let us rewrite this code using [publish] coroutine builder from `kotlinx-coroutines-reactive` module -instead of [produce] from `kotlinx-coroutines-core` module. The code stays the same, -but where `source` used to have [ReceiveChannel] type, it now has reactive streams -[Publisher](http://www.reactive-streams.org/reactive-streams-1.0.0-javadoc/org/reactivestreams/Publisher.html) -type. - - - -```kotlin -fun main(args: Array) = runBlocking { - // create a publisher that produces numbers from 1 to 3 with 200ms delays between them - val source = publish(context) { - // ^^^^^^^ <--- Difference from the previous examples is here - println("Begin") // mark the beginning of this coroutine in output - for (x in 1..3) { - delay(200) // wait for 200ms - send(x) // send number x to the channel - } - } - // print elements from the source - println("Elements:") - source.consumeEach { // consume elements from it - println(it) - } - // print elements from the source AGAIN - println("Again:") - source.consumeEach { // consume elements from it - println(it) - } -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-02.kt) - -Now the output of this code changes to: - -```text -Elements: -Begin -1 -2 -3 -Again: -Begin -1 -2 -3 -``` - - - -This example highlights the key difference between a reactive stream and a channel. A reactive stream is a higher-order -functional concept. While the channel _is_ a stream of elements, the reactive stream defines a recipe on how the stream of -elements is produced. It becomes the actual stream of elements on _subscription_. Each subscriber may receive the same or -a different stream of elements, depending on how the corresponding implementation of `Publisher` works. - -The [publish] coroutine builder, that is used in the above example, launches a fresh coroutine on each subscription. -Every [Publisher.consumeEach][org.reactivestreams.Publisher.consumeEach] invocation creates a fresh subscription. -We have two of them in this code and that is why we see "Begin" printed twice. - -In Rx lingo this is called a _cold_ publisher. Many standard Rx operators produce cold streams, too. We can iterate -over them from a coroutine, and every subscription produces the same stream of elements. - -> Note, that we can replicate the same behaviour that we saw with channels by using Rx -[publish](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#publish()) -operator and [connect](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/flowables/ConnectableFlowable.html#connect()) -method with it. - -### Subscription and cancellation - -An example in the previous section uses `source.consumeEach { ... }` snippet to open a subscription -and receive all the elements from it. If we need more control on how what to do with -the elements that are being received from the channel, we can use [Publisher.open][org.reactivestreams.Publisher.open] -as shown in the following example: - - - -```kotlin -fun main(args: Array) = runBlocking { - val source = Flowable.range(1, 5) // a range of five numbers - .doOnSubscribe { println("OnSubscribe") } // provide some insight - .doFinally { println("Finally") } // ... into what's going on - var cnt = 0 - source.open().use { channel -> // open channel to the source - for (x in channel) { // iterate over the channel to receive elements from it - println(x) - if (++cnt >= 3) break // break when 3 elements are printed - } - // `use` will close the channel when this block of code is complete - } -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-03.kt) - -It produces the following output: - -```text -OnSubscribe -1 -2 -3 -Finally -``` - - - -With an explicit `open` we should [close][SubscriptionReceiveChannel.close] the corresponding -subscription to unsubscribe from the source. However, instead of invoking `close` explicitly, -this code relies on [use](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/use.html) -function from Kotlin's standard library. -The installed -[doFinally](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#doFinally(io.reactivex.functions.Action)) -listener prints "Finally" to confirm that the subscription is actually being closed. - -We do not need to use an explicit `close` if iteration is performed over all the items that are emitted -by the publisher, because it is being closed automatically by `consumeEach`: - - - -```kotlin -fun main(args: Array) = runBlocking { - val source = Flowable.range(1, 5) // a range of five numbers - .doOnSubscribe { println("OnSubscribe") } // provide some insight - .doFinally { println("Finally") } // ... into what's going on - // iterate over the source fully - source.consumeEach { println(it) } -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-04.kt) - -We get the following output: - -```text -OnSubscribe -1 -2 -3 -4 -Finally -5 -``` - - - -Notice, how "Finally" is printed before the last element "5". It happens because our `main` function in this -example is a coroutine that we start with [runBlocking] coroutine builder. -Our main coroutine receives on the channel using `source.consumeEach { ... }` expression. -The main coroutine is _suspended_ while it waits for the source to emit an item. -When the last item is emitted by `Flowable.range(1, 5)` it -_resumes_ the main coroutine, which gets dispatched onto the main thread to print this - last element at a later point in time, while the source completes and prints "Finally". - -### Backpressure - -Backpressure is one of the most interesting and complex aspects of reactive streams. Coroutines can -_suspend_ and they provide a natural answer to handling backpressure. - -In Rx Java 2.x a backpressure-capable class is called -[Flowable](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html). -In the following example we use [rxFlowable] coroutine builder from `kotlinx-coroutines-rx2` module to define a -flowable that sends three integers from 1 to 3. -It prints a message to the output before invocation of -suspending [send][SendChannel.send] function, so that we can study how it operates. - -The integers are generated in the context of the main thread, but subscription is shifted -to another thread using Rx -[observeOn](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#observeOn(io.reactivex.Scheduler,%20boolean,%20int)) -operator with a buffer of size 1. -The subscriber is slow. It takes 500 ms to process each item, which is simulated using `Thread.sleep`. - - - -```kotlin -fun main(args: Array) = runBlocking { - // coroutine -- fast producer of elements in the context of the main thread - val source = rxFlowable(context) { - for (x in 1..3) { - send(x) // this is a suspending function - println("Sent $x") // print after successfully sent item - } - } - // subscribe on another thread with a slow subscriber using Rx - source - .observeOn(Schedulers.io(), false, 1) // specify buffer size of 1 item - .doOnComplete { println("Complete") } - .subscribe { x -> - Thread.sleep(500) // 500ms to process each item - println("Processed $x") - } - delay(2000) // suspend the main thread for a few seconds -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-05.kt) - -The output of this code nicely illustrates how backpressure works with coroutines: - -```text -Sent 1 -Processed 1 -Sent 2 -Processed 2 -Sent 3 -Processed 3 -Complete -``` - - - -We see here how producer coroutine puts the first element in the buffer and is suspended while trying to send another -one. Only after consumer processes the first item, producer sends the second one and resumes, etc. - - -### Rx Subject vs BroadcastChannel - -RxJava has a concept of [Subject](https://github.com/ReactiveX/RxJava/wiki/Subject) which is an object that -effectively broadcasts elements to all its subscribers. The matching concept in coroutines world is called a -[BroadcastChannel]. There is a variety of subjects in Rx with -[BehaviorSubject](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/subjects/BehaviorSubject.html) being the -the one used to manage state: - - - -```kotlin -fun main(args: Array) { - val subject = BehaviorSubject.create() - subject.onNext("one") - subject.onNext("two") // updates the state of BehaviorSubject, "one" value is lost - // now subscribe to this subject and print everything - subject.subscribe(System.out::println) - subject.onNext("three") - subject.onNext("four") -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-06.kt) - -This code prints the current state of the subject on subscription and all its further updates: - - -```text -two -three -four -``` - - - -You can subscribe to subjects from a coroutine just as with any other reactive stream: - - - -```kotlin -fun main(args: Array) = runBlocking { - val subject = BehaviorSubject.create() - subject.onNext("one") - subject.onNext("two") - // now launch a coroutine to print everything - launch(context) { // use the context of the main thread for a coroutine - subject.consumeEach { println(it) } - } - subject.onNext("three") - subject.onNext("four") -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-07.kt) - -The result is different, though: - -```text -four -``` - - - -It prints only the value value, because the coroutine is working in the main thread, which is busy updating the -subject value. Only when the main thread completes, the subscribing coroutine has a change to print anything. By that -time, the subject had already updated its value to "four". - -The coroutines in the main thread are scheduled cooperatively. There is a [yield] function to explicitly relinquish -the control of the thread to other coroutines. We can add it to the last example: - - - -```kotlin -fun main(args: Array) = runBlocking { - val subject = BehaviorSubject.create() - subject.onNext("one") - subject.onNext("two") - // now launch a coroutine to print everything - launch(context) { // use the context of the main thread for a coroutine - subject.consumeEach { println(it) } - } - subject.onNext("three") - yield() // yield the main thread to the launched coroutine <--- HERE - subject.onNext("four") -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-08.kt) - -Now coroutine has a chance to process (print) the "three" state of the subject, too: - -```text -three -four -``` - - - -This is quite the desired behavior for any kind of state-holding variable that needs to processed to update UI or -other linked state, for example. There is no reason to react to back-to-back updates of the state. -Only the most recent state is relevant. - -The corresponding behavior in coroutines world is implemented by [ConflatedBroadcastChannel] that provides the same logic -on top of coroutine channels directly, without going through the bridge to the reactive streams: - - - -```kotlin -fun main(args: Array) = runBlocking { - val broadcast = ConflatedBroadcastChannel() - broadcast.offer("one") - broadcast.offer("two") - // now launch a coroutine to print everything - launch(context) { // use the context of the main thread for a coroutine - broadcast.consumeEach { println(it) } - } - broadcast.offer("three") - yield() // yield the main thread to the launched coroutine - broadcast.offer("four") -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-09.kt) - -It produces the same output as the version based on `BehaviorSubject`: - -```text -three -four -``` - -Another implementation of [BroadcastChannel] is [ArrayBroadcastChannel]. It delivers every event to every -subscriber since the moment the corresponding subscription is open. It corresponds to -[PublishSubject][http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/subjects/PublishSubject.html] in Rx. -The capacity of the buffer in the constructor of `ArrayBroadcastChannel` controls the numbers of elements -that can be sent before the sender is suspended waiting for receiver to receive those elements. - - - -## Operators - -Full-featured reactive stream libraries, like Rx, come with -[a very large set of operators](http://reactivex.io/documentation/operators.html) to create, transform, combine -and otherwise process the corresponding streams. Creating your own operators with support for -back-pressure is [notoriously](http://akarnokd.blogspot.ru/2015/05/pitfalls-of-operator-implementations.html) -[difficult](https://github.com/ReactiveX/RxJava/wiki/Writing-operators-for-2.0). - -Coroutines and channels are designed to provide an opposite experience. There are no built-in operators, -but processing streams of elements is extremely simple and back-pressure is supported automatically -without you having to explicitly think about it. - -This section shows coroutine-based implementation of several reactive stream operators. - -### Range - -Let's roll out own implementation of -[range](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#range(int,%20int)) -operator for reactive streams `Publisher` interface. The asynchronous clean-slate implementation of this operator for -reactive streams is explained in -[this blog post](http://akarnokd.blogspot.ru/2017/03/java-9-flow-api-asynchronous-integer.html). -It takes a lot of code. -Here is the corresponding code with coroutines: - - - -```kotlin -fun range(context: CoroutineContext, start: Int, count: Int) = publish(context) { - for (x in start until start + count) send(x) -} -``` - -In this code `CoroutineContext` is used instead of an `Executor` and all the backpressure aspects are taken care -of by the coroutines machinery. Note, that this implementation depends only on the small reactive streams library -that defines `Publisher` interface and its friends. - -It is straightforward to use from a coroutine: - -```kotlin -fun main(args: Array) = runBlocking { - range(CommonPool, 1, 5).consumeEach { println(it) } -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-01.kt) - -The result of this code is quite expected: - -```text -1 -2 -3 -4 -5 -``` - - - -### Fused filter-map hybrid - -Reactive operators like -[filter](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#filter(io.reactivex.functions.Predicate)) and -[map](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#map(io.reactivex.functions.Function)) -are trivial to implement with coroutines. For a bit of challenge and showcase, let us combine them -into the single `fusedFilterMap` operator: - - - -```kotlin -fun Publisher.fusedFilterMap( - context: CoroutineContext, // the context to execute this coroutine in - predicate: (T) -> Boolean, // the filter predicate - mapper: (T) -> R // the mapper function -) = publish(context) { - consumeEach { // consume the source stream - if (predicate(it)) // filter part - send(mapper(it)) // map part - } -} -``` - -Using `range` from the previous example we can test our `fusedFilterMap` -by filtering for even numbers and mapping them to strings: - - - -```kotlin -fun main(args: Array) = runBlocking { - range(context, 1, 5) - .fusedFilterMap(context, { it % 2 == 0}, { "$it is even" }) - .consumeEach { println(it) } // print all the resulting strings -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-02.kt) - -It is not hard to see, that the result is going to be: - -```text -2 is even -4 is even -``` - - - -### Take until - -Let's implement our own version of -[takeUntil](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#takeUntil(org.reactivestreams.Publisher)) -operator. It is quite a [tricky one](http://akarnokd.blogspot.ru/2015/05/pitfalls-of-operator-implementations.html) -to implement, because of the need to track and manage subscription to two streams. -We need to relay all the elements from the source stream until the other stream either completes or -emits anything. However, we have [select] expression to rescue us in coroutines implementation: - - - -```kotlin -fun Publisher.takeUntil(context: CoroutineContext, other: Publisher) = publish(context) { - this@takeUntil.open().use { thisChannel -> // explicitly open channel to Publisher - other.open().use { otherChannel -> // explicitly open channel to Publisher - whileSelect { - otherChannel.onReceive { false } // bail out on any received element from `other` - thisChannel.onReceive { send(it); true } // resend element from this channel and continue - } - } - } -} -``` - -This code is using [whileSelect] as a nicer shortcut to `while(select{...}) {}` loop and Kotlin's -[use](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/use.html) -expression to close the channels on exit, which unsubscribes from the corresponding publishers. - -The following hand-written combination of -[range](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#range(int,%20int)) with -[interval](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#interval(long,%20java.util.concurrent.TimeUnit,%20io.reactivex.Scheduler)) -is used for testing. It is coded using a `publish` coroutine builder -(its pure-Rx implementation is shown in later sections): - -```kotlin -fun rangeWithInterval(context: CoroutineContext, time: Long, start: Int, count: Int) = publish(context) { - for (x in start until start + count) { - delay(time) // wait before sending each number - send(x) - } -} -``` - -The following code shows how `takeUntil` works: - -```kotlin -fun main(args: Array) = runBlocking { - val slowNums = rangeWithInterval(context, 200, 1, 10) // numbers with 200ms interval - val stop = rangeWithInterval(context, 500, 1, 10) // the first one after 500ms - slowNums.takeUntil(context, stop).consumeEach { println(it) } // let's test it -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-03.kt) - -Producing - -```text -1 -2 -``` - - - -### Merge - -There are always at least two ways for processing multiple streams of data with coroutines. One way involving -[select] was shown in the previous example. The other way is just to launch multiple coroutines. Let -us implement -[merge](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#merge(org.reactivestreams.Publisher)) -operator using the later approach: - - - -```kotlin -fun Publisher>.merge(context: CoroutineContext) = publish(context) { - consumeEach { pub -> // for each publisher received on the source channel - launch(this.context) { // launch a child coroutine - pub.consumeEach { send(it) } // resend all element from this publisher - } - } -} -``` - -Notice, the use of `this.context` in the invocation of [launch] coroutine builder. It is used to refer -to the [CoroutineScope.context] that is provided by [publish] builder. This way, all the coroutines that are -being launched here are [children](../coroutines-guide.md#children-of-a-coroutine) of the `publish` -coroutine and will get cancelled when the `publish` coroutine is cancelled or is otherwise completed. -This implementation completes as soon as the original publisher completes. - -For a test, let us start with `rangeWithInterval` function from the previous example and write a -producer that sends its results twice with some delay: - - - -```kotlin -fun testPub(context: CoroutineContext) = publish>(context) { - send(rangeWithInterval(context, 250, 1, 4)) // number 1 at 250ms, 2 at 500ms, 3 at 750ms, 4 at 1000ms - delay(100) // wait for 100 ms - send(rangeWithInterval(context, 500, 11, 3)) // number 11 at 600ms, 12 at 1100ms, 13 at 1600ms - delay(1100) // wait for 1.1s - done in 1.2 sec after start -} -``` - -The test code is to use `merge` on `testPub` and to display the results: - -```kotlin -fun main(args: Array) = runBlocking { - testPub(context).merge(context).consumeEach { println(it) } // print the whole stream -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-04.kt) - -And the results should be: - -```text -1 -2 -11 -3 -4 -12 -``` - - - -## Coroutine context - -All the example operators that are shown in the previous section have an explicit -[CoroutineContext](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.coroutines.experimental/-coroutine-context/) -parameter. In Rx world it roughly corresponds to -a [Scheduler](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Scheduler.html). - -### Threads with Rx - -The following example shows the basics of threading context management with Rx. -Here `rangeWithIntervalRx` is an implementation of `rangeWithInterval` function using Rx -`zip`, `range`, and `interval` operators. - - - -```kotlin -fun rangeWithIntervalRx(scheduler: Scheduler, time: Long, start: Int, count: Int): Flowable = - Flowable.zip( - Flowable.range(start, count), - Flowable.interval(time, TimeUnit.MILLISECONDS, scheduler), - BiFunction { x, _ -> x }) - -fun main(args: Array) { - rangeWithIntervalRx(Schedulers.computation(), 100, 1, 3) - .subscribe { println("$it on thread ${Thread.currentThread().name}") } - Thread.sleep(1000) -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-01.kt) - -We are explicitly passing the -[Schedulers.computation()](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/schedulers/Schedulers.html#computation()) -scheduler to our `rangeWithIntervalRx` operator and -it is going to be executed in Rx computation thread pool. The output is going to be similar to the following one: - -```text -1 on thread RxComputationThreadPool-1 -2 on thread RxComputationThreadPool-1 -3 on thread RxComputationThreadPool-1 -``` - - - -### Threads with coroutines - -In the world of coroutines `Schedulers.computation()` roughly corresponds to [CommonPool], -so the previous example is similar to the following one: - - - -```kotlin -fun rangeWithInterval(context: CoroutineContext, time: Long, start: Int, count: Int) = publish(context) { - for (x in start until start + count) { - delay(time) // wait before sending each number - send(x) - } -} - -fun main(args: Array) { - Flowable.fromPublisher(rangeWithInterval(CommonPool, 100, 1, 3)) - .subscribe { println("$it on thread ${Thread.currentThread().name}") } - Thread.sleep(1000) -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-02.kt) - -The produced output is going to be similar to: - -```text -1 on thread ForkJoinPool.commonPool-worker-1 -2 on thread ForkJoinPool.commonPool-worker-1 -3 on thread ForkJoinPool.commonPool-worker-1 -``` - - - -Here we've used Rx -[subscribe](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#subscribe(io.reactivex.functions.Consumer)) -operator that does not have its own scheduler and operates on the same thread that the publisher -- on a `CommonPool` -in this example. - -### Rx observeOn - -In Rx you use special operators to modify the threading context for operations in the chain. You -can find some [good guides](http://tomstechnicalblog.blogspot.ru/2016/02/rxjava-understanding-observeon-and.html) -about them, if you are not familiar. - -For example, there is -[observeOn](http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#observeOn(io.reactivex.Scheduler)) -operator. Let us modify the previous example to observe using `Schedulers.computation()`: - - - -```kotlin -fun rangeWithInterval(context: CoroutineContext, time: Long, start: Int, count: Int) = publish(context) { - for (x in start until start + count) { - delay(time) // wait before sending each number - send(x) - } -} - -fun main(args: Array) { - Flowable.fromPublisher(rangeWithInterval(CommonPool, 100, 1, 3)) - .observeOn(Schedulers.computation()) // <-- THIS LINE IS ADDED - .subscribe { println("$it on thread ${Thread.currentThread().name}") } - Thread.sleep(1000) -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-03.kt) - -Here is the difference in output, notice "RxComputationThreadPool": - -```text -1 on thread RxComputationThreadPool-1 -2 on thread RxComputationThreadPool-1 -3 on thread RxComputationThreadPool-1 -``` - - - -### Coroutine context to rule them all - -A coroutine is always working in some context. For example, let us start a coroutine -in the main thread with [runBlocking] and iterate over the result of the Rx version of `rangeWithIntervalRx` operator, -instead of using Rx `subscribe` operator: - - - -```kotlin -fun rangeWithIntervalRx(scheduler: Scheduler, time: Long, start: Int, count: Int): Flowable = - Flowable.zip( - Flowable.range(start, count), - Flowable.interval(time, TimeUnit.MILLISECONDS, scheduler), - BiFunction { x, _ -> x }) - -fun main(args: Array) = runBlocking { - rangeWithIntervalRx(Schedulers.computation(), 100, 1, 3) - .consumeEach { println("$it on thread ${Thread.currentThread().name}") } -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-04.kt) - -The resulting messages are going to be printed in the main thread: - -```text -1 on thread main -2 on thread main -3 on thread main -``` - - - -### Unconfined context - -Most Rx operators do not have any specific thread (scheduler) associated with them and are working -in whatever thread that they happen to be invoked in. We've seen it on the example of `subscribe` operator -in the [threads with Rx](#threads-with-rx) section. - -In the world of coroutines, [Unconfined] context serves a similar role. Let us modify our previous example, -but instead of iterating over the source `Flowable` from the `runBlocking` coroutine that is confined -to the main thread, we launch a new coroutine in `Unconfined` context, while the main coroutine -simply waits its completion using [Job.join]: - - - -```kotlin -fun rangeWithIntervalRx(scheduler: Scheduler, time: Long, start: Int, count: Int): Flowable = - Flowable.zip( - Flowable.range(start, count), - Flowable.interval(time, TimeUnit.MILLISECONDS, scheduler), - BiFunction { x, _ -> x }) - -fun main(args: Array) = runBlocking { - val job = launch(Unconfined) { // launch new coroutine in Unconfined context (without its own thread pool) - rangeWithIntervalRx(Schedulers.computation(), 100, 1, 3) - .consumeEach { println("$it on thread ${Thread.currentThread().name}") } - } - job.join() // wait for our coroutine to complete -} -``` - -> You can get full code [here](kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-05.kt) - -Now, the output shows that the code of the coroutine is executing in the Rx computation thread pool, just -like our initial example using Rx `subscribe` operator. - -```text -1 on thread RxComputationThreadPool-1 -2 on thread RxComputationThreadPool-1 -3 on thread RxComputationThreadPool-1 -``` - - - -Note, that [Unconfined] context shall be used with care. It may improve the overall performance on certain tests, -due to the increased stack-locality of operations and less scheduling overhead, but it also produces deeper stacks -and makes it harder to reason about asynchronicity of the code that is using it. - -If a coroutine sends an element to a channel, then the thread that invoked the -[send][SendChannel.send] may start executing the code of a coroutine with [Unconfined] dispatcher. -The original producer coroutine that invoked `send` is paused until the unconfined consumer coroutine hits its next -suspension point. This is very similar to a lock-step single-threaded `onNext` execution in Rx world in the absense -of thread-shifting operators. It is a normal default for Rx, because operators are usually doing very small chunks -of work and you have to combine many operators for a complex processing. However, this is unusual with coroutines, -where you can have an arbitrary complex processing in a coroutine. Usually, you only need to chain stream-processing -coroutines for complex pipelines with fan-in and fan-out between multiple worker coroutines. - - - - -[runBlocking]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/run-blocking.html -[yield]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/yield.html -[launch]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/launch.html -[CoroutineScope.context]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-scope/context.html -[CommonPool]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-common-pool/index.html -[Unconfined]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-unconfined/index.html -[Job.join]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/join.html - -[Channel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel/index.html -[ReceiveChannel.receive]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/receive.html -[produce]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/produce.html -[consumeEach]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/consume-each.html -[ReceiveChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/index.html -[SubscriptionReceiveChannel.close]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-subscription-receive-channel/close.html -[SendChannel.send]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-send-channel/send.html -[BroadcastChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-broadcast-channel/index.html -[ConflatedBroadcastChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-conflated-broadcast-channel/index.html -[ArrayBroadcastChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-array-broadcast-channel/index.html - -[select]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/select.html -[whileSelect]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.selects/while-select.html - - - -[publish]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/publish.html -[org.reactivestreams.Publisher.consumeEach]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/org.reactivestreams.-publisher/consume-each.html -[org.reactivestreams.Publisher.open]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/org.reactivestreams.-publisher/open.html - - - -[rxFlowable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/rx-flowable.html - - - diff --git a/reactive/knit.properties b/reactive/knit.properties new file mode 100644 index 0000000000..1a28ea50f7 --- /dev/null +++ b/reactive/knit.properties @@ -0,0 +1,2 @@ +knit.package=kotlinx.coroutines.rx2.guide +knit.dir=kotlinx-coroutines-rx2/test/guide/ diff --git a/reactive/knit.test.include b/reactive/knit.test.include new file mode 100644 index 0000000000..8b62dcd4b5 --- /dev/null +++ b/reactive/knit.test.include @@ -0,0 +1,8 @@ +// This file was automatically generated from ${file.name} by Knit tool. Do not edit. +package ${test.package} + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.guide.test.* +import org.junit.Test + +class ${test.name} : ReactiveTestBase() { \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-jdk9/README.md b/reactive/kotlinx-coroutines-jdk9/README.md new file mode 100644 index 0000000000..fcabd7da15 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/README.md @@ -0,0 +1,10 @@ +# Module kotlinx-coroutines-jdk9 + +Utilities for [Java Flow](https://docs.oracle.com/javase/9/docs/api/java/util/concurrent/Flow.html). + +Implemented as a collection of thin wrappers over [kotlinx-coroutines-reactive](../kotlinx-coroutines-reactive), +an equivalent package for the Reactive Streams. + +# Package kotlinx.coroutines.jdk9 + +Utilities for [Java Flow](https://docs.oracle.com/javase/9/docs/api/java/util/concurrent/Flow.html). diff --git a/reactive/kotlinx-coroutines-jdk9/api/kotlinx-coroutines-jdk9.api b/reactive/kotlinx-coroutines-jdk9/api/kotlinx-coroutines-jdk9.api new file mode 100644 index 0000000000..1f5bdec7d0 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/api/kotlinx-coroutines-jdk9.api @@ -0,0 +1,22 @@ +public final class kotlinx/coroutines/jdk9/AwaitKt { + public static final fun awaitFirst (Ljava/util/concurrent/Flow$Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrDefault (Ljava/util/concurrent/Flow$Publisher;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrElse (Ljava/util/concurrent/Flow$Publisher;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrNull (Ljava/util/concurrent/Flow$Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitLast (Ljava/util/concurrent/Flow$Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingle (Ljava/util/concurrent/Flow$Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/jdk9/PublishKt { + public static final fun flowPublish (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Ljava/util/concurrent/Flow$Publisher; + public static synthetic fun flowPublish$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Ljava/util/concurrent/Flow$Publisher; +} + +public final class kotlinx/coroutines/jdk9/ReactiveFlowKt { + public static final fun asFlow (Ljava/util/concurrent/Flow$Publisher;)Lkotlinx/coroutines/flow/Flow; + public static final fun asPublisher (Lkotlinx/coroutines/flow/Flow;)Ljava/util/concurrent/Flow$Publisher; + public static final fun asPublisher (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Ljava/util/concurrent/Flow$Publisher; + public static synthetic fun asPublisher$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Ljava/util/concurrent/Flow$Publisher; + public static final fun collect (Ljava/util/concurrent/Flow$Publisher;Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + diff --git a/reactive/kotlinx-coroutines-jdk9/build.gradle.kts b/reactive/kotlinx-coroutines-jdk9/build.gradle.kts new file mode 100644 index 0000000000..b8e05d61fe --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/build.gradle.kts @@ -0,0 +1,24 @@ +import org.jetbrains.kotlin.gradle.dsl.* + +dependencies { + implementation(project(":kotlinx-coroutines-reactive")) +} + +java { + sourceCompatibility = JavaVersion.VERSION_1_9 + targetCompatibility = JavaVersion.VERSION_1_9 +} + +tasks { + compileKotlin { + compilerOptions.jvmTarget = JvmTarget.JVM_9 + } + + compileTestKotlin { + compilerOptions.jvmTarget = JvmTarget.JVM_9 + } +} + +externalDocumentationLink( + url = "https://docs.oracle.com/javase/9/docs/api/java/util/concurrent/Flow.html" +) diff --git a/reactive/kotlinx-coroutines-jdk9/package.list b/reactive/kotlinx-coroutines-jdk9/package.list new file mode 100644 index 0000000000..43e8ff22c7 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/package.list @@ -0,0 +1 @@ +java.util.concurrent.Flow diff --git a/reactive/kotlinx-coroutines-jdk9/src/Await.kt b/reactive/kotlinx-coroutines-jdk9/src/Await.kt new file mode 100644 index 0000000000..2fba53dd1f --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/src/Await.kt @@ -0,0 +1,81 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.* +import java.util.concurrent.* +import org.reactivestreams.FlowAdapters +import kotlinx.coroutines.reactive.* + +/** + * Awaits the first value from the given publisher without blocking the thread and returns the resulting value, or, if + * the publisher has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Flow.Subscription] and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the publisher does not emit any value + */ +public suspend fun Flow.Publisher.awaitFirst(): T = + FlowAdapters.toPublisher(this).awaitFirst() + +/** + * Awaits the first value from the given publisher, or returns the [default] value if none is emitted, without blocking + * the thread, and returns the resulting value, or, if this publisher has produced an error, throws the corresponding + * exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Flow.Subscription] and resumes with [CancellationException]. + */ +public suspend fun Flow.Publisher.awaitFirstOrDefault(default: T): T = + FlowAdapters.toPublisher(this).awaitFirstOrDefault(default) + +/** + * Awaits the first value from the given publisher, or returns `null` if none is emitted, without blocking the thread, + * and returns the resulting value, or, if this publisher has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Flow.Subscription] and resumes with [CancellationException]. + */ +public suspend fun Flow.Publisher.awaitFirstOrNull(): T? = + FlowAdapters.toPublisher(this).awaitFirstOrNull() + +/** + * Awaits the first value from the given publisher, or calls [defaultValue] to get a value if none is emitted, without + * blocking the thread, and returns the resulting value, or, if this publisher has produced an error, throws the + * corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Flow.Subscription] and resumes with [CancellationException]. + */ +public suspend fun Flow.Publisher.awaitFirstOrElse(defaultValue: () -> T): T = + FlowAdapters.toPublisher(this).awaitFirstOrElse(defaultValue) + +/** + * Awaits the last value from the given publisher without blocking the thread and + * returns the resulting value, or, if this publisher has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Flow.Subscription] and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the publisher does not emit any value + */ +public suspend fun Flow.Publisher.awaitLast(): T = + FlowAdapters.toPublisher(this).awaitLast() + +/** + * Awaits the single value from the given publisher without blocking the thread and returns the resulting value, or, + * if this publisher has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Flow.Subscription] and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the publisher does not emit any value + * @throws IllegalArgumentException if the publisher emits more than one value + */ +public suspend fun Flow.Publisher.awaitSingle(): T = + FlowAdapters.toPublisher(this).awaitSingle() diff --git a/reactive/kotlinx-coroutines-jdk9/src/Publish.kt b/reactive/kotlinx-coroutines-jdk9/src/Publish.kt new file mode 100644 index 0000000000..019e604888 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/src/Publish.kt @@ -0,0 +1,34 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.reactive.* +import java.util.concurrent.* +import kotlin.coroutines.* +import org.reactivestreams.FlowAdapters + +/** + * Creates a cold reactive [Flow.Publisher] that runs a given [block] in a coroutine. + * + * Every time the returned flux is subscribed, it starts a new coroutine in the specified [context]. + * The coroutine emits (via [Flow.Subscriber.onNext]) values with [send][ProducerScope.send], + * completes (via [Flow.Subscriber.onComplete]) when the coroutine completes or channel is explicitly closed, and emits + * errors (via [Flow.Subscriber.onError]) if the coroutine throws an exception or closes channel with a cause. + * Unsubscribing cancels the running coroutine. + * + * Invocations of [send][ProducerScope.send] are suspended appropriately when subscribers apply back-pressure and to + * ensure that [onNext][Flow.Subscriber.onNext] is not invoked concurrently. + * + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is + * used. + * + * **Note: This is an experimental api.** Behaviour of publishers that work as children in a parent scope with respect + * to cancellation and error handling may change in the future. + * + * @throws IllegalArgumentException if the provided [context] contains a [Job] instance. + */ +public fun flowPublish( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Flow.Publisher = FlowAdapters.toFlowPublisher(publish(context, block)) diff --git a/reactive/kotlinx-coroutines-jdk9/src/ReactiveFlow.kt b/reactive/kotlinx-coroutines-jdk9/src/ReactiveFlow.kt new file mode 100644 index 0000000000..11f3a95fe1 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/src/ReactiveFlow.kt @@ -0,0 +1,45 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.asFlow +import kotlinx.coroutines.reactive.asPublisher as asReactivePublisher +import kotlinx.coroutines.reactive.collect +import kotlinx.coroutines.channels.* +import org.reactivestreams.* +import kotlin.coroutines.* +import java.util.concurrent.Flow as JFlow + +/** + * Transforms the given reactive [Flow Publisher][JFlow.Publisher] into [Flow]. + * Use the [buffer] operator on the resulting flow to specify the size of the back-pressure. + * In effect, it specifies the value of the subscription's [request][JFlow.Subscription.request]. + * The [default buffer capacity][Channel.BUFFERED] for a suspending channel is used by default. + * + * If any of the resulting flow transformations fails, the subscription is immediately cancelled and all the in-flight + * elements are discarded. + */ +public fun JFlow.Publisher.asFlow(): Flow = + FlowAdapters.toPublisher(this).asFlow() + +/** + * Transforms the given flow into a reactive specification compliant [Flow Publisher][JFlow.Publisher]. + * + * An optional [context] can be specified to control the execution context of calls to the [Flow Subscriber][Subscriber] + * methods. + * A [CoroutineDispatcher] can be set to confine them to a specific thread; various [ThreadContextElement] can be set to + * inject additional context into the caller thread. By default, the [Unconfined][Dispatchers.Unconfined] dispatcher + * is used, so calls are performed from an arbitrary thread. + */ +@JvmOverloads // binary compatibility +public fun Flow.asPublisher(context: CoroutineContext = EmptyCoroutineContext): JFlow.Publisher = + FlowAdapters.toFlowPublisher(asReactivePublisher(context)) + +/** + * Subscribes to this [Flow Publisher][JFlow.Publisher] and performs the specified action for each received element. + * + * If [action] throws an exception at some point, the subscription is cancelled, and the exception is rethrown from + * [collect]. Also, if the publisher signals an error, that error is rethrown from [collect]. + */ +public suspend inline fun JFlow.Publisher.collect(action: (T) -> Unit): Unit = + FlowAdapters.toPublisher(this).collect(action) diff --git a/reactive/kotlinx-coroutines-jdk9/src/module-info.java b/reactive/kotlinx-coroutines-jdk9/src/module-info.java new file mode 100644 index 0000000000..ce31d81015 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/src/module-info.java @@ -0,0 +1,9 @@ +@SuppressWarnings("JavaModuleNaming") +module kotlinx.coroutines.jdk9 { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.coroutines.reactive; + requires org.reactivestreams; + + exports kotlinx.coroutines.jdk9; +} diff --git a/reactive/kotlinx-coroutines-jdk9/test/AwaitTest.kt b/reactive/kotlinx-coroutines-jdk9/test/AwaitTest.kt new file mode 100644 index 0000000000..4b0b77c801 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/test/AwaitTest.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.concurrent.Flow as JFlow + +class AwaitTest: TestBase() { + + /** Tests that calls to [awaitFirst] (and, thus, to the rest of these functions) throw [CancellationException] and + * unsubscribe from the publisher when their [Job] is cancelled. */ + @Test + fun testAwaitCancellation() = runTest { + expect(1) + val publisher = JFlow.Publisher { s -> + s.onSubscribe(object : JFlow.Subscription { + override fun request(n: Long) { + expect(3) + } + + override fun cancel() { + expect(5) + } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + publisher.awaitFirst() + } catch (e: CancellationException) { + expect(6) + throw e + } + } + expect(4) + job.cancelAndJoin() + finish(7) + } + +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-jdk9/test/FlowAsPublisherTest.kt b/reactive/kotlinx-coroutines-jdk9/test/FlowAsPublisherTest.kt new file mode 100644 index 0000000000..b09324cff2 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/test/FlowAsPublisherTest.kt @@ -0,0 +1,90 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.junit.Test +import java.util.concurrent.Flow as JFlow +import kotlin.test.* + +class FlowAsPublisherTest : TestBase() { + + @Test + fun testErrorOnCancellationIsReported() { + expect(1) + flow { + try { + emit(2) + } finally { + expect(3) + throw TestException() + } + }.asPublisher().subscribe(object : JFlow.Subscriber { + private lateinit var subscription: JFlow.Subscription + + override fun onComplete() { + expectUnreached() + } + + override fun onSubscribe(s: JFlow.Subscription?) { + subscription = s!! + subscription.request(2) + } + + override fun onNext(t: Int) { + expect(t) + subscription.cancel() + } + + override fun onError(t: Throwable?) { + assertIs(t) + expect(4) + } + }) + finish(5) + } + + @Test + fun testCancellationIsNotReported() { + expect(1) + flow { + emit(2) + }.asPublisher().subscribe(object : JFlow.Subscriber { + private lateinit var subscription: JFlow.Subscription + + override fun onComplete() { + expectUnreached() + } + + override fun onSubscribe(s: JFlow.Subscription?) { + subscription = s!! + subscription.request(2) + } + + override fun onNext(t: Int) { + expect(t) + subscription.cancel() + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + finish(3) + } + + @Test + fun testFlowWithTimeout() = runTest { + val publisher = flow { + expect(2) + withTimeout(1) { delay(Long.MAX_VALUE) } + }.asPublisher() + try { + expect(1) + publisher.awaitFirstOrNull() + } catch (e: CancellationException) { + expect(3) + } + finish(4) + } +} diff --git a/reactive/kotlinx-coroutines-jdk9/test/IntegrationTest.kt b/reactive/kotlinx-coroutines-jdk9/test/IntegrationTest.kt new file mode 100644 index 0000000000..400d11bbec --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/test/IntegrationTest.kt @@ -0,0 +1,131 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlinx.coroutines.flow.flowOn +import kotlinx.coroutines.testing.exceptions.* +import org.junit.runner.* +import org.junit.runners.* +import java.util.concurrent.Flow as JFlow +import kotlin.coroutines.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class IntegrationTest( + private val ctx: Ctx, + private val delay: Boolean +) : TestBase() { + + enum class Ctx { + MAIN { override fun invoke(context: CoroutineContext): CoroutineContext = context.minusKey(Job) }, + DEFAULT { override fun invoke(context: CoroutineContext): CoroutineContext = Dispatchers.Default }, + UNCONFINED { override fun invoke(context: CoroutineContext): CoroutineContext = Dispatchers.Unconfined }; + + abstract operator fun invoke(context: CoroutineContext): CoroutineContext + } + + companion object { + @Parameterized.Parameters(name = "ctx={0}, delay={1}") + @JvmStatic + fun params(): Collection> = Ctx.values().flatMap { ctx -> + listOf(false, true).map { delay -> + arrayOf(ctx, delay) + } + } + } + + @Test + fun testEmpty(): Unit = runBlocking { + val pub = flowPublish(ctx(coroutineContext)) { + if (delay) delay(1) + // does not send anything + } + assertFailsWith { pub.awaitFirst() } + assertEquals("OK", pub.awaitFirstOrDefault("OK")) + assertNull(pub.awaitFirstOrNull()) + assertEquals("ELSE", pub.awaitFirstOrElse { "ELSE" }) + assertFailsWith { pub.awaitLast() } + assertFailsWith { pub.awaitSingle() } + var cnt = 0 + pub.collect { cnt++ } + assertEquals(0, cnt) + } + + @Test + fun testSingle() = runBlocking { + val pub = flowPublish(ctx(coroutineContext)) { + if (delay) delay(1) + send("OK") + } + assertEquals("OK", pub.awaitFirst()) + assertEquals("OK", pub.awaitFirstOrDefault("!")) + assertEquals("OK", pub.awaitFirstOrNull()) + assertEquals("OK", pub.awaitFirstOrElse { "ELSE" }) + assertEquals("OK", pub.awaitLast()) + assertEquals("OK", pub.awaitSingle()) + var cnt = 0 + pub.collect { + assertEquals("OK", it) + cnt++ + } + assertEquals(1, cnt) + } + + @Test + fun testNumbers() = runBlocking { + val n = 100 * stressTestMultiplier + val pub = flowPublish(ctx(coroutineContext)) { + for (i in 1..n) { + send(i) + if (delay) delay(1) + } + } + assertEquals(1, pub.awaitFirst()) + assertEquals(1, pub.awaitFirstOrDefault(0)) + assertEquals(n, pub.awaitLast()) + assertEquals(1, pub.awaitFirstOrNull()) + assertEquals(1, pub.awaitFirstOrElse { 0 }) + assertFailsWith { pub.awaitSingle() } + checkNumbers(n, pub) + val flow = pub.asFlow() + checkNumbers(n, flow.flowOn(ctx(coroutineContext)).asPublisher()) + } + + @Test + fun testCancelWithoutValue() = runTest { + val job = launch(Job(), start = CoroutineStart.UNDISPATCHED) { + flowPublish { + hang {} + }.awaitFirst() + } + + job.cancel() + job.join() + } + + @Test + fun testEmptySingle() = runTest(unhandled = listOf { e -> e is NoSuchElementException}) { + expect(1) + val job = launch(Job(), start = CoroutineStart.UNDISPATCHED) { + flowPublish { + yield() + expect(2) + // Nothing to emit + }.awaitFirst() + } + + job.join() + finish(3) + } + + private suspend fun checkNumbers(n: Int, pub: JFlow.Publisher) { + var last = 0 + pub.collect { + assertEquals(++last, it) + } + assertEquals(n, last) + } + +} + diff --git a/reactive/kotlinx-coroutines-jdk9/test/PublishTest.kt b/reactive/kotlinx-coroutines-jdk9/test/PublishTest.kt new file mode 100644 index 0000000000..595c6788df --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/test/PublishTest.kt @@ -0,0 +1,288 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.testing.exceptions.* +import org.junit.Test +import java.util.concurrent.Flow as JFlow +import kotlin.test.* + +class PublishTest : TestBase() { + @Test + fun testBasicEmpty() = runTest { + expect(1) + val publisher = flowPublish(currentDispatcher()) { + expect(5) + } + expect(2) + publisher.subscribe(object : JFlow.Subscriber { + override fun onSubscribe(s: JFlow.Subscription?) { expect(3) } + override fun onNext(t: Int?) { expectUnreached() } + override fun onComplete() { expect(6) } + override fun onError(t: Throwable?) { expectUnreached() } + }) + expect(4) + yield() // to publish coroutine + finish(7) + } + + @Test + fun testBasicSingle() = runTest { + expect(1) + val publisher = flowPublish(currentDispatcher()) { + expect(5) + send(42) + expect(7) + } + expect(2) + publisher.subscribe(object : JFlow.Subscriber { + override fun onSubscribe(s: JFlow.Subscription) { + expect(3) + s.request(1) + } + override fun onNext(t: Int) { + expect(6) + assertEquals(42, t) + } + override fun onComplete() { expect(8) } + override fun onError(t: Throwable?) { expectUnreached() } + }) + expect(4) + yield() // to publish coroutine + finish(9) + } + + @Test + fun testBasicError() = runTest { + expect(1) + val publisher = flowPublish(currentDispatcher()) { + expect(5) + throw RuntimeException("OK") + } + expect(2) + publisher.subscribe(object : JFlow.Subscriber { + override fun onSubscribe(s: JFlow.Subscription) { + expect(3) + s.request(1) + } + override fun onNext(t: Int) { expectUnreached() } + override fun onComplete() { expectUnreached() } + override fun onError(t: Throwable) { + expect(6) + assertIs(t) + assertEquals("OK", t.message) + } + }) + expect(4) + yield() // to publish coroutine + finish(7) + } + + @Test + fun testHandleFailureAfterCancel() = runTest { + expect(1) + + val eh = CoroutineExceptionHandler { _, t -> + assertIs(t) + expect(6) + } + val publisher = flowPublish(Dispatchers.Unconfined + eh) { + try { + expect(3) + delay(10000) + } finally { + expect(5) + throw RuntimeException("FAILED") // crash after cancel + } + } + var sub: JFlow.Subscription? = null + publisher.subscribe(object : JFlow.Subscriber { + override fun onComplete() { + expectUnreached() + } + + override fun onSubscribe(s: JFlow.Subscription) { + expect(2) + sub = s + } + + override fun onNext(t: Unit?) { + expectUnreached() + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + expect(4) + sub!!.cancel() + finish(7) + } + + /** Tests that, as soon as `ProducerScope.close` is called, `isClosedForSend` starts returning `true`. */ + @Test + fun testChannelClosing() = runTest { + expect(1) + val publisher = flowPublish(Dispatchers.Unconfined) { + expect(3) + close() + assert(isClosedForSend) + expect(4) + } + try { + expect(2) + publisher.awaitFirstOrNull() + } catch (e: CancellationException) { + expect(5) + } + finish(6) + } + + @Test + fun testOnNextError() = runTest { + val latch = CompletableDeferred() + expect(1) + assertCallsExceptionHandlerWith { exceptionHandler -> + val publisher = flowPublish(currentDispatcher() + exceptionHandler) { + expect(4) + try { + send("OK") + } catch (e: Throwable) { + expect(6) + assert(e is TestException) + assert(isClosedForSend) + latch.complete(Unit) + } + } + expect(2) + publisher.subscribe(object : JFlow.Subscriber { + override fun onComplete() { + expectUnreached() + } + + override fun onSubscribe(s: JFlow.Subscription) { + expect(3) + s.request(1) + } + + override fun onNext(t: String) { + expect(5) + assertEquals("OK", t) + throw TestException() + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + latch.await() + } + finish(7) + } + + /** Tests the behavior when a call to `onNext` fails after the channel is already closed. */ + @Test + fun testOnNextErrorAfterCancellation() = runTest { + assertCallsExceptionHandlerWith { handler -> + var producerScope: ProducerScope? = null + CompletableDeferred() + expect(1) + var job: Job? = null + val publisher = flowPublish(handler + Dispatchers.Unconfined) { + producerScope = this + expect(4) + job = launch { + delay(Long.MAX_VALUE) + } + } + expect(2) + publisher.subscribe(object : JFlow.Subscriber { + override fun onSubscribe(s: JFlow.Subscription) { + expect(3) + s.request(Long.MAX_VALUE) + } + + override fun onNext(t: Int) { + expect(6) + assertEquals(1, t) + job!!.cancel() + throw TestException() + } + + override fun onError(t: Throwable?) { + /* Correct changes to the implementation could lead to us entering or not entering this method, but + it only matters that if we do, it is the "correct" exception that was validly used to cancel the + coroutine that gets passed here and not `TestException`. */ + assertIs(t) + } + + override fun onComplete() { + expectUnreached() + } + }) + expect(5) + val result: ChannelResult = producerScope!!.trySend(1) + val e = result.exceptionOrNull()!! + assertIs(e, "The actual error: $e") + assertTrue(producerScope!!.isClosedForSend) + assertTrue(result.isFailure) + } + finish(7) + } + + @Test + fun testFailingConsumer() = runTest { + val pub = flowPublish(currentDispatcher()) { + repeat(3) { + expect(it + 1) // expect(1), expect(2) *should* be invoked + send(it) + } + } + try { + pub.collect { + throw TestException() + } + } catch (e: TestException) { + finish(3) + } + } + + @Test + fun testIllegalArgumentException() { + assertFailsWith { flowPublish(Job()) { } } + } + + /** Tests that `trySend` doesn't throw in `flowPublish`. */ + @Test + fun testTrySendNotThrowing() = runTest { + var producerScope: ProducerScope? = null + expect(1) + val publisher = flowPublish(Dispatchers.Unconfined) { + producerScope = this + expect(3) + delay(Long.MAX_VALUE) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + publisher.awaitFirstOrNull() + expectUnreached() + } + job.cancel() + expect(4) + val result = producerScope!!.trySend(1) + assertTrue(result.isFailure) + finish(5) + } + + /** Tests that all methods on `flowPublish` fail without closing the channel when attempting to emit `null`. */ + @Test + fun testEmittingNull() = runTest { + val publisher = flowPublish { + assertFailsWith { send(null) } + assertFailsWith { trySend(null) } + send("OK") + } + assertEquals("OK", publisher.awaitFirstOrNull()) + } +} diff --git a/reactive/kotlinx-coroutines-jdk9/test/PublisherAsFlowTest.kt b/reactive/kotlinx-coroutines-jdk9/test/PublisherAsFlowTest.kt new file mode 100644 index 0000000000..ae1708b99f --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/test/PublisherAsFlowTest.kt @@ -0,0 +1,182 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.testing.flow.* +import kotlin.test.* + +class PublisherAsFlowTest : TestBase() { + @Test + fun testCancellation() = runTest { + var onNext = 0 + var onCancelled = 0 + var onError = 0 + + val publisher = flowPublish(currentDispatcher()) { + coroutineContext[Job]?.invokeOnCompletion { + if (it is CancellationException) ++onCancelled + } + + repeat(100) { + send(it) + } + } + + publisher.asFlow().launchIn(CoroutineScope(Dispatchers.Unconfined)) { + onEach { + ++onNext + throw RuntimeException() + } + catch { + ++onError + } + }.join() + + + assertEquals(1, onNext) + assertEquals(1, onError) + assertEquals(1, onCancelled) + } + + @Test + fun testBufferSize1() = runTest { + val publisher = flowPublish(currentDispatcher()) { + expect(1) + send(3) + + expect(2) + send(5) + + expect(4) + send(7) + expect(6) + } + + publisher.asFlow().buffer(1).collect { + expect(it) + } + + finish(8) + } + + @Test + fun testBufferSizeDefault() = runTest { + val publisher = flowPublish(currentDispatcher()) { + repeat(64) { + send(it + 1) + expect(it + 1) + } + assertFalse { trySend(-1).isSuccess } + } + + publisher.asFlow().collect { + expect(64 + it) + } + + finish(129) + } + + @Test + fun testDefaultCapacityIsProperlyOverwritten() = runTest { + val publisher = flowPublish(currentDispatcher()) { + expect(1) + send(3) + expect(2) + send(5) + expect(4) + send(7) + expect(6) + } + + publisher.asFlow().flowOn(wrapperDispatcher()).buffer(1).collect { + expect(it) + } + + finish(8) + } + + @Test + fun testBufferSize10() = runTest { + val publisher = flowPublish(currentDispatcher()) { + expect(1) + send(5) + + expect(2) + send(6) + + expect(3) + send(7) + expect(4) + } + + publisher.asFlow().buffer(10).collect { + expect(it) + } + + finish(8) + } + + @Test + fun testConflated() = runTest { + val publisher = flowPublish(currentDispatcher()) { + for (i in 1..5) send(i) + } + val list = publisher.asFlow().conflate().toList() + assertEquals(listOf(1, 5), list) + } + + @Test + fun testProduce() = runTest { + val flow = flowPublish(currentDispatcher()) { repeat(10) { send(it) } }.asFlow() + check((0..9).toList(), flow.produceIn(this)) + check((0..9).toList(), flow.buffer(2).produceIn(this)) + check((0..9).toList(), flow.buffer(Channel.UNLIMITED).produceIn(this)) + check(listOf(0, 9), flow.conflate().produceIn(this)) + } + + private suspend fun check(expected: List, channel: ReceiveChannel) { + val result = ArrayList(10) + channel.consumeEach { result.add(it) } + assertEquals(expected, result) + } + + @Test + fun testProduceCancellation() = runTest { + expect(1) + // publisher is an async coroutine, so it overproduces to the channel, but still gets cancelled + val flow = flowPublish(currentDispatcher()) { + expect(3) + repeat(10) { value -> + when (value) { + in 0..6 -> send(value) + 7 -> try { + send(value) + } catch (e: CancellationException) { + expect(5) + throw e + } + else -> expectUnreached() + } + } + }.asFlow().buffer(1) + assertFailsWith { + coroutineScope { + expect(2) + val channel = flow.produceIn(this) + channel.consumeEach { value -> + when (value) { + in 0..4 -> {} + 5 -> { + expect(4) + throw TestException() + } + else -> expectUnreached() + } + } + } + } + finish(6) + } +} diff --git a/reactive/kotlinx-coroutines-jdk9/test/PublisherBackpressureTest.kt b/reactive/kotlinx-coroutines-jdk9/test/PublisherBackpressureTest.kt new file mode 100644 index 0000000000..b7bbcbd261 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/test/PublisherBackpressureTest.kt @@ -0,0 +1,58 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.concurrent.Flow as JFlow + +class PublisherBackpressureTest : TestBase() { + @Test + fun testCancelWhileBPSuspended() = runBlocking { + expect(1) + val observable = flowPublish(currentDispatcher()) { + expect(5) + send("A") // will not suspend, because an item was requested + expect(7) + send("B") // second requested item + expect(9) + try { + send("C") // will suspend (no more requested) + } finally { + expect(12) + } + expectUnreached() + } + expect(2) + var sub: JFlow.Subscription? = null + observable.subscribe(object : JFlow.Subscriber { + override fun onSubscribe(s: JFlow.Subscription) { + sub = s + expect(3) + s.request(2) // request two items + } + + override fun onNext(t: String) { + when (t) { + "A" -> expect(6) + "B" -> expect(8) + else -> error("Should not happen") + } + } + + override fun onComplete() { + expectUnreached() + } + + override fun onError(e: Throwable) { + expectUnreached() + } + }) + expect(4) + yield() // yield to observable coroutine + expect(10) + sub!!.cancel() // now unsubscribe -- shall cancel coroutine (& do not signal) + expect(11) + yield() // shall perform finally in coroutine + finish(13) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-jdk9/test/PublisherCollectTest.kt b/reactive/kotlinx-coroutines-jdk9/test/PublisherCollectTest.kt new file mode 100644 index 0000000000..aa2b679304 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/test/PublisherCollectTest.kt @@ -0,0 +1,143 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import org.reactivestreams.* +import kotlin.test.* +import java.util.concurrent.Flow as JFlow + +class PublisherCollectTest: TestBase() { + + /** Tests the simple scenario where the publisher outputs a bounded stream of values to collect. */ + @Test + fun testCollect() = runTest { + val x = 100 + val xSum = x * (x + 1) / 2 + val publisher = JFlow.Publisher { subscriber -> + var requested = 0L + var lastOutput = 0 + subscriber.onSubscribe(object: JFlow.Subscription { + + override fun request(n: Long) { + requested += n + if (n <= 0) { + subscriber.onError(IllegalArgumentException()) + return + } + while (lastOutput < x && lastOutput < requested) { + lastOutput += 1 + subscriber.onNext(lastOutput) + } + if (lastOutput == x) + subscriber.onComplete() + } + + override fun cancel() { + /** According to rule 3.5 of the + * [reactive spec](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.3/README.md#3.5), + * this method can be called by the subscriber at any point, so it's not an error if it's called + * in this scenario. */ + } + + }) + } + var sum = 0 + publisher.collect { + sum += it + } + assertEquals(xSum, sum) + } + + /** Tests the behavior of [collect] when the publisher raises an error. */ + @Test + fun testCollectThrowingPublisher() = runTest { + val errorString = "Too many elements requested" + val x = 100 + val xSum = x * (x + 1) / 2 + val publisher = Publisher { subscriber -> + var requested = 0L + var lastOutput = 0 + subscriber.onSubscribe(object: Subscription { + + override fun request(n: Long) { + requested += n + if (n <= 0) { + subscriber.onError(IllegalArgumentException()) + return + } + while (lastOutput < x && lastOutput < requested) { + lastOutput += 1 + subscriber.onNext(lastOutput) + } + if (lastOutput == x) + subscriber.onError(IllegalArgumentException(errorString)) + } + + override fun cancel() { + /** See the comment for the corresponding part of [testCollect]. */ + } + + }) + } + var sum = 0 + try { + publisher.collect { + sum += it + } + } catch (e: IllegalArgumentException) { + assertEquals(errorString, e.message) + } + assertEquals(xSum, sum) + } + + /** Tests the behavior of [collect] when the action throws. */ + @Test + fun testCollectThrowingAction() = runTest { + val errorString = "Too many elements produced" + val x = 100 + val xSum = x * (x + 1) / 2 + val publisher = Publisher { subscriber -> + var requested = 0L + var lastOutput = 0 + subscriber.onSubscribe(object: Subscription { + + override fun request(n: Long) { + requested += n + if (n <= 0) { + subscriber.onError(IllegalArgumentException()) + return + } + while (lastOutput < x && lastOutput < requested) { + lastOutput += 1 + subscriber.onNext(lastOutput) + } + } + + override fun cancel() { + assertEquals(x, lastOutput) + expect(x + 2) + } + + }) + } + var sum = 0 + try { + expect(1) + var i = 1 + publisher.collect { + sum += it + i += 1 + expect(i) + if (sum >= xSum) { + throw IllegalArgumentException(errorString) + } + } + } catch (e: IllegalArgumentException) { + expect(x + 3) + assertEquals(errorString, e.message) + } + finish(x + 4) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-jdk9/test/PublisherCompletionStressTest.kt b/reactive/kotlinx-coroutines-jdk9/test/PublisherCompletionStressTest.kt new file mode 100644 index 0000000000..abacdec38b --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/test/PublisherCompletionStressTest.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.* +import kotlin.coroutines.* + +class PublisherCompletionStressTest : TestBase() { + private val N_REPEATS = 10_000 * stressTestMultiplier + + private fun CoroutineScope.range(context: CoroutineContext, start: Int, count: Int) = flowPublish(context) { + for (x in start until start + count) send(x) + } + + @Test + fun testCompletion() { + val rnd = Random() + repeat(N_REPEATS) { + val count = rnd.nextInt(5) + runBlocking { + withTimeout(5000) { + var received = 0 + range(Dispatchers.Default, 1, count).collect { x -> + received++ + if (x != received) error("$x != $received") + } + if (received != count) error("$received != $count") + } + } + } + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-jdk9/test/PublisherMultiTest.kt b/reactive/kotlinx-coroutines-jdk9/test/PublisherMultiTest.kt new file mode 100644 index 0000000000..90fa5dda86 --- /dev/null +++ b/reactive/kotlinx-coroutines-jdk9/test/PublisherMultiTest.kt @@ -0,0 +1,28 @@ +package kotlinx.coroutines.jdk9 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class PublisherMultiTest : TestBase() { + @Test + fun testConcurrentStress() = runBlocking { + val n = 10_000 * stressTestMultiplier + val observable = flowPublish { + // concurrent emitters (many coroutines) + val jobs = List(n) { + // launch + launch { + send(it) + } + } + jobs.forEach { it.join() } + } + val resultSet = mutableSetOf() + observable.collect { + assertTrue(resultSet.add(it)) + } + assertEquals(n, resultSet.size) + } +} diff --git a/reactive/kotlinx-coroutines-reactive/README.md b/reactive/kotlinx-coroutines-reactive/README.md index e54bd5cd8d..cd27b27d97 100644 --- a/reactive/kotlinx-coroutines-reactive/README.md +++ b/reactive/kotlinx-coroutines-reactive/README.md @@ -1,12 +1,22 @@ # Module kotlinx-coroutines-reactive -Utilities for [Reactive Streams](http://www.reactive-streams.org). +Utilities for [Reactive Streams](https://www.reactive-streams.org). Coroutine builders: | **Name** | **Result** | **Scope** | **Description** | --------------- | ----------------------------- | ---------------- | --------------- -| [publish] | `Publisher` | [ProducerScope] | Cold reactive publisher that starts coroutine on subscribe +| [kotlinx.coroutines.reactive.publish] | `Publisher` | [ProducerScope] | Cold reactive publisher that starts the coroutine on subscribe + +Integration with [Flow]: + +| **Name** | **Result** | **Description** +| --------------- | -------------- | --------------- +| [Publisher.asFlow] | `Flow` | Converts the given publisher to a flow +| [Flow.asPublisher] | `Publisher` | Converts the given flow to a TCK-compliant publisher + +If these adapters are used along with `kotlinx-coroutines-reactor` in the classpath, then Reactor's `Context` is properly +propagated as coroutine context element (`ReactorContext`) and vice versa. Suspending extension functions and suspending iteration: @@ -14,36 +24,35 @@ Suspending extension functions and suspending iteration: | -------- | --------------- | [Publisher.awaitFirst][org.reactivestreams.Publisher.awaitFirst] | Returns the first value from the given publisher | [Publisher.awaitFirstOrDefault][org.reactivestreams.Publisher.awaitFirstOrDefault] | Returns the first value from the given publisher or default +| [Publisher.awaitFirstOrElse][org.reactivestreams.Publisher.awaitFirstOrElse] | Returns the first value from the given publisher or default from a function +| [Publisher.awaitFirstOrNull][org.reactivestreams.Publisher.awaitFirstOrNull] | Returns the first value from the given publisher or null | [Publisher.awaitLast][org.reactivestreams.Publisher.awaitFirst] | Returns the last value from the given publisher | [Publisher.awaitSingle][org.reactivestreams.Publisher.awaitSingle] | Returns the single value from the given publisher -| [Publisher.open][org.reactivestreams.Publisher.open] | Subscribes to publisher and returns [ReceiveChannel] -| [Publisher.iterator][org.reactivestreams.Publisher.iterator] | Subscribes to publisher and returns [ChannelIterator] -Conversion functions: + + + + +[Flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html + + + +[ProducerScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-producer-scope/index.html + + + + +[kotlinx.coroutines.reactive.publish]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/publish.html +[Publisher.asFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/as-flow.html +[Flow.asPublisher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/as-publisher.html +[org.reactivestreams.Publisher.awaitFirst]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/await-first.html +[org.reactivestreams.Publisher.awaitFirstOrDefault]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/await-first-or-default.html +[org.reactivestreams.Publisher.awaitFirstOrElse]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/await-first-or-else.html +[org.reactivestreams.Publisher.awaitFirstOrNull]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/await-first-or-null.html +[org.reactivestreams.Publisher.awaitSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.reactive/await-single.html -| **Name** | **Description** -| -------- | --------------- -| [ReceiveChannel.asPublisher][kotlinx.coroutines.experimental.channels.ReceiveChannel.asPublisher] | Converts streaming channel to hot publisher - - - - - -[ProducerScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-producer-scope/index.html -[ReceiveChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/index.html -[ChannelIterator]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel-iterator/index.html - - - -[publish]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/publish.html -[org.reactivestreams.Publisher.awaitFirst]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/org.reactivestreams.-publisher/await-first.html -[org.reactivestreams.Publisher.awaitFirstOrDefault]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/org.reactivestreams.-publisher/await-first-or-default.html -[org.reactivestreams.Publisher.awaitSingle]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/org.reactivestreams.-publisher/await-single.html -[org.reactivestreams.Publisher.open]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/org.reactivestreams.-publisher/open.html -[org.reactivestreams.Publisher.iterator]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/org.reactivestreams.-publisher/iterator.html -[kotlinx.coroutines.experimental.channels.ReceiveChannel.asPublisher]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactive/kotlinx.coroutines.experimental.reactive/kotlinx.coroutines.experimental.channels.-receive-channel/as-publisher.html -# Package kotlinx.coroutines.experimental.reactive +# Package kotlinx.coroutines.reactive -Utilities for [Reactive Streams](http://www.reactive-streams.org). +Utilities for [Reactive Streams](https://www.reactive-streams.org). diff --git a/reactive/kotlinx-coroutines-reactive/api/kotlinx-coroutines-reactive.api b/reactive/kotlinx-coroutines-reactive/api/kotlinx-coroutines-reactive.api new file mode 100644 index 0000000000..7436696aed --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/api/kotlinx-coroutines-reactive.api @@ -0,0 +1,74 @@ +public final class kotlinx/coroutines/reactive/AwaitKt { + public static final fun awaitFirst (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrDefault (Lorg/reactivestreams/Publisher;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrElse (Lorg/reactivestreams/Publisher;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrNull (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitLast (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingle (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitSingleOrDefault (Lorg/reactivestreams/Publisher;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitSingleOrElse (Lorg/reactivestreams/Publisher;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitSingleOrNull (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/reactive/ChannelKt { + public static final fun collect (Lorg/reactivestreams/Publisher;Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun openSubscription (Lorg/reactivestreams/Publisher;I)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun openSubscription$default (Lorg/reactivestreams/Publisher;IILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun toChannel (Lorg/reactivestreams/Publisher;I)Lkotlinx/coroutines/channels/ReceiveChannel; + public static synthetic fun toChannel$default (Lorg/reactivestreams/Publisher;IILjava/lang/Object;)Lkotlinx/coroutines/channels/ReceiveChannel; +} + +public abstract interface class kotlinx/coroutines/reactive/ContextInjector { + public abstract fun injectCoroutineContext (Lorg/reactivestreams/Publisher;Lkotlin/coroutines/CoroutineContext;)Lorg/reactivestreams/Publisher; +} + +public final class kotlinx/coroutines/reactive/ConvertKt { + public static final synthetic fun asPublisher (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;)Lorg/reactivestreams/Publisher; + public static synthetic fun asPublisher$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lorg/reactivestreams/Publisher; +} + +public final class kotlinx/coroutines/reactive/FlowKt { + public static final synthetic fun asFlow (Lorg/reactivestreams/Publisher;)Lkotlinx/coroutines/flow/Flow; + public static final synthetic fun asFlow (Lorg/reactivestreams/Publisher;I)Lkotlinx/coroutines/flow/Flow; + public static final synthetic fun asPublisher (Lkotlinx/coroutines/flow/Flow;)Lorg/reactivestreams/Publisher; +} + +public final class kotlinx/coroutines/reactive/FlowSubscription : kotlinx/coroutines/AbstractCoroutine, org/reactivestreams/Subscription { + public final field flow Lkotlinx/coroutines/flow/Flow; + public final field subscriber Lorg/reactivestreams/Subscriber; + public fun (Lkotlinx/coroutines/flow/Flow;Lorg/reactivestreams/Subscriber;Lkotlin/coroutines/CoroutineContext;)V + public synthetic fun cancel ()V + public fun request (J)V +} + +public final class kotlinx/coroutines/reactive/PublishKt { + public static final fun publish (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lorg/reactivestreams/Publisher; + public static final synthetic fun publish (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lorg/reactivestreams/Publisher; + public static synthetic fun publish$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lorg/reactivestreams/Publisher; + public static synthetic fun publish$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lorg/reactivestreams/Publisher; + public static final fun publishInternal (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;Lkotlin/jvm/functions/Function2;)Lorg/reactivestreams/Publisher; +} + +public final class kotlinx/coroutines/reactive/PublisherCoroutine : kotlinx/coroutines/AbstractCoroutine, kotlinx/coroutines/channels/ProducerScope, org/reactivestreams/Subscription { + public fun (Lkotlin/coroutines/CoroutineContext;Lorg/reactivestreams/Subscriber;Lkotlin/jvm/functions/Function2;)V + public fun cancel ()V + public fun close (Ljava/lang/Throwable;)Z + public fun getChannel ()Lkotlinx/coroutines/channels/SendChannel; + public fun getOnSend ()Lkotlinx/coroutines/selects/SelectClause2; + public fun invokeOnClose (Lkotlin/jvm/functions/Function1;)Ljava/lang/Void; + public synthetic fun invokeOnClose (Lkotlin/jvm/functions/Function1;)V + public fun isClosedForSend ()Z + public fun offer (Ljava/lang/Object;)Z + public synthetic fun onCompleted (Ljava/lang/Object;)V + public fun request (J)V + public fun send (Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun trySend-JP2dKIU (Ljava/lang/Object;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/reactive/ReactiveFlowKt { + public static final fun asFlow (Lorg/reactivestreams/Publisher;)Lkotlinx/coroutines/flow/Flow; + public static final fun asPublisher (Lkotlinx/coroutines/flow/Flow;)Lorg/reactivestreams/Publisher; + public static final fun asPublisher (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lorg/reactivestreams/Publisher; + public static synthetic fun asPublisher$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lorg/reactivestreams/Publisher; +} + diff --git a/reactive/kotlinx-coroutines-reactive/build.gradle.kts b/reactive/kotlinx-coroutines-reactive/build.gradle.kts new file mode 100644 index 0000000000..3e6ecccf3e --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/build.gradle.kts @@ -0,0 +1,51 @@ +plugins { + // apply plugin to use autocomplete for Kover DSL + id("org.jetbrains.kotlinx.kover") +} + +val reactiveStreamsVersion = property("reactive_streams_version") + +dependencies { + api("org.reactivestreams:reactive-streams:$reactiveStreamsVersion") + testImplementation("org.reactivestreams:reactive-streams-tck:$reactiveStreamsVersion") +} + +val testNG by tasks.registering(Test::class) { + useTestNG() + reports.html.outputLocation = layout.buildDirectory.dir("reports/testng") + include("**/*ReactiveStreamTckTest.*") + // Skip testNG when tests are filtered with --tests, otherwise it simply fails + onlyIf { + filter.includePatterns.isEmpty() + } + doFirst { + // Classic gradle, nothing works without doFirst + println("TestNG tests: ($includes)") + } +} + +tasks.test { + reports.html.outputLocation = layout.buildDirectory.dir("reports/junit") +} + +tasks.check { + dependsOn(testNG) +} + +externalDocumentationLink( + url = "https://www.reactive-streams.org/reactive-streams-$reactiveStreamsVersion-javadoc/" +) + +kover { + reports { + filters { + excludes { + classes( + "kotlinx.coroutines.reactive.FlowKt", // Deprecated + "kotlinx.coroutines.reactive.FlowKt__MigrationKt", // Deprecated + "kotlinx.coroutines.reactive.ConvertKt" // Deprecated + ) + } + } + } +} diff --git a/reactive/kotlinx-coroutines-reactive/package.list b/reactive/kotlinx-coroutines-reactive/package.list new file mode 100644 index 0000000000..6a8ba62f50 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/package.list @@ -0,0 +1 @@ +org.reactivestreams diff --git a/reactive/kotlinx-coroutines-reactive/pom.xml b/reactive/kotlinx-coroutines-reactive/pom.xml deleted file mode 100644 index 007e2f373c..0000000000 --- a/reactive/kotlinx-coroutines-reactive/pom.xml +++ /dev/null @@ -1,58 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - ../../pom.xml - - - kotlinx-coroutines-reactive - jar - - - src/main/kotlin - src/test/kotlin - - - - - org.reactivestreams - reactive-streams - 1.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - tests - test - - - - diff --git a/reactive/kotlinx-coroutines-reactive/src/Await.kt b/reactive/kotlinx-coroutines-reactive/src/Await.kt new file mode 100644 index 0000000000..73f0ca220b --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/src/Await.kt @@ -0,0 +1,324 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.* +import org.reactivestreams.Publisher +import org.reactivestreams.Subscriber +import org.reactivestreams.Subscription +import java.lang.IllegalStateException +import kotlin.coroutines.* + +/** + * Awaits the first value from the given publisher without blocking the thread and returns the resulting value, or, if + * the publisher has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the publisher does not emit any value + */ +public suspend fun Publisher.awaitFirst(): T = awaitOne(Mode.FIRST) + +/** + * Awaits the first value from the given publisher, or returns the [default] value if none is emitted, without blocking + * the thread, and returns the resulting value, or, if this publisher has produced an error, throws the corresponding + * exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + */ +public suspend fun Publisher.awaitFirstOrDefault(default: T): T = awaitOne(Mode.FIRST_OR_DEFAULT, default) + +/** + * Awaits the first value from the given publisher, or returns `null` if none is emitted, without blocking the thread, + * and returns the resulting value, or, if this publisher has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + */ +public suspend fun Publisher.awaitFirstOrNull(): T? = awaitOne(Mode.FIRST_OR_DEFAULT) + +/** + * Awaits the first value from the given publisher, or calls [defaultValue] to get a value if none is emitted, without + * blocking the thread, and returns the resulting value, or, if this publisher has produced an error, throws the + * corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + */ +public suspend fun Publisher.awaitFirstOrElse(defaultValue: () -> T): T = awaitOne(Mode.FIRST_OR_DEFAULT) ?: defaultValue() + +/** + * Awaits the last value from the given publisher without blocking the thread and + * returns the resulting value, or, if this publisher has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the publisher does not emit any value + */ +public suspend fun Publisher.awaitLast(): T = awaitOne(Mode.LAST) + +/** + * Awaits the single value from the given publisher without blocking the thread and returns the resulting value, or, + * if this publisher has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the publisher does not emit any value + * @throws IllegalArgumentException if the publisher emits more than one value + */ +public suspend fun Publisher.awaitSingle(): T = awaitOne(Mode.SINGLE) + +/** + * Awaits the single value from the given publisher, or returns the [default] value if none is emitted, without + * blocking the thread, and returns the resulting value, or, if this publisher has produced an error, throws the + * corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + * + * ### Deprecation + * + * This method is deprecated because the conventions established in Kotlin mandate that an operation with the name + * `awaitSingleOrDefault` returns the default value instead of throwing in case there is an error; however, this would + * also mean that this method would return the default value if there are *too many* values. This could be confusing to + * those who expect this function to validate that there is a single element or none at all emitted, and cases where + * there are no elements are indistinguishable from those where there are too many, though these cases have different + * meaning. + * + * @throws NoSuchElementException if the publisher does not emit any value + * @throws IllegalArgumentException if the publisher emits more than one value + * + * @suppress + */ +@Deprecated( + message = "Deprecated without a replacement due to its name incorrectly conveying the behavior. " + + "Please consider using awaitFirstOrDefault().", + level = DeprecationLevel.HIDDEN +) // Warning since 1.5, error in 1.6, hidden in 1.7 +public suspend fun Publisher.awaitSingleOrDefault(default: T): T = awaitOne(Mode.SINGLE_OR_DEFAULT, default) + +/** + * Awaits the single value from the given publisher without blocking the thread and returns the resulting value, or, if + * this publisher has produced an error, throws the corresponding exception. If more than one value or none were + * produced by the publisher, `null` is returned. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + * + * ### Deprecation + * + * This method is deprecated because the conventions established in Kotlin mandate that an operation with the name + * `awaitSingleOrNull` returns `null` instead of throwing in case there is an error; however, this would + * also mean that this method would return `null` if there are *too many* values. This could be confusing to + * those who expect this function to validate that there is a single element or none at all emitted, and cases where + * there are no elements are indistinguishable from those where there are too many, though these cases have different + * meaning. + * + * @throws IllegalArgumentException if the publisher emits more than one value + * @suppress + */ +@Deprecated( + message = "Deprecated without a replacement due to its name incorrectly conveying the behavior. " + + "There is a specialized version for Reactor's Mono, please use that where applicable. " + + "Alternatively, please consider using awaitFirstOrNull().", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingleOrNull()", "kotlinx.coroutines.reactor") +) // Warning since 1.5, error in 1.6, hidden in 1.7 +public suspend fun Publisher.awaitSingleOrNull(): T? = awaitOne(Mode.SINGLE_OR_DEFAULT) + +/** + * Awaits the single value from the given publisher, or calls [defaultValue] to get a value if none is emitted, without + * blocking the thread, and returns the resulting value, or, if this publisher has produced an error, throws the + * corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + * + * ### Deprecation + * + * This method is deprecated because the conventions established in Kotlin mandate that an operation with the name + * `awaitSingleOrElse` returns the calculated value instead of throwing in case there is an error; however, this would + * also mean that this method would return the calculated value if there are *too many* values. This could be confusing + * to those who expect this function to validate that there is a single element or none at all emitted, and cases where + * there are no elements are indistinguishable from those where there are too many, though these cases have different + * meaning. + * + * @throws IllegalArgumentException if the publisher emits more than one value + * @suppress + */ +@Deprecated( + message = "Deprecated without a replacement due to its name incorrectly conveying the behavior. " + + "Please consider using awaitFirstOrElse().", + level = DeprecationLevel.HIDDEN +) // Warning since 1.5, error in 1.6, hidden in 1.7 +public suspend fun Publisher.awaitSingleOrElse(defaultValue: () -> T): T = + awaitOne(Mode.SINGLE_OR_DEFAULT) ?: defaultValue() + +// ------------------------ private ------------------------ + +private enum class Mode(val s: String) { + FIRST("awaitFirst"), + FIRST_OR_DEFAULT("awaitFirstOrDefault"), + LAST("awaitLast"), + SINGLE("awaitSingle"), + SINGLE_OR_DEFAULT("awaitSingleOrDefault"); + override fun toString(): String = s +} + +private suspend fun Publisher.awaitOne( + mode: Mode, + default: T? = null +): T = suspendCancellableCoroutine { cont -> + /* This implementation must obey + https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.3/README.md#2-subscriber-code + The numbers of rules are taken from there. */ + injectCoroutineContext(cont.context).subscribe(object : Subscriber { + // It is unclear whether 2.13 implies (T: Any), but if so, it seems that we don't break anything by not adhering + private var subscription: Subscription? = null + private var value: T? = null + private var seenValue = false + private var inTerminalState = false + + override fun onSubscribe(sub: Subscription) { + /** cancelling the new subscription due to rule 2.5, though the publisher would either have to + * subscribe more than once, which would break 2.12, or leak this [Subscriber]. */ + if (subscription != null) { + withSubscriptionLock { + sub.cancel() + } + return + } + subscription = sub + cont.invokeOnCancellation { + withSubscriptionLock { + sub.cancel() + } + } + withSubscriptionLock { + sub.request(if (mode == Mode.FIRST || mode == Mode.FIRST_OR_DEFAULT) 1 else Long.MAX_VALUE) + } + } + + override fun onNext(t: T) { + val sub = subscription.let { + if (it == null) { + /** Enforce rule 1.9: expect [Subscriber.onSubscribe] before any other signals. */ + handleCoroutineException(cont.context, + IllegalStateException("'onNext' was called before 'onSubscribe'")) + return + } else { + it + } + } + if (inTerminalState) { + gotSignalInTerminalStateException(cont.context, "onNext") + return + } + when (mode) { + Mode.FIRST, Mode.FIRST_OR_DEFAULT -> { + if (seenValue) { + moreThanOneValueProvidedException(cont.context, mode) + return + } + seenValue = true + withSubscriptionLock { + sub.cancel() + } + cont.resume(t) + } + Mode.LAST, Mode.SINGLE, Mode.SINGLE_OR_DEFAULT -> { + if ((mode == Mode.SINGLE || mode == Mode.SINGLE_OR_DEFAULT) && seenValue) { + withSubscriptionLock { + sub.cancel() + } + /* the check for `cont.isActive` is needed in case `sub.cancel() above calls `onComplete` or + `onError` on its own. */ + if (cont.isActive) { + cont.resumeWithException(IllegalArgumentException("More than one onNext value for $mode")) + } + } else { + value = t + seenValue = true + } + } + } + } + + @Suppress("UNCHECKED_CAST") + override fun onComplete() { + if (!tryEnterTerminalState("onComplete")) { + return + } + if (seenValue) { + /* the check for `cont.isActive` is needed because, otherwise, if the publisher doesn't acknowledge the + call to `cancel` for modes `SINGLE*` when more than one value was seen, it may call `onComplete`, and + here `cont.resume` would fail. */ + if (mode != Mode.FIRST_OR_DEFAULT && mode != Mode.FIRST && cont.isActive) { + cont.resume(value as T) + } + return + } + when { + (mode == Mode.FIRST_OR_DEFAULT || mode == Mode.SINGLE_OR_DEFAULT) -> { + cont.resume(default as T) + } + cont.isActive -> { + // the check for `cont.isActive` is just a slight optimization and doesn't affect correctness + cont.resumeWithException(NoSuchElementException("No value received via onNext for $mode")) + } + } + } + + override fun onError(e: Throwable) { + if (tryEnterTerminalState("onError")) { + cont.resumeWithException(e) + } + } + + /** + * Enforce rule 2.4: assume that the [Publisher] is in a terminal state after [onError] or [onComplete]. + */ + private fun tryEnterTerminalState(signalName: String): Boolean { + if (inTerminalState) { + gotSignalInTerminalStateException(cont.context, signalName) + return false + } + inTerminalState = true + return true + } + + /** + * Enforce rule 2.7: [Subscription.request] and [Subscription.cancel] must be executed serially + */ + @Synchronized + private fun withSubscriptionLock(block: () -> Unit) { + block() + } + }) +} + +/** + * Enforce rule 2.4 (detect publishers that don't respect rule 1.7): don't process anything after a terminal + * state was reached. + */ +private fun gotSignalInTerminalStateException(context: CoroutineContext, signalName: String) = + handleCoroutineException(context, + IllegalStateException("'$signalName' was called after the publisher already signalled being in a terminal state")) + +/** + * Enforce rule 1.1: it is invalid for a publisher to provide more values than requested. + */ +private fun moreThanOneValueProvidedException(context: CoroutineContext, mode: Mode) = + handleCoroutineException(context, + IllegalStateException("Only a single value was requested in '$mode', but the publisher provided more")) diff --git a/reactive/kotlinx-coroutines-reactive/src/Channel.kt b/reactive/kotlinx-coroutines-reactive/src/Channel.kt new file mode 100644 index 0000000000..4b775f7272 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/src/Channel.kt @@ -0,0 +1,106 @@ +package kotlinx.coroutines.reactive + +import kotlinx.atomicfu.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import org.reactivestreams.* + +/** + * Subscribes to this [Publisher] and performs the specified action for each received element. + * + * If [action] throws an exception at some point, the subscription is cancelled, and the exception is rethrown from + * [collect]. Also, if the publisher signals an error, that error is rethrown from [collect]. + */ +public suspend inline fun Publisher.collect(action: (T) -> Unit): Unit = + toChannel().consumeEach(action) + +@PublishedApi +internal fun Publisher.toChannel(request: Int = 1): ReceiveChannel { + val channel = SubscriptionChannel(request) + subscribe(channel) + return channel +} + +@Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER", "SubscriberImplementation") +private class SubscriptionChannel( + private val request: Int +) : BufferedChannel(capacity = Channel.UNLIMITED), Subscriber { + init { + require(request >= 0) { "Invalid request size: $request" } + } + + private val _subscription = atomic(null) + + // requested from subscription minus number of received minus number of enqueued receivers, + // can be negative if we have receivers, but no subscription yet + private val _requested = atomic(0) + + // --------------------- BufferedChannel overrides ------------------------------- + @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") + override fun onReceiveEnqueued() { + _requested.loop { wasRequested -> + val subscription = _subscription.value + val needRequested = wasRequested - 1 + if (subscription != null && needRequested < 0) { // need to request more from subscription + // try to fixup by making request + if (wasRequested != request && !_requested.compareAndSet(wasRequested, request)) + return@loop // continue looping if failed + subscription.request((request - needRequested).toLong()) + return + } + // just do book-keeping + if (_requested.compareAndSet(wasRequested, needRequested)) return + } + } + + @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") + override fun onReceiveDequeued() { + _requested.incrementAndGet() + } + + @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") + override fun onClosedIdempotent() { + _subscription.getAndSet(null)?.cancel() // cancel exactly once + } + + // --------------------- Subscriber overrides ------------------------------- + override fun onSubscribe(s: Subscription) { + _subscription.value = s + while (true) { // lock-free loop on _requested + if (isClosedForSend) { + s.cancel() + return + } + val wasRequested = _requested.value + if (wasRequested >= request) return // ok -- normal story + // otherwise, receivers came before we had subscription or need to make initial request + // try to fixup by making request + if (!_requested.compareAndSet(wasRequested, request)) continue + s.request((request - wasRequested).toLong()) + return + } + } + + override fun onNext(t: T) { + _requested.decrementAndGet() + trySend(t) // Safe to ignore return value here, expectedly racing with cancellation + } + + override fun onComplete() { + close(cause = null) + } + + override fun onError(e: Throwable) { + close(cause = e) + } +} + +/** @suppress */ +@Deprecated( + message = "Transforming publisher to channel is deprecated, use asFlow() instead", + level = DeprecationLevel.HIDDEN) // ERROR in 1.4, HIDDEN in 1.6.0 +public fun Publisher.openSubscription(request: Int = 1): ReceiveChannel { + val channel = SubscriptionChannel(request) + subscribe(channel) + return channel +} diff --git a/reactive/kotlinx-coroutines-reactive/src/ContextInjector.kt b/reactive/kotlinx-coroutines-reactive/src/ContextInjector.kt new file mode 100644 index 0000000000..abf0946377 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/src/ContextInjector.kt @@ -0,0 +1,15 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.InternalCoroutinesApi +import org.reactivestreams.Publisher +import kotlin.coroutines.CoroutineContext + +/** @suppress */ +@InternalCoroutinesApi +public interface ContextInjector { + /** + * Injects `ReactorContext` element from the given context into the `SubscriberContext` of the publisher. + * This API used as an indirection layer between `reactive` and `reactor` modules. + */ + public fun injectCoroutineContext(publisher: Publisher, coroutineContext: CoroutineContext): Publisher +} diff --git a/reactive/kotlinx-coroutines-reactive/src/Convert.kt b/reactive/kotlinx-coroutines-reactive/src/Convert.kt new file mode 100644 index 0000000000..55bce6c08a --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/src/Convert.kt @@ -0,0 +1,14 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.channels.* +import org.reactivestreams.* +import kotlin.coroutines.* + +/** @suppress */ +@Deprecated(message = "Deprecated in the favour of consumeAsFlow()", + level = DeprecationLevel.HIDDEN, // Error in 1.4, HIDDEN in 1.6.0 + replaceWith = ReplaceWith("this.consumeAsFlow().asPublisher(context)", imports = ["kotlinx.coroutines.flow.consumeAsFlow"])) +public fun ReceiveChannel.asPublisher(context: CoroutineContext = EmptyCoroutineContext): Publisher = publish(context) { + for (t in this@asPublisher) + send(t) +} diff --git a/reactive/kotlinx-coroutines-reactive/src/Migration.kt b/reactive/kotlinx-coroutines-reactive/src/Migration.kt new file mode 100644 index 0000000000..3fc94d5afd --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/src/Migration.kt @@ -0,0 +1,34 @@ +@file:JvmMultifileClass +@file:JvmName("FlowKt") + +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.reactivestreams.* + +// Binary compatibility with Spring 5.2 RC +/** @suppress */ +@Deprecated( + message = "Replaced in favor of ReactiveFlow extension, please import kotlinx.coroutines.reactive.* instead of kotlinx.coroutines.reactive.FlowKt", + level = DeprecationLevel.HIDDEN +) +@JvmName("asFlow") +public fun Publisher.asFlowDeprecated(): Flow = asFlow() + +// Binary compatibility with Spring 5.2 RC +/** @suppress */ +@Deprecated( + message = "Replaced in favor of ReactiveFlow extension, please import kotlinx.coroutines.reactive.* instead of kotlinx.coroutines.reactive.FlowKt", + level = DeprecationLevel.HIDDEN +) +@JvmName("asPublisher") +public fun Flow.asPublisherDeprecated(): Publisher = asPublisher() + +/** @suppress */ +@Deprecated( + message = "batchSize parameter is deprecated, use .buffer() instead to control the backpressure", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("asFlow().buffer(batchSize)", imports = ["kotlinx.coroutines.flow.*"]) +) +public fun Publisher.asFlow(batchSize: Int): Flow = asFlow().buffer(batchSize) diff --git a/reactive/kotlinx-coroutines-reactive/src/Publish.kt b/reactive/kotlinx-coroutines-reactive/src/Publish.kt new file mode 100644 index 0000000000..cb3e596268 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/src/Publish.kt @@ -0,0 +1,336 @@ +package kotlinx.coroutines.reactive + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import kotlinx.coroutines.sync.* +import org.reactivestreams.* +import kotlin.coroutines.* + +/** + * Creates a cold reactive [Publisher] that runs a given [block] in a coroutine. + * + * Every time the returned flux is subscribed, it starts a new coroutine in the specified [context]. + * The coroutine emits (via [Subscriber.onNext]) values with [send][ProducerScope.send], + * completes (via [Subscriber.onComplete]) when the coroutine completes or channel is explicitly closed, and emits + * errors (via [Subscriber.onError]) if the coroutine throws an exception or closes channel with a cause. + * Unsubscribing cancels the running coroutine. + * + * Invocations of [send][ProducerScope.send] are suspended appropriately when subscribers apply back-pressure and to + * ensure that [onNext][Subscriber.onNext] is not invoked concurrently. + * + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is + * used. + * + * **Note: This is an experimental api.** Behaviour of publishers that work as children in a parent scope with respect + * to cancellation and error handling may change in the future. + * + * @throws IllegalArgumentException if the provided [context] contains a [Job] instance. + */ +public fun publish( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Publisher { + require(context[Job] === null) { "Publisher context cannot contain job in it." + + "Its lifecycle should be managed via subscription. Had $context" } + return publishInternal(GlobalScope, context, DEFAULT_HANDLER, block) +} + +/** @suppress For internal use from other reactive integration modules only */ +@InternalCoroutinesApi +public fun publishInternal( + scope: CoroutineScope, // support for legacy publish in scope + context: CoroutineContext, + exceptionOnCancelHandler: (Throwable, CoroutineContext) -> Unit, + block: suspend ProducerScope.() -> Unit +): Publisher = Publisher { subscriber -> + // specification requires NPE on null subscriber + if (subscriber == null) throw NullPointerException("Subscriber cannot be null") + val newContext = scope.newCoroutineContext(context) + val coroutine = PublisherCoroutine(newContext, subscriber, exceptionOnCancelHandler) + subscriber.onSubscribe(coroutine) // do it first (before starting coroutine), to avoid unnecessary suspensions + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private const val CLOSED = -1L // closed, but have not signalled onCompleted/onError yet +private const val SIGNALLED = -2L // already signalled subscriber onCompleted/onError +private val DEFAULT_HANDLER: (Throwable, CoroutineContext) -> Unit = { t, ctx -> if (t !is CancellationException) handleCoroutineException(ctx, t) } + +/** @suppress */ +@Suppress("CONFLICTING_JVM_DECLARATIONS", "RETURN_TYPE_MISMATCH_ON_INHERITANCE") +@InternalCoroutinesApi +public class PublisherCoroutine( + parentContext: CoroutineContext, + private val subscriber: Subscriber, + private val exceptionOnCancelHandler: (Throwable, CoroutineContext) -> Unit +) : AbstractCoroutine(parentContext, false, true), ProducerScope, Subscription { + override val channel: SendChannel get() = this + + private val _nRequested = atomic(0L) // < 0 when closed (CLOSED or SIGNALLED) + + @Volatile + private var cancelled = false // true after Subscription.cancel() is invoked + + override val isClosedForSend: Boolean get() = !isActive + override fun close(cause: Throwable?): Boolean = cancelCoroutine(cause) + override fun invokeOnClose(handler: (Throwable?) -> Unit): Nothing = + throw UnsupportedOperationException("PublisherCoroutine doesn't support invokeOnClose") + + // Mutex is locked when either nRequested == 0 or while subscriber.onXXX is being invoked + private val mutex: Mutex = Mutex(locked = true) + + @Suppress("UNCHECKED_CAST", "INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + override val onSend: SelectClause2> get() = SelectClause2Impl( + clauseObject = this, + regFunc = PublisherCoroutine<*>::registerSelectForSend as RegistrationFunction, + processResFunc = PublisherCoroutine<*>::processResultSelectSend as ProcessResultFunction + ) + + @Suppress("UNCHECKED_CAST", "UNUSED_PARAMETER") + private fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // Try to acquire the mutex and complete in the registration phase. + if (mutex.tryLock()) { + select.selectInRegistrationPhase(Unit) + return + } + // Start a new coroutine that waits for the mutex, invoking `trySelect(..)` after that. + // Please note that at the point of the `trySelect(..)` invocation the corresponding + // `select` can still be in the registration phase, making this `trySelect(..)` bound to fail. + // In this case, the `onSend` clause will be re-registered, which alongside with the mutex + // manipulation makes the resulting solution obstruction-free. + launch { + mutex.lock() + if (!select.trySelect(this@PublisherCoroutine, Unit)) { + mutex.unlock() + } + } + } + + @Suppress("RedundantNullableReturnType", "UNUSED_PARAMETER", "UNCHECKED_CAST") + private fun processResultSelectSend(element: Any?, selectResult: Any?): Any? { + doLockedNext(element as T)?.let { throw it } + return this@PublisherCoroutine + } + + override fun trySend(element: T): ChannelResult = + if (!mutex.tryLock()) { + ChannelResult.failure() + } else { + when (val throwable = doLockedNext(element)) { + null -> ChannelResult.success(Unit) + else -> ChannelResult.closed(throwable) + } + } + + public override suspend fun send(element: T) { + mutex.lock() + doLockedNext(element)?.let { throw it } + } + + /* + * This code is not trivial because of the following properties: + * 1. It ensures conformance to the reactive specification that mandates that onXXX invocations should not + * be concurrent. It uses Mutex to protect all onXXX invocation and ensure conformance even when multiple + * coroutines are invoking `send` function. + * 2. Normally, `onComplete/onError` notification is sent only when coroutine and all its children are complete. + * However, nothing prevents `publish` coroutine from leaking reference to it send channel to some + * globally-scoped coroutine that is invoking `send` outside of this context. Without extra precaution this may + * lead to `onNext` that is concurrent with `onComplete/onError`, so that is why signalling for + * `onComplete/onError` is also done under the same mutex. + * 3. The reactive specification forbids emitting more elements than requested, so `onNext` is forbidden until the + * subscriber actually requests some elements. This is implemented by the mutex being locked when emitting + * elements is not permitted (`_nRequested.value == 0`). + */ + + /** + * Attempts to emit a value to the subscriber and, if back-pressure permits this, unlock the mutex. + * + * Requires that the caller has locked the mutex before this invocation. + * + * If the channel is closed, returns the corresponding [Throwable]; otherwise, returns `null` to denote success. + * + * @throws NullPointerException if the passed element is `null` + */ + private fun doLockedNext(elem: T): Throwable? { + if (elem == null) { + unlockAndCheckCompleted() + throw NullPointerException("Attempted to emit `null` inside a reactive publisher") + } + /** This guards against the case when the caller of this function managed to lock the mutex not because some + * elements were requested--and thus it is permitted to call `onNext`--but because the channel was closed. + * + * It may look like there is a race condition here between `isActive` and a concurrent cancellation, but it's + * okay for a cancellation to happen during `onNext`, as the reactive spec only requires that we *eventually* + * stop signalling the subscriber. */ + if (!isActive) { + unlockAndCheckCompleted() + return getCancellationException() + } + // notify the subscriber + try { + subscriber.onNext(elem) + } catch (cause: Throwable) { + /** The reactive streams spec forbids the subscribers from throwing from [Subscriber.onNext] unless the + * element is `null`, which we check not to be the case. Therefore, we report this exception to the handler + * for uncaught exceptions and consider the subscription cancelled, as mandated by + * https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.3/README.md#2.13. + * + * Some reactive implementations, like RxJava or Reactor, are known to throw from [Subscriber.onNext] if the + * execution encounters an exception they consider to be "fatal", like [VirtualMachineError] or + * [ThreadDeath]. Us using the handler for the undeliverable exceptions to signal "fatal" exceptions is + * inconsistent with RxJava and Reactor, which attempt to bubble the exception up the call chain as soon as + * possible. However, we can't do much better here, as simply throwing from all methods indiscriminately + * would violate the contracts we place on them. */ + cancelled = true + val causeDelivered = close(cause) + unlockAndCheckCompleted() + return if (causeDelivered) { + // `cause` is the reason this channel is closed + cause + } else { + // Someone else closed the channel during `onNext`. We report `cause` as an undeliverable exception. + exceptionOnCancelHandler(cause, context) + getCancellationException() + } + } + // now update nRequested + while (true) { // lock-free loop on nRequested + val current = _nRequested.value + if (current < 0) break // closed from inside onNext => unlock + if (current == Long.MAX_VALUE) break // no back-pressure => unlock + val updated = current - 1 + if (_nRequested.compareAndSet(current, updated)) { + if (updated == 0L) { + // return to keep locked due to back-pressure + return null + } + break // unlock if updated > 0 + } + } + unlockAndCheckCompleted() + return null + } + + private fun unlockAndCheckCompleted() { + /* + * There is no sense to check completion before doing `unlock`, because completion might + * happen after this check and before `unlock` (see `signalCompleted` that does not do anything + * if it fails to acquire the lock that we are still holding). + * We have to recheck `isCompleted` after `unlock` anyway. + */ + mutex.unlock() + // check isCompleted and try to regain lock to signal completion + if (isCompleted && mutex.tryLock()) { + doLockedSignalCompleted(completionCause, completionCauseHandled) + } + } + + // assert: mutex.isLocked() & isCompleted + private fun doLockedSignalCompleted(cause: Throwable?, handled: Boolean) { + try { + if (_nRequested.value == SIGNALLED) + return + _nRequested.value = SIGNALLED // we'll signal onError/onCompleted (the final state, so no CAS needed) + // Specification requires that after the cancellation is requested we eventually stop calling onXXX + if (cancelled) { + // If the parent failed to handle this exception, then we must not lose the exception + if (cause != null && !handled) exceptionOnCancelHandler(cause, context) + return + } + if (cause == null) { + try { + subscriber.onComplete() + } catch (e: Throwable) { + handleCoroutineException(context, e) + } + } else { + try { + // This can't be the cancellation exception from `cancel`, as then `cancelled` would be `true`. + subscriber.onError(cause) + } catch (e: Throwable) { + if (e !== cause) { + cause.addSuppressed(e) + } + handleCoroutineException(context, cause) + } + } + } finally { + mutex.unlock() + } + } + + override fun request(n: Long) { + if (n <= 0) { + // Specification requires to call onError with IAE for n <= 0 + cancelCoroutine(IllegalArgumentException("non-positive subscription request $n")) + return + } + while (true) { // lock-free loop for nRequested + val cur = _nRequested.value + if (cur < 0) return // already closed for send, ignore requests, as mandated by the reactive streams spec + var upd = cur + n + if (upd < 0 || n == Long.MAX_VALUE) + upd = Long.MAX_VALUE + if (cur == upd) return // nothing to do + if (_nRequested.compareAndSet(cur, upd)) { + // unlock the mutex when we don't have back-pressure anymore + if (cur == 0L) { + /** In a sense, after a successful CAS, it is this invocation, not the coroutine itself, that owns + * the lock, given that `upd` is necessarily strictly positive. Thus, no other operation has the + * right to lower the value on [_nRequested], it can only grow or become [CLOSED]. Therefore, it is + * impossible for any other operations to assume that they own the lock without actually acquiring + * it. */ + unlockAndCheckCompleted() + } + return + } + } + } + + // assert: isCompleted + private fun signalCompleted(cause: Throwable?, handled: Boolean) { + while (true) { // lock-free loop for nRequested + val current = _nRequested.value + if (current == SIGNALLED) return // some other thread holding lock already signalled cancellation/completion + check(current >= 0) // no other thread could have marked it as CLOSED, because onCompleted[Exceptionally] is invoked once + if (!_nRequested.compareAndSet(current, CLOSED)) continue // retry on failed CAS + // Ok -- marked as CLOSED, now can unlock the mutex if it was locked due to backpressure + if (current == 0L) { + doLockedSignalCompleted(cause, handled) + } else { + // otherwise mutex was either not locked or locked in concurrent onNext... try lock it to signal completion + if (mutex.tryLock()) doLockedSignalCompleted(cause, handled) + // Note: if failed `tryLock`, then `doLockedNext` will signal after performing `unlock` + } + return // done anyway + } + } + + override fun onCompleted(value: Unit) { + signalCompleted(null, false) + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + signalCompleted(cause, handled) + } + + @Suppress("OVERRIDE_DEPRECATION") // Remove after 2.2.0 + override fun cancel() { + // Specification requires that after cancellation publisher stops signalling + // This flag distinguishes subscription cancellation request from the job crash + cancelled = true + super.cancel(null) + } +} + +@Deprecated( + message = "CoroutineScope.publish is deprecated in favour of top-level publish", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("publish(context, block)") +) // Since 1.3.0, will be error in 1.3.1 and hidden in 1.4.0. Binary compatibility with Spring +public fun CoroutineScope.publish( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Publisher = publishInternal(this, context, DEFAULT_HANDLER, block) diff --git a/reactive/kotlinx-coroutines-reactive/src/ReactiveFlow.kt b/reactive/kotlinx-coroutines-reactive/src/ReactiveFlow.kt new file mode 100644 index 0000000000..e1dcf0f63e --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/src/ReactiveFlow.kt @@ -0,0 +1,269 @@ +package kotlinx.coroutines.reactive + +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.flow.internal.* +import kotlinx.coroutines.intrinsics.* +import org.reactivestreams.* +import java.util.* +import kotlin.coroutines.* +import kotlinx.coroutines.internal.* + +/** + * Transforms the given reactive [Publisher] into [Flow]. + * Use the [buffer] operator on the resulting flow to specify the size of the back-pressure. + * In effect, it specifies the value of the subscription's [request][Subscription.request]. + * The [default buffer capacity][Channel.BUFFERED] for a suspending channel is used by default. + * + * If any of the resulting flow transformations fails, the subscription is immediately cancelled and all the in-flight + * elements are discarded. + * + * This function is integrated with `ReactorContext` from `kotlinx-coroutines-reactor` module, + * see its documentation for additional details. + */ +public fun Publisher.asFlow(): Flow = + PublisherAsFlow(this) + +/** + * Transforms the given flow into a reactive specification compliant [Publisher]. + * + * This function is integrated with `ReactorContext` from `kotlinx-coroutines-reactor` module, + * see its documentation for additional details. + * + * An optional [context] can be specified to control the execution context of calls to the [Subscriber] methods. + * A [CoroutineDispatcher] can be set to confine them to a specific thread; various [ThreadContextElement] can be set to + * inject additional context into the caller thread. By default, the [Unconfined][Dispatchers.Unconfined] dispatcher + * is used, so calls are performed from an arbitrary thread. + */ +@JvmOverloads // binary compatibility +public fun Flow.asPublisher(context: CoroutineContext = EmptyCoroutineContext): Publisher = + FlowAsPublisher(this, Dispatchers.Unconfined + context) + +private class PublisherAsFlow( + private val publisher: Publisher, + context: CoroutineContext = EmptyCoroutineContext, + capacity: Int = Channel.BUFFERED, + onBufferOverflow: BufferOverflow = BufferOverflow.SUSPEND +) : ChannelFlow(context, capacity, onBufferOverflow) { + override fun create(context: CoroutineContext, capacity: Int, onBufferOverflow: BufferOverflow): ChannelFlow = + PublisherAsFlow(publisher, context, capacity, onBufferOverflow) + + /* + * The @Suppress is for Channel.CHANNEL_DEFAULT_CAPACITY. + * It's too counter-intuitive to be public, and moving it to Flow companion + * will also create undesired effect. + */ + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + private val requestSize: Long + get() = + if (onBufferOverflow != BufferOverflow.SUSPEND) { + Long.MAX_VALUE // request all, since buffering strategy is to never suspend + } else when (capacity) { + Channel.RENDEZVOUS -> 1L // need to request at least one anyway + Channel.UNLIMITED -> Long.MAX_VALUE // reactive streams way to say "give all", must be Long.MAX_VALUE + Channel.BUFFERED -> Channel.CHANNEL_DEFAULT_CAPACITY.toLong() + else -> capacity.toLong().also { check(it >= 1) } + } + + override suspend fun collect(collector: FlowCollector) { + val collectContext = coroutineContext + val newDispatcher = context[ContinuationInterceptor] + if (newDispatcher == null || newDispatcher == collectContext[ContinuationInterceptor]) { + // fast path -- subscribe directly in this dispatcher + return collectImpl(collectContext + context, collector) + } + // slow path -- produce in a separate dispatcher + collectSlowPath(collector) + } + + private suspend fun collectSlowPath(collector: FlowCollector) { + coroutineScope { + collector.emitAll(produceImpl(this + context)) + } + } + + private suspend fun collectImpl(injectContext: CoroutineContext, collector: FlowCollector) { + val subscriber = ReactiveSubscriber(capacity, onBufferOverflow, requestSize) + // inject subscribe context into publisher + publisher.injectCoroutineContext(injectContext).subscribe(subscriber) + try { + var consumed = 0L + while (true) { + val value = subscriber.takeNextOrNull() ?: break + coroutineContext.ensureActive() + collector.emit(value) + if (++consumed == requestSize) { + consumed = 0L + subscriber.makeRequest() + } + } + } finally { + subscriber.cancel() + } + } + + // The second channel here is used for produceIn/broadcastIn and slow-path (dispatcher change) + override suspend fun collectTo(scope: ProducerScope) = + collectImpl(scope.coroutineContext, SendingCollector(scope.channel)) +} + +@Suppress("ReactiveStreamsSubscriberImplementation") +private class ReactiveSubscriber( + capacity: Int, + onBufferOverflow: BufferOverflow, + private val requestSize: Long +) : Subscriber { + private lateinit var subscription: Subscription + + // This implementation of ReactiveSubscriber always uses "offer" in its onNext implementation and it cannot + // be reliable with rendezvous channel, so a rendezvous channel is replaced with buffer=1 channel + private val channel = Channel(if (capacity == Channel.RENDEZVOUS) 1 else capacity, onBufferOverflow) + + suspend fun takeNextOrNull(): T? { + val result = channel.receiveCatching() + result.exceptionOrNull()?.let { throw it } + return result.getOrElse { null } // Closed channel + } + + override fun onNext(value: T) { + // Controlled by requestSize + require(channel.trySend(value).isSuccess) { "Element $value was not added to channel because it was full, $channel" } + } + + override fun onComplete() { + channel.close() + } + + override fun onError(t: Throwable?) { + channel.close(t) + } + + override fun onSubscribe(s: Subscription) { + subscription = s + makeRequest() + } + + fun makeRequest() { + subscription.request(requestSize) + } + + fun cancel() { + subscription.cancel() + } +} + +// ContextInjector service is implemented in `kotlinx-coroutines-reactor` module only. +// If `kotlinx-coroutines-reactor` module is not included, the list is empty. +private val contextInjectors: Array = + ServiceLoader.load(ContextInjector::class.java, ContextInjector::class.java.classLoader) + .iterator().asSequence() + .toList().toTypedArray() // R8 opto + +internal fun Publisher.injectCoroutineContext(coroutineContext: CoroutineContext) = + contextInjectors.fold(this) { pub, contextInjector -> contextInjector.injectCoroutineContext(pub, coroutineContext) } + +/** + * Adapter that transforms [Flow] into TCK-complaint [Publisher]. + * [cancel] invocation cancels the original flow. + */ +@Suppress("ReactiveStreamsPublisherImplementation") +private class FlowAsPublisher( + private val flow: Flow, + private val context: CoroutineContext +) : Publisher { + override fun subscribe(subscriber: Subscriber?) { + if (subscriber == null) throw NullPointerException() + subscriber.onSubscribe(FlowSubscription(flow, subscriber, context)) + } +} + +/** @suppress */ +@InternalCoroutinesApi +public class FlowSubscription( + @JvmField public val flow: Flow, + @JvmField public val subscriber: Subscriber, + context: CoroutineContext +) : Subscription, AbstractCoroutine(context, initParentJob = false, true) { + /* + * We deliberately set initParentJob to false and do not establish parent-child + * relationship because FlowSubscription doesn't support it + */ + private val requested = atomic(0L) + private val producer = atomic?>(createInitialContinuation()) + @Volatile + private var cancellationRequested = false + + // This code wraps startCoroutineCancellable into continuation + private fun createInitialContinuation(): Continuation = Continuation(coroutineContext) { + ::flowProcessing.startCoroutineCancellable(this) + } + + private suspend fun flowProcessing() { + try { + consumeFlow() + } catch (cause: Throwable) { + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + val unwrappedCause = unwrap(cause) + if (!cancellationRequested || isActive || unwrappedCause !== getCancellationException()) { + try { + subscriber.onError(cause) + } catch (e: Throwable) { + // Last ditch report + cause.addSuppressed(e) + handleCoroutineException(coroutineContext, cause) + } + } + return + } + // We only call this if `consumeFlow()` finished successfully + try { + subscriber.onComplete() + } catch (e: Throwable) { + handleCoroutineException(coroutineContext, e) + } + } + + /* + * This method has at most one caller at any time (triggered from the `request` method) + */ + private suspend fun consumeFlow() { + flow.collect { value -> + // Emit the value + subscriber.onNext(value) + // Suspend if needed before requesting the next value + if (requested.decrementAndGet() <= 0) { + suspendCancellableCoroutine { + producer.value = it + } + } else { + // check for cancellation if we don't suspend + coroutineContext.ensureActive() + } + } + } + + @Deprecated("Since 1.2.0, binary compatibility with versions <= 1.1.x", level = DeprecationLevel.HIDDEN) + override fun cancel() { + cancellationRequested = true + cancel(null) + } + + override fun request(n: Long) { + if (n <= 0) return + val old = requested.getAndUpdate { value -> + val newValue = value + n + if (newValue <= 0L) Long.MAX_VALUE else newValue + } + if (old <= 0L) { + assert(old == 0L) + // Emitter is not started yet or has suspended -- spin on race with suspendCancellableCoroutine + while (true) { + val producer = producer.getAndSet(null) ?: continue // spin if not set yet + producer.resume(Unit) + break + } + } + } +} diff --git a/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Await.kt b/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Await.kt deleted file mode 100644 index ad3ff89b7d..0000000000 --- a/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Await.kt +++ /dev/null @@ -1,138 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactive - -import kotlinx.coroutines.experimental.CancellationException -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.suspendCancellableCoroutine -import org.reactivestreams.Publisher -import org.reactivestreams.Subscriber -import org.reactivestreams.Subscription - -/** - * Awaits for the first value from the given publisher without blocking a thread and - * returns the resulting value or throws the corresponding exception if this publisher had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * @throws NoSuchElementException if publisher does not emit any value - */ -public suspend fun Publisher.awaitFirst(): T = awaitOne(Mode.FIRST) - -/** - * Awaits for the first value from the given observable or the [default] value if none is emitted without blocking a - * thread and returns the resulting value or throws the corresponding exception if this observable had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - */ -public suspend fun Publisher.awaitFirstOrDefault(default: T): T = awaitOne(Mode.FIRST_OR_DEFAULT, default) - -/** - * Awaits for the last value from the given publisher without blocking a thread and - * returns the resulting value or throws the corresponding exception if this publisher had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * @throws NoSuchElementException if publisher does not emit any value - */ -public suspend fun Publisher.awaitLast(): T = awaitOne(Mode.LAST) - -/** - * Awaits for the single value from the given publisher without blocking a thread and - * returns the resulting value or throws the corresponding exception if this publisher had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * @throws NoSuchElementException if publisher does not emit any value - * @throws IllegalArgumentException if publisher emits more than one value - */ -public suspend fun Publisher.awaitSingle(): T = awaitOne(Mode.SINGLE) - -// ------------------------ private ------------------------ - -private enum class Mode(val s: String) { - FIRST("awaitFirst"), - FIRST_OR_DEFAULT("awaitFirstOrDefault"), - LAST("awaitLast"), - SINGLE("awaitSingle"); - override fun toString(): String = s -} - -private suspend fun Publisher.awaitOne( - mode: Mode, - default: T? = null -): T = suspendCancellableCoroutine { cont -> - subscribe(object : Subscriber { - private lateinit var subscription: Subscription - private var value: T? = null - private var seenValue = false - - override fun onSubscribe(sub: Subscription) { - subscription = sub - cont.invokeOnCompletion { sub.cancel() } - sub.request(if (mode == Mode.FIRST) 1 else Long.MAX_VALUE) - } - - override fun onNext(t: T) { - when (mode) { - Mode.FIRST, Mode.FIRST_OR_DEFAULT -> { - seenValue = true - cont.resume(t) - subscription.cancel() - } - Mode.LAST, Mode.SINGLE -> { - if (mode == Mode.SINGLE && seenValue) { - if (cont.isActive) - cont.resumeWithException(IllegalArgumentException("More that one onNext value for $mode")) - subscription.cancel() - } else { - value = t - seenValue = true - } - } - } - } - - override fun onComplete() { - if (seenValue) { - if (cont.isActive) cont.resume(value as T) - return - } - when { - mode == Mode.FIRST_OR_DEFAULT -> { - cont.resume(default as T) - } - cont.isActive -> { - cont.resumeWithException(NoSuchElementException("No value received via onNext for $mode")) - } - } - } - - override fun onError(e: Throwable) { - cont.resumeWithException(e) - } - }) -} - diff --git a/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Channel.kt b/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Channel.kt deleted file mode 100644 index 4f464c99bd..0000000000 --- a/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Channel.kt +++ /dev/null @@ -1,140 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactive - -import kotlinx.coroutines.experimental.channels.LinkedListChannel -import kotlinx.coroutines.experimental.channels.SubscriptionReceiveChannel -import org.reactivestreams.Publisher -import org.reactivestreams.Subscriber -import org.reactivestreams.Subscription -import java.util.concurrent.atomic.AtomicIntegerFieldUpdater - -/** - * Subscribes to this [Publisher] and returns a channel to receive elements emitted by it. - * The resulting channel shall be [closed][SubscriptionReceiveChannel.close] to unsubscribe from this publisher. - */ -public fun Publisher.open(): SubscriptionReceiveChannel { - val channel = SubscriptionChannel() - subscribe(channel) - return channel -} - -/** - * Subscribes to this [Publisher] and returns an iterator to receive elements emitted by it. - * - * This is a shortcut for `open().iterator()`. See [open] if you need an ability to manually - * unsubscribe from the observable. - */ - -@Suppress("DeprecatedCallableAddReplaceWith") -@Deprecated(message = - "This iteration operator for `for (x in source) { ... }` loop is deprecated, " + - "because it leaves code vulnerable to leaving unclosed subscriptions on exception. " + - "Use `source.consumeEach { x -> ... }`.") -public operator fun Publisher.iterator() = open().iterator() - -/** - * Subscribes to this [Publisher] and performs the specified action for each received element. - */ -// :todo: make it inline when this bug is fixed: https://youtrack.jetbrains.com/issue/KT-16448 -public suspend fun Publisher.consumeEach(action: suspend (T) -> Unit) { - open().use { channel -> - for (x in channel) action(x) - } -} - -private class SubscriptionChannel : LinkedListChannel(), SubscriptionReceiveChannel, Subscriber { - @Volatile - @JvmField - var subscription: Subscription? = null - - @Volatile - @JvmField - // request balance from cancelled receivers, balance is negative if we have receivers, but no subscription yet - var balance = 0 - - private companion object { - @JvmField - val BALANCE: AtomicIntegerFieldUpdater> = - AtomicIntegerFieldUpdater.newUpdater(SubscriptionChannel::class.java, "balance") - } - - // AbstractChannel overrides - override fun onEnqueuedReceive() { - loop@ while (true) { // lock-free loop on balance - val balance = this.balance - val subscription = this.subscription - if (subscription != null) { - if (balance < 0) { // receivers came before we had subscription - // try to fixup by making request - if (!BALANCE.compareAndSet(this, balance, 0)) continue@loop - subscription.request(-balance.toLong()) - return - } - if (balance == 0) { // normal story - subscription.request(1) - return - } - } - if (BALANCE.compareAndSet(this, balance, balance - 1)) return - } - } - - override fun onCancelledReceive() { - BALANCE.incrementAndGet(this) - } - - override fun afterClose(cause: Throwable?) { - subscription?.cancel() - } - - // Subscription overrides - override fun close() { - close(cause = null) - } - - // Subscriber overrides - override fun onSubscribe(s: Subscription) { - subscription = s - while (true) { // lock-free loop on balance - if (isClosedForSend) { - s.cancel() - return - } - val balance = this.balance - if (balance >= 0) return // ok -- normal story - // otherwise, receivers came before we had subscription - // try to fixup by making request - if (!BALANCE.compareAndSet(this, balance, 0)) continue - s.request(-balance.toLong()) - return - } - } - - override fun onNext(t: T) { - offer(t) - } - - override fun onComplete() { - close(cause = null) - } - - override fun onError(e: Throwable) { - close(cause = e) - } -} - diff --git a/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Convert.kt b/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Convert.kt deleted file mode 100644 index 1dbec54300..0000000000 --- a/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Convert.kt +++ /dev/null @@ -1,41 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactive - -import kotlinx.coroutines.experimental.channels.ReceiveChannel -import org.reactivestreams.Publisher -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Converts a stream of elements received from the channel to the hot reactive publisher. - * - * Every subscriber receives values from this channel in **fan-out** fashion. If the are multiple subscribers, - * they'll receive values in round-robin way. - * - * @param context -- the coroutine context from which the resulting observable is going to be signalled - */ -public fun ReceiveChannel.asPublisher(context: CoroutineContext): Publisher = publish(context) { - for (t in this@asPublisher) - send(t) -} - -/** - * @suppress **Deprecated**: Renamed to [asPublisher] - */ -@Deprecated(message = "Renamed to `asPublisher`", - replaceWith = ReplaceWith("asPublisher(context)")) -public fun ReceiveChannel.toPublisher(context: CoroutineContext): Publisher = asPublisher(context) \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Publish.kt b/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Publish.kt deleted file mode 100644 index d2dfd7d1ce..0000000000 --- a/reactive/kotlinx-coroutines-reactive/src/main/kotlin/kotlinx/coroutines/experimental/reactive/Publish.kt +++ /dev/null @@ -1,223 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactive - -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.channels.ClosedSendChannelException -import kotlinx.coroutines.experimental.channels.ProducerScope -import kotlinx.coroutines.experimental.channels.SendChannel -import kotlinx.coroutines.experimental.handleCoroutineException -import kotlinx.coroutines.experimental.newCoroutineContext -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlinx.coroutines.experimental.sync.Mutex -import org.reactivestreams.Publisher -import org.reactivestreams.Subscriber -import org.reactivestreams.Subscription -import java.util.concurrent.atomic.AtomicLongFieldUpdater -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Creates cold reactive [Publisher] that runs a given [block] in a coroutine. - * Every time the returned publisher is subscribed, it starts a new coroutine in the specified [context]. - * Coroutine emits items with `send`. Unsubscribing cancels running coroutine. - * - * Invocations of `send` are suspended appropriately when subscribers apply back-pressure and to ensure that - * `onNext` is not invoked concurrently. - * - * | **Coroutine action** | **Signal to subscriber** - * | -------------------------------------------- | ------------------------ - * | `send` | `onNext` - * | Normal completion or `close` without cause | `onComplete` - * | Failure with exception or `close` with cause | `onError` - */ -public fun publish( - context: CoroutineContext, - block: suspend ProducerScope.() -> Unit -): Publisher = Publisher { subscriber -> - val newContext = newCoroutineContext(context) - val coroutine = PublisherCoroutine(newContext, subscriber) - coroutine.initParentJob(context[Job]) - subscriber.onSubscribe(coroutine) // do it first (before starting coroutine), to avoid unnecessary suspensions - block.startCoroutine(coroutine, coroutine) -} - -private class PublisherCoroutine( - override val parentContext: CoroutineContext, - private val subscriber: Subscriber -) : AbstractCoroutine(true), ProducerScope, Subscription { - override val channel: SendChannel get() = this - - // Mutex is locked when either nRequested == 0 or while subscriber.onXXX is being invoked - private val mutex = Mutex(locked = true) - - @Volatile - private var nRequested: Long = 0 // < 0 when closed (CLOSED or SIGNALLED) - - companion object { - private val N_REQUESTED = AtomicLongFieldUpdater - .newUpdater(PublisherCoroutine::class.java, "nRequested") - - private const val CLOSED_MESSAGE = "This subscription had already closed (completed or failed)" - - private const val CLOSED = -1L // closed, but have not signalled onCompleted/onError yet - private const val SIGNALLED = -2L // already signalled subscriber onCompleted/onError - } - - override val isClosedForSend: Boolean get() = isCompleted - override val isFull: Boolean = mutex.isLocked - override fun close(cause: Throwable?): Boolean = cancel(cause) - - private fun sendException() = - (state as? CompletedExceptionally)?.cause ?: ClosedSendChannelException(CLOSED_MESSAGE) - - override fun offer(element: T): Boolean { - if (!mutex.tryLock()) return false - doLockedNext(element) - return true - } - - public suspend override fun send(element: T): Unit { - // fast-path -- try send without suspension - if (offer(element)) return - // slow-path does suspend - return sendSuspend(element) - } - - private suspend fun sendSuspend(element: T) { - mutex.lock() - doLockedNext(element) - } - - override fun registerSelectSend(select: SelectInstance, element: T, block: suspend () -> R) = - mutex.registerSelectLock(select, null) { - doLockedNext(element) - block() - } - - // assert: mutex.isLocked() - private fun doLockedNext(elem: T) { - // check if already closed for send - if (isCompleted) { - doLockedSignalCompleted() - throw sendException() - } - // notify subscriber - try { - subscriber.onNext(elem) - } catch (e: Throwable) { - try { - if (!cancel(e)) - handleCoroutineException(context, e) - } finally { - doLockedSignalCompleted() - } - throw sendException() - } - // now update nRequested - while (true) { // lock-free loop on nRequested - val cur = nRequested - if (cur < 0) break // closed from inside onNext => unlock - if (cur == Long.MAX_VALUE) break // no back-pressure => unlock - val upd = cur - 1 - if (N_REQUESTED.compareAndSet(this, cur, upd)) { - if (upd == 0L) return // return to keep locked due to back-pressure - break // unlock if upd > 0 - } - } - /* - There is no sense to check for `isCompleted` before doing `unlock`, because completion might - happen after this check and before `unlock` (see `afterCompleted` that does not do anything - if it fails to acquire the lock that we are still holding). - We have to recheck `isCompleted` after `unlock` anyway. - */ - mutex.unlock() - // recheck isCompleted - if (isCompleted && mutex.tryLock()) - doLockedSignalCompleted() - } - - // assert: mutex.isLocked() - private fun doLockedSignalCompleted() { - try { - if (nRequested >= CLOSED) { - nRequested = SIGNALLED // we'll signal onError/onCompleted (that the final state -- no CAS needed) - val state = this.state - try { - if (state is CompletedExceptionally && state.cause != null) - subscriber.onError(state.cause) - else - subscriber.onComplete() - } catch (e: Throwable) { - handleCoroutineException(context, e) - } - } - } finally { - mutex.unlock() - } - } - - override fun request(n: Long) { - if (n < 0) { - cancel(IllegalArgumentException("Must request non-negative number, but $n requested")) - return - } - while (true) { // lock-free loop for nRequested - val cur = nRequested - if (cur < 0) return // already closed for send, ignore requests - var upd = cur + n - if (upd < 0 || n == Long.MAX_VALUE) - upd = Long.MAX_VALUE - if (cur == upd) return // nothing to do - if (N_REQUESTED.compareAndSet(this, cur, upd)) { - // unlock the mutex when we don't have back-pressure anymore - if (cur == 0L) { - mutex.unlock() - // recheck isCompleted - if (isCompleted && mutex.tryLock()) - doLockedSignalCompleted() - } - return - } - } - } - - override fun afterCompletion(state: Any?, mode: Int) { - while (true) { // lock-free loop for nRequested - val cur = nRequested - if (cur == SIGNALLED) return // some other thread holding lock already signalled completion - check(cur >= 0) // no other thread could have marked it as CLOSED, because afterCompletion is invoked once - if (!N_REQUESTED.compareAndSet(this, cur, CLOSED)) continue // retry on failed CAS - // Ok -- marked as CLOSED, now can unlock the mutex if it was locked due to backpressure - if (cur == 0L) { - doLockedSignalCompleted() - } else { - // otherwise mutex was either not locked or locked in concurrent onNext... try lock it to signal completion - if (mutex.tryLock()) - doLockedSignalCompleted() - // Note: if failed `tryLock`, then `doLockedNext` will signal after performing `unlock` - } - return // done anyway - } - } - - // Subscription impl - override fun cancel() { - cancel(cause = null) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/src/module-info.java b/reactive/kotlinx-coroutines-reactive/src/module-info.java new file mode 100644 index 0000000000..67fcc26b40 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/src/module-info.java @@ -0,0 +1,10 @@ +module kotlinx.coroutines.reactive { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.atomicfu; + requires org.reactivestreams; + + exports kotlinx.coroutines.reactive; + + uses kotlinx.coroutines.reactive.ContextInjector; +} diff --git a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/IntegrationTest.kt b/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/IntegrationTest.kt deleted file mode 100644 index f80fe1c3af..0000000000 --- a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/IntegrationTest.kt +++ /dev/null @@ -1,130 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactive - -import kotlinx.coroutines.experimental.* -import org.hamcrest.MatcherAssert.assertThat -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Test -import org.junit.runner.RunWith -import org.junit.runners.Parameterized -import org.reactivestreams.Publisher -import kotlin.coroutines.experimental.CoroutineContext - -@RunWith(Parameterized::class) -class IntegrationTest( - val ctx: Ctx, - val delay: Boolean -) : TestBase() { - - enum class Ctx { - MAIN { override fun invoke(context: CoroutineContext): CoroutineContext = context }, - COMMON_POOL { override fun invoke(context: CoroutineContext): CoroutineContext = CommonPool }, - UNCONFINED { override fun invoke(context: CoroutineContext): CoroutineContext = Unconfined }; - - abstract operator fun invoke(context: CoroutineContext): CoroutineContext - } - - companion object { - @Parameterized.Parameters(name = "ctx={0}, delay={1}") - @JvmStatic - fun params(): Collection> = Ctx.values().flatMap { ctx -> - listOf(false, true).map { delay -> - arrayOf(ctx, delay) - } - } - } - - @Test - fun testEmpty(): Unit = runBlocking { - val pub = publish(ctx(context)) { - if (delay) delay(1) - // does not send anything - } - assertNSE { pub.awaitFirst() } - assertThat(pub.awaitFirstOrDefault("OK"), IsEqual("OK")) - assertNSE { pub.awaitLast() } - assertNSE { pub.awaitSingle() } - var cnt = 0 - pub.consumeEach { cnt++ } - assertThat(cnt, IsEqual(0)) - } - - @Test - fun testSingle() = runBlocking { - val pub = publish(ctx(context)) { - if (delay) delay(1) - send("OK") - } - assertThat(pub.awaitFirst(), IsEqual("OK")) - assertThat(pub.awaitFirstOrDefault("!"), IsEqual("OK")) - assertThat(pub.awaitLast(), IsEqual("OK")) - assertThat(pub.awaitSingle(), IsEqual("OK")) - var cnt = 0 - pub.consumeEach { - assertThat(it, IsEqual("OK")) - cnt++ - } - assertThat(cnt, IsEqual(1)) - } - - @Test - fun testNumbers() = runBlocking { - val n = 100 * stressTestMultiplier - val pub = publish(ctx(context)) { - for (i in 1..n) { - send(i) - if (delay) delay(1) - } - } - assertThat(pub.awaitFirst(), IsEqual(1)) - assertThat(pub.awaitFirstOrDefault(0), IsEqual(1)) - assertThat(pub.awaitLast(), IsEqual(n)) - assertIAE { pub.awaitSingle() } - checkNumbers(n, pub) - val channel = pub.open() - checkNumbers(n, channel.asPublisher(ctx(context))) - channel.close() - } - - private suspend fun checkNumbers(n: Int, pub: Publisher) { - var last = 0 - pub.consumeEach { - assertThat(it, IsEqual(++last)) - } - assertThat(last, IsEqual(n)) - } - - inline fun assertIAE(block: () -> Unit) { - try { - block() - expectUnreached() - } catch (e: Throwable) { - assertThat(e, IsInstanceOf(IllegalArgumentException::class.java)) - } - } - - inline fun assertNSE(block: () -> Unit) { - try { - block() - expectUnreached() - } catch (e: Throwable) { - assertThat(e, IsInstanceOf(NoSuchElementException::class.java)) - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublishTest.kt b/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublishTest.kt deleted file mode 100644 index efc365ff39..0000000000 --- a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublishTest.kt +++ /dev/null @@ -1,99 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactive - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert.assertThat -import org.junit.Test -import org.reactivestreams.Subscriber -import org.reactivestreams.Subscription - -class PublishTest : TestBase() { - @Test - fun testBasicEmpty() = runBlocking { - expect(1) - val publisher = publish(context) { - expect(5) - } - expect(2) - publisher.subscribe(object : Subscriber { - override fun onSubscribe(s: Subscription?) { expect(3) } - override fun onNext(t: Int?) { expectUnreached() } - override fun onComplete() { expect(6) } - override fun onError(t: Throwable?) { expectUnreached() } - }) - expect(4) - yield() // to publish coroutine - finish(7) - } - - @Test - fun testBasicSingle() = runBlocking { - expect(1) - val publisher = publish(context) { - expect(5) - send(42) - expect(7) - } - expect(2) - publisher.subscribe(object : Subscriber { - override fun onSubscribe(s: Subscription) { - expect(3) - s.request(1) - } - override fun onNext(t: Int) { - expect(6) - assertThat(t, IsEqual(42)) - } - override fun onComplete() { expect(8) } - override fun onError(t: Throwable?) { expectUnreached() } - }) - expect(4) - yield() // to publish coroutine - finish(9) - } - - @Test - fun testBasicError() = runBlocking { - expect(1) - val publisher = publish(context) { - expect(5) - throw RuntimeException("OK") - } - expect(2) - publisher.subscribe(object : Subscriber { - override fun onSubscribe(s: Subscription) { - expect(3) - s.request(1) - } - override fun onNext(t: Int) { expectUnreached() } - override fun onComplete() { expectUnreached() } - override fun onError(t: Throwable) { - expect(6) - assertThat(t, IsInstanceOf(RuntimeException::class.java)) - assertThat(t.message, IsEqual("OK")) - } - }) - expect(4) - yield() // to publish coroutine - finish(7) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublisherBackpressureTest.kt b/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublisherBackpressureTest.kt deleted file mode 100644 index b04f25828f..0000000000 --- a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublisherBackpressureTest.kt +++ /dev/null @@ -1,76 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactive - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.junit.Test -import org.reactivestreams.Subscriber -import org.reactivestreams.Subscription - -class PublisherBackpressureTest : TestBase() { - @Test - fun testCancelWhileBPSuspended() = runBlocking { - expect(1) - val observable = publish(context) { - expect(5) - send("A") // will not suspend, because an item was requested - expect(7) - send("B") // second requested item - expect(9) - try { - send("C") // will suspend (no more requested) - } finally { - expect(13) - } - expectUnreached() - } - expect(2) - var sub: Subscription? = null - observable.subscribe(object : Subscriber { - override fun onSubscribe(s: Subscription) { - sub = s - expect(3) - s.request(2) // request two items - } - - override fun onNext(t: String) { - when (t) { - "A" -> expect(6) - "B" -> expect(8) - else -> error("Should not happen") - } - } - - override fun onComplete() { - expect(11) - } - - override fun onError(e: Throwable) { - expectUnreached() - } - }) - expect(4) - yield() // yield to observable coroutine - expect(10) - sub!!.cancel() // now unsubscribe -- shall cancel coroutine & invoke onCompleted - expect(12) - yield() // shall perform finally in coroutine - finish(14) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublisherCompletionStressTest.kt b/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublisherCompletionStressTest.kt deleted file mode 100644 index a1f5fa1aed..0000000000 --- a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublisherCompletionStressTest.kt +++ /dev/null @@ -1,51 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactive - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.withTimeout -import org.junit.Test -import java.util.* -import kotlin.coroutines.experimental.CoroutineContext - -class PublisherCompletionStressTest : TestBase() { - val N_REPEATS = 10_000 * stressTestMultiplier - - fun range(context: CoroutineContext, start: Int, count: Int) = publish(context) { - for (x in start until start + count) send(x) - } - - @Test - fun testCompletion() { - val rnd = Random() - repeat(N_REPEATS) { - val count = rnd.nextInt(5) - runBlocking { - withTimeout(5000) { - var received = 0 - range(CommonPool, 1, count).consumeEach { x -> - received++ - if (x != received) error("$x != $received") - } - if (received != count) error("$received != $count") - } - } - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublisherMultiTest.kt b/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublisherMultiTest.kt deleted file mode 100644 index 954aae1b48..0000000000 --- a/reactive/kotlinx-coroutines-reactive/src/test/kotlin/kotlinx/coroutines/experimental/reactive/PublisherMultiTest.kt +++ /dev/null @@ -1,51 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactive - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import org.hamcrest.core.IsEqual -import org.junit.Assert.assertThat -import org.junit.Assert.assertTrue -import org.junit.Test - -/** - * Test emitting multiple values with [publish]. - */ -class PublisherMultiTest : TestBase() { - @Test - fun testConcurrentStress() = runBlocking { - val n = 10_000 * stressTestMultiplier - val observable = publish(CommonPool) { - // concurrent emitters (many coroutines) - val jobs = List(n) { - // launch - launch(CommonPool) { - send(it) - } - } - jobs.forEach { it.join() } - } - val resultSet = mutableSetOf() - observable.consumeEach { - assertTrue(resultSet.add(it)) - } - assertThat(resultSet.size, IsEqual(n)) - } -} diff --git a/reactive/kotlinx-coroutines-reactive/test/AwaitCancellationStressTest.kt b/reactive/kotlinx-coroutines-reactive/test/AwaitCancellationStressTest.kt new file mode 100644 index 0000000000..03676c7071 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/AwaitCancellationStressTest.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.reactivestreams.* +import java.util.concurrent.locks.* + +/** + * This test checks implementation of rule 2.7 for await methods - serial execution of subscription methods + */ +class AwaitCancellationStressTest : TestBase() { + private val iterations = 10_000 * stressTestMultiplier + + @Test + fun testAwaitCancellationOrder() = runTest { + repeat(iterations) { + val job = launch(Dispatchers.Default) { + testPublisher().awaitFirst() + } + job.cancelAndJoin() + } + } + + private fun testPublisher() = Publisher { s -> + val lock = ReentrantLock() + s.onSubscribe(object : Subscription { + override fun request(n: Long) { + check(lock.tryLock()) + s.onNext(42) + lock.unlock() + } + + override fun cancel() { + check(lock.tryLock()) + lock.unlock() + } + }) + } +} diff --git a/reactive/kotlinx-coroutines-reactive/test/AwaitTest.kt b/reactive/kotlinx-coroutines-reactive/test/AwaitTest.kt new file mode 100644 index 0000000000..227720489c --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/AwaitTest.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.reactivestreams.* + +class AwaitTest: TestBase() { + + /** Tests that calls to [awaitFirst] (and, thus, to the rest of these functions) throw [CancellationException] and + * unsubscribe from the publisher when their [Job] is cancelled. */ + @Test + fun testAwaitCancellation() = runTest { + expect(1) + val publisher = Publisher { s -> + s.onSubscribe(object: Subscription { + override fun request(n: Long) { + expect(3) + } + + override fun cancel() { + expect(5) + } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + publisher.awaitFirst() + } catch (e: CancellationException) { + expect(6) + throw e + } + } + expect(4) + job.cancelAndJoin() + finish(7) + } + +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/test/CancelledParentAttachTest.kt b/reactive/kotlinx-coroutines-reactive/test/CancelledParentAttachTest.kt new file mode 100644 index 0000000000..05bb8789bc --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/CancelledParentAttachTest.kt @@ -0,0 +1,18 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.junit.* + + +class CancelledParentAttachTest : TestBase() {; + + @Test + fun testFlow() = runTest { + val f = flowOf(1, 2, 3).cancellable() + val j = Job().also { it.cancel() } + f.asPublisher(j).asFlow().collect() + } + +} diff --git a/reactive/kotlinx-coroutines-reactive/test/FlowAsPublisherTest.kt b/reactive/kotlinx-coroutines-reactive/test/FlowAsPublisherTest.kt new file mode 100644 index 0000000000..18615e2b36 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/FlowAsPublisherTest.kt @@ -0,0 +1,165 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.flow.* +import org.junit.Test +import org.reactivestreams.* +import java.util.concurrent.* +import kotlin.test.* + +class FlowAsPublisherTest : TestBase() { + @Test + fun testErrorOnCancellationIsReported() { + expect(1) + flow { + try { + emit(2) + } finally { + expect(3) + throw TestException() + } + }.asPublisher().subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onComplete() { + expectUnreached() + } + + override fun onSubscribe(s: Subscription?) { + subscription = s!! + subscription.request(2) + } + + override fun onNext(t: Int) { + expect(t) + subscription.cancel() + } + + override fun onError(t: Throwable?) { + assertIs(t) + expect(4) + } + }) + finish(5) + } + + @Test + fun testCancellationIsNotReported() { + expect(1) + flow { + emit(2) + }.asPublisher().subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onComplete() { + expectUnreached() + } + + override fun onSubscribe(s: Subscription?) { + subscription = s!! + subscription.request(2) + } + + override fun onNext(t: Int) { + expect(t) + subscription.cancel() + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + finish(3) + } + + @Test + fun testUnconfinedDefaultContext() { + expect(1) + val thread = Thread.currentThread() + fun checkThread() { + assertSame(thread, Thread.currentThread()) + } + flowOf(42).asPublisher().subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onSubscribe(s: Subscription) { + expect(2) + subscription = s + subscription.request(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + finish(5) + } + + @Test + fun testConfinedContext() { + expect(1) + val threadName = "FlowAsPublisherTest.testConfinedContext" + fun checkThread() { + val currentThread = Thread.currentThread() + assertTrue(currentThread.name.startsWith(threadName), "Unexpected thread $currentThread") + } + val completed = CountDownLatch(1) + newSingleThreadContext(threadName).use { dispatcher -> + flowOf(42).asPublisher(dispatcher).subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onSubscribe(s: Subscription) { + expect(2) + subscription = s + subscription.request(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + completed.countDown() + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + completed.await() + } + finish(5) + } + + @Test + fun testFlowWithTimeout() = runTest { + val publisher = flow { + expect(2) + withTimeout(1) { delay(Long.MAX_VALUE) } + }.asPublisher() + try { + expect(1) + publisher.awaitFirstOrNull() + } catch (e: CancellationException) { + expect(3) + } + finish(4) + } +} diff --git a/reactive/kotlinx-coroutines-reactive/test/IntegrationTest.kt b/reactive/kotlinx-coroutines-reactive/test/IntegrationTest.kt new file mode 100644 index 0000000000..51e9562a4b --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/IntegrationTest.kt @@ -0,0 +1,225 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.testing.exceptions.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import org.reactivestreams.* +import java.lang.IllegalStateException +import java.lang.RuntimeException +import kotlin.coroutines.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class IntegrationTest( + private val ctx: Ctx, + private val delay: Boolean +) : TestBase() { + + enum class Ctx { + MAIN { override fun invoke(context: CoroutineContext): CoroutineContext = context.minusKey(Job) }, + DEFAULT { override fun invoke(context: CoroutineContext): CoroutineContext = Dispatchers.Default }, + UNCONFINED { override fun invoke(context: CoroutineContext): CoroutineContext = Dispatchers.Unconfined }; + + abstract operator fun invoke(context: CoroutineContext): CoroutineContext + } + + companion object { + @Parameterized.Parameters(name = "ctx={0}, delay={1}") + @JvmStatic + fun params(): Collection> = Ctx.values().flatMap { ctx -> + listOf(false, true).map { delay -> + arrayOf(ctx, delay) + } + } + } + + @Test + fun testEmpty(): Unit = runBlocking { + val pub = publish(ctx(coroutineContext)) { + if (delay) delay(1) + // does not send anything + } + assertFailsWith { pub.awaitFirst() } + assertEquals("OK", pub.awaitFirstOrDefault("OK")) + assertNull(pub.awaitFirstOrNull()) + assertEquals("ELSE", pub.awaitFirstOrElse { "ELSE" }) + assertFailsWith { pub.awaitLast() } + assertFailsWith { pub.awaitSingle() } + var cnt = 0 + pub.collect { cnt++ } + assertEquals(0, cnt) + } + + @Test + fun testSingle() = runBlocking { + val pub = publish(ctx(coroutineContext)) { + if (delay) delay(1) + send("OK") + } + assertEquals("OK", pub.awaitFirst()) + assertEquals("OK", pub.awaitFirstOrDefault("!")) + assertEquals("OK", pub.awaitFirstOrNull()) + assertEquals("OK", pub.awaitFirstOrElse { "ELSE" }) + assertEquals("OK", pub.awaitLast()) + assertEquals("OK", pub.awaitSingle()) + var cnt = 0 + pub.collect { + assertEquals("OK", it) + cnt++ + } + assertEquals(1, cnt) + } + + @Test + fun testCancelWithoutValue() = runTest { + val job = launch(Job(), start = CoroutineStart.UNDISPATCHED) { + publish { + hang {} + }.awaitFirst() + } + + job.cancel() + job.join() + } + + @Test + fun testEmptySingle() = runTest(unhandled = listOf { e -> e is NoSuchElementException }) { + expect(1) + val job = launch(Job(), start = CoroutineStart.UNDISPATCHED) { + publish { + yield() + expect(2) + // Nothing to emit + }.awaitFirst() + } + + job.join() + finish(3) + } + + /** + * Test that the continuation is not being resumed after it has already failed due to there having been too many + * values passed. + */ + @Test + fun testNotCompletingFailedAwait() = runTest { + try { + expect(1) + Publisher { sub -> + sub.onSubscribe(object: Subscription { + override fun request(n: Long) { + expect(2) + sub.onNext(1) + sub.onNext(2) + expect(4) + sub.onComplete() + } + + override fun cancel() { + expect(3) + } + }) + }.awaitSingle() + } catch (e: java.lang.IllegalArgumentException) { + expect(5) + } + finish(6) + } + + /** + * Test the behavior of [awaitOne] on unconforming publishers. + */ + @Test + fun testAwaitOnNonconformingPublishers() = runTest { + fun publisher(block: Subscriber.(n: Long) -> Unit) = + Publisher { subscriber -> + subscriber.onSubscribe(object: Subscription { + override fun request(n: Long) { + subscriber.block(n) + } + + override fun cancel() { + } + }) + } + val dummyMessage = "dummy" + val dummyThrowable = RuntimeException(dummyMessage) + suspend fun assertDetectsBadPublisher( + operation: suspend Publisher.() -> T, + message: String, + block: Subscriber.(n: Long) -> Unit, + ) { + assertCallsExceptionHandlerWith { + try { + publisher(block).operation() + } catch (e: Throwable) { + if (e.message != dummyMessage) + throw e + } + }.let { + assertTrue("Expected the message to contain '$message', got '${it.message}'") { + it.message?.contains(message) ?: false + } + } + } + + // Rule 1.1 broken: the publisher produces more values than requested. + assertDetectsBadPublisher({ awaitFirst() }, "provided more") { + onNext(1) + onNext(2) + onComplete() + } + + // Rule 1.7 broken: the publisher calls a method on a subscriber after reaching the terminal state. + assertDetectsBadPublisher({ awaitSingle() }, "terminal state") { + onNext(1) + onError(dummyThrowable) + onComplete() + } + assertDetectsBadPublisher({ awaitFirst() }, "terminal state") { + onNext(0) + onComplete() + onComplete() + } + assertDetectsBadPublisher({ awaitFirstOrDefault(1) }, "terminal state") { + onComplete() + onNext(3) + } + assertDetectsBadPublisher({ awaitSingle() }, "terminal state") { + onError(dummyThrowable) + onNext(3) + } + + // Rule 1.9 broken (the first signal to the subscriber was not 'onSubscribe') + assertCallsExceptionHandlerWith { + try { + Publisher { subscriber -> + subscriber.onNext(3) + subscriber.onComplete() + }.awaitFirst() + } catch (e: NoSuchElementException) { + // intentionally blank + } + }.let { assertTrue(it.message?.contains("onSubscribe") ?: false) } + } + + @Test + fun testPublishWithTimeout() = runTest { + val publisher = publish { + expect(2) + withTimeout(1) { delay(100) } + } + try { + expect(1) + publisher.awaitFirstOrNull() + } catch (e: CancellationException) { + expect(3) + } + finish(4) + } + +} + diff --git a/reactive/kotlinx-coroutines-reactive/test/IterableFlowTckTest.kt b/reactive/kotlinx-coroutines-reactive/test/IterableFlowTckTest.kt new file mode 100644 index 0000000000..13313d2909 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/IterableFlowTckTest.kt @@ -0,0 +1,124 @@ +@file:Suppress("UNCHECKED_CAST") + +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.flow.* +import org.junit.Ignore +import org.junit.Test +import org.reactivestreams.* +import org.reactivestreams.tck.* +import java.util.concurrent.* +import java.util.concurrent.ForkJoinPool.* +import kotlin.test.* + +class IterableFlowTckTest : PublisherVerification(TestEnvironment()) { + + private fun generate(num: Long): Array { + return Array(if (num >= Integer.MAX_VALUE) 1000000 else num.toInt()) { it.toLong() } + } + + override fun createPublisher(elements: Long): Publisher { + return generate(elements).asIterable().asFlow().asPublisher() + } + + @Suppress("SubscriberImplementation") + override fun createFailedPublisher(): Publisher? { + /* + * This is a hack for our adapter structure: + * Tests assume that calling "collect" is enough for publisher to fail and it is not + * true for our implementation + */ + val pub = { error(42) }.asFlow().asPublisher() + return Publisher { subscriber -> + pub.subscribe(object : Subscriber by subscriber as Subscriber { + override fun onSubscribe(s: Subscription) { + subscriber.onSubscribe(s) + s.request(1) + } + }) + } + } + + @Test + fun testStackOverflowTrampoline() { + val latch = CountDownLatch(1) + val collected = ArrayList() + val toRequest = 1000L + val array = generate(toRequest) + val publisher = array.asIterable().asFlow().asPublisher() + + publisher.subscribe(object : Subscriber { + private lateinit var s: Subscription + + override fun onSubscribe(s: Subscription) { + this.s = s + s.request(1) + } + + override fun onNext(aLong: Long) { + collected.add(aLong) + + s.request(1) + } + + override fun onError(t: Throwable) { + + } + + override fun onComplete() { + latch.countDown() + } + }) + + latch.await(5, TimeUnit.SECONDS) + assertEquals(collected, array.toList()) + } + + @Test + fun testConcurrentRequest() { + val latch = CountDownLatch(1) + val collected = ArrayList() + val n = 50000L + val array = generate(n) + val publisher = array.asIterable().asFlow().asPublisher() + + publisher.subscribe(object : Subscriber { + private var s: Subscription? = null + + override fun onSubscribe(s: Subscription) { + this.s = s + for (i in 0..n) { + commonPool().execute { s.request(1) } + } + } + + override fun onNext(aLong: Long) { + collected.add(aLong) + } + + override fun onError(t: Throwable) { + + } + + override fun onComplete() { + latch.countDown() + } + }) + + latch.await() + assertEquals(array.toList(), collected) + } + + @Ignore + override fun required_spec309_requestZeroMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec309_requestNegativeNumberMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec312_cancelMustMakeThePublisherToEventuallyStopSignaling() { + // This test has a bug in it + } +} diff --git a/reactive/kotlinx-coroutines-reactive/test/PublishTest.kt b/reactive/kotlinx-coroutines-reactive/test/PublishTest.kt new file mode 100644 index 0000000000..df01e471e5 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/PublishTest.kt @@ -0,0 +1,324 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.sync.* +import kotlinx.coroutines.testing.exceptions.* +import org.junit.Test +import org.reactivestreams.* +import java.util.concurrent.* +import kotlin.test.* + +class PublishTest : TestBase() { + @Test + fun testBasicEmpty() = runTest { + expect(1) + val publisher = publish(currentDispatcher()) { + expect(5) + } + expect(2) + publisher.subscribe(object : Subscriber { + override fun onSubscribe(s: Subscription?) { expect(3) } + override fun onNext(t: Int?) { expectUnreached() } + override fun onComplete() { expect(6) } + override fun onError(t: Throwable?) { expectUnreached() } + }) + expect(4) + yield() // to publish coroutine + finish(7) + } + + @Test + fun testBasicSingle() = runTest { + expect(1) + val publisher = publish(currentDispatcher()) { + expect(5) + send(42) + expect(7) + } + expect(2) + publisher.subscribe(object : Subscriber { + override fun onSubscribe(s: Subscription) { + expect(3) + s.request(1) + } + override fun onNext(t: Int) { + expect(6) + assertEquals(42, t) + } + override fun onComplete() { expect(8) } + override fun onError(t: Throwable?) { expectUnreached() } + }) + expect(4) + yield() // to publish coroutine + finish(9) + } + + @Test + fun testBasicError() = runTest { + expect(1) + val publisher = publish(currentDispatcher()) { + expect(5) + throw RuntimeException("OK") + } + expect(2) + publisher.subscribe(object : Subscriber { + override fun onSubscribe(s: Subscription) { + expect(3) + s.request(1) + } + override fun onNext(t: Int) { expectUnreached() } + override fun onComplete() { expectUnreached() } + override fun onError(t: Throwable) { + expect(6) + assertIs(t) + assertEquals("OK", t.message) + } + }) + expect(4) + yield() // to publish coroutine + finish(7) + } + + @Test + fun testHandleFailureAfterCancel() = runTest { + expect(1) + + val eh = CoroutineExceptionHandler { _, t -> + assertIs(t) + expect(6) + } + val publisher = publish(Dispatchers.Unconfined + eh) { + try { + expect(3) + delay(10000) + } finally { + expect(5) + throw RuntimeException("FAILED") // crash after cancel + } + } + var sub: Subscription? = null + publisher.subscribe(object : Subscriber { + override fun onComplete() { + expectUnreached() + } + + override fun onSubscribe(s: Subscription) { + expect(2) + sub = s + } + + override fun onNext(t: Unit?) { + expectUnreached() + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + expect(4) + sub!!.cancel() + finish(7) + } + + /** Tests that, as soon as `ProducerScope.close` is called, `isClosedForSend` starts returning `true`. */ + @Test + fun testChannelClosing() = runTest { + expect(1) + val publisher = publish(Dispatchers.Unconfined) { + expect(3) + close() + assert(isClosedForSend) + expect(4) + } + try { + expect(2) + publisher.awaitFirstOrNull() + } catch (e: CancellationException) { + expect(5) + } + finish(6) + } + + @Test + fun testOnNextError() = runTest { + val latch = CompletableDeferred() + expect(1) + assertCallsExceptionHandlerWith { exceptionHandler -> + val publisher = publish(currentDispatcher() + exceptionHandler) { + expect(4) + try { + send("OK") + } catch (e: Throwable) { + expect(6) + assert(e is TestException) + assert(isClosedForSend) + latch.complete(Unit) + } + } + expect(2) + publisher.subscribe(object : Subscriber { + override fun onComplete() { + expectUnreached() + } + + override fun onSubscribe(s: Subscription) { + expect(3) + s.request(1) + } + + override fun onNext(t: String) { + expect(5) + assertEquals("OK", t) + throw TestException() + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + latch.await() + } + finish(7) + } + + /** Tests the behavior when a call to `onNext` fails after the channel is already closed. */ + @Test + fun testOnNextErrorAfterCancellation() = runTest { + assertCallsExceptionHandlerWith { handler -> + var producerScope: ProducerScope? = null + CompletableDeferred() + expect(1) + var job: Job? = null + val publisher = publish(handler + Dispatchers.Unconfined) { + producerScope = this + expect(4) + job = launch { + delay(Long.MAX_VALUE) + } + } + expect(2) + publisher.subscribe(object : Subscriber { + override fun onSubscribe(s: Subscription) { + expect(3) + s.request(Long.MAX_VALUE) + } + + override fun onNext(t: Int) { + expect(6) + assertEquals(1, t) + job!!.cancel() + throw TestException() + } + + override fun onError(t: Throwable?) { + /* Correct changes to the implementation could lead to us entering or not entering this method, but + it only matters that if we do, it is the "correct" exception that was validly used to cancel the + coroutine that gets passed here and not `TestException`. */ + assertIs(t) + } + + override fun onComplete() { + expectUnreached() + } + }) + expect(5) + val result: ChannelResult = producerScope!!.trySend(1) + val e = result.exceptionOrNull()!! + assertIs(e, "The actual error: $e") + assertTrue(producerScope!!.isClosedForSend) + assertTrue(result.isFailure) + } + finish(7) + } + + @Test + fun testFailingConsumer() = runTest { + val pub = publish(currentDispatcher()) { + repeat(3) { + expect(it + 1) // expect(1), expect(2) *should* be invoked + send(it) + } + } + try { + pub.collect { + throw TestException() + } + } catch (e: TestException) { + finish(3) + } + } + + @Test + fun testIllegalArgumentException() { + assertFailsWith { publish(Job()) { } } + } + + /** Tests that `trySend` doesn't throw in `publish`. */ + @Test + fun testTrySendNotThrowing() = runTest { + var producerScope: ProducerScope? = null + expect(1) + val publisher = publish(Dispatchers.Unconfined) { + producerScope = this + expect(3) + delay(Long.MAX_VALUE) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + publisher.awaitFirstOrNull() + expectUnreached() + } + job.cancel() + expect(4) + val result = producerScope!!.trySend(1) + assertTrue(result.isFailure) + finish(5) + } + + /** Tests that all methods on `publish` fail without closing the channel when attempting to emit `null`. */ + @Test + fun testEmittingNull() = runTest { + val publisher = publish { + assertFailsWith { send(null) } + assertFailsWith { trySend(null) } + send("OK") + } + assertEquals("OK", publisher.awaitFirstOrNull()) + } + + @Test + fun testOnSendCancelled() = runTest { + val latch = CountDownLatch(1) + val published = publish(Dispatchers.Default) { + expect(2) + // Collector is ready + send(1) + try { + send(2) + expectUnreached() + } catch (e: CancellationException) { + // publisher cancellation is async + latch.countDown() + throw e + } + } + + expect(1) + val collectorLatch = Mutex(true) + val job = launch { + published.asFlow().buffer(0).collect { + collectorLatch.unlock() + hang { expect(4) } + } + } + collectorLatch.lock() + expect(3) + job.cancelAndJoin() + latch.await() + finish(5) + } +} diff --git a/reactive/kotlinx-coroutines-reactive/test/PublisherAsFlowTest.kt b/reactive/kotlinx-coroutines-reactive/test/PublisherAsFlowTest.kt new file mode 100644 index 0000000000..42dd6817a2 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/PublisherAsFlowTest.kt @@ -0,0 +1,274 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.testing.flow.* +import org.reactivestreams.* +import kotlin.test.* + +class PublisherAsFlowTest : TestBase() { + @Test + fun testCancellation() = runTest { + var onNext = 0 + var onCancelled = 0 + var onError = 0 + + val publisher = publish(currentDispatcher()) { + coroutineContext[Job]?.invokeOnCompletion { + if (it is CancellationException) ++onCancelled + } + + repeat(100) { + send(it) + } + } + + publisher.asFlow().launchIn(CoroutineScope(Dispatchers.Unconfined)) { + onEach { + ++onNext + throw RuntimeException() + } + catch { + ++onError + } + }.join() + + + assertEquals(1, onNext) + assertEquals(1, onError) + assertEquals(1, onCancelled) + } + + @Test + fun testBufferSize1() = runTest { + val publisher = publish(currentDispatcher()) { + expect(1) + send(3) + + expect(2) + send(5) + + expect(4) + send(7) + expect(6) + } + + publisher.asFlow().buffer(1).collect { + expect(it) + } + + finish(8) + } + + @Test + fun testBufferSizeDefault() = runTest { + val publisher = publish(currentDispatcher()) { + repeat(64) { + send(it + 1) + expect(it + 1) + } + assertFalse { trySend(-1).isSuccess } + } + + publisher.asFlow().collect { + expect(64 + it) + } + + finish(129) + } + + @Test + fun testDefaultCapacityIsProperlyOverwritten() = runTest { + val publisher = publish(currentDispatcher()) { + expect(1) + send(3) + expect(2) + send(5) + expect(4) + send(7) + expect(6) + } + + publisher.asFlow().flowOn(wrapperDispatcher()).buffer(1).collect { + expect(it) + } + + finish(8) + } + + @Test + fun testBufferSize10() = runTest { + val publisher = publish(currentDispatcher()) { + expect(1) + send(5) + + expect(2) + send(6) + + expect(3) + send(7) + expect(4) + } + + publisher.asFlow().buffer(10).collect { + expect(it) + } + + finish(8) + } + + @Test + fun testConflated() = runTest { + val publisher = publish(currentDispatcher()) { + for (i in 1..5) send(i) + } + val list = publisher.asFlow().conflate().toList() + assertEquals(listOf(1, 5), list) + } + + @Test + fun testProduce() = runTest { + val flow = publish(currentDispatcher()) { repeat(10) { send(it) } }.asFlow() + check((0..9).toList(), flow.produceIn(this)) + check((0..9).toList(), flow.buffer(2).produceIn(this)) + check((0..9).toList(), flow.buffer(Channel.UNLIMITED).produceIn(this)) + check(listOf(0, 9), flow.conflate().produceIn(this)) + } + + private suspend fun check(expected: List, channel: ReceiveChannel) { + val result = ArrayList(10) + channel.consumeEach { result.add(it) } + assertEquals(expected, result) + } + + @Test + fun testProduceCancellation() = runTest { + expect(1) + // publisher is an async coroutine, so it overproduces to the channel, but still gets cancelled + val flow = publish(currentDispatcher()) { + expect(3) + repeat(10) { value -> + when (value) { + in 0..6 -> send(value) + 7 -> try { + send(value) + } catch (e: CancellationException) { + expect(5) + throw e + } + else -> expectUnreached() + } + } + }.asFlow().buffer(1) + assertFailsWith { + coroutineScope { + expect(2) + val channel = flow.produceIn(this) + channel.consumeEach { value -> + when (value) { + in 0..4 -> {} + 5 -> { + expect(4) + throw TestException() + } + else -> expectUnreached() + } + } + } + } + finish(6) + } + + @Test + fun testRequestRendezvous() = + testRequestSizeWithBuffer(Channel.RENDEZVOUS, BufferOverflow.SUSPEND, 1) + + @Test + fun testRequestBuffer1() = + testRequestSizeWithBuffer(1, BufferOverflow.SUSPEND, 1) + + @Test + fun testRequestBuffer10() = + testRequestSizeWithBuffer(10, BufferOverflow.SUSPEND, 10) + + @Test + fun testRequestBufferUnlimited() = + testRequestSizeWithBuffer(Channel.UNLIMITED, BufferOverflow.SUSPEND, Long.MAX_VALUE) + + @Test + fun testRequestBufferOverflowSuspend() = + testRequestSizeWithBuffer(Channel.BUFFERED, BufferOverflow.SUSPEND, 64) + + @Test + fun testRequestBufferOverflowDropOldest() = + testRequestSizeWithBuffer(Channel.BUFFERED, BufferOverflow.DROP_OLDEST, Long.MAX_VALUE) + + @Test + fun testRequestBufferOverflowDropLatest() = + testRequestSizeWithBuffer(Channel.BUFFERED, BufferOverflow.DROP_LATEST, Long.MAX_VALUE) + + @Test + fun testRequestBuffer10OverflowDropOldest() = + testRequestSizeWithBuffer(10, BufferOverflow.DROP_OLDEST, Long.MAX_VALUE) + + @Test + fun testRequestBuffer10OverflowDropLatest() = + testRequestSizeWithBuffer(10, BufferOverflow.DROP_LATEST, Long.MAX_VALUE) + + /** + * Tests `publisher.asFlow.buffer(...)` chain, verifying expected requests size and that only expected + * values are delivered. + */ + private fun testRequestSizeWithBuffer( + capacity: Int, + onBufferOverflow: BufferOverflow, + expectedRequestSize: Long + ) = runTest { + val m = 50 + // publishers numbers from 1 to m + val publisher = Publisher { s -> + s.onSubscribe(object : Subscription { + var lastSent = 0 + var remaining = 0L + override fun request(n: Long) { + assertEquals(expectedRequestSize, n) + remaining += n + check(remaining >= 0) + while (lastSent < m && remaining > 0) { + s.onNext(++lastSent) + remaining-- + } + if (lastSent == m) s.onComplete() + } + + override fun cancel() {} + }) + } + val flow = publisher + .asFlow() + .buffer(capacity, onBufferOverflow) + val list = flow.toList() + val runSize = if (capacity == Channel.BUFFERED) 1 else capacity + val expected = when (onBufferOverflow) { + // Everything is expected to be delivered + BufferOverflow.SUSPEND -> (1..m).toList() + // Only the last one (by default) or the last "capacity" items delivered + BufferOverflow.DROP_OLDEST -> (m - runSize + 1..m).toList() + // Only the first one (by default) or the first "capacity" items delivered + BufferOverflow.DROP_LATEST -> (1..runSize).toList() + } + assertEquals(expected, list) + } + + @Test + fun testException() = runTest { + expect(1) + val p = publish { throw TestException() }.asFlow() + p.catch { + assertTrue { it is TestException } + finish(2) + }.collect() + } +} diff --git a/reactive/kotlinx-coroutines-reactive/test/PublisherBackpressureTest.kt b/reactive/kotlinx-coroutines-reactive/test/PublisherBackpressureTest.kt new file mode 100644 index 0000000000..8b8d0fc7a1 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/PublisherBackpressureTest.kt @@ -0,0 +1,58 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import org.reactivestreams.* + +class PublisherBackpressureTest : TestBase() { + @Test + fun testCancelWhileBPSuspended() = runBlocking { + expect(1) + val observable = publish(currentDispatcher()) { + expect(5) + send("A") // will not suspend, because an item was requested + expect(7) + send("B") // second requested item + expect(9) + try { + send("C") // will suspend (no more requested) + } finally { + expect(12) + } + expectUnreached() + } + expect(2) + var sub: Subscription? = null + observable.subscribe(object : Subscriber { + override fun onSubscribe(s: Subscription) { + sub = s + expect(3) + s.request(2) // request two items + } + + override fun onNext(t: String) { + when (t) { + "A" -> expect(6) + "B" -> expect(8) + else -> error("Should not happen") + } + } + + override fun onComplete() { + expectUnreached() + } + + override fun onError(e: Throwable) { + expectUnreached() + } + }) + expect(4) + yield() // yield to observable coroutine + expect(10) + sub!!.cancel() // now unsubscribe -- shall cancel coroutine (& do not signal) + expect(11) + yield() // shall perform finally in coroutine + finish(13) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/test/PublisherCollectTest.kt b/reactive/kotlinx-coroutines-reactive/test/PublisherCollectTest.kt new file mode 100644 index 0000000000..a7f94dfe98 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/PublisherCollectTest.kt @@ -0,0 +1,141 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import org.reactivestreams.* +import kotlin.test.* + +class PublisherCollectTest: TestBase() { + + /** Tests the simple scenario where the publisher outputs a bounded stream of values to collect. */ + @Test + fun testCollect() = runTest { + val x = 100 + val xSum = x * (x + 1) / 2 + val publisher = Publisher { subscriber -> + var requested = 0L + var lastOutput = 0 + subscriber.onSubscribe(object: Subscription { + + override fun request(n: Long) { + requested += n + if (n <= 0) { + subscriber.onError(IllegalArgumentException()) + return + } + while (lastOutput < x && lastOutput < requested) { + lastOutput += 1 + subscriber.onNext(lastOutput) + } + if (lastOutput == x) + subscriber.onComplete() + } + + override fun cancel() { + /** According to rule 3.5 of the + * [reactive spec](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.3/README.md#3.5), + * this method can be called by the subscriber at any point, so it's not an error if it's called + * in this scenario. */ + } + + }) + } + var sum = 0 + publisher.collect { + sum += it + } + assertEquals(xSum, sum) + } + + /** Tests the behavior of [collect] when the publisher raises an error. */ + @Test + fun testCollectThrowingPublisher() = runTest { + val errorString = "Too many elements requested" + val x = 100 + val xSum = x * (x + 1) / 2 + val publisher = Publisher { subscriber -> + var requested = 0L + var lastOutput = 0 + subscriber.onSubscribe(object: Subscription { + + override fun request(n: Long) { + requested += n + if (n <= 0) { + subscriber.onError(IllegalArgumentException()) + return + } + while (lastOutput < x && lastOutput < requested) { + lastOutput += 1 + subscriber.onNext(lastOutput) + } + if (lastOutput == x) + subscriber.onError(IllegalArgumentException(errorString)) + } + + override fun cancel() { + /** See the comment for the corresponding part of [testCollect]. */ + } + + }) + } + var sum = 0 + try { + publisher.collect { + sum += it + } + } catch (e: IllegalArgumentException) { + assertEquals(errorString, e.message) + } + assertEquals(xSum, sum) + } + + /** Tests the behavior of [collect] when the action throws. */ + @Test + fun testCollectThrowingAction() = runTest { + val errorString = "Too many elements produced" + val x = 100 + val xSum = x * (x + 1) / 2 + val publisher = Publisher { subscriber -> + var requested = 0L + var lastOutput = 0 + subscriber.onSubscribe(object: Subscription { + + override fun request(n: Long) { + requested += n + if (n <= 0) { + subscriber.onError(IllegalArgumentException()) + return + } + while (lastOutput < x && lastOutput < requested) { + lastOutput += 1 + subscriber.onNext(lastOutput) + } + } + + override fun cancel() { + assertEquals(x, lastOutput) + expect(x + 2) + } + + }) + } + var sum = 0 + try { + expect(1) + var i = 1 + publisher.collect { + sum += it + i += 1 + expect(i) + if (sum >= xSum) { + throw IllegalArgumentException(errorString) + } + } + } catch (e: IllegalArgumentException) { + expect(x + 3) + assertEquals(errorString, e.message) + } + finish(x + 4) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/test/PublisherCompletionStressTest.kt b/reactive/kotlinx-coroutines-reactive/test/PublisherCompletionStressTest.kt new file mode 100644 index 0000000000..f3ea685459 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/PublisherCompletionStressTest.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.* +import kotlin.coroutines.* + +class PublisherCompletionStressTest : TestBase() { + private val N_REPEATS = 10_000 * stressTestMultiplier + + private fun CoroutineScope.range(context: CoroutineContext, start: Int, count: Int) = publish(context) { + for (x in start until start + count) send(x) + } + + @Test + fun testCompletion() { + val rnd = Random() + repeat(N_REPEATS) { + val count = rnd.nextInt(5) + runBlocking { + withTimeout(5000) { + var received = 0 + range(Dispatchers.Default, 1, count).collect { x -> + received++ + if (x != received) error("$x != $received") + } + if (received != count) error("$received != $count") + } + } + } + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/test/PublisherMultiTest.kt b/reactive/kotlinx-coroutines-reactive/test/PublisherMultiTest.kt new file mode 100644 index 0000000000..2ac658d0ea --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/PublisherMultiTest.kt @@ -0,0 +1,51 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.Test +import kotlin.test.* + +class PublisherMultiTest : TestBase() { + @Test + fun testConcurrentStress() = runBlocking { + val n = 10_000 * stressTestMultiplier + val observable = publish { + // concurrent emitters (many coroutines) + val jobs = List(n) { + // launch + launch(Dispatchers.Default) { + send(it) + } + } + jobs.forEach { it.join() } + } + val resultSet = mutableSetOf() + observable.collect { + assertTrue(resultSet.add(it)) + } + assertEquals(n, resultSet.size) + } + + @Test + fun testConcurrentStressOnSend() = runBlocking { + val n = 10_000 * stressTestMultiplier + val observable = publish { + // concurrent emitters (many coroutines) + val jobs = List(n) { + // launch + launch(Dispatchers.Default) { + select { + onSend(it) {} + } + } + } + jobs.forEach { it.join() } + } + val resultSet = mutableSetOf() + observable.collect { + assertTrue(resultSet.add(it)) + } + assertEquals(n, resultSet.size) + } +} diff --git a/reactive/kotlinx-coroutines-reactive/test/PublisherRequestStressTest.kt b/reactive/kotlinx-coroutines-reactive/test/PublisherRequestStressTest.kt new file mode 100644 index 0000000000..18ab7c31e5 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/PublisherRequestStressTest.kt @@ -0,0 +1,140 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.flow.Flow +import org.junit.* +import org.reactivestreams.* +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import kotlin.coroutines.* +import kotlin.random.* + +/** + * This stress-test is self-contained reproducer for the race in [Flow.asPublisher] extension + * that was originally reported in the issue + * [#2109](https://github.com/Kotlin/kotlinx.coroutines/issues/2109). + * The original reproducer used a flow that loads a file using AsynchronousFileChannel + * (that issues completion callbacks from multiple threads) + * and uploads it to S3 via Amazon SDK, which internally uses netty for I/O + * (which uses a single thread for connection-related callbacks). + * + * This stress-test essentially mimics the logic in multiple interacting threads: several emitter threads that form + * the flow and a single requesting thread works on the subscriber's side to periodically request more + * values when the number of items requested drops below the threshold. + */ +@Suppress("ReactiveStreamsSubscriberImplementation") +class PublisherRequestStressTest : TestBase() { + + private val testDurationSec = 3 * stressTestMultiplier + + // Original code in Amazon SDK uses 4 and 16 as low/high watermarks. + // These constants were chosen so that problem reproduces asap with particular this code. + private val minDemand = 8L + private val maxDemand = 16L + + private val nEmitThreads = 4 + + private val emitThreadNo = AtomicInteger() + + private val emitPool = Executors.newFixedThreadPool(nEmitThreads) { r -> + Thread(r, "PublisherRequestStressTest-emit-${emitThreadNo.incrementAndGet()}") + } + + private val reqPool = Executors.newSingleThreadExecutor { r -> + Thread(r, "PublisherRequestStressTest-req") + } + + private val nextValue = AtomicLong(0) + + @After + fun tearDown() { + emitPool.shutdown() + reqPool.shutdown() + emitPool.awaitTermination(10, TimeUnit.SECONDS) + reqPool.awaitTermination(10, TimeUnit.SECONDS) + } + + private lateinit var subscription: Subscription + + @Test + fun testRequestStress() { + val expectedValue = AtomicLong(0) + val requestedTill = AtomicLong(0) + val callingOnNext = AtomicInteger() + + val publisher = mtFlow().asPublisher() + var error = false + + publisher.subscribe(object : Subscriber { + private var demand = 0L // only updated from reqPool + + override fun onComplete() { + // Typically unreached, but, rarely, `emitPool` may shut down before the cancellation is performed. + } + + override fun onSubscribe(sub: Subscription) { + subscription = sub + maybeRequestMore() + } + + private fun maybeRequestMore() { + if (demand >= minDemand) return + val nextDemand = Random.nextLong(minDemand + 1..maxDemand) + val more = nextDemand - demand + demand = nextDemand + requestedTill.addAndGet(more) + subscription.request(more) + } + + override fun onNext(value: Long) { + check(callingOnNext.getAndIncrement() == 0) // make sure it is not concurrent + // check for expected value + check(value == expectedValue.get()) + // check that it does not exceed requested values + check(value < requestedTill.get()) + val nextExpected = value + 1 + expectedValue.set(nextExpected) + // send more requests from request thread + reqPool.execute { + demand-- // processed an item + maybeRequestMore() + } + callingOnNext.decrementAndGet() + } + + override fun onError(ex: Throwable?) { + error = true + error("Failed", ex) + } + }) + var prevExpected = -1L + for (second in 1..testDurationSec) { + if (error) break + Thread.sleep(1000) + val expected = expectedValue.get() + println("$second: expectedValue = $expected") + check(expected > prevExpected) // should have progress + prevExpected = expected + } + if (!error) { + subscription.cancel() + runBlocking { + (subscription as AbstractCoroutine<*>).join() + } + } + } + + private fun mtFlow(): Flow = flow { + while (currentCoroutineContext().isActive) { + emit(aWait()) + } + } + + private suspend fun aWait(): Long = suspendCancellableCoroutine { cont -> + emitPool.execute(Runnable { + cont.resume(nextValue.getAndIncrement()) + }) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/test/PublisherSubscriptionSelectTest.kt b/reactive/kotlinx-coroutines-reactive/test/PublisherSubscriptionSelectTest.kt new file mode 100644 index 0000000000..ab22d12c05 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/PublisherSubscriptionSelectTest.kt @@ -0,0 +1,59 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class PublisherSubscriptionSelectTest(private val request: Int) : TestBase() { + companion object { + @Parameterized.Parameters(name = "request = {0}") + @JvmStatic + fun params(): Collection> = listOf(0, 1, 10).map { arrayOf(it) } + } + + @Test + fun testSelect() = runTest { + // source with n ints + val n = 1000 * stressTestMultiplier + val source = publish { repeat(n) { send(it) } } + var a = 0 + var b = 0 + // open two subs + val channelA = source.toChannel(request) + val channelB = source.toChannel(request) + loop@ while (true) { + val done: Int = select { + channelA.onReceiveCatching { result -> + result.onSuccess { assertEquals(a++, it) } + if (result.isSuccess) 1 else 0 + } + channelB.onReceiveCatching { result -> + result.onSuccess { assertEquals(b++, it) } + if (result.isSuccess) 2 else 0 + } + } + when (done) { + 0 -> break@loop + 1 -> { + val r = channelB.receiveCatching().getOrNull() + if (r != null) assertEquals(b++, r) + } + 2 -> { + val r = channelA.receiveCatching().getOrNull() + if (r != null) assertEquals(a++, r) + } + } + } + + channelA.cancel() + channelB.cancel() + // should receive one of them fully + assertTrue(a == n || b == n) + } +} diff --git a/reactive/kotlinx-coroutines-reactive/test/RangePublisherBufferedTest.kt b/reactive/kotlinx-coroutines-reactive/test/RangePublisherBufferedTest.kt new file mode 100644 index 0000000000..da1f3cced9 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/RangePublisherBufferedTest.kt @@ -0,0 +1,27 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.flow.* +import org.junit.* +import org.reactivestreams.* +import org.reactivestreams.example.unicast.* +import org.reactivestreams.tck.* + +class RangePublisherBufferedTest : + PublisherVerification(TestEnvironment(50, 50)) +{ + override fun createPublisher(elements: Long): Publisher { + return RangePublisher(1, elements.toInt()).asFlow().buffer(2).asPublisher() + } + + override fun createFailedPublisher(): Publisher? { + return null + } + + @Ignore + override fun required_spec309_requestZeroMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec309_requestNegativeNumberMustSignalIllegalArgumentException() { + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/test/RangePublisherTest.kt b/reactive/kotlinx-coroutines-reactive/test/RangePublisherTest.kt new file mode 100644 index 0000000000..9d1f85a2fa --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/RangePublisherTest.kt @@ -0,0 +1,44 @@ +package kotlinx.coroutines.reactive + +import org.junit.* +import org.reactivestreams.* +import org.reactivestreams.example.unicast.* +import org.reactivestreams.tck.* + +class RangePublisherTest : PublisherVerification(TestEnvironment(50, 50)) { + + override fun createPublisher(elements: Long): Publisher { + return RangePublisher(1, elements.toInt()).asFlow().asPublisher() + } + + override fun createFailedPublisher(): Publisher? { + return null + } + + @Ignore + override fun required_spec309_requestZeroMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec309_requestNegativeNumberMustSignalIllegalArgumentException() { + } +} + +class RangePublisherWrappedTwiceTest : PublisherVerification(TestEnvironment(50, 50)) { + + override fun createPublisher(elements: Long): Publisher { + return RangePublisher(1, elements.toInt()).asFlow().asPublisher().asFlow().asPublisher() + } + + override fun createFailedPublisher(): Publisher? { + return null + } + + @Ignore + override fun required_spec309_requestZeroMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec309_requestNegativeNumberMustSignalIllegalArgumentException() { + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactive/test/ReactiveStreamTckTest.kt b/reactive/kotlinx-coroutines-reactive/test/ReactiveStreamTckTest.kt new file mode 100644 index 0000000000..9a4fc5230d --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/ReactiveStreamTckTest.kt @@ -0,0 +1,48 @@ +package kotlinx.coroutines.reactive + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.reactivestreams.* +import org.reactivestreams.tck.* +import org.testng.* +import org.testng.annotations.* + + +class ReactiveStreamTckTest : TestBase() { + + @Factory(dataProvider = "dispatchers") + fun createTests(dispatcher: Dispatcher): Array { + return arrayOf(ReactiveStreamTckTestSuite(dispatcher)) + } + + @DataProvider(name = "dispatchers") + public fun dispatchers(): Array> = Dispatcher.values().map { arrayOf(it) }.toTypedArray() + + + public class ReactiveStreamTckTestSuite( + private val dispatcher: Dispatcher + ) : PublisherVerification(TestEnvironment(500, 500)) { + + override fun createPublisher(elements: Long): Publisher = + publish(dispatcher.dispatcher) { + for (i in 1..elements) send(i) + } + + override fun createFailedPublisher(): Publisher = + publish(dispatcher.dispatcher) { + throw TestException() + } + + @Test + public override fun optional_spec105_emptyStreamMustTerminateBySignallingOnComplete() { + throw SkipException("Skipped") + } + + class TestException : Exception() + } +} + +enum class Dispatcher(val dispatcher: CoroutineDispatcher) { + DEFAULT(Dispatchers.Default), + UNCONFINED(Dispatchers.Unconfined) +} diff --git a/reactive/kotlinx-coroutines-reactive/test/UnboundedIntegerIncrementPublisherTest.kt b/reactive/kotlinx-coroutines-reactive/test/UnboundedIntegerIncrementPublisherTest.kt new file mode 100644 index 0000000000..c602a0cb1c --- /dev/null +++ b/reactive/kotlinx-coroutines-reactive/test/UnboundedIntegerIncrementPublisherTest.kt @@ -0,0 +1,53 @@ +package kotlinx.coroutines.reactive + +import org.junit.* +import org.reactivestreams.example.unicast.AsyncIterablePublisher +import org.reactivestreams.Publisher +import org.reactivestreams.example.unicast.InfiniteIncrementNumberPublisher +import org.reactivestreams.tck.TestEnvironment +import java.util.concurrent.Executors +import java.util.concurrent.ExecutorService +import org.reactivestreams.tck.PublisherVerification +import org.testng.annotations.AfterClass +import org.testng.annotations.BeforeClass +import org.testng.annotations.Test + +@Test +class UnboundedIntegerIncrementPublisherTest : PublisherVerification(TestEnvironment()) { + + private var e: ExecutorService? = null + + @BeforeClass + internal fun before() { + e = Executors.newFixedThreadPool(4) + } + + @AfterClass + internal fun after() { + if (e != null) e!!.shutdown() + } + + override fun createPublisher(elements: Long): Publisher { + return InfiniteIncrementNumberPublisher(e!!).asFlow().asPublisher() + } + + override fun createFailedPublisher(): Publisher { + return AsyncIterablePublisher(object : Iterable { + override fun iterator(): Iterator { + throw RuntimeException("Error state signal!") + } + }, e!!) + } + + override fun maxElementsFromPublisher(): Long { + return super.publisherUnableToSignalOnComplete() + } + + @Ignore + override fun required_spec309_requestZeroMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec309_requestNegativeNumberMustSignalIllegalArgumentException() { + } +} diff --git a/reactive/kotlinx-coroutines-reactor/README.md b/reactive/kotlinx-coroutines-reactor/README.md index b68aad89e6..577385a074 100644 --- a/reactive/kotlinx-coroutines-reactor/README.md +++ b/reactive/kotlinx-coroutines-reactor/README.md @@ -4,46 +4,55 @@ Utilities for [Reactor](https://projectreactor.io). Coroutine builders: -| **Name** | **Result** | **Scope** | **Description** -| --------------- | -------------------------------------- | ---------------- | --------------- -| [mono] | `Mono` | [CoroutineScope] | Cold mono that starts coroutine on subscribe -| [flux] | `Flux` | [CoroutineScope] | Cold flux that starts coroutine on subscribe +| **Name** | **Result** | **Scope** | **Description** +| --------------- | ------------| ---------------- | --------------- +| [mono] | `Mono` | [CoroutineScope] | A cold Mono that starts the coroutine on subscription +| [flux] | `Flux` | [CoroutineScope] | A cold Flux that starts the coroutine on subscription -Note, that `Mono` and `Flux` are a subclass of [Reactive Streams](http://www.reactive-streams.org) -`Publisher` and extensions for it are covered by +Note that `Mono` and `Flux` are subclasses of [Reactive Streams](https://www.reactive-streams.org)' +`Publisher` and extensions for it are covered by the [kotlinx-coroutines-reactive](../kotlinx-coroutines-reactive) module. +Integration with [Flow]: + +| **Name** | **Result** | **Description** +| --------------- | -------------- | --------------- +| [Flow.asFlux] | `Flux` | Converts the given flow to a TCK-compliant Flux. + +This adapter is integrated with Reactor's `Context` and coroutines' [ReactorContext]. + Conversion functions: | **Name** | **Description** | -------- | --------------- -| [Job.asMono][kotlinx.coroutines.experimental.Job.asMono] | Converts job to hot mono -| [Deferred.asMono][kotlinx.coroutines.experimental.Deferred.asMono] | Converts deferred value to hot mono -| [ReceiveChannel.asFlux][kotlinx.coroutines.experimental.channels.ReceiveChannel.asFlux] | Converts streaming channel to hot flux -| [Scheduler.asCoroutineDispatcher][reactor.core.scheduler.Scheduler.asCoroutineDispatcher] | Converts scheduler to [CoroutineDispatcher] -| [TimedScheduler.asCoroutineDispatcher][reactor.core.scheduler.TimedScheduler.asCoroutineDispatcher] | Converts scheduler to [CoroutineDispatcher] supporting [Delay] - - - - -[CoroutineScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-scope/index.html -[CoroutineDispatcher]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-dispatcher/index.html - -[ProducerScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-producer-scope/index.html -[ReceiveChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/index.html -[ChannelIterator]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel-iterator/index.html - - - -[mono]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.experimental.reactor/mono.html -[flux]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.experimental.reactor/flux.html -[kotlinx.coroutines.experimental.Job.asMono]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.experimental.reactor/kotlinx.coroutines.experimental.-job/as-mono.html -[kotlinx.coroutines.experimental.Deferred.asMono]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.experimental.reactor/kotlinx.coroutines.experimental.-deferred/as-mono.html -[kotlinx.coroutines.experimental.channels.ReceiveChannel.asFlux]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.experimental.reactor/kotlinx.coroutines.experimental.channels.-receive-channel/as-flux.html -[reactor.core.scheduler.Scheduler.asCoroutineDispatcher]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.experimental.reactor/reactor.core.scheduler.-scheduler/as-coroutine-dispatcher.html -[reactor.core.scheduler.TimedScheduler.asCoroutineDispatcher]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.experimental.reactor/reactor.core.scheduler.-timed-scheduler/as-coroutine-dispatcher.html +| [Job.asMono][kotlinx.coroutines.Job.asMono] | Converts a job to a hot Mono +| [Deferred.asMono][kotlinx.coroutines.Deferred.asMono] | Converts a deferred value to a hot Mono +| [Scheduler.asCoroutineDispatcher][reactor.core.scheduler.Scheduler.asCoroutineDispatcher] | Converts a scheduler to a [CoroutineDispatcher] + + + + +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[CoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/index.html + + + + +[Flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html + + + + +[mono]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.reactor/mono.html +[flux]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.reactor/flux.html +[Flow.asFlux]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.reactor/as-flux.html +[ReactorContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.reactor/-reactor-context/index.html +[kotlinx.coroutines.Job.asMono]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.reactor/as-mono.html +[kotlinx.coroutines.Deferred.asMono]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.reactor/as-mono.html +[reactor.core.scheduler.Scheduler.asCoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactor/kotlinx.coroutines.reactor/as-coroutine-dispatcher.html + -# Package kotlinx.coroutines.experimental.reactor +# Package kotlinx.coroutines.reactor Utilities for [Reactor](https://projectreactor.io). diff --git a/reactive/kotlinx-coroutines-reactor/api/kotlinx-coroutines-reactor.api b/reactive/kotlinx-coroutines-reactor/api/kotlinx-coroutines-reactor.api new file mode 100644 index 0000000000..5a881a128e --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/api/kotlinx-coroutines-reactor.api @@ -0,0 +1,70 @@ +public final class kotlinx/coroutines/reactor/ConvertKt { + public static final synthetic fun asFlux (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;)Lreactor/core/publisher/Flux; + public static synthetic fun asFlux$default (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lreactor/core/publisher/Flux; + public static final fun asMono (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext;)Lreactor/core/publisher/Mono; + public static final fun asMono (Lkotlinx/coroutines/Job;Lkotlin/coroutines/CoroutineContext;)Lreactor/core/publisher/Mono; +} + +public final class kotlinx/coroutines/reactor/FlowKt { + public static final synthetic fun asFlux (Lkotlinx/coroutines/flow/Flow;)Lreactor/core/publisher/Flux; +} + +public final class kotlinx/coroutines/reactor/FluxKt { + public static final fun flux (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lreactor/core/publisher/Flux; + public static final synthetic fun flux (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lreactor/core/publisher/Flux; + public static synthetic fun flux$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lreactor/core/publisher/Flux; + public static synthetic fun flux$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lreactor/core/publisher/Flux; +} + +public final class kotlinx/coroutines/reactor/MonoKt { + public static final synthetic fun awaitFirst (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitFirstOrDefault (Lreactor/core/publisher/Mono;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitFirstOrElse (Lreactor/core/publisher/Mono;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitFirstOrNull (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitLast (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingle (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingleOrNull (Lreactor/core/publisher/Mono;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun mono (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lreactor/core/publisher/Mono; + public static final synthetic fun mono (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lreactor/core/publisher/Mono; + public static synthetic fun mono$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lreactor/core/publisher/Mono; + public static synthetic fun mono$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lreactor/core/publisher/Mono; +} + +public final class kotlinx/coroutines/reactor/ReactorContext : kotlin/coroutines/AbstractCoroutineContextElement { + public static final field Key Lkotlinx/coroutines/reactor/ReactorContext$Key; + public fun (Lreactor/util/context/Context;)V + public fun (Lreactor/util/context/ContextView;)V + public final fun getContext ()Lreactor/util/context/Context; + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/reactor/ReactorContext$Key : kotlin/coroutines/CoroutineContext$Key { +} + +public final class kotlinx/coroutines/reactor/ReactorContextKt { + public static final synthetic fun asCoroutineContext (Lreactor/util/context/Context;)Lkotlinx/coroutines/reactor/ReactorContext; + public static final fun asCoroutineContext (Lreactor/util/context/ContextView;)Lkotlinx/coroutines/reactor/ReactorContext; +} + +public final class kotlinx/coroutines/reactor/ReactorFlowKt { + public static final fun asFlux (Lkotlinx/coroutines/flow/Flow;)Lreactor/core/publisher/Flux; + public static final fun asFlux (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lreactor/core/publisher/Flux; + public static synthetic fun asFlux$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lreactor/core/publisher/Flux; +} + +public final class kotlinx/coroutines/reactor/SchedulerCoroutineDispatcher : kotlinx/coroutines/CoroutineDispatcher, kotlinx/coroutines/Delay { + public fun (Lreactor/core/scheduler/Scheduler;)V + public fun delay (JLkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun dispatch (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Runnable;)V + public fun equals (Ljava/lang/Object;)Z + public final fun getScheduler ()Lreactor/core/scheduler/Scheduler; + public fun hashCode ()I + public fun invokeOnTimeout (JLjava/lang/Runnable;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/DisposableHandle; + public fun scheduleResumeAfterDelay (JLkotlinx/coroutines/CancellableContinuation;)V + public fun toString ()Ljava/lang/String; +} + +public final class kotlinx/coroutines/reactor/SchedulerKt { + public static final fun asCoroutineDispatcher (Lreactor/core/scheduler/Scheduler;)Lkotlinx/coroutines/reactor/SchedulerCoroutineDispatcher; +} + diff --git a/reactive/kotlinx-coroutines-reactor/build.gradle.kts b/reactive/kotlinx-coroutines-reactor/build.gradle.kts new file mode 100644 index 0000000000..6e46d912b5 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/build.gradle.kts @@ -0,0 +1,46 @@ +import org.jetbrains.kotlin.gradle.dsl.* + +plugins { + // apply plugin to use autocomplete for Kover DSL + id("org.jetbrains.kotlinx.kover") +} + +dependencies { + api("io.projectreactor:reactor-core:${version("reactor")}") + api(project(":kotlinx-coroutines-reactive")) +} + +java { + targetCompatibility = JavaVersion.VERSION_1_8 + sourceCompatibility = JavaVersion.VERSION_1_8 +} + +tasks { + compileKotlin { + compilerOptions.jvmTarget = JvmTarget.JVM_1_8 + } + + compileTestKotlin { + compilerOptions.jvmTarget = JvmTarget.JVM_1_8 + } +} + +// the version of the docs can be different from the version of the Reactor +// library itself: https://github.com/reactor/reactor-core/issues/3794 +externalDocumentationLink( + url = "https://projectreactor.io/docs/core/$%7Bversion("reactor_docs")}/api/" +) + + +kover { + reports { + filters { + excludes { + classes( + "kotlinx.coroutines.reactor.FlowKt", // Deprecated + "kotlinx.coroutines.reactor.ConvertKt\$asFlux$1" // Deprecated + ) + } + } + } +} diff --git a/reactive/kotlinx-coroutines-reactor/package.list b/reactive/kotlinx-coroutines-reactor/package.list new file mode 100644 index 0000000000..9809a3f5f1 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/package.list @@ -0,0 +1,9 @@ +reactor.adapter +reactor.core +reactor.core.publisher +reactor.core.scheduler +reactor.util +reactor.util.annotation +reactor.util.concurrent +reactor.util.context +reactor.util.function diff --git a/reactive/kotlinx-coroutines-reactor/pom.xml b/reactive/kotlinx-coroutines-reactor/pom.xml deleted file mode 100644 index d221dce958..0000000000 --- a/reactive/kotlinx-coroutines-reactor/pom.xml +++ /dev/null @@ -1,82 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - ../../pom.xml - - - kotlinx-coroutines-reactor - jar - - - src/main/kotlin - src/test/kotlin - - - - - - io.projectreactor - reactor-bom - Aluminium-RELEASE - pom - import - - - - - - - io.projectreactor - reactor-core - - - io.projectreactor.addons - reactor-test - test - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - tests - test - - - org.jetbrains.kotlinx - kotlinx-coroutines-reactive - ${project.version} - compile - - - - - diff --git a/reactive/kotlinx-coroutines-reactor/resources/META-INF/services/kotlinx.coroutines.reactive.ContextInjector b/reactive/kotlinx-coroutines-reactor/resources/META-INF/services/kotlinx.coroutines.reactive.ContextInjector new file mode 100644 index 0000000000..9838c12bd4 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/resources/META-INF/services/kotlinx.coroutines.reactive.ContextInjector @@ -0,0 +1 @@ +kotlinx.coroutines.reactor.ReactorContextInjector diff --git a/reactive/kotlinx-coroutines-reactor/src/Convert.kt b/reactive/kotlinx-coroutines-reactor/src/Convert.kt new file mode 100644 index 0000000000..97a2649f1b --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/Convert.kt @@ -0,0 +1,49 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import reactor.core.publisher.* +import kotlin.coroutines.* + +/** + * Converts this job to the hot reactive mono that signals + * with [success][MonoSink.success] when the corresponding job completes. + * + * Every subscriber gets the signal at the same time. + * Unsubscribing from the resulting mono **does not** affect the original job in any way. + * + * **Note: This is an experimental api.** Conversion of coroutines primitives to reactive entities may change + * in the future to account for the concept of structured concurrency. + * + * @param context -- the coroutine context from which the resulting mono is going to be signalled + */ +public fun Job.asMono(context: CoroutineContext): Mono = mono(context) { this@asMono.join() } +/** + * Converts this deferred value to the hot reactive mono that signals + * [success][MonoSink.success] or [error][MonoSink.error]. + * + * Every subscriber gets the same completion value. + * Unsubscribing from the resulting mono **does not** affect the original deferred value in any way. + * + * **Note: This is an experimental api.** Conversion of coroutines primitives to reactive entities may change + * in the future to account for the concept of structured concurrency. + * + * @param context -- the coroutine context from which the resulting mono is going to be signalled + */ +public fun Deferred.asMono(context: CoroutineContext): Mono = mono(context) { this@asMono.await() } + +/** + * Converts a stream of elements received from the channel to the hot reactive flux. + * + * Every subscriber receives values from this channel in a **fan-out** fashion. If the are multiple subscribers, + * they'll receive values in a round-robin way. + * @param context -- the coroutine context from which the resulting flux is going to be signalled + * @suppress + */ +@Deprecated(message = "Deprecated in the favour of consumeAsFlow()", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.consumeAsFlow().asFlux(context)", imports = ["kotlinx.coroutines.flow.consumeAsFlow"])) +public fun ReceiveChannel.asFlux(context: CoroutineContext = EmptyCoroutineContext): Flux = flux(context) { + for (t in this@asFlux) + send(t) +} diff --git a/reactive/kotlinx-coroutines-reactor/src/Flux.kt b/reactive/kotlinx-coroutines-reactor/src/Flux.kt new file mode 100644 index 0000000000..e0355e2e30 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/Flux.kt @@ -0,0 +1,94 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.reactive.* +import org.reactivestreams.* +import reactor.core.* +import reactor.core.publisher.* +import reactor.util.context.* +import kotlin.coroutines.* + +/** + * Creates a cold reactive [Flux] that runs the given [block] in a coroutine. + * Every time the returned flux is subscribed, it starts a new coroutine in the specified [context]. + * The coroutine emits ([Subscriber.onNext]) values with [send][ProducerScope.send], completes ([Subscriber.onComplete]) + * when the coroutine completes, or, in case the coroutine throws an exception or the channel is closed, + * emits the error ([Subscriber.onError]) and closes the channel with the cause. + * Unsubscribing cancels the running coroutine. + * + * Invocations of [send][ProducerScope.send] are suspended appropriately when subscribers apply back-pressure and to + * ensure that [onNext][Subscriber.onNext] is not invoked concurrently. + * + * **Note: This is an experimental api.** Behaviour of publishers that work as children in a parent scope with respect + * to cancellation and error handling may change in the future. + * + * @throws IllegalArgumentException if the provided [context] contains a [Job] instance. + */ +public fun flux( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Flux { + require(context[Job] === null) { "Flux context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return Flux.from(reactorPublish(GlobalScope, context, block)) +} + +private fun reactorPublish( + scope: CoroutineScope, + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Publisher = Publisher onSubscribe@{ subscriber: Subscriber? -> + if (subscriber !is CoreSubscriber) { + subscriber.reject(IllegalArgumentException("Subscriber is not an instance of CoreSubscriber, context can not be extracted.")) + return@onSubscribe + } + val currentContext = subscriber.currentContext() + val reactorContext = context.extendReactorContext(currentContext) + val newContext = scope.newCoroutineContext(context + reactorContext) + val coroutine = PublisherCoroutine(newContext, subscriber, REACTOR_HANDLER) + subscriber.onSubscribe(coroutine) // do it first (before starting coroutine), to avoid unnecessary suspensions + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private val REACTOR_HANDLER: (Throwable, CoroutineContext) -> Unit = { cause, ctx -> + if (cause !is CancellationException) { + try { + Operators.onOperatorError(cause, ctx[ReactorContext]?.context ?: Context.empty()) + } catch (e: Throwable) { + cause.addSuppressed(e) + handleCoroutineException(ctx, cause) + } + } +} + +/** The proper way to reject the subscriber, according to + * [the reactive spec](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.3/README.md#1.9) + */ +private fun Subscriber?.reject(t: Throwable) { + if (this == null) + throw NullPointerException("The subscriber can not be null") + onSubscribe(object: Subscription { + override fun request(n: Long) { + // intentionally left blank + } + override fun cancel() { + // intentionally left blank + } + }) + onError(t) +} + +/** + * @suppress + */ +@Deprecated( + message = "CoroutineScope.flux is deprecated in favour of top-level flux", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("flux(context, block)") +) // Since 1.3.0, will be error in 1.3.1 and hidden in 1.4.0. Binary compatibility with Spring +public fun CoroutineScope.flux( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Flux = + Flux.from(reactorPublish(this, context, block)) diff --git a/reactive/kotlinx-coroutines-reactor/src/Migration.kt b/reactive/kotlinx-coroutines-reactor/src/Migration.kt new file mode 100644 index 0000000000..dc13142b76 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/Migration.kt @@ -0,0 +1,14 @@ +@file:JvmName("FlowKt") + +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.flow.* +import reactor.core.publisher.* + +/** @suppress **/ +@Deprecated( + message = "Replaced in favor of ReactiveFlow extension, please import kotlinx.coroutines.reactor.* instead of kotlinx.coroutines.reactor.FlowKt", + level = DeprecationLevel.HIDDEN +) // Compatibility with Spring 5.2-RC +@JvmName("asFlux") +public fun Flow.asFluxDeprecated(): Flux = asFlux() diff --git a/reactive/kotlinx-coroutines-reactor/src/Mono.kt b/reactive/kotlinx-coroutines-reactor/src/Mono.kt new file mode 100644 index 0000000000..94b925ab8e --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/Mono.kt @@ -0,0 +1,257 @@ +@file:Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.* +import kotlinx.coroutines.reactive.* +import org.reactivestreams.* +import reactor.core.* +import reactor.core.publisher.* +import kotlin.coroutines.* +import kotlinx.coroutines.internal.* + +/** + * Creates a cold [mono][Mono] that runs a given [block] in a coroutine and emits its result. + * Every time the returned mono is subscribed, it starts a new coroutine. + * If the result of [block] is `null`, [MonoSink.success] is invoked without a value. + * Unsubscribing cancels the running coroutine. + * + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * + * @throws IllegalArgumentException if the provided [context] contains a [Job] instance. + */ +public fun mono( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> T? +): Mono { + require(context[Job] === null) { "Mono context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return monoInternal(GlobalScope, context, block) +} + +/** + * Awaits the single value from the given [Mono] without blocking the thread and returns the resulting value, or, if + * this publisher has produced an error, throws the corresponding exception. If the Mono completed without a value, + * `null` is returned. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + */ +public suspend fun Mono.awaitSingleOrNull(): T? = suspendCancellableCoroutine { cont -> + injectCoroutineContext(cont.context).subscribe(object : Subscriber { + private var value: T? = null + + override fun onSubscribe(s: Subscription) { + cont.invokeOnCancellation { s.cancel() } + s.request(Long.MAX_VALUE) + } + + override fun onComplete() { + cont.resume(value) + value = null + } + + override fun onNext(t: T) { + // We don't return the value immediately because the process that emitted it may not be finished yet. + // Resuming now could lead to race conditions between emitter and the awaiting code. + value = t + } + + override fun onError(error: Throwable) { cont.resumeWithException(error) } + }) +} + +/** + * Awaits the single value from the given [Mono] without blocking the thread and returns the resulting value, or, + * if this Mono has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately cancels its [Subscription] and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the Mono does not emit any value + */ +// TODO: consider using https://github.com/Kotlin/kotlinx.coroutines/issues/2607 once that lands +public suspend fun Mono.awaitSingle(): T = awaitSingleOrNull() ?: throw NoSuchElementException() + +private fun monoInternal( + scope: CoroutineScope, // support for legacy mono in scope + context: CoroutineContext, + block: suspend CoroutineScope.() -> T? +): Mono = Mono.create { sink -> + val reactorContext = context.extendReactorContext(sink.currentContext()) + val newContext = scope.newCoroutineContext(context + reactorContext) + val coroutine = MonoCoroutine(newContext, sink) + sink.onDispose(coroutine) + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private class MonoCoroutine( + parentContext: CoroutineContext, + private val sink: MonoSink +) : AbstractCoroutine(parentContext, false, true), Disposable { + @Volatile + private var disposed = false + + override fun onCompleted(value: T) { + if (value == null) sink.success() else sink.success(value) + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + /** Cancellation exceptions that were caused by [dispose], that is, came from downstream, are not errors. */ + val unwrappedCause = unwrap(cause) + if (getCancellationException() !== unwrappedCause || !disposed) { + try { + /** If [sink] turns out to already be in a terminal state, this exception will be passed through the + * [Hooks.onOperatorError] hook, which is the way to signal undeliverable exceptions in Reactor. */ + sink.error(cause) + } catch (e: Throwable) { + // In case of improper error implementation or fatal exceptions + cause.addSuppressed(e) + handleCoroutineException(context, cause) + } + } + } + + override fun dispose() { + disposed = true + cancel() + } + + override fun isDisposed(): Boolean = disposed +} + +/** + * @suppress + */ +@Deprecated( + message = "CoroutineScope.mono is deprecated in favour of top-level mono", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("mono(context, block)") +) // Since 1.3.0, will be error in 1.3.1 and hidden in 1.4.0 +public fun CoroutineScope.mono( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> T? +): Mono = monoInternal(this, context, block) + +/** + * This is a lint function that was added already deprecated in order to guard against confusing usages on [Mono]. + * On [Publisher] instances other than [Mono], this function is not deprecated. + * + * Both [awaitFirst] and [awaitSingle] await the first value, or throw [NoSuchElementException] if there is none, but + * the name [Mono.awaitSingle] better reflects the semantics of [Mono]. + * + * For example, consider this code: + * ``` + * myDbClient.findById(uniqueId).awaitFirst() // findById returns a `Mono` + * ``` + * It looks like more than one value could be returned from `findById` and [awaitFirst] discards the extra elements, + * when in fact, at most a single value can be present. + * + * @suppress + */ +@Deprecated( + message = "Mono produces at most one value, so the semantics of dropping the remaining elements are not useful. " + + "Please use awaitSingle() instead.", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingle()") +) // Warning since 1.5, error in 1.6 +public suspend fun Mono.awaitFirst(): T = awaitSingle() + +/** + * This is a lint function that was added already deprecated in order to guard against confusing usages on [Mono]. + * On [Publisher] instances other than [Mono], this function is not deprecated. + * + * Both [awaitFirstOrDefault] and [awaitSingleOrNull] await the first value, or return some special value if there + * is none, but the name [Mono.awaitSingleOrNull] better reflects the semantics of [Mono]. + * + * For example, consider this code: + * ``` + * myDbClient.findById(uniqueId).awaitFirstOrDefault(default) // findById returns a `Mono` + * ``` + * It looks like more than one value could be returned from `findById` and [awaitFirstOrDefault] discards the extra + * elements, when in fact, at most a single value can be present. + * + * @suppress + */ +@Deprecated( + message = "Mono produces at most one value, so the semantics of dropping the remaining elements are not useful. " + + "Please use awaitSingleOrNull() instead.", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingleOrNull() ?: default") +) // Warning since 1.5, error in 1.6 +public suspend fun Mono.awaitFirstOrDefault(default: T): T = awaitSingleOrNull() ?: default + +/** + * This is a lint function that was added already deprecated in order to guard against confusing usages on [Mono]. + * On [Publisher] instances other than [Mono], this function is not deprecated. + * + * Both [awaitFirstOrNull] and [awaitSingleOrNull] await the first value, or return some special value if there + * is none, but the name [Mono.awaitSingleOrNull] better reflects the semantics of [Mono]. + * + * For example, consider this code: + * ``` + * myDbClient.findById(uniqueId).awaitFirstOrNull() // findById returns a `Mono` + * ``` + * It looks like more than one value could be returned from `findById` and [awaitFirstOrNull] discards the extra + * elements, when in fact, at most a single value can be present. + * + * @suppress + */ +@Deprecated( + message = "Mono produces at most one value, so the semantics of dropping the remaining elements are not useful. " + + "Please use awaitSingleOrNull() instead.", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingleOrNull()") +) // Warning since 1.5, error in 1.6 +public suspend fun Mono.awaitFirstOrNull(): T? = awaitSingleOrNull() + +/** + * This is a lint function that was added already deprecated in order to guard against confusing usages on [Mono]. + * On [Publisher] instances other than [Mono], this function is not deprecated. + * + * Both [awaitFirstOrElse] and [awaitSingleOrNull] await the first value, or return some special value if there + * is none, but the name [Mono.awaitSingleOrNull] better reflects the semantics of [Mono]. + * + * For example, consider this code: + * ``` + * myDbClient.findById(uniqueId).awaitFirstOrElse(defaultValue) // findById returns a `Mono` + * ``` + * It looks like more than one value could be returned from `findById` and [awaitFirstOrElse] discards the extra + * elements, when in fact, at most a single value can be present. + * + * @suppress + */ +@Deprecated( + message = "Mono produces at most one value, so the semantics of dropping the remaining elements are not useful. " + + "Please use awaitSingleOrNull() instead.", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingleOrNull() ?: defaultValue()") +) // Warning since 1.5, error in 1.6 +public suspend fun Mono.awaitFirstOrElse(defaultValue: () -> T): T = awaitSingleOrNull() ?: defaultValue() + +/** + * This is a lint function that was added already deprecated in order to guard against confusing usages on [Mono]. + * On [Publisher] instances other than [Mono], this function is not deprecated. + * + * Both [awaitLast] and [awaitSingle] await the single value, or throw [NoSuchElementException] if there is none, but + * the name [Mono.awaitSingle] better reflects the semantics of [Mono]. + * + * For example, consider this code: + * ``` + * myDbClient.findById(uniqueId).awaitLast() // findById returns a `Mono` + * ``` + * It looks like more than one value could be returned from `findById` and [awaitLast] discards the initial elements, + * when in fact, at most a single value can be present. + * + * @suppress + */ +@Deprecated( + message = "Mono produces at most one value, so the last element is the same as the first. " + + "Please use awaitSingle() instead.", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingle()") +) // Warning since 1.5, error in 1.6 +public suspend fun Mono.awaitLast(): T = awaitSingle() diff --git a/reactive/kotlinx-coroutines-reactor/src/ReactorContext.kt b/reactive/kotlinx-coroutines-reactor/src/ReactorContext.kt new file mode 100644 index 0000000000..44541eecc5 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/ReactorContext.kt @@ -0,0 +1,71 @@ +package kotlinx.coroutines.reactor + +import kotlin.coroutines.* +import kotlinx.coroutines.reactive.* +import reactor.util.context.* + +/** + * Wraps Reactor's [Context] into a [CoroutineContext] element for seamless integration between + * Reactor and kotlinx.coroutines. + * [Context.asCoroutineContext] puts Reactor's [Context] elements into a [CoroutineContext], + * which can be used to propagate the information about Reactor's [Context] through coroutines. + * + * This context element is implicitly propagated through subscribers' context by all Reactive integrations, + * such as [mono], [flux], [Publisher.asFlow][asFlow], [Flow.asPublisher][asPublisher] and [Flow.asFlux][asFlux]. + * Functions that subscribe to a reactive stream + * (e.g. [Publisher.awaitFirst][kotlinx.coroutines.reactive.awaitFirst]), too, propagate [ReactorContext] + * to the subscriber's [Context]. + ** + * ### Examples of Reactive context integration. + * + * #### Propagating ReactorContext to Reactor's Context + * ``` + * val flux = myDatabaseService.getUsers() + * .contextWrite { ctx -> println(ctx); ctx } + * flux.awaitFirst() // Will print "null" + * + * // Now add ReactorContext + * withContext(Context.of("answer", "42").asCoroutineContext()) { + * flux.awaitFirst() // Will print "Context{'key'='value'}" + * } + * ``` + * + * #### Propagating subscriber's Context to ReactorContext: + * ``` + * val flow = flow { + * println("Reactor context in Flow: " + currentCoroutineContext()[ReactorContext]) + * } + * // No context + * flow.asFlux() + * .subscribe() // Will print 'Reactor context in Flow: null' + * // Add subscriber's context + * flow.asFlux() + * .contextWrite { ctx -> ctx.put("answer", 42) } + * .subscribe() // Will print "Reactor context in Flow: Context{'answer'=42}" + * ``` + */ +public class ReactorContext(public val context: Context) : AbstractCoroutineContextElement(ReactorContext) { + + // `Context.of` is zero-cost if the argument is a `Context` + public constructor(contextView: ContextView): this(Context.of(contextView)) + + public companion object Key : CoroutineContext.Key + + override fun toString(): String = context.toString() +} + +/** + * Wraps the given [ContextView] into [ReactorContext], so it can be added to the coroutine's context + * and later used via `coroutineContext[ReactorContext]`. + */ +public fun ContextView.asCoroutineContext(): ReactorContext = ReactorContext(this) + +/** @suppress */ +@Deprecated("The more general version for ContextView should be used instead", level = DeprecationLevel.HIDDEN) +public fun Context.asCoroutineContext(): ReactorContext = readOnly().asCoroutineContext() // `readOnly()` is zero-cost. + +/** + * Updates the Reactor context in this [CoroutineContext], adding (or possibly replacing) some values. + */ +internal fun CoroutineContext.extendReactorContext(extensions: ContextView): CoroutineContext = + (this[ReactorContext]?.context?.putAll(extensions) ?: extensions).asCoroutineContext() diff --git a/reactive/kotlinx-coroutines-reactor/src/ReactorContextInjector.kt b/reactive/kotlinx-coroutines-reactor/src/ReactorContextInjector.kt new file mode 100644 index 0000000000..7a5debf001 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/ReactorContextInjector.kt @@ -0,0 +1,22 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.reactive.* +import org.reactivestreams.* +import reactor.core.publisher.* +import reactor.util.context.* +import kotlin.coroutines.* + +internal class ReactorContextInjector : ContextInjector { + /** + * Injects all values from the [ReactorContext] entry of the given coroutine context + * into the downstream [Context] of Reactor's [Publisher] instances of [Mono] or [Flux]. + */ + override fun injectCoroutineContext(publisher: Publisher, coroutineContext: CoroutineContext): Publisher { + val reactorContext = coroutineContext[ReactorContext]?.context ?: return publisher + return when(publisher) { + is Mono -> publisher.contextWrite(reactorContext) + is Flux -> publisher.contextWrite(reactorContext) + else -> publisher + } + } +} diff --git a/reactive/kotlinx-coroutines-reactor/src/ReactorFlow.kt b/reactive/kotlinx-coroutines-reactor/src/ReactorFlow.kt new file mode 100644 index 0000000000..ba819774ec --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/ReactorFlow.kt @@ -0,0 +1,36 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.Flow +import kotlinx.coroutines.flow.flowOn +import kotlinx.coroutines.reactive.FlowSubscription +import org.reactivestreams.* +import reactor.core.CoreSubscriber +import reactor.core.publisher.Flux +import kotlin.coroutines.* + +/** + * Converts the given flow to a cold flux. + * The original flow is cancelled when the flux subscriber is disposed. + * + * This function is integrated with [ReactorContext], see its documentation for additional details. + * + * An optional [context] can be specified to control the execution context of calls to [Subscriber] methods. + * You can set a [CoroutineDispatcher] to confine them to a specific thread and/or various [ThreadContextElement] to + * inject additional context into the caller thread. By default, the [Unconfined][Dispatchers.Unconfined] dispatcher + * is used, so calls are performed from an arbitrary thread. + */ +@JvmOverloads // binary compatibility +public fun Flow.asFlux(context: CoroutineContext = EmptyCoroutineContext): Flux = + FlowAsFlux(this, Dispatchers.Unconfined + context) + +private class FlowAsFlux( + private val flow: Flow, + private val context: CoroutineContext +) : Flux() { + override fun subscribe(subscriber: CoreSubscriber) { + val hasContext = !subscriber.currentContext().isEmpty + val source = if (hasContext) flow.flowOn(subscriber.currentContext().asCoroutineContext()) else flow + subscriber.onSubscribe(FlowSubscription(source, subscriber, context)) + } +} diff --git a/reactive/kotlinx-coroutines-reactor/src/Scheduler.kt b/reactive/kotlinx-coroutines-reactor/src/Scheduler.kt new file mode 100644 index 0000000000..5371ff39d7 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/Scheduler.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.* +import reactor.core.Disposable +import reactor.core.scheduler.Scheduler +import java.util.concurrent.TimeUnit +import kotlin.coroutines.CoroutineContext + +/** + * Converts an instance of [Scheduler] to an implementation of [CoroutineDispatcher]. + */ +public fun Scheduler.asCoroutineDispatcher(): SchedulerCoroutineDispatcher = SchedulerCoroutineDispatcher(this) + +/** + * Implements [CoroutineDispatcher] on top of an arbitrary [Scheduler]. + * @param scheduler a scheduler. + */ +public class SchedulerCoroutineDispatcher( + /** + * Underlying scheduler of current [CoroutineDispatcher]. + */ + public val scheduler: Scheduler +) : CoroutineDispatcher(), Delay { + /** @suppress */ + override fun dispatch(context: CoroutineContext, block: Runnable) { + scheduler.schedule(block) + } + + /** @suppress */ + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val disposable = scheduler.schedule({ + with(continuation) { resumeUndispatched(Unit) } + }, timeMillis, TimeUnit.MILLISECONDS) + continuation.disposeOnCancellation(disposable.asDisposableHandle()) + } + + /** @suppress */ + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle = + scheduler.schedule(block, timeMillis, TimeUnit.MILLISECONDS).asDisposableHandle() + + /** @suppress */ + override fun toString(): String = scheduler.toString() + /** @suppress */ + override fun equals(other: Any?): Boolean = other is SchedulerCoroutineDispatcher && other.scheduler === scheduler + /** @suppress */ + override fun hashCode(): Int = System.identityHashCode(scheduler) +} + +private fun Disposable.asDisposableHandle(): DisposableHandle = + DisposableHandle { this@asDisposableHandle.dispose() } diff --git a/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Convert.kt b/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Convert.kt deleted file mode 100644 index c343b5c264..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Convert.kt +++ /dev/null @@ -1,43 +0,0 @@ -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.Deferred -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.channels.ReceiveChannel -import reactor.core.publisher.Flux -import reactor.core.publisher.Mono -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Converts this job to the hot reactive mono that signals - * with [success][MonoSink.success] when the corresponding job completes. - * - * Every subscriber gets the signal at the same time. - * Unsubscribing from the resulting mono **does not** affect the original job in any way. - * - * @param context -- the coroutine context from which the resulting mono is going to be signalled - */ -public fun Job.asMono(context: CoroutineContext): Mono = mono(context) { this@asMono.join() } - -/** - * Converts this deferred value to the hot reactive mono that signals - * [success][MonoSink.success] or [error][MonoSink.error]. - * - * Every subscriber gets the same completion value. - * Unsubscribing from the resulting mono **does not** affect the original deferred value in any way. - * - * @param context -- the coroutine context from which the resulting mono is going to be signalled - */ -public fun Deferred.asMono(context: CoroutineContext): Mono = mono(context) { this@asMono.await() } - -/** - * Converts a stream of elements received from the channel to the hot reactive flux. - * - * Every subscriber receives values from this channel in **fan-out** fashion. If the are multiple subscribers, - * they'll receive values in round-robin way. - * - * @param context -- the coroutine context from which the resulting flux is going to be signalled - */ -public fun ReceiveChannel.asFlux(context: CoroutineContext): Flux = flux(context) { - for (t in this@asFlux) - send(t) -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Flux.kt b/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Flux.kt deleted file mode 100644 index 30d3f3db46..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Flux.kt +++ /dev/null @@ -1,25 +0,0 @@ -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.channels.ProducerScope -import kotlinx.coroutines.experimental.reactive.publish -import reactor.core.publisher.Flux -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Creates cold reactive [Flux] that runs a given [block] in a coroutine. - * Every time the returned flux is subscribed, it starts a new coroutine in the specified [context]. - * Coroutine emits items with `send`. Unsubscribing cancels running coroutine. - * - * Invocations of `send` are suspended appropriately when subscribers apply back-pressure and to ensure that - * `onNext` is not invoked concurrently. - * - * | **Coroutine action** | **Signal to subscriber** - * | -------------------------------------------- | ------------------------ - * | `send` | `onNext` - * | Normal completion or `close` without cause | `onComplete` - * | Failure with exception or `close` with cause | `onError` - */ -fun flux( - context: CoroutineContext, - block: suspend ProducerScope.() -> Unit -): Flux = Flux.from(publish(context, block)) diff --git a/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Mono.kt b/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Mono.kt deleted file mode 100644 index 2c4bb0cfb6..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Mono.kt +++ /dev/null @@ -1,73 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.CoroutineScope -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.newCoroutineContext -import reactor.core.Disposable -import reactor.core.publisher.Mono -import reactor.core.publisher.MonoSink -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Creates cold [mono][Mono] that will run a given [block] in a coroutine. - * Every time the returned mono is subscribed, it starts a new coroutine in the specified [context]. - * Coroutine returns a single, possibly null value. Unsubscribing cancels running coroutine. - * - * | **Coroutine action** | **Signal to sink** - * | ------------------------------------- | ------------------------ - * | Returns a non-null value | `success(value)` - * | Returns a null | `success` - * | Failure with exception or unsubscribe | `error` - */ -fun mono( - context: CoroutineContext, - block: suspend CoroutineScope.() -> T? -): Mono = Mono.create { sink -> - val newContext = newCoroutineContext(context) - val coroutine = MonoCoroutine(newContext, sink) - coroutine.initParentJob(context[Job]) - sink.setCancellation(coroutine) - block.startCoroutine(coroutine, coroutine) -} - -private class MonoCoroutine( - override val parentContext: CoroutineContext, - private val sink: MonoSink -) : AbstractCoroutine(true), Disposable { - var disposed = false - - @Suppress("UNCHECKED_CAST") - override fun afterCompletion(state: Any?, mode: Int) { - when { - disposed -> {} - state is CompletedExceptionally -> sink.error(state.exception) - state != null -> sink.success(state as T) - else -> sink.success() - } - } - - override fun dispose() { - disposed = true - cancel(cause = null) - } - - override fun isDisposed(): Boolean = disposed -} - diff --git a/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Scheduler.kt b/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Scheduler.kt deleted file mode 100644 index 6db0e47cca..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/main/kotlin/kotlinx/coroutines/experimental/reactor/Scheduler.kt +++ /dev/null @@ -1,60 +0,0 @@ -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.CancellableContinuation -import kotlinx.coroutines.experimental.CoroutineDispatcher -import kotlinx.coroutines.experimental.Delay -import kotlinx.coroutines.experimental.DisposableHandle -import kotlinx.coroutines.experimental.disposeOnCompletion -import reactor.core.Cancellation -import reactor.core.scheduler.Scheduler -import reactor.core.scheduler.TimedScheduler -import java.util.concurrent.TimeUnit -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Converts an instance of [Scheduler] to an implementation of [CoroutineDispatcher]. - */ -fun Scheduler.asCoroutineDispatcher() = SchedulerCoroutineDispatcher(this) - -/** - * Converts an instance of [TimedScheduler] to an implementation of [CoroutineDispatcher] - * and provides native [delay][Delay.delay] support. - */ -fun TimedScheduler.asCoroutineDispatcher() = TimedSchedulerCoroutineDispatcher(this) - -/** - * Implements [CoroutineDispatcher] on top of an arbitrary [Scheduler]. - * @param scheduler a scheduler. - */ -open class SchedulerCoroutineDispatcher(private val scheduler: Scheduler) : CoroutineDispatcher() { - override fun dispatch(context: CoroutineContext, block: Runnable) { - scheduler.schedule(block) - } - - override fun toString(): String = scheduler.toString() - override fun equals(other: Any?): Boolean = other is SchedulerCoroutineDispatcher && other.scheduler === scheduler - override fun hashCode(): Int = System.identityHashCode(scheduler) -} - -/** - * Implements [CoroutineDispatcher] on top of an arbitrary [TimedScheduler]. - * @param scheduler a timed scheduler. - */ -open class TimedSchedulerCoroutineDispatcher(private val scheduler: TimedScheduler) : SchedulerCoroutineDispatcher(scheduler), Delay { - - override fun scheduleResumeAfterDelay(time: Long, unit: TimeUnit, continuation: CancellableContinuation) { - val disposable = scheduler.schedule({ - with(continuation) { resumeUndispatched(Unit) } - }, time, unit) - - continuation.disposeOnCompletion(disposable.asDisposableHandle()) - } - - override fun invokeOnTimeout(time: Long, unit: TimeUnit, block: Runnable): DisposableHandle = - scheduler.schedule(block, time, unit).asDisposableHandle() -} - -private fun Cancellation.asDisposableHandle(): DisposableHandle = - object : DisposableHandle { - override fun dispose() = this@asDisposableHandle.dispose() - } \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/src/module-info.java b/reactive/kotlinx-coroutines-reactor/src/module-info.java new file mode 100644 index 0000000000..b75308b521 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/src/module-info.java @@ -0,0 +1,14 @@ +import kotlinx.coroutines.reactive.ContextInjector; +import kotlinx.coroutines.reactor.ReactorContextInjector; + +module kotlinx.coroutines.reactor { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.coroutines.reactive; + requires org.reactivestreams; + requires reactor.core; + + exports kotlinx.coroutines.reactor; + + provides ContextInjector with ReactorContextInjector; +} diff --git a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/Check.kt b/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/Check.kt deleted file mode 100644 index 168284321d..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/Check.kt +++ /dev/null @@ -1,40 +0,0 @@ -package kotlinx.coroutines.experimental.reactor - -import reactor.core.publisher.Flux -import reactor.core.publisher.Mono - -fun checkMonoValue( - mono: Mono, - checker: (T) -> Unit -) { - val monoValue = mono.block() - checker(monoValue) -} - -fun checkErroneous( - mono: Mono<*>, - checker: (Throwable) -> Unit -) { - try { - mono.block() - error("Should have failed") - } catch (e: Throwable) { - checker(e) - } -} - -fun checkSingleValue( - flux: Flux, - checker: (T) -> Unit -) { - val singleValue = flux.toIterable().single() - checker(singleValue) -} - -fun checkErroneous( - flux: Flux<*>, - checker: (Throwable) -> Unit -) { - val singleNotification = flux.materialize().toIterable().single() - checker(singleNotification.throwable) -} diff --git a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/ConvertTest.kt b/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/ConvertTest.kt deleted file mode 100644 index 0b5362353b..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/ConvertTest.kt +++ /dev/null @@ -1,151 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.NonCancellable -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.Unconfined -import kotlinx.coroutines.experimental.async -import kotlinx.coroutines.experimental.channels.produce -import kotlinx.coroutines.experimental.delay -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.reactive.consumeEach -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.junit.Assert.assertEquals -import org.junit.Assert.assertNull -import org.junit.Assert.fail -import org.junit.Test - -class ConvertTest : TestBase() { - class TestException(s: String): RuntimeException(s) - - @Test - fun testJobToMonoSuccess() = runBlocking { - expect(1) - val job = launch(context) { - expect(3) - } - val mono = job.asMono(context) - mono.subscribe { - expect(4) - } - expect(2) - yield() - finish(5) - } - - @Test - fun testJobToMonoFail() = runBlocking { - expect(1) - val job = async(context + NonCancellable) { // don't kill parent on exception - expect(3) - throw RuntimeException("OK") - } - val mono = job.asMono(context) - mono.subscribe( - { fail("no item should be emitted") }, - { expect(4) } - ) - expect(2) - yield() - finish(5) - } - - @Test - fun testDeferredToMono() { - val d = async(CommonPool) { - delay(50) - "OK" - } - val mono1 = d.asMono(Unconfined) - checkMonoValue(mono1) { - assertEquals("OK", it) - } - val mono2 = d.asMono(Unconfined) - checkMonoValue(mono2) { - assertEquals("OK", it) - } - } - - @Test - fun testDeferredToMonoEmpty() { - val d = async(CommonPool) { - delay(50) - null - } - val mono1 = d.asMono(Unconfined) - checkMonoValue(mono1, ::assertNull) - val mono2 = d.asMono(Unconfined) - checkMonoValue(mono2, ::assertNull) - } - - @Test - fun testDeferredToMonoFail() { - val d = async(CommonPool) { - delay(50) - throw TestException("OK") - } - val mono1 = d.asMono(Unconfined) - checkErroneous(mono1) { - check(it is TestException && it.message == "OK") { "$it" } - } - val mono2 = d.asMono(Unconfined) - checkErroneous(mono2) { - check(it is TestException && it.message == "OK") { "$it" } - } - } - - @Test - fun testToFlux() { - val c = produce(CommonPool) { - delay(50) - send("O") - delay(50) - send("K") - } - val flux = c.asFlux(Unconfined) - checkMonoValue(flux.reduce { t1, t2 -> t1 + t2 }) { - assertEquals("OK", it) - } - } - - @Test - fun testToFluxFail() { - val c = produce(CommonPool) { - delay(50) - send("O") - delay(50) - throw TestException("K") - } - val flux = c.asFlux(Unconfined) - val mono = mono(Unconfined) { - var result = "" - try { - flux.consumeEach { result += it } - } catch(e: Throwable) { - check(e is TestException) - result += e.message - } - result - } - checkMonoValue(mono) { - assertEquals("OK", it) - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxCompletionStressTest.kt b/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxCompletionStressTest.kt deleted file mode 100644 index 8336187452..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxCompletionStressTest.kt +++ /dev/null @@ -1,52 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.reactive.consumeEach -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.withTimeout -import org.junit.Test -import java.util.Random -import kotlin.coroutines.experimental.CoroutineContext - -class FluxCompletionStressTest : TestBase() { - val N_REPEATS = 10_000 * stressTestMultiplier - - fun range(context: CoroutineContext, start: Int, count: Int) = flux(context) { - for (x in start until start + count) send(x) - } - - @Test - fun testCompletion() { - val rnd = Random() - repeat(N_REPEATS) { - val count = rnd.nextInt(5) - runBlocking { - withTimeout(5000) { - var received = 0 - range(CommonPool, 1, count).consumeEach { x -> - received++ - if (x != received) error("$x != $received") - } - if (received != count) error("$received != $count") - } - } - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxMultiTest.kt b/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxMultiTest.kt deleted file mode 100644 index 797532ce2e..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxMultiTest.kt +++ /dev/null @@ -1,104 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.Unconfined -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.reactive.consumeEach -import org.junit.Assert.assertEquals -import org.junit.Test -import reactor.core.publisher.Flux -import java.io.IOException - -/** - * Test emitting multiple values with [flux]. - */ -class FluxMultiTest : TestBase() { - @Test - fun testNumbers() { - val n = 100 * stressTestMultiplier - val flux = flux(CommonPool) { - repeat(n) { send(it) } - } - checkMonoValue(flux.collectList()) { list -> - assertEquals((0..n - 1).toList(), list) - } - } - - @Test - fun testConcurrentStress() { - val n = 10_000 * stressTestMultiplier - val flux = flux(CommonPool) { - // concurrent emitters (many coroutines) - val jobs = List(n) { - // launch - launch(CommonPool) { - send(it) - } - } - jobs.forEach { it.join() } - } - checkMonoValue(flux.collectList()) { list -> - assertEquals(n, list.size) - assertEquals((0..n - 1).toList(), list.sorted()) - } - } - - @Test - fun testIteratorResendUnconfined() { - val n = 10_000 * stressTestMultiplier - val flux = flux(Unconfined) { - Flux.range(0, n).consumeEach { send(it) } - } - checkMonoValue(flux.collectList()) { list -> - assertEquals((0..n - 1).toList(), list) - } - } - - @Test - fun testIteratorResendPool() { - val n = 10_000 * stressTestMultiplier - val flux = flux(CommonPool) { - Flux.range(0, n).consumeEach { send(it) } - } - checkMonoValue(flux.collectList()) { list -> - assertEquals((0..n - 1).toList(), list) - } - } - - @Test - fun testSendAndCrash() { - val flux = flux(CommonPool) { - send("O") - throw IOException("K") - } - val mono = mono(CommonPool) { - var result = "" - try { - flux.consumeEach { result += it } - } catch(e: IOException) { - result += e.message - } - result - } - checkMonoValue(mono) { - assertEquals("OK", it) - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxSingleTest.kt b/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxSingleTest.kt deleted file mode 100644 index 03eda404e0..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxSingleTest.kt +++ /dev/null @@ -1,183 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.reactive.awaitFirst -import kotlinx.coroutines.experimental.reactive.awaitFirstOrDefault -import kotlinx.coroutines.experimental.reactive.awaitLast -import kotlinx.coroutines.experimental.reactive.awaitSingle -import kotlinx.coroutines.experimental.reactive.consumeEach -import kotlinx.coroutines.experimental.runBlocking -import org.junit.Assert.assertEquals -import org.junit.Assert.fail -import org.junit.Test -import reactor.core.publisher.Flux - -/** - * Tests emitting single item with [flux]. - */ -class FluxSingleTest { - @Test - fun testSingleNoWait() { - val flux = flux(CommonPool) { - send("OK") - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleAwait() = runBlocking { - assertEquals("OK", Flux.just("O").awaitSingle() + "K") - } - - @Test - fun testSingleEmitAndAwait() { - val flux = flux(CommonPool) { - send(Flux.just("O").awaitSingle() + "K") - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleWithDelay() { - val flux = flux(CommonPool) { - send(Flux.just("O").delayMillis(50).awaitSingle() + "K") - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleException() { - val flux = flux(CommonPool) { - send(Flux.just("O", "K").awaitSingle() + "K") - } - - checkErroneous(flux) { - assert(it is IllegalArgumentException) - } - } - - @Test - fun testAwaitFirst() { - val flux = flux(CommonPool) { - send(Flux.just("O", "#").awaitFirst() + "K") - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitFirstOrDefault() { - val flux = flux(CommonPool) { - send(Flux.empty().awaitFirstOrDefault("O") + "K") - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitFirstOrDefaultWithValues() { - val flux = flux(CommonPool) { - send(Flux.just("O", "#").awaitFirstOrDefault("!") + "K") - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitLast() { - val flux = flux(CommonPool) { - send(Flux.just("#", "O").awaitLast() + "K") - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromObservable() { - val flux = flux(CommonPool) { - try { - send(Flux.error(RuntimeException("O")).awaitFirst()) - } catch (e: RuntimeException) { - send(Flux.just(e.message!!).awaitLast() + "K") - } - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromCoroutine() { - val flux = flux(CommonPool) { - error(Flux.just("O").awaitSingle() + "K") - } - - checkErroneous(flux) { - assert(it is IllegalStateException) - assertEquals("OK", it.message) - } - } - - @Test - fun testFluxIteration() { - val flux = flux(CommonPool) { - var result = "" - Flux.just("O", "K").consumeEach { result += it } - send(result) - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } - - @Test - fun testFluxIterationFailure() { - val flux = flux(CommonPool) { - try { - Flux.error(RuntimeException("OK")).consumeEach { fail("Should not be here") } - send("Fail") - } catch (e: RuntimeException) { - send(e.message!!) - } - } - - checkSingleValue(flux) { - assertEquals("OK", it) - } - } -} diff --git a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxTest.kt b/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxTest.kt deleted file mode 100644 index 01640b89d4..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/FluxTest.kt +++ /dev/null @@ -1,86 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert -import org.junit.Test - -class FluxTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val flux = flux(context) { - expect(4) - send("OK") - } - expect(2) - flux.subscribe { value -> - expect(5) - Assert.assertThat(value, IsEqual("OK")) - } - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val flux = flux(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - flux.subscribe({ - expectUnreached() - }, { error -> - expect(5) - Assert.assertThat(error, IsInstanceOf(RuntimeException::class.java)) - Assert.assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val flux = flux(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - val sub = flux.subscribe({ - expectUnreached() - }, { - expectUnreached() - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.dispose() // will cancel coroutine - yield() - finish(6) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/MonoTest.kt b/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/MonoTest.kt deleted file mode 100644 index a2667e9b64..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/MonoTest.kt +++ /dev/null @@ -1,211 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.runBlocking -import org.junit.Assert.assertEquals -import org.junit.Test -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.reactive.awaitFirst -import kotlinx.coroutines.experimental.reactive.awaitLast -import kotlinx.coroutines.experimental.reactive.awaitSingle -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert -import reactor.core.publisher.Flux -import reactor.core.publisher.Mono - -/** - * Tests emitting single item with [mono]. - */ -class MonoTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val mono = mono(context) { - expect(4) - "OK" - } - expect(2) - mono.subscribe { value -> - expect(5) - Assert.assertThat(value, IsEqual("OK")) - } - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val mono = mono(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - mono.subscribe({ - expectUnreached() - }, { error -> - expect(5) - Assert.assertThat(error, IsInstanceOf(RuntimeException::class.java)) - Assert.assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicEmpty() = runBlocking { - expect(1) - val mono = mono(context) { - expect(4) - null - } - expect(2) - mono.subscribe ({}, { throw it }, { - expect(5) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val mono = mono(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - // nothing is called on a disposed mono - val sub = mono.subscribe({ - expectUnreached() - }, { - expectUnreached() - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.dispose() // will cancel coroutine - yield() - finish(6) - } - - @Test - fun testMonoNoWait() { - val mono = mono(CommonPool) { - "OK" - } - - checkMonoValue(mono) { - assertEquals("OK", it) - } - } - - @Test - fun testMonoAwait() = runBlocking { - assertEquals("OK", Mono.just("O").awaitSingle() + "K") - } - - @Test - fun testMonoEmitAndAwait() { - val mono = mono(CommonPool) { - Mono.just("O").awaitSingle() + "K" - } - - checkMonoValue(mono) { - assertEquals("OK", it) - } - } - - @Test - fun testMonoWithDelay() { - val mono = mono(CommonPool) { - Flux.just("O").delayMillis(50).awaitSingle() + "K" - } - - checkMonoValue(mono) { - assertEquals("OK", it) - } - } - - @Test - fun testMonoException() { - val mono = mono(CommonPool) { - Flux.just("O", "K").awaitSingle() + "K" - } - - checkErroneous(mono) { - assert(it is IllegalArgumentException) - } - } - - @Test - fun testAwaitFirst() { - val mono = mono(CommonPool) { - Flux.just("O", "#").awaitFirst() + "K" - } - - checkMonoValue(mono) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitLast() { - val mono = mono(CommonPool) { - Flux.just("#", "O").awaitLast() + "K" - } - - checkMonoValue(mono) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromFlux() { - val mono = mono(CommonPool) { - try { - Flux.error(RuntimeException("O")).awaitFirst() - } catch (e: RuntimeException) { - Flux.just(e.message!!).awaitLast() + "K" - } - } - - checkMonoValue(mono) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromCoroutine() { - val mono = mono(CommonPool) { - throw IllegalStateException(Flux.just("O").awaitSingle() + "K") - } - - checkErroneous(mono) { - assert(it is IllegalStateException) - assertEquals("OK", it.message) - } - } -} diff --git a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/SchedulerTest.kt b/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/SchedulerTest.kt deleted file mode 100644 index 20f31cc1b4..0000000000 --- a/reactive/kotlinx-coroutines-reactor/src/test/kotlin/kotlinx/coroutines/experimental/reactor/SchedulerTest.kt +++ /dev/null @@ -1,47 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.reactor - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.delay -import kotlinx.coroutines.experimental.run -import kotlinx.coroutines.experimental.runBlocking -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsNot -import org.junit.Assert.assertThat -import org.junit.Test -import reactor.core.scheduler.Schedulers - -class SchedulerTest : TestBase() { - @Test - fun testSingleScheduler(): Unit = runBlocking { - expect(1) - val mainThread = Thread.currentThread() - run(Schedulers.single().asCoroutineDispatcher()) { - val t1 = Thread.currentThread() - println(t1) - assertThat(t1, IsNot(IsEqual(mainThread))) - expect(2) - delay(100) - val t2 = Thread.currentThread() - println(t2) - assertThat(t2, IsNot(IsEqual(mainThread))) - expect(3) - } - finish(4) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/test/BackpressureTest.kt b/reactive/kotlinx-coroutines-reactor/test/BackpressureTest.kt new file mode 100644 index 0000000000..16775eb56d --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/BackpressureTest.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import reactor.core.publisher.* +import kotlin.test.* + +class BackpressureTest : TestBase() { + @Test + fun testBackpressureDropDirect() = runTest { + expect(1) + Flux.fromArray(arrayOf(1)) + .onBackpressureDrop() + .collect { + assertEquals(1, it) + expect(2) + } + finish(3) + } + + @Test + fun testBackpressureDropFlow() = runTest { + expect(1) + Flux.fromArray(arrayOf(1)) + .onBackpressureDrop() + .asFlow() + .collect { + assertEquals(1, it) + expect(2) + } + finish(3) + } + + @Test + fun testCooperativeCancellation() = runTest { + val flow = Flux.fromIterable((0L..Long.MAX_VALUE)).asFlow() + flow.onEach { if (it > 10) currentCoroutineContext().cancel() }.launchIn(this + Dispatchers.Default).join() + } + + @Test + fun testCooperativeCancellationForBuffered() = runTest(expected = { it is CancellationException }) { + val flow = Flux.fromIterable((0L..Long.MAX_VALUE)).asFlow() + val channel = flow.onEach { if (it > 10) currentCoroutineContext().cancel() }.produceIn(this + Dispatchers.Default) + channel.consumeEach { /* Do nothing, just consume elements */ } + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/test/Check.kt b/reactive/kotlinx-coroutines-reactor/test/Check.kt new file mode 100644 index 0000000000..d5a9a57df6 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/Check.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.reactor + +import reactor.core.publisher.Flux +import reactor.core.publisher.Mono + +fun checkMonoValue( + mono: Mono, + checker: (T) -> Unit +) { + val monoValue = mono.block() + checker(monoValue) +} + +fun checkErroneous( + mono: Mono<*>, + checker: (Throwable) -> Unit +) { + try { + mono.block() + error("Should have failed") + } catch (e: Throwable) { + checker(e) + } +} + +fun checkSingleValue( + flux: Flux, + checker: (T) -> Unit +) { + val singleValue = flux.toIterable().single() + checker(singleValue) +} + +fun checkErroneous( + flux: Flux<*>, + checker: (Throwable) -> Unit +) { + val singleNotification = flux.materialize().toIterable().single() + checker(singleNotification.throwable) +} diff --git a/reactive/kotlinx-coroutines-reactor/test/ConvertTest.kt b/reactive/kotlinx-coroutines-reactor/test/ConvertTest.kt new file mode 100644 index 0000000000..7da27cf2ab --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/ConvertTest.kt @@ -0,0 +1,126 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class ConvertTest : TestBase() { + @Test + fun testJobToMonoSuccess() = runBlocking { + expect(1) + val job = launch { + expect(3) + } + val mono = job.asMono(coroutineContext.minusKey(Job)) + mono.subscribe { + expect(4) + } + expect(2) + yield() + finish(5) + } + + @Test + fun testJobToMonoFail() = runBlocking { + expect(1) + val job = async(NonCancellable) { + expect(3) + throw RuntimeException("OK") + } + val mono = job.asMono(coroutineContext.minusKey(Job)) + mono.subscribe( + { fail("no item should be emitted") }, + { expect(4) } + ) + expect(2) + yield() + finish(5) + } + + @Test + fun testDeferredToMono() { + val d = GlobalScope.async { + delay(50) + "OK" + } + val mono1 = d.asMono(Dispatchers.Unconfined) + checkMonoValue(mono1) { + assertEquals("OK", it) + } + val mono2 = d.asMono(Dispatchers.Unconfined) + checkMonoValue(mono2) { + assertEquals("OK", it) + } + } + + @Test + fun testDeferredToMonoEmpty() { + val d = GlobalScope.async { + delay(50) + null + } + val mono1 = d.asMono(Dispatchers.Unconfined) + checkMonoValue(mono1, Assert::assertNull) + val mono2 = d.asMono(Dispatchers.Unconfined) + checkMonoValue(mono2, Assert::assertNull) + } + + @Test + fun testDeferredToMonoFail() { + val d = GlobalScope.async { + delay(50) + throw TestRuntimeException("OK") + } + val mono1 = d.asMono(Dispatchers.Unconfined) + checkErroneous(mono1) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + val mono2 = d.asMono(Dispatchers.Unconfined) + checkErroneous(mono2) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + } + + @Test + fun testToFlux() { + val c = GlobalScope.produce { + delay(50) + send("O") + delay(50) + send("K") + } + val flux = c.consumeAsFlow().asFlux(Dispatchers.Unconfined) + checkMonoValue(flux.reduce { t1, t2 -> t1 + t2 }) { + assertEquals("OK", it) + } + } + + @Test + fun testToFluxFail() { + val c = GlobalScope.produce { + delay(50) + send("O") + delay(50) + throw TestException("K") + } + val flux = c.consumeAsFlow().asFlux(Dispatchers.Unconfined) + val mono = mono(Dispatchers.Unconfined) { + var result = "" + try { + flux.collect { result += it } + } catch(e: Throwable) { + check(e is TestException) + result += e.message + } + result + } + checkMonoValue(mono) { + assertEquals("OK", it) + } + } +} diff --git a/reactive/kotlinx-coroutines-reactor/test/FlowAsFluxTest.kt b/reactive/kotlinx-coroutines-reactor/test/FlowAsFluxTest.kt new file mode 100644 index 0000000000..0a94c8e43f --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/FlowAsFluxTest.kt @@ -0,0 +1,149 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import org.reactivestreams.* +import reactor.core.publisher.* +import reactor.util.context.Context +import java.util.concurrent.* +import kotlin.test.* + +@Suppress("ReactiveStreamsSubscriberImplementation") +class FlowAsFluxTest : TestBase() { + @Test + fun testFlowAsFluxContextPropagation() { + val flux = flow { + (1..4).forEach { i -> emit(createMono(i).awaitSingle()) } + } + .asFlux() + .contextWrite(Context.of(1, "1")) + .contextWrite(Context.of(2, "2", 3, "3", 4, "4")) + val list = flux.collectList().block()!! + assertEquals(listOf("1", "2", "3", "4"), list) + } + + private fun createMono(i: Int): Mono = mono { + val ctx = coroutineContext[ReactorContext]!!.context + ctx.getOrDefault(i, "noValue") + } + + @Test + fun testFluxAsFlowContextPropagationWithFlowOn() = runTest { + expect(1) + Flux.create { + it.next("OK") + it.complete() + } + .contextWrite { ctx -> + expect(2) + assertEquals("CTX", ctx.get(1)) + ctx + } + .asFlow() + .flowOn(ReactorContext(Context.of(1, "CTX"))) + .collect { + expect(3) + assertEquals("OK", it) + } + finish(4) + } + + @Test + fun testFluxAsFlowContextPropagationFromScope() = runTest { + expect(1) + withContext(ReactorContext(Context.of(1, "CTX"))) { + Flux.create { + it.next("OK") + it.complete() + } + .contextWrite { ctx -> + expect(2) + assertEquals("CTX", ctx.get(1)) + ctx + } + .asFlow() + .collect { + expect(3) + assertEquals("OK", it) + } + } + finish(4) + } + + @Test + fun testUnconfinedDefaultContext() { + expect(1) + val thread = Thread.currentThread() + fun checkThread() { + assertSame(thread, Thread.currentThread()) + } + flowOf(42).asFlux().subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onSubscribe(s: Subscription) { + expect(2) + subscription = s + subscription.request(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + finish(5) + } + + @Test + fun testConfinedContext() { + expect(1) + val threadName = "FlowAsFluxTest.testConfinedContext" + fun checkThread() { + val currentThread = Thread.currentThread() + assertTrue(currentThread.name.startsWith(threadName), "Unexpected thread $currentThread") + } + val completed = CountDownLatch(1) + newSingleThreadContext(threadName).use { dispatcher -> + flowOf(42).asFlux(dispatcher).subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onSubscribe(s: Subscription) { + expect(2) + subscription = s + subscription.request(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + completed.countDown() + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + completed.await() + } + finish(5) + } +} diff --git a/reactive/kotlinx-coroutines-reactor/test/FluxCompletionStressTest.kt b/reactive/kotlinx-coroutines-reactor/test/FluxCompletionStressTest.kt new file mode 100644 index 0000000000..6c3ca4c6e9 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/FluxCompletionStressTest.kt @@ -0,0 +1,34 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.reactive.* +import org.junit.* +import java.util.* +import kotlin.coroutines.* + +class FluxCompletionStressTest : TestBase() { + private val N_REPEATS = 10_000 * stressTestMultiplier + + private fun CoroutineScope.range(context: CoroutineContext, start: Int, count: Int) = flux(context) { + for (x in start until start + count) send(x) + } + + @Test + fun testCompletion() { + val rnd = Random() + repeat(N_REPEATS) { + val count = rnd.nextInt(5) + runBlocking { + withTimeout(5000) { + var received = 0 + range(Dispatchers.Default, 1, count).collect { x -> + received++ + if (x != received) error("$x != $received") + } + if (received != count) error("$received != $count") + } + } + } + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/test/FluxContextTest.kt b/reactive/kotlinx-coroutines-reactor/test/FluxContextTest.kt new file mode 100644 index 0000000000..a9eb522672 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/FluxContextTest.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.* +import org.junit.Test +import reactor.core.publisher.* +import kotlin.test.* + +class FluxContextTest : TestBase() { + private val dispatcher = newSingleThreadContext("FluxContextTest") + + @After + fun tearDown() { + dispatcher.close() + } + + @Test + fun testFluxCreateAsFlowThread() = runTest { + expect(1) + val mainThread = Thread.currentThread() + val dispatcherThread = withContext(dispatcher) { Thread.currentThread() } + assertTrue(dispatcherThread != mainThread) + Flux.create { + assertEquals(dispatcherThread, Thread.currentThread()) + it.next("OK") + it.complete() + } + .asFlow() + .flowOn(dispatcher) + .collect { + expect(2) + assertEquals("OK", it) + assertEquals(mainThread, Thread.currentThread()) + } + finish(3) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/test/FluxMultiTest.kt b/reactive/kotlinx-coroutines-reactor/test/FluxMultiTest.kt new file mode 100644 index 0000000000..562f0a9b0e --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/FluxMultiTest.kt @@ -0,0 +1,84 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.reactive.* +import org.junit.* +import org.junit.Test +import reactor.core.publisher.* +import java.io.* +import kotlin.test.* + +class FluxMultiTest : TestBase() { + @Test + fun testNumbers() { + val n = 100 * stressTestMultiplier + val flux = flux { + repeat(n) { send(it) } + } + checkMonoValue(flux.collectList()) { list -> + assertEquals((0 until n).toList(), list) + } + } + + @Test + fun testConcurrentStress() { + val n = 10_000 * stressTestMultiplier + val flux = flux { + // concurrent emitters (many coroutines) + val jobs = List(n) { + // launch + launch { + send(it) + } + } + jobs.forEach { it.join() } + } + checkMonoValue(flux.collectList()) { list -> + assertEquals(n, list.size) + assertEquals((0 until n).toList(), list.sorted()) + } + } + + @Test + fun testIteratorResendUnconfined() { + val n = 10_000 * stressTestMultiplier + val flux = flux(Dispatchers.Unconfined) { + Flux.range(0, n).collect { send(it) } + } + checkMonoValue(flux.collectList()) { list -> + assertEquals((0 until n).toList(), list) + } + } + + @Test + fun testIteratorResendPool() { + val n = 10_000 * stressTestMultiplier + val flux = flux { + Flux.range(0, n).collect { send(it) } + } + checkMonoValue(flux.collectList()) { list -> + assertEquals((0 until n).toList(), list) + } + } + + @Test + fun testSendAndCrash() { + val flux = flux { + send("O") + throw IOException("K") + } + val mono = mono { + var result = "" + try { + flux.collect { result += it } + } catch(e: IOException) { + result += e.message + } + result + } + checkMonoValue(mono) { + assertEquals("OK", it) + } + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/test/FluxSingleTest.kt b/reactive/kotlinx-coroutines-reactor/test/FluxSingleTest.kt new file mode 100644 index 0000000000..6bc6dc1ec4 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/FluxSingleTest.kt @@ -0,0 +1,211 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.reactive.* +import org.junit.* +import org.junit.Test +import reactor.core.publisher.* +import java.time.Duration.* +import kotlin.test.* + +class FluxSingleTest : TestBase() { + + @Before + fun setup() { + ignoreLostThreads("parallel-") + } + + @Test + fun testSingleNoWait() { + val flux = flux { + send("OK") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleAwait() = runBlocking { + assertEquals("OK", Flux.just("O").awaitSingle() + "K") + } + + @Test + fun testSingleEmitAndAwait() { + val flux = flux { + send(Flux.just("O").awaitSingle() + "K") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleWithDelay() { + val flux = flux { + send(Flux.just("O").delayElements(ofMillis(50)).awaitSingle() + "K") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleException() { + val flux = flux { + send(Flux.just("O", "K").awaitSingle() + "K") + } + + checkErroneous(flux) { + assert(it is IllegalArgumentException) + } + } + + @Test + fun testAwaitFirst() { + val flux = flux { + send(Flux.just("O", "#").awaitFirst() + "K") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrDefault() { + val flux = flux { + send(Flux.empty().awaitFirstOrDefault("O") + "K") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrDefaultWithValues() { + val flux = flux { + send(Flux.just("O", "#").awaitFirstOrDefault("!") + "K") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrNull() { + val flux = flux { + send(Flux.empty().awaitFirstOrNull() ?: "OK") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrNullWithValues() { + val flux = flux { + send((Flux.just("O", "#").awaitFirstOrNull() ?: "!") + "K") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrElse() { + val flux = flux { + send(Flux.empty().awaitFirstOrElse { "O" } + "K") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrElseWithValues() { + val flux = flux { + send(Flux.just("O", "#").awaitFirstOrElse { "!" } + "K") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitLast() { + val flux = flux { + send(Flux.just("#", "O").awaitLast() + "K") + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromObservable() { + val flux = flux { + try { + send(Flux.error(RuntimeException("O")).awaitFirst()) + } catch (e: RuntimeException) { + send(Flux.just(e.message!!).awaitLast() + "K") + } + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromCoroutine() { + val flux = flux { + throw IllegalStateException(Flux.just("O").awaitSingle() + "K") + } + + checkErroneous(flux) { + assert(it is IllegalStateException) + assertEquals("OK", it.message) + } + } + + @Test + fun testFluxIteration() { + val flux = flux { + var result = "" + Flux.just("O", "K").collect { result += it } + send(result) + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } + + @Test + fun testFluxIterationFailure() { + val flux = flux { + try { + Flux.error(RuntimeException("OK")).collect { fail("Should not be here") } + send("Fail") + } catch (e: RuntimeException) { + send(e.message!!) + } + } + + checkSingleValue(flux) { + assertEquals("OK", it) + } + } +} diff --git a/reactive/kotlinx-coroutines-reactor/test/FluxTest.kt b/reactive/kotlinx-coroutines-reactor/test/FluxTest.kt new file mode 100644 index 0000000000..634bbcd51e --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/FluxTest.kt @@ -0,0 +1,174 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import kotlin.test.* + +class FluxTest : TestBase() { + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val flux = flux(currentDispatcher()) { + expect(4) + send("OK") + } + expect(2) + flux.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val flux = flux(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + flux.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val flux = flux(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + val sub = flux.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testNotifyOnceOnCancellation() = runTest { + expect(1) + val observable = + flux(currentDispatcher()) { + expect(5) + send("OK") + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + expect(11) + } + } + .doOnNext { + expect(6) + assertEquals("OK", it) + } + .doOnCancel { + expect(10) // notified once! + } + expect(2) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(3) + observable.collect { + expect(8) + assertEquals("OK", it) + } + } + expect(4) + yield() // to observable code + expect(7) + yield() // to consuming coroutines + expect(9) + job.cancel() + job.join() + finish(12) + } + + @Test + fun testFailingConsumer() = runTest { + val pub = flux(currentDispatcher()) { + repeat(3) { + expect(it + 1) // expect(1), expect(2) *should* be invoked + send(it) + } + } + try { + pub.collect { + throw TestException() + } + } catch (e: TestException) { + finish(3) + } + } + + @Test + fun testIllegalArgumentException() { + assertFailsWith { flux(Job()) { } } + } + + @Test + fun testLeakedException() = runBlocking { + // Test exception is not reported to global handler + val flow = flux { throw TestException() }.asFlow() + repeat(2000) { + combine(flow, flow) { _, _ -> Unit } + .catch {} + .collect { } + } + } + + /** Tests that `trySend` doesn't throw in `flux`. */ + @Test + fun testTrySendNotThrowing() = runTest { + var producerScope: ProducerScope? = null + expect(1) + val flux = flux(Dispatchers.Unconfined) { + producerScope = this + expect(3) + delay(Long.MAX_VALUE) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + flux.awaitFirstOrNull() + expectUnreached() + } + job.cancel() + expect(4) + val result = producerScope!!.trySend(1) + assertTrue(result.isFailure) + finish(5) + } + + /** Tests that all methods on `flux` fail without closing the channel when attempting to emit `null`. */ + @Test + fun testEmittingNull() = runTest { + val flux = flux { + assertFailsWith { send(null) } + assertFailsWith { trySend(null) } + send("OK") + } + assertEquals("OK", flux.awaitFirstOrNull()) + } +} diff --git a/reactive/kotlinx-coroutines-reactor/test/MonoAwaitStressTest.kt b/reactive/kotlinx-coroutines-reactor/test/MonoAwaitStressTest.kt new file mode 100644 index 0000000000..a40ced83b5 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/MonoAwaitStressTest.kt @@ -0,0 +1,51 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Test +import org.reactivestreams.* +import reactor.core.* +import reactor.core.publisher.* +import kotlin.concurrent.* +import kotlin.test.* + +class MonoAwaitStressTest: TestBase() { + private val N_REPEATS = 10_000 * stressTestMultiplier + + private var completed: Boolean = false + + private var thread: Thread? = null + + /** + * Tests that [Mono.awaitSingleOrNull] does await [CoreSubscriber.onComplete] and does not return + * the value as soon as it has it. + */ + @Test + fun testAwaitingRacingWithCompletion() = runTest { + val mono = object: Mono() { + override fun subscribe(s: CoreSubscriber) { + s.onSubscribe(object : Subscription { + override fun request(n: Long) { + thread = thread { + s.onNext(1) + Thread.yield() + completed = true + s.onComplete() + } + } + + override fun cancel() { + } + }) + } + } + repeat(N_REPEATS) { + thread = null + completed = false + val value = mono.awaitSingleOrNull() + assertTrue(completed, "iteration $it") + assertEquals(1, value) + thread!!.join() + } + } +} diff --git a/reactive/kotlinx-coroutines-reactor/test/MonoTest.kt b/reactive/kotlinx-coroutines-reactor/test/MonoTest.kt new file mode 100644 index 0000000000..80603570a3 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/MonoTest.kt @@ -0,0 +1,404 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.* +import org.junit.Test +import org.reactivestreams.* +import reactor.core.publisher.* +import reactor.util.context.* +import java.time.Duration.* +import java.util.function.* +import kotlin.test.* + +class MonoTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("timer-", "parallel-") + Hooks.onErrorDropped { expectUnreached() } + } + + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val mono = mono(currentDispatcher()) { + expect(4) + "OK" + } + expect(2) + mono.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val mono = mono(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + mono.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicEmpty() = runBlocking { + expect(1) + val mono = mono(currentDispatcher()) { + expect(4) + null + } + expect(2) + mono.subscribe({}, { throw it }, { + expect(5) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val mono = mono(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + // nothing is called on a disposed mono + val sub = mono.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testMonoNoWait() { + val mono = mono { + "OK" + } + + checkMonoValue(mono) { + assertEquals("OK", it) + } + } + + @Test + fun testMonoAwait() = runBlocking { + assertEquals("OK", Mono.just("O").awaitSingle() + "K") + assertEquals("OK", Mono.just("O").awaitSingleOrNull() + "K") + assertFailsWith{ Mono.empty().awaitSingle() } + assertNull(Mono.empty().awaitSingleOrNull()) + } + + /** Tests that calls to [awaitSingleOrNull] (and, thus, to the rest of such functions) throw [CancellationException] + * and unsubscribe from the publisher when their [Job] is cancelled. */ + @Test + fun testAwaitCancellation() = runTest { + expect(1) + val mono = mono { delay(Long.MAX_VALUE) }.doOnSubscribe { expect(3) }.doOnCancel { expect(5) } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + mono.awaitSingleOrNull() + } catch (e: CancellationException) { + expect(6) + throw e + } + } + expect(4) + job.cancelAndJoin() + finish(7) + } + + @Test + fun testMonoEmitAndAwait() { + val mono = mono { + Mono.just("O").awaitSingle() + "K" + } + + checkMonoValue(mono) { + assertEquals("OK", it) + } + } + + @Test + fun testMonoWithDelay() { + val mono = mono { + Flux.just("O").delayElements(ofMillis(50)).awaitSingle() + "K" + } + + checkMonoValue(mono) { + assertEquals("OK", it) + } + } + + @Test + fun testMonoException() { + val mono = mono { + Flux.just("O", "K").awaitSingle() + "K" + } + + checkErroneous(mono) { + assert(it is IllegalArgumentException) + } + } + + @Test + fun testAwaitFirst() { + val mono = mono { + Flux.just("O", "#").awaitFirst() + "K" + } + + checkMonoValue(mono) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitLast() { + val mono = mono { + Flux.just("#", "O").awaitLast() + "K" + } + + checkMonoValue(mono) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromFlux() { + val mono = mono { + try { + Flux.error(RuntimeException("O")).awaitFirst() + } catch (e: RuntimeException) { + Flux.just(e.message!!).awaitLast() + "K" + } + } + + checkMonoValue(mono) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromCoroutine() { + val mono = mono { + throw IllegalStateException(Flux.just("O").awaitSingle() + "K") + } + + checkErroneous(mono) { + assert(it is IllegalStateException) + assertEquals("OK", it.message) + } + } + + @Test + fun testSuppressedException() = runTest { + val mono = mono(currentDispatcher()) { + launch(start = CoroutineStart.ATOMIC) { + throw TestException() // child coroutine fails + } + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() // but parent throws another exception while cleaning up + } + } + try { + mono.awaitSingle() + expectUnreached() + } catch (e: TestException) { + assertIs(e.suppressed[0]) + } + } + + @Test + fun testUnhandledException() = runTest { + expect(1) + var subscription: Subscription? = null + val handler = BiFunction { t, _ -> + assertIs(t) + expect(5) + t + } + + val mono = mono(currentDispatcher()) { + expect(4) + subscription!!.cancel() // cancel our own subscription, so that delay will get cancelled + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException() // would not be able to handle it since mono is disposed + } + }.contextWrite { Context.of("reactor.onOperatorError.local", handler) } + mono.subscribe(object : Subscriber { + override fun onSubscribe(s: Subscription) { + expect(2) + subscription = s + } + override fun onNext(t: Unit?) { expectUnreached() } + override fun onComplete() { expectUnreached() } + override fun onError(t: Throwable) { expectUnreached() } + }) + expect(3) + yield() // run coroutine + finish(6) + } + + @Test + fun testIllegalArgumentException() { + assertFailsWith { mono(Job()) { } } + } + + @Test + fun testExceptionAfterCancellation() = runTest { + // Test exception is not reported to global handler + Flux + .interval(ofMillis(1)) + .switchMap { + mono(coroutineContext) { + timeBomb().awaitSingle() + } + } + .onErrorReturn({ + expect(1) + true + }, 42) + .blockLast() + finish(2) + } + + private fun timeBomb() = Mono.delay(ofMillis(1)).doOnSuccess { throw Exception("something went wrong") } + + @Test + fun testLeakedException() = runBlocking { + // Test exception is not reported to global handler + val flow = mono { throw TestException() }.toFlux().asFlow() + repeat(10000) { + combine(flow, flow) { _, _ -> } + .catch {} + .collect { } + } + } + + /** Test that cancelling a [mono] due to a timeout does throw an exception. */ + @Test + fun testTimeout() { + val mono = mono { + withTimeout(1) { delay(100) } + } + try { + mono.doOnSubscribe { expect(1) } + .doOnNext { expectUnreached() } + .doOnSuccess { expectUnreached() } + .doOnError { expect(2) } + .doOnCancel { expectUnreached() } + .block() + } catch (e: CancellationException) { + expect(3) + } + finish(4) + } + + /** Test that when the reason for cancellation of a [mono] is that the downstream doesn't want its results anymore, + * this is considered normal behavior and exceptions are not propagated. */ + @Test + fun testDownstreamCancellationDoesNotThrow() = runTest { + var i = 0 + /** Attach a hook that handles exceptions from publishers that are known to be disposed of. We don't expect it + * to be fired in this case, as the reason for the publisher in this test to accept an exception is simply + * cancellation from the downstream. */ + Hooks.onOperatorError("testDownstreamCancellationDoesNotThrow") { t, a -> + expectUnreached() + t + } + /** A Mono that doesn't emit a value and instead waits indefinitely. */ + val mono = mono(Dispatchers.Unconfined) { expect(5 * i + 3); delay(Long.MAX_VALUE) } + .doOnSubscribe { expect(5 * i + 2) } + .doOnNext { expectUnreached() } + .doOnSuccess { expectUnreached() } + .doOnError { expectUnreached() } + .doOnCancel { expect(5 * i + 4) } + val n = 1000 + repeat(n) { + i = it + expect(5 * i + 1) + mono.awaitCancelAndJoin() + expect(5 * i + 5) + } + finish(5 * n + 1) + Hooks.resetOnOperatorError("testDownstreamCancellationDoesNotThrow") + } + + /** Test that, when [Mono] is cancelled by the downstream and throws during handling the cancellation, the resulting + * error is propagated to [Hooks.onOperatorError]. */ + @Test + fun testRethrowingDownstreamCancellation() = runTest { + var i = 0 + /** Attach a hook that handles exceptions from publishers that are known to be disposed of. We expect it + * to be fired in this case. */ + Hooks.onOperatorError("testDownstreamCancellationDoesNotThrow") { t, a -> + expect(i * 6 + 5) + t + } + /** A Mono that doesn't emit a value and instead waits indefinitely, and, when cancelled, throws. */ + val mono = mono(Dispatchers.Unconfined) { + expect(i * 6 + 3) + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + throw TestException() + } + } + .doOnSubscribe { expect(i * 6 + 2) } + .doOnNext { expectUnreached() } + .doOnSuccess { expectUnreached() } + .doOnError { expectUnreached() } + .doOnCancel { expect(i * 6 + 4) } + val n = 1000 + repeat(n) { + i = it + expect(i * 6 + 1) + mono.awaitCancelAndJoin() + expect(i * 6 + 6) + } + finish(n * 6 + 1) + Hooks.resetOnOperatorError("testDownstreamCancellationDoesNotThrow") + } + + /** Run the given [Mono], cancel it, wait for the cancellation handler to finish, and return only then. + * + * Will not work in the general case, but here, when the publisher uses [Dispatchers.Unconfined], this seems to + * ensure that the cancellation handler will have nowhere to execute but serially with the cancellation. */ + private suspend fun Mono.awaitCancelAndJoin() = coroutineScope { + async(start = CoroutineStart.UNDISPATCHED) { + awaitSingleOrNull() + }.cancelAndJoin() + } +} diff --git a/reactive/kotlinx-coroutines-reactor/test/ReactorContextTest.kt b/reactive/kotlinx-coroutines-reactor/test/ReactorContextTest.kt new file mode 100644 index 0000000000..5bc6d45115 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/ReactorContextTest.kt @@ -0,0 +1,108 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import reactor.core.publisher.* +import reactor.util.context.* +import kotlin.coroutines.* +import kotlin.test.* + +class ReactorContextTest : TestBase() { + @Test + fun testMonoHookedContext() = runBlocking { + val mono = mono(Context.of(1, "1", 7, "7").asCoroutineContext()) { + val ctx = reactorContext() + buildString { + (1..7).forEach { append(ctx.getOrDefault(it, "noValue")) } + } + } .contextWrite(Context.of(2, "2", 3, "3", 4, "4", 5, "5")) + .contextWrite { ctx -> ctx.put(6, "6") } + assertEquals(mono.awaitSingle(), "1234567") + } + + @Test + fun testFluxContext() { + val flux = flux(Context.of(1, "1", 7, "7").asCoroutineContext()) { + val ctx = reactorContext() + (1..7).forEach { send(ctx.getOrDefault(it, "noValue")) } + } + .contextWrite(Context.of(2, "2", 3, "3", 4, "4", 5, "5")) + .contextWrite { ctx -> ctx.put(6, "6") } + val list = flux.collectList().block()!! + assertEquals((1..7).map { it.toString() }, list) + } + + @Test + fun testAwait() = runBlocking(Context.of(3, "3").asCoroutineContext()) { + val result = mono(Context.of(1, "1").asCoroutineContext()) { + val ctx = reactorContext() + buildString { + (1..3).forEach { append(ctx.getOrDefault(it, "noValue")) } + } + } .contextWrite(Context.of(2, "2")) + .awaitSingle() + assertEquals(result, "123") + } + + @Test + fun testMonoAwaitContextPropagation() = runBlocking(Context.of(7, "7").asCoroutineContext()) { + assertEquals(createMono().awaitSingle(), "7") + assertEquals(createMono().awaitSingleOrNull(), "7") + } + + @Test + fun testFluxAwaitContextPropagation() = runBlocking( + Context.of(1, "1", 2, "2", 3, "3").asCoroutineContext() + ) { + assertEquals(createFlux().awaitFirst(), "1") + assertEquals(createFlux().awaitFirstOrDefault("noValue"), "1") + assertEquals(createFlux().awaitFirstOrNull(), "1") + assertEquals(createFlux().awaitFirstOrElse { "noValue" }, "1") + assertEquals(createFlux().awaitLast(), "3") + } + + private fun createMono(): Mono = mono { + val ctx = reactorContext() + ctx.getOrDefault(7, "noValue") + } + + + private fun createFlux(): Flux = flux { + val ctx = reactorContext() + (1..3).forEach { send(ctx.getOrDefault(it, "noValue")) } + } + + @Test + fun testFlowToFluxContextPropagation() = runBlocking( + Context.of(1, "1", 2, "2", 3, "3").asCoroutineContext() + ) { + var i = 0 + // call "collect" on the converted Flow + bar().collect { str -> + i++; assertEquals(str, i.toString()) + } + assertEquals(i, 3) + } + + @Test + fun testFlowToFluxDirectContextPropagation() = runBlocking( + Context.of(1, "1", 2, "2", 3, "3").asCoroutineContext() + ) { + // convert resulting flow to channel using "produceIn" + val channel = bar().produceIn(this) + val list = channel.toList() + assertEquals(listOf("1", "2", "3"), list) + } + + private fun bar(): Flow = flux { + val ctx = reactorContext() + (1..3).forEach { send(ctx.getOrDefault(it, "noValue")) } + }.asFlow() + + private suspend fun reactorContext() = + coroutineContext[ReactorContext]!!.context +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-reactor/test/SchedulerTest.kt b/reactive/kotlinx-coroutines-reactor/test/SchedulerTest.kt new file mode 100644 index 0000000000..7437d3a1b8 --- /dev/null +++ b/reactive/kotlinx-coroutines-reactor/test/SchedulerTest.kt @@ -0,0 +1,31 @@ +package kotlinx.coroutines.reactor + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.Before +import org.junit.Test +import reactor.core.scheduler.Schedulers +import kotlin.test.* + +class SchedulerTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("single-") + } + + @Test + fun testSingleScheduler(): Unit = runBlocking { + expect(1) + val mainThread = Thread.currentThread() + withContext(Schedulers.single().asCoroutineDispatcher()) { + val t1 = Thread.currentThread() + assertNotSame(t1, mainThread) + expect(2) + delay(100) + val t2 = Thread.currentThread() + assertNotSame(t2, mainThread) + expect(3) + } + finish(4) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx-example/pom.xml b/reactive/kotlinx-coroutines-rx-example/pom.xml deleted file mode 100644 index 9c8ea5fd3d..0000000000 --- a/reactive/kotlinx-coroutines-rx-example/pom.xml +++ /dev/null @@ -1,74 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - ../../pom.xml - - - kotlinx-coroutines-rx-example - jar - - - src/main/kotlin - - - org.apache.maven.plugins - maven-deploy-plugin - - true - - - - - - - - io.reactivex - rxjava - 1.1.5 - - - com.squareup.retrofit2 - retrofit - 2.1.0 - - - com.squareup.retrofit2 - converter-gson - 2.1.0 - - - com.squareup.retrofit2 - adapter-rxjava - 2.1.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines-rx1 - ${project.version} - compile - - - - diff --git a/reactive/kotlinx-coroutines-rx-example/src/main/kotlin/main.kt b/reactive/kotlinx-coroutines-rx-example/src/main/kotlin/main.kt deleted file mode 100644 index 5dbe9fedd4..0000000000 --- a/reactive/kotlinx-coroutines-rx-example/src/main/kotlin/main.kt +++ /dev/null @@ -1,66 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.rx1.awaitSingle -import retrofit2.Retrofit -import retrofit2.adapter.rxjava.RxJavaCallAdapterFactory -import retrofit2.converter.gson.GsonConverterFactory -import retrofit2.http.GET -import retrofit2.http.Path -import rx.Observable - -interface GitHub { - @GET("/repos/{owner}/{repo}/contributors") - fun contributors( - @Path("owner") owner: String, - @Path("repo") repo: String - ): Observable> - - @GET("users/{user}/repos") - fun listRepos(@Path("user") user: String): Observable> -} - -data class Contributor(val login: String, val contributions: Int) -data class Repo(val name: String) - -fun main(args: Array) { - val retrofit = Retrofit.Builder().apply { - baseUrl("https://api.github.com/") - addConverterFactory(GsonConverterFactory.create()) - addCallAdapterFactory(RxJavaCallAdapterFactory.create()) - }.build() - - val github = retrofit.create(GitHub::class.java) - - launch(CommonPool) { - val contributors = - github.contributors("JetBrains", "Kotlin") - .awaitSingle().take(10) - - for ((name, contributions) in contributors) { - println("$name has $contributions contributions, other repos: ") - - val otherRepos = - github.listRepos(name).awaitSingle() - .map(Repo::name).joinToString(", ") - - println(otherRepos) - } - } -} diff --git a/reactive/kotlinx-coroutines-rx1/README.md b/reactive/kotlinx-coroutines-rx1/README.md deleted file mode 100644 index d5f4df481a..0000000000 --- a/reactive/kotlinx-coroutines-rx1/README.md +++ /dev/null @@ -1,62 +0,0 @@ -# Module kotlinx-coroutines-rx1 - -Utilities for [RxJava 1.x](https://github.com/ReactiveX/RxJava/tree/1.x). - -Coroutine builders: - -| **Name** | **Result** | **Scope** | **Description** -| --------------- | ----------------------------- | ---------------- | --------------- -| [rxCompletable] | `Completable` | [CoroutineScope] | Cold completable that starts coroutine on subscribe -| [rxSingle] | `Single` | [CoroutineScope] | Cold single that starts coroutine on subscribe -| [rxObservable] | `Observable` | [ProducerScope] | Cold observable that starts coroutine on subscribe - -Suspending extension functions and suspending iteration: - -| **Name** | **Description** -| -------- | --------------- -| [Completable.awaitCompleted][rx.Completable.awaitCompleted] | Awaits for completion of the completable value -| [Single.await][rx.Single.await] | Awaits for completion of the single value and returns it -| [Observable.awaitFirst][rx.Observable.awaitFirst] | Returns the first value from the given observable -| [Observable.awaitFirstOrDefault][rx.Observable.awaitFirstOrDefault] | Returns the first value from the given observable or default -| [Observable.awaitLast][rx.Observable.awaitFirst] | Returns the last value from the given observable -| [Observable.awaitSingle][rx.Observable.awaitSingle] | Returns the single value from the given observable -| [Observable.open][rx.Observable.open] | Subscribes to observable and returns [ReceiveChannel] -| [Observable.iterator][rx.Observable.iterator] | Subscribes to observable and returns [ChannelIterator] - -Conversion functions: - -| **Name** | **Description** -| -------- | --------------- -| [Job.asCompletable][kotlinx.coroutines.experimental.Job.asCompletable] | Converts job to hot completable -| [Deferred.asSingle][kotlinx.coroutines.experimental.Deferred.asSingle] | Converts deferred value to hot single -| [ReceiveChannel.asObservable][kotlinx.coroutines.experimental.channels.ReceiveChannel.asObservable] | Converts streaming channel to hot observable - - - - -[CoroutineScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-scope/index.html - -[ProducerScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-producer-scope/index.html -[ReceiveChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/index.html -[ChannelIterator]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel-iterator/index.html - - - -[rxCompletable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx-completable.html -[rxSingle]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx-single.html -[rxObservable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx-observable.html -[rx.Completable.awaitCompleted]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx.-completable/await-completed.html -[rx.Single.await]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx.-single/await.html -[rx.Observable.awaitFirst]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx.-observable/await-first.html -[rx.Observable.awaitFirstOrDefault]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx.-observable/await-first-or-default.html -[rx.Observable.awaitSingle]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx.-observable/await-single.html -[rx.Observable.open]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx.-observable/open.html -[rx.Observable.iterator]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/rx.-observable/iterator.html -[kotlinx.coroutines.experimental.Job.asCompletable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/kotlinx.coroutines.experimental.-job/as-completable.html -[kotlinx.coroutines.experimental.Deferred.asSingle]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/kotlinx.coroutines.experimental.-deferred/as-single.html -[kotlinx.coroutines.experimental.channels.ReceiveChannel.asObservable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx1/kotlinx.coroutines.experimental.rx1/kotlinx.coroutines.experimental.channels.-receive-channel/as-observable.html - - -# Package kotlinx.coroutines.experimental.rx1 - -Utilities for [RxJava 1.x](https://github.com/ReactiveX/RxJava/tree/1.x). diff --git a/reactive/kotlinx-coroutines-rx1/pom.xml b/reactive/kotlinx-coroutines-rx1/pom.xml deleted file mode 100644 index e7acc64999..0000000000 --- a/reactive/kotlinx-coroutines-rx1/pom.xml +++ /dev/null @@ -1,58 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - ../../pom.xml - - - kotlinx-coroutines-rx1 - jar - - - src/main/kotlin - src/test/kotlin - - - - - io.reactivex - rxjava - 1.2.6 - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - tests - test - - - - diff --git a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxAwait.kt b/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxAwait.kt deleted file mode 100644 index bd0b271bf0..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxAwait.kt +++ /dev/null @@ -1,121 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.CancellableContinuation -import kotlinx.coroutines.experimental.CancellationException -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.suspendCancellableCoroutine -import rx.* - -// ------------------------ Completable ------------------------ - -/** - * Awaits for completion of this completable without blocking a thread. - * Returns `Unit` or throws the corresponding exception if this completable had produced error. - * - * This suspending function is cancellable. If the [Job] of the invoking coroutine is completed while this - * suspending function is suspended, this function immediately resumes with [CancellationException]. - */ -public suspend fun Completable.awaitCompleted(): Unit = suspendCancellableCoroutine { cont -> - subscribe(object : CompletableSubscriber { - override fun onSubscribe(s: Subscription) { cont.unsubscribeOnCompletion(s) } - override fun onCompleted() { cont.resume(Unit) } - override fun onError(e: Throwable) { cont.resumeWithException(e) } - }) -} - -// ------------------------ Single ------------------------ - -/** - * Awaits for completion of the single value without blocking a thread and - * returns the resulting value or throws the corresponding exception if this single had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - */ -public suspend fun Single.await(): T = suspendCancellableCoroutine { cont -> - cont.unsubscribeOnCompletion(subscribe(object : SingleSubscriber() { - override fun onSuccess(t: T) { cont.resume(t) } - override fun onError(error: Throwable) { cont.resumeWithException(error) } - })) -} - -// ------------------------ Observable ------------------------ - -/** - * Awaits for the first value from the given observable without blocking a thread and - * returns the resulting value or throws the corresponding exception if this observable had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * @throws NoSuchElementException if observable does not emit any value - */ -public suspend fun Observable.awaitFirst(): T = first().awaitOne() - -/** - * Awaits for the first value from the given observable or the [default] value if none is emitted without blocking a - * thread and returns the resulting value or throws the corresponding exception if this observable had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - */ -public suspend fun Observable.awaitFirstOrDefault(default: T): T = firstOrDefault(default).awaitOne() - -/** - * Awaits for the last value from the given observable without blocking a thread and - * returns the resulting value or throws the corresponding exception if this observable had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * @throws NoSuchElementException if observable does not emit any value - */ -public suspend fun Observable.awaitLast(): T = last().awaitOne() - -/** - * Awaits for the single value from the given observable without blocking a thread and - * returns the resulting value or throws the corresponding exception if this observable had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * @throws NoSuchElementException if observable does not emit any value - * @throws IllegalArgumentException if publisher emits more than one value - */ -public suspend fun Observable.awaitSingle(): T = single().awaitOne() - -// ------------------------ private ------------------------ - -private suspend fun Observable.awaitOne(): T = suspendCancellableCoroutine { cont -> - cont.unsubscribeOnCompletion(subscribe(object : Subscriber() { - override fun onStart() { request(1) } - override fun onNext(t: T) { cont.resume(t) } - override fun onCompleted() { if (cont.isActive) cont.resumeWithException(IllegalStateException("Should have invoked onNext")) } - override fun onError(e: Throwable) { cont.resumeWithException(e) } - })) -} - -private fun CancellableContinuation.unsubscribeOnCompletion(sub: Subscription) { - invokeOnCompletion { sub.unsubscribe() } -} diff --git a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxChannel.kt b/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxChannel.kt deleted file mode 100644 index fa6d45932e..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxChannel.kt +++ /dev/null @@ -1,125 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.channels.LinkedListChannel -import kotlinx.coroutines.experimental.channels.SubscriptionReceiveChannel -import rx.Observable -import rx.Subscriber -import rx.Subscription -import java.util.concurrent.atomic.AtomicIntegerFieldUpdater - -/** - * Subscribes to this [Observable] and returns a channel to receive elements emitted by it. - * The resulting channel shall be [closed][SubscriptionReceiveChannel.close] to unsubscribe from this observable. - */ -public fun Observable.open(): SubscriptionReceiveChannel { - val channel = SubscriptionChannel() - val subscription = subscribe(channel.subscriber) - channel.subscription = subscription - if (channel.isClosedForSend) subscription.unsubscribe() - return channel -} - -/** - * Subscribes to this [Observable] and returns an iterator to receive elements emitted by it. - * - * This is a shortcut for `open().iterator()`. See [open] if you need an ability to manually - * unsubscribe from the observable. - */ -@Suppress("DeprecatedCallableAddReplaceWith") -@Deprecated(message = -"This iteration operator for `for (x in source) { ... }` loop is deprecated, " + - "because it leaves code vulnerable to leaving unclosed subscriptions on exception. " + - "Use `source.consumeEach { x -> ... }`.") -public operator fun Observable.iterator() = open().iterator() - -/** - * Subscribes to this [Observable] and performs the specified action for each received element. - */ -// :todo: make it inline when this bug is fixed: https://youtrack.jetbrains.com/issue/KT-16448 -public suspend fun Observable.consumeEach(action: suspend (T) -> Unit) { - open().use { channel -> - for (x in channel) action(x) - } -} - -private class SubscriptionChannel : LinkedListChannel(), SubscriptionReceiveChannel { - @JvmField - val subscriber: ChannelSubscriber = ChannelSubscriber() - - @Volatile - @JvmField - var subscription: Subscription? = null - - @Volatile - @JvmField - var balance = 0 // request balance from cancelled receivers - - private companion object { - @JvmField - val BALANCE: AtomicIntegerFieldUpdater> = - AtomicIntegerFieldUpdater.newUpdater(SubscriptionChannel::class.java, "balance") - } - - // AbstractChannel overrides - override fun onEnqueuedReceive() { - while (true) { // lock-free loop on balance - val balance = this.balance - if (balance == 0) { - subscriber.requestOne() - return - } - if (BALANCE.compareAndSet(this, balance, balance - 1)) return - } - } - - override fun onCancelledReceive() { - BALANCE.incrementAndGet(this) - } - - override fun afterClose(cause: Throwable?) { - subscription?.unsubscribe() - } - - // Subscription overrides - override fun close() { - close(cause = null) - } - - inner class ChannelSubscriber: Subscriber() { - fun requestOne() { - request(1) - } - - override fun onStart() { - request(0) // init backpressure, but don't request anything yet - } - - override fun onNext(t: T) { - offer(t) - } - - override fun onCompleted() { - close(cause = null) - } - - override fun onError(e: Throwable) { - close(cause = e) - } - } -} diff --git a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxCompletable.kt b/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxCompletable.kt deleted file mode 100644 index 4e7a203bea..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxCompletable.kt +++ /dev/null @@ -1,63 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.CoroutineScope -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.newCoroutineContext -import rx.* -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Creates cold [Completable] that runs a given [block] in a coroutine. - * Every time the returned completable is subscribed, it starts a new coroutine in the specified [context]. - * Unsubscribing cancels running coroutine. - * - * | **Coroutine action** | **Signal to subscriber** - * | ------------------------------------- | ------------------------ - * | Completes successfully | `onCompleted` - * | Failure with exception or unsubscribe | `onError` - */ -public fun rxCompletable( - context: CoroutineContext, - block: suspend CoroutineScope.() -> Unit -): Completable = Completable.create { subscriber -> - val newContext = newCoroutineContext(context) - val coroutine = RxCompletableCoroutine(newContext, subscriber) - coroutine.initParentJob(context[Job]) - subscriber.onSubscribe(coroutine) - block.startCoroutine(coroutine, coroutine) -} - -private class RxCompletableCoroutine( - override val parentContext: CoroutineContext, - private val subscriber: CompletableSubscriber -) : AbstractCoroutine(true), Subscription { - @Suppress("UNCHECKED_CAST") - override fun afterCompletion(state: Any?, mode: Int) { - if (state is CompletedExceptionally) - subscriber.onError(state.exception) - else - subscriber.onCompleted() - } - - // Subscription impl - override fun isUnsubscribed(): Boolean = isCompleted - override fun unsubscribe() { cancel() } -} diff --git a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxConvert.kt b/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxConvert.kt deleted file mode 100644 index e06868038c..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxConvert.kt +++ /dev/null @@ -1,83 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.Deferred -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.channels.ReceiveChannel -import rx.* -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Converts this job to the hot reactive completable that signals - * with [onCompleted][CompletableSubscriber.onCompleted] when the corresponding job completes. - * - * Every subscriber gets the signal at the same time. - * Unsubscribing from the resulting completable **does not** affect the original job in any way. - * - * @param context -- the coroutine context from which the resulting completable is going to be signalled - */ -public fun Job.asCompletable(context: CoroutineContext): Completable = rxCompletable(context) { - this@asCompletable.join() -} - -/** - * Converts this deferred value to the hot reactive single that signals either - * [onSuccess][SingleSubscriber.onSuccess] or [onError][SingleSubscriber.onError]. - * - * Every subscriber gets the same completion value. - * Unsubscribing from the resulting single **does not** affect the original deferred value in any way. - * - * @param context -- the coroutine context from which the resulting single is going to be signalled - */ -public fun Deferred.asSingle(context: CoroutineContext): Single = rxSingle(context) { - this@asSingle.await() -} - -/** - * Converts a stream of elements received from the channel to the hot reactive observable. - * - * Every subscriber receives values from this channel in **fan-out** fashion. If the are multiple subscribers, - * they'll receive values in round-robin way. - * - * @param context -- the coroutine context from which the resulting observable is going to be signalled - */ -public fun ReceiveChannel.asObservable(context: CoroutineContext): Observable = rxObservable(context) { - for (t in this@asObservable) - send(t) -} - -/** - * @suppress **Deprecated**: Renamed to [asCompletable] - */ -@Deprecated(message = "Renamed to `asCompletable`", - replaceWith = ReplaceWith("asCompletable(context)")) -public fun Job.toCompletable(context: CoroutineContext): Completable = asCompletable(context) - -/** - * @suppress **Deprecated**: Renamed to [asSingle] - */ -@Deprecated(message = "Renamed to `asSingle`", - replaceWith = ReplaceWith("asSingle(context)")) -public fun Deferred.toSingle(context: CoroutineContext): Single = asSingle(context) - -/** - * @suppress **Deprecated**: Renamed to [asObservable] - */ -@Deprecated(message = "Renamed to `asObservable`", - replaceWith = ReplaceWith("asObservable(context)")) -public fun ReceiveChannel.toObservable(context: CoroutineContext): Observable = asObservable(context) diff --git a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxObservable.kt b/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxObservable.kt deleted file mode 100644 index 60ad4cdfa6..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxObservable.kt +++ /dev/null @@ -1,224 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.channels.ClosedSendChannelException -import kotlinx.coroutines.experimental.channels.ProducerScope -import kotlinx.coroutines.experimental.channels.SendChannel -import kotlinx.coroutines.experimental.handleCoroutineException -import kotlinx.coroutines.experimental.newCoroutineContext -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlinx.coroutines.experimental.sync.Mutex -import rx.Observable -import rx.Producer -import rx.Subscriber -import rx.Subscription -import java.util.concurrent.atomic.AtomicLongFieldUpdater -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Creates cold [Observable] that runs a given [block] in a coroutine. - * Every time the returned observable is subscribed, it starts a new coroutine in the specified [context]. - * Coroutine emits items with `send`. Unsubscribing cancels running coroutine. - * - * Invocations of `send` are suspended appropriately when subscribers apply back-pressure and to ensure that - * `onNext` is not invoked concurrently. - * - * | **Coroutine action** | **Signal to subscriber** - * | -------------------------------------------- | ------------------------ - * | `send` | `onNext` - * | Normal completion or `close` without cause | `onCompleted` - * | Failure with exception or `close` with cause | `onError` - */ -public fun rxObservable( - context: CoroutineContext, - block: suspend ProducerScope.() -> Unit -): Observable = Observable.create { subscriber -> - val newContext = newCoroutineContext(context) - val coroutine = RxObservableCoroutine(newContext, subscriber) - coroutine.initParentJob(context[Job]) - subscriber.setProducer(coroutine) // do it first (before starting coroutine), to avoid unnecessary suspensions - subscriber.add(coroutine) - block.startCoroutine(coroutine, coroutine) -} - -private class RxObservableCoroutine( - override val parentContext: CoroutineContext, - private val subscriber: Subscriber -) : AbstractCoroutine(true), ProducerScope, Producer, Subscription { - override val channel: SendChannel get() = this - - // Mutex is locked when either nRequested == 0 or while subscriber.onXXX is being invoked - private val mutex = Mutex(locked = true) - - @Volatile - private var nRequested: Long = 0 // < 0 when closed (CLOSED or SIGNALLED) - - companion object { - private val N_REQUESTED = AtomicLongFieldUpdater - .newUpdater(RxObservableCoroutine::class.java, "nRequested") - - private const val CLOSED_MESSAGE = "This subscription had already closed (completed or failed)" - - private const val CLOSED = -1L // closed, but have not signalled onCompleted/onError yet - private const val SIGNALLED = -2L // already signalled subscriber onCompleted/onError - } - - override val isClosedForSend: Boolean get() = isCompleted - override val isFull: Boolean = mutex.isLocked - override fun close(cause: Throwable?): Boolean = cancel(cause) - - private fun sendException() = - (state as? CompletedExceptionally)?.cause ?: ClosedSendChannelException(CLOSED_MESSAGE) - - override fun offer(element: T): Boolean { - if (!mutex.tryLock()) return false - doLockedNext(element) - return true - } - - public suspend override fun send(element: T): Unit { - // fast-path -- try send without suspension - if (offer(element)) return - // slow-path does suspend - return sendSuspend(element) - } - - private suspend fun sendSuspend(element: T) { - mutex.lock() - doLockedNext(element) - } - - override fun registerSelectSend(select: SelectInstance, element: T, block: suspend () -> R) = - mutex.registerSelectLock(select, null) { - doLockedNext(element) - block() - } - - // assert: mutex.isLocked() - private fun doLockedNext(elem: T) { - // check if already closed for send - if (isCompleted) { - doLockedSignalCompleted() - throw sendException() - } - // notify subscriber - try { - subscriber.onNext(elem) - } catch (e: Throwable) { - try { - if (!cancel(e)) - handleCoroutineException(context, e) - } finally { - doLockedSignalCompleted() - } - throw sendException() - } - // now update nRequested - while (true) { // lock-free loop on nRequested - val cur = nRequested - if (cur < 0) break // closed from inside onNext => unlock - if (cur == Long.MAX_VALUE) break // no back-pressure => unlock - val upd = cur - 1 - if (N_REQUESTED.compareAndSet(this, cur, upd)) { - if (upd == 0L) return // return to keep locked due to back-pressure - break // unlock if upd > 0 - } - } - /* - There is no sense to check for `isCompleted` before doing `unlock`, because completion might - happen after this check and before `unlock` (see `afterCompleted` that does not do anything - if it fails to acquire the lock that we are still holding). - We have to recheck `isCompleted` after `unlock` anyway. - */ - mutex.unlock() - // recheck isCompleted - if (isCompleted && mutex.tryLock()) - doLockedSignalCompleted() - } - - // assert: mutex.isLocked() - private fun doLockedSignalCompleted() { - try { - if (nRequested >= CLOSED) { - nRequested = SIGNALLED // we'll signal onError/onCompleted (that the final state -- no CAS needed) - val state = this.state - try { - if (state is CompletedExceptionally && state.cause != null) - subscriber.onError(state.cause) - else - subscriber.onCompleted() - } catch (e: Throwable) { - handleCoroutineException(context, e) - } - } - } finally { - mutex.unlock() - } - } - - override fun request(n: Long) { - if (n < 0) { - cancel(IllegalArgumentException("Must request non-negative number, but $n requested")) - return - } - while (true) { // lock-free loop for nRequested - val cur = nRequested - if (cur < 0) return // already closed for send, ignore requests - var upd = cur + n - if (upd < 0 || n == Long.MAX_VALUE) - upd = Long.MAX_VALUE - if (cur == upd) return // nothing to do - if (N_REQUESTED.compareAndSet(this, cur, upd)) { - // unlock the mutex when we don't have back-pressure anymore - if (cur == 0L) { - mutex.unlock() - // recheck isCompleted - if (isCompleted && mutex.tryLock()) - doLockedSignalCompleted() - } - return - } - } - } - - override fun afterCompletion(state: Any?, mode: Int) { - while (true) { // lock-free loop for nRequested - val cur = nRequested - if (cur == SIGNALLED) return // some other thread holding lock already signalled completion - check(cur >= 0) // no other thread could have marked it as CLOSED, because afterCompletion is invoked once - if (!N_REQUESTED.compareAndSet(this, cur, CLOSED)) continue // retry on failed CAS - // Ok -- marked as CLOSED, now can unlock the mutex if it was locked due to backpressure - if (cur == 0L) { - doLockedSignalCompleted() - } else { - // otherwise mutex was either not locked or locked in concurrent onNext... try lock it to signal completion - if (mutex.tryLock()) - doLockedSignalCompleted() - // Note: if failed `tryLock`, then `doLockedNext` will signal after performing `unlock` - } - return // done anyway - } - } - - // Subscription impl - override fun isUnsubscribed(): Boolean = isCompleted - override fun unsubscribe() { cancel() } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxSingle.kt b/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxSingle.kt deleted file mode 100644 index e4322f060a..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/main/kotlin/kotlinx/coroutines/experimental/rx1/RxSingle.kt +++ /dev/null @@ -1,65 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.CoroutineScope -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.newCoroutineContext -import rx.Single -import rx.SingleSubscriber -import rx.Subscription -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Creates cold [Single] that runs a given [block] in a coroutine. - * Every time the returned single is subscribed, it starts a new coroutine in the specified [context]. - * Coroutine returns a single value. Unsubscribing cancels running coroutine. - * - * | **Coroutine action** | **Signal to subscriber** - * | ------------------------------------- | ------------------------ - * | Returns a value | `onSuccess` - * | Failure with exception or unsubscribe | `onError` - */ -public fun rxSingle( - context: CoroutineContext, - block: suspend CoroutineScope.() -> T -): Single = Single.create { subscriber -> - val newContext = newCoroutineContext(context) - val coroutine = RxSingleCoroutine(newContext, subscriber) - coroutine.initParentJob(context[Job]) - subscriber.add(coroutine) - block.startCoroutine(coroutine, coroutine) -} - -private class RxSingleCoroutine( - override val parentContext: CoroutineContext, - private val subscriber: SingleSubscriber -) : AbstractCoroutine(true), Subscription { - @Suppress("UNCHECKED_CAST") - override fun afterCompletion(state: Any?, mode: Int) { - if (state is CompletedExceptionally) - subscriber.onError(state.exception) - else - subscriber.onSuccess(state as T) - } - - // Subscription impl - override fun isUnsubscribed(): Boolean = isCompleted - override fun unsubscribe() { cancel() } -} diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/Check.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/Check.kt deleted file mode 100644 index ffb91ffe18..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/Check.kt +++ /dev/null @@ -1,57 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import rx.Observable -import rx.Single - -fun checkSingleValue( - observable: Observable, - checker: (T) -> Unit -) { - val singleValue = observable.toBlocking().single() - checker(singleValue) -} - -fun checkErroneous( - observable: Observable<*>, - checker: (Throwable) -> Unit -) { - val singleNotification = observable.materialize().toBlocking().single() - checker(singleNotification.throwable) -} - -fun checkSingleValue( - single: Single, - checker: (T) -> Unit -) { - val singleValue = single.toBlocking().value() - checker(singleValue) -} - -fun checkErroneous( - single: Single<*>, - checker: (Throwable) -> Unit -) { - try { - single.toBlocking().value() - error("Should have failed") - } catch (e: Throwable) { - checker(e) - } -} - diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/CompletableTest.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/CompletableTest.kt deleted file mode 100644 index c3a952a223..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/CompletableTest.kt +++ /dev/null @@ -1,114 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.CancellationException -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert.assertThat -import org.junit.Test - -class CompletableTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(4) - } - expect(2) - completable.subscribe { - expect(5) - } - expect(3) - yield() // to completable coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - completable.subscribe({ - expectUnreached() - }, { error -> - expect(5) - assertThat(error, IsInstanceOf(RuntimeException::class.java)) - assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to completable coroutine - finish(6) - } - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - val sub = completable.subscribe({ - expectUnreached() - }, { error -> - expect(6) - assertThat(error, IsInstanceOf(CancellationException::class.java)) - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.unsubscribe() // will cancel coroutine - yield() - finish(7) - } - - @Test - fun testAwaitSuccess() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(3) - } - expect(2) - completable.awaitCompleted() // shall launch coroutine - finish(4) - } - - @Test - fun testAwaitFailure() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(3) - throw RuntimeException("OK") - } - expect(2) - try { - completable.awaitCompleted() // shall launch coroutine and throw exception - expectUnreached() - } catch (e: RuntimeException) { - finish(4) - assertThat(e.message, IsEqual("OK")) - } - } -} diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ConvertTest.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ConvertTest.kt deleted file mode 100644 index eed771f815..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ConvertTest.kt +++ /dev/null @@ -1,131 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.produce -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert.assertThat -import org.junit.Test - -class ConvertTest : TestBase() { - class TestException(s: String): RuntimeException(s) - - @Test - fun testToCompletableSuccess() = runBlocking { - expect(1) - val job = launch(context) { - expect(3) - } - val completable = job.asCompletable(context) - completable.subscribe { - expect(4) - } - expect(2) - yield() - finish(5) - } - - @Test - fun testToCompletableFail() = runBlocking { - expect(1) - val job = async(context + NonCancellable) { // don't kill parent on exception - expect(3) - throw RuntimeException("OK") - } - val completable = job.asCompletable(context) - completable.subscribe { - expect(4) - } - expect(2) - yield() - finish(5) - } - - @Test - fun testToSingle() { - val d = async(CommonPool) { - delay(50) - "OK" - } - val single1 = d.asSingle(Unconfined) - checkSingleValue(single1) { - assertThat(it, IsEqual("OK")) - } - val single2 = d.asSingle(Unconfined) - checkSingleValue(single2) { - assertThat(it, IsEqual("OK")) - } - } - - @Test - fun testToSingleFail() { - val d = async(CommonPool) { - delay(50) - throw TestException("OK") - } - val single1 = d.asSingle(Unconfined) - checkErroneous(single1) { - assertThat(it, IsInstanceOf(TestException::class.java)) - assertThat(it.message, IsEqual("OK")) - } - val single2 = d.asSingle(Unconfined) - checkErroneous(single2) { - assertThat(it, IsInstanceOf(TestException::class.java)) - assertThat(it.message, IsEqual("OK")) - } - } - - @Test - fun testToObservable() { - val c = produce(CommonPool) { - delay(50) - send("O") - delay(50) - send("K") - } - val observable = c.asObservable(Unconfined) - checkSingleValue(observable.reduce { t1, t2 -> t1 + t2 }) { - assertThat(it, IsEqual("OK")) - } - } - - @Test - fun testToObservableFail() { - val c = produce(CommonPool) { - delay(50) - send("O") - delay(50) - throw TestException("K") - } - val observable = c.asObservable(Unconfined) - val single = rxSingle(Unconfined) { - var result = "" - try { - observable.consumeEach { result += it } - } catch(e: Throwable) { - check(e is TestException) - result += e.message - } - result - } - checkSingleValue(single) { - assertThat(it, IsEqual("OK")) - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/IntegrationTest.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/IntegrationTest.kt deleted file mode 100644 index 410c38d279..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/IntegrationTest.kt +++ /dev/null @@ -1,130 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.* -import org.hamcrest.MatcherAssert.assertThat -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Test -import org.junit.runner.RunWith -import org.junit.runners.Parameterized -import rx.Observable -import kotlin.coroutines.experimental.CoroutineContext - -@RunWith(Parameterized::class) -class IntegrationTest( - val ctx: Ctx, - val delay: Boolean -) : TestBase() { - - enum class Ctx { - MAIN { override fun invoke(context: CoroutineContext): CoroutineContext = context }, - COMMON_POOL { override fun invoke(context: CoroutineContext): CoroutineContext = CommonPool }, - UNCONFINED { override fun invoke(context: CoroutineContext): CoroutineContext = Unconfined }; - - abstract operator fun invoke(context: CoroutineContext): CoroutineContext - } - - companion object { - @Parameterized.Parameters(name = "ctx={0}, delay={1}") - @JvmStatic - fun params(): Collection> = Ctx.values().flatMap { ctx -> - listOf(false, true).map { delay -> - arrayOf(ctx, delay) - } - } - } - - @Test - fun testEmpty(): Unit = runBlocking { - val observable = rxObservable(ctx(context)) { - if (delay) delay(1) - // does not send anything - } - assertNSE { observable.awaitFirst() } - assertThat(observable.awaitFirstOrDefault("OK"), IsEqual("OK")) - assertNSE { observable.awaitLast() } - assertNSE { observable.awaitSingle() } - var cnt = 0 - observable.consumeEach { - cnt++ - } - assertThat(cnt, IsEqual(0)) - } - - @Test - fun testSingle() = runBlocking { - val observable = rxObservable(ctx(context)) { - if (delay) delay(1) - send("OK") - } - assertThat(observable.awaitFirst(), IsEqual("OK")) - assertThat(observable.awaitLast(), IsEqual("OK")) - assertThat(observable.awaitSingle(), IsEqual("OK")) - var cnt = 0 - observable.consumeEach { - assertThat(it, IsEqual("OK")) - cnt++ - } - assertThat(cnt, IsEqual(1)) - } - - @Test - fun testNumbers() = runBlocking { - val n = 100 * stressTestMultiplier - val observable = rxObservable(ctx(context)) { - for (i in 1..n) { - send(i) - if (delay) delay(1) - } - } - assertThat(observable.awaitFirst(), IsEqual(1)) - assertThat(observable.awaitLast(), IsEqual(n)) - assertIAE { observable.awaitSingle() } - checkNumbers(n, observable) - val channel = observable.open() - checkNumbers(n, channel.asObservable(ctx(context))) - channel.close() - } - - private suspend fun checkNumbers(n: Int, observable: Observable) { - var last = 0 - observable.consumeEach { - assertThat(it, IsEqual(++last)) - } - assertThat(last, IsEqual(n)) - } - - inline fun assertIAE(block: () -> Unit) { - try { - block() - expectUnreached() - } catch (e: Throwable) { - assertThat(e, IsInstanceOf(IllegalArgumentException::class.java)) - } - } - - inline fun assertNSE(block: () -> Unit) { - try { - block() - expectUnreached() - } catch (e: Throwable) { - assertThat(e, IsInstanceOf(NoSuchElementException::class.java)) - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableBackpressureTest.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableBackpressureTest.kt deleted file mode 100644 index 163414075d..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableBackpressureTest.kt +++ /dev/null @@ -1,73 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.junit.Test -import rx.Subscriber - -class ObservableBackpressureTest : TestBase() { - @Test - fun testCancelWhileBPSuspended() = runBlocking { - expect(1) - val observable = rxObservable(context) { - expect(5) - send("A") // will not suspend, because an item was requested - expect(7) - send("B") // second requested item - expect(9) - try { - send("C") // will suspend (no more requested) - } finally { - expect(13) - } - expectUnreached() - } - expect(2) - val sub = observable.subscribe(object : Subscriber() { - override fun onStart() { - expect(3) - request(2) // request two items - } - - override fun onNext(t: String) { - when (t) { - "A" -> expect(6) - "B" -> expect(8) - else -> error("Should not happen") - } - } - - override fun onCompleted() { - expect(11) - } - - override fun onError(e: Throwable) { - expectUnreached() - } - }) - expect(4) - yield() // yield to observable coroutine - expect(10) - sub.unsubscribe() // now unsubscribe -- shall cancel coroutine & invoke onCompleted - expect(12) - yield() // shall perform finally in coroutine - finish(14) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableCompletionStressTest.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableCompletionStressTest.kt deleted file mode 100644 index bd2d915d0b..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableCompletionStressTest.kt +++ /dev/null @@ -1,51 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.withTimeout -import org.junit.Test -import java.util.* -import kotlin.coroutines.experimental.CoroutineContext - -class ObservableCompletionStressTest : TestBase() { - val N_REPEATS = 10_000 * stressTestMultiplier - - fun range(context: CoroutineContext, start: Int, count: Int) = rxObservable(context) { - for (x in start until start + count) send(x) - } - - @Test - fun testCompletion() { - val rnd = Random() - repeat(N_REPEATS) { - val count = rnd.nextInt(5) - runBlocking { - withTimeout(5000) { - var received = 0 - range(CommonPool, 1, count).consumeEach { x -> - received++ - if (x != received) error("$x != $received") - } - if (received != count) error("$received != $count") - } - } - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableMultiTest.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableMultiTest.kt deleted file mode 100644 index acf1b7f5ae..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableMultiTest.kt +++ /dev/null @@ -1,103 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.Unconfined -import kotlinx.coroutines.experimental.launch -import org.junit.Assert.assertEquals -import org.junit.Test -import rx.Observable -import java.io.IOException - -/** - * Test emitting multiple values with [rxObservable]. - */ -class ObservableMultiTest : TestBase() { - @Test - fun testNumbers() { - val n = 100 * stressTestMultiplier - val observable = rxObservable(CommonPool) { - repeat(n) { send(it) } - } - checkSingleValue(observable.toList()) { list -> - assertEquals((0..n - 1).toList(), list) - } - } - - @Test - fun testConcurrentStress() { - val n = 10_000 * stressTestMultiplier - val observable = rxObservable(CommonPool) { - // concurrent emitters (many coroutines) - val jobs = List(n) { - // launch - launch(CommonPool) { - send(it) - } - } - jobs.forEach { it.join() } - } - checkSingleValue(observable.toList()) { list -> - assertEquals(n, list.size) - assertEquals((0..n - 1).toList(), list.sorted()) - } - } - - @Test - fun testIteratorResendUnconfined() { - val n = 10_000 * stressTestMultiplier - val observable = rxObservable(Unconfined) { - Observable.range(0, n).consumeEach { send(it) } - } - checkSingleValue(observable.toList()) { list -> - assertEquals((0..n - 1).toList(), list) - } - } - - @Test - fun testIteratorResendPool() { - val n = 10_000 * stressTestMultiplier - val observable = rxObservable(CommonPool) { - Observable.range(0, n).consumeEach { send(it) } - } - checkSingleValue(observable.toList()) { list -> - assertEquals((0..n - 1).toList(), list) - } - } - - @Test - fun testSendAndCrash() { - val observable = rxObservable(CommonPool) { - send("O") - throw IOException("K") - } - val single = rxSingle(CommonPool) { - var result = "" - try { - observable.consumeEach { result += it } - } catch(e: IOException) { - result += e.message - } - result - } - checkSingleValue(single) { - assertEquals("OK", it) - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableSingleTest.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableSingleTest.kt deleted file mode 100644 index 873810d8af..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableSingleTest.kt +++ /dev/null @@ -1,190 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.runBlocking -import org.junit.Assert.assertEquals -import org.junit.Assert.fail -import org.junit.Test -import rx.Observable -import java.util.concurrent.TimeUnit - -/** - * Tests emitting single item with [rxObservable]. - */ -class ObservableSingleTest { - @Test - fun testSingleNoWait() { - val observable = rxObservable(CommonPool) { - send("OK") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleNullNoWait() { - val observable = rxObservable(CommonPool) { - send(null) - } - - checkSingleValue(observable) { - assertEquals(null, it) - } - } - - @Test - fun testSingleAwait() = runBlocking { - assertEquals("OK", Observable.just("O").awaitSingle() + "K") - } - - @Test - fun testSingleEmitAndAwait() { - val observable = rxObservable(CommonPool) { - send(Observable.just("O").awaitSingle() + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleWithDelay() { - val observable = rxObservable(CommonPool) { - send(Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleException() { - val observable = rxObservable(CommonPool) { - send(Observable.just("O", "K").awaitSingle() + "K") - } - - checkErroneous(observable) { - assert(it is IllegalArgumentException) - } - } - - @Test - fun testAwaitFirst() { - val observable = rxObservable(CommonPool) { - send(Observable.just("O", "#").awaitFirst() + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitFirstOrDefault() { - val observable = rxObservable(CommonPool) { - send(Observable.empty().awaitFirstOrDefault("O") + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitFirstOrDefaultWithValues() { - val observable = rxObservable(CommonPool) { - send(Observable.just("O", "#").awaitFirstOrDefault("!") + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitLast() { - val observable = rxObservable(CommonPool) { - send(Observable.just("#", "O").awaitLast() + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromObservable() { - val observable = rxObservable(CommonPool) { - try { - send(Observable.error(RuntimeException("O")).awaitFirst()) - } catch (e: RuntimeException) { - send(Observable.just(e.message!!).awaitLast() + "K") - } - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromCoroutine() { - val observable = rxObservable(CommonPool) { - error(Observable.just("O").awaitSingle() + "K") - } - - checkErroneous(observable) { - assert(it is IllegalStateException) - assertEquals("OK", it.message) - } - } - - @Test - fun testObservableIteration() { - val observable = rxObservable(CommonPool) { - var result = "" - Observable.just("O", "K").consumeEach {result += it } - send(result) - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testObservableIterationFailure() { - val observable = rxObservable(CommonPool) { - try { - Observable.error(RuntimeException("OK")).consumeEach { fail("Should not be here") } - send("Fail") - } catch (e: RuntimeException) { - send(e.message!!) - } - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } -} diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableTest.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableTest.kt deleted file mode 100644 index c7f7dc3c1a..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/ObservableTest.kt +++ /dev/null @@ -1,86 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert -import org.junit.Test - -class ObservableTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val observable = rxObservable(context) { - expect(4) - send("OK") - } - expect(2) - observable.subscribe { value -> - expect(5) - Assert.assertThat(value, IsEqual("OK")) - } - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val observable = rxObservable(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - observable.subscribe({ - expectUnreached() - }, { error -> - expect(5) - Assert.assertThat(error, IsInstanceOf(RuntimeException::class.java)) - Assert.assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val observable = rxObservable(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - val sub = observable.subscribe({ - expectUnreached() - }, { - expectUnreached() - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.unsubscribe() // will cancel coroutine - yield() - finish(6) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/SingleTest.kt b/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/SingleTest.kt deleted file mode 100644 index f2a1832f3d..0000000000 --- a/reactive/kotlinx-coroutines-rx1/src/test/kotlin/kotlinx/coroutines/experimental/rx1/SingleTest.kt +++ /dev/null @@ -1,202 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx1 - -import kotlinx.coroutines.experimental.* -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.hamcrest.core.IsNull -import org.junit.Assert.* -import org.junit.Test -import rx.Observable -import rx.Single -import java.util.concurrent.TimeUnit - -/** - * Tests emitting single item with [rxSingle]. - */ -class SingleTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val single = rxSingle(context) { - expect(4) - "OK" - } - expect(2) - single.subscribe { value -> - expect(5) - assertThat(value, IsEqual("OK")) - } - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val single = rxSingle(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - single.subscribe({ - expectUnreached() - }, { error -> - expect(5) - assertThat(error, IsInstanceOf(RuntimeException::class.java)) - assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val single = rxSingle(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - val sub = single.subscribe({ - expectUnreached() - }, { error -> - expect(6) - assertThat(error, IsInstanceOf(CancellationException::class.java)) - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.unsubscribe() // will cancel coroutine - yield() - finish(7) - } - - @Test - fun testSingleNoWait() { - val single = rxSingle(CommonPool) { - "OK" - } - - checkSingleValue(single) { - assertThat(it, IsEqual("OK")) - } - } - - @Test - fun testSingleNullNoWait() { - val single = rxSingle(CommonPool) { - null - } - - checkSingleValue(single) { - assertThat(it, IsNull()) - } - } - - @Test - fun testSingleAwait() = runBlocking { - assertEquals("OK", Single.just("O").await() + "K") - } - - @Test - fun testSingleEmitAndAwait() { - val single = rxSingle(CommonPool) { - Single.just("O").await() + "K" - } - - checkSingleValue(single) { - assertThat(it, IsEqual("OK")) - } - } - - @Test - fun testSingleWithDelay() { - val single = rxSingle(CommonPool) { - Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K" - } - - checkSingleValue(single) { - assertThat(it, IsEqual("OK")) - } - } - - @Test - fun testSingleException() { - val single = rxSingle(CommonPool) { - Observable.just("O", "K").awaitSingle() + "K" - } - - checkErroneous(single) { - assertThat(it, IsInstanceOf(IllegalArgumentException::class.java)) - } - } - - @Test - fun testAwaitFirst() { - val single = rxSingle(CommonPool) { - Observable.just("O", "#").awaitFirst() + "K" - } - - checkSingleValue(single) { - assertThat(it, IsEqual("OK")) - } - } - - @Test - fun testAwaitLast() { - val single = rxSingle(CommonPool) { - Observable.just("#", "O").awaitLast() + "K" - } - - checkSingleValue(single) { - assertThat(it, IsEqual("OK")) - } - } - - @Test - fun testExceptionFromObservable() { - val single = rxSingle(CommonPool) { - try { - Observable.error(RuntimeException("O")).awaitFirst() - } catch (e: RuntimeException) { - Observable.just(e.message!!).awaitLast() + "K" - } - } - - checkSingleValue(single) { - assertThat(it, IsEqual("OK")) - } - } - - @Test - fun testExceptionFromCoroutine() { - val single = rxSingle(CommonPool) { - throw RuntimeException(Observable.just("O").awaitSingle() + "K") - } - - checkErroneous(single) { - assertThat(it, IsInstanceOf(RuntimeException::class.java)) - assertEquals("OK", it.message) - } - } -} diff --git a/reactive/kotlinx-coroutines-rx2/README.md b/reactive/kotlinx-coroutines-rx2/README.md index 11b338145b..4131716554 100644 --- a/reactive/kotlinx-coroutines-rx2/README.md +++ b/reactive/kotlinx-coroutines-rx2/README.md @@ -12,23 +12,30 @@ Coroutine builders: | [rxObservable] | `Observable` | [ProducerScope] | Cold observable that starts coroutine on subscribe | [rxFlowable] | `Flowable` | [ProducerScope] | Cold observable that starts coroutine on subscribe with **backpressure** support +Integration with [Flow]: + +| **Name** | **Result** | **Description** +| --------------- | -------------- | --------------- +| [Flow.asFlowable] | `Flowable` | Converts the given flow to a cold Flowable. +| [Flow.asObservable] | `Observable` | Converts the given flow to a cold Observable. +| [ObservableSource.asFlow] | `Flow` | Converts the given cold ObservableSource to flow + Suspending extension functions and suspending iteration: | **Name** | **Description** | -------- | --------------- | [CompletableSource.await][io.reactivex.CompletableSource.await] | Awaits for completion of the completable value -| [MaybeSource.await][io.reactivex.MaybeSource.await] | Awaits for the value of the maybe and returns it or null -| [MaybeSource.awaitOrDefault][io.reactivex.MaybeSource.awaitOrDefault] | Awaits for the value of the maybe and returns it or default -| [MaybeSource.open][io.reactivex.MaybeSource.open] | Subscribes to maybe and returns [ReceiveChannel] +| [MaybeSource.awaitSingle][io.reactivex.MaybeSource.awaitSingle] | Awaits for the value of the maybe and returns it or throws an exception +| [MaybeSource.awaitSingleOrNull][io.reactivex.MaybeSource.awaitSingleOrNull] | Awaits for the value of the maybe and returns it or null | [SingleSource.await][io.reactivex.SingleSource.await] | Awaits for completion of the single value and returns it | [ObservableSource.awaitFirst][io.reactivex.ObservableSource.awaitFirst] | Awaits for the first value from the given observable | [ObservableSource.awaitFirstOrDefault][io.reactivex.ObservableSource.awaitFirstOrDefault] | Awaits for the first value from the given observable or default +| [ObservableSource.awaitFirstOrElse][io.reactivex.ObservableSource.awaitFirstOrElse] | Awaits for the first value from the given observable or default from a function +| [ObservableSource.awaitFirstOrNull][io.reactivex.ObservableSource.awaitFirstOrNull] | Awaits for the first value from the given observable or null | [ObservableSource.awaitLast][io.reactivex.ObservableSource.awaitFirst] | Awaits for the last value from the given observable | [ObservableSource.awaitSingle][io.reactivex.ObservableSource.awaitSingle] | Awaits for the single value from the given observable -| [ObservableSource.open][io.reactivex.ObservableSource.open] | Subscribes to observable and returns [ReceiveChannel] -| [ObservableSource.iterator][io.reactivex.ObservableSource.iterator] | Subscribes to observable and returns [ChannelIterator] -Note, that `Flowable` is a subclass of [Reactive Streams](http://www.reactive-streams.org) +Note that `Flowable` is a subclass of [Reactive Streams](https://www.reactive-streams.org) `Publisher` and extensions for it are covered by [kotlinx-coroutines-reactive](../kotlinx-coroutines-reactive) module. @@ -36,44 +43,50 @@ Conversion functions: | **Name** | **Description** | -------- | --------------- -| [Job.asCompletable][kotlinx.coroutines.experimental.Job.asCompletable] | Converts job to hot completable -| [Deferred.asSingle][kotlinx.coroutines.experimental.Deferred.asSingle] | Converts deferred value to hot single -| [ReceiveChannel.asObservable][kotlinx.coroutines.experimental.channels.ReceiveChannel.asObservable] | Converts streaming channel to hot observable +| [Job.asCompletable][kotlinx.coroutines.Job.asCompletable] | Converts job to hot completable +| [Deferred.asSingle][kotlinx.coroutines.Deferred.asSingle] | Converts deferred value to hot single | [Scheduler.asCoroutineDispatcher][io.reactivex.Scheduler.asCoroutineDispatcher] | Converts scheduler to [CoroutineDispatcher] - - - -[CoroutineScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-scope/index.html -[CoroutineDispatcher]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-dispatcher/index.html - -[ProducerScope]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-producer-scope/index.html -[ReceiveChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-receive-channel/index.html -[ChannelIterator]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel-iterator/index.html - - - -[rxCompletable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/rx-completable.html -[rxMaybe]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/rx-maybe.html -[rxSingle]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/rx-single.html -[rxObservable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/rx-observable.html -[rxFlowable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/rx-flowable.html -[io.reactivex.CompletableSource.await]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-completable-source/await.html -[io.reactivex.MaybeSource.await]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-maybe-source/await.html -[io.reactivex.MaybeSource.awaitOrDefault]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-maybe-source/await-or-default.html -[io.reactivex.MaybeSource.open]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-maybe-source/open.html -[io.reactivex.SingleSource.await]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-single-source/await.html -[io.reactivex.ObservableSource.awaitFirst]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-observable-source/await-first.html -[io.reactivex.ObservableSource.awaitFirstOrDefault]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-observable-source/await-first-or-default.html -[io.reactivex.ObservableSource.awaitSingle]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-observable-source/await-single.html -[io.reactivex.ObservableSource.open]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-observable-source/open.html -[io.reactivex.ObservableSource.iterator]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-observable-source/iterator.html -[kotlinx.coroutines.experimental.Job.asCompletable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/kotlinx.coroutines.experimental.-job/as-completable.html -[kotlinx.coroutines.experimental.Deferred.asSingle]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/kotlinx.coroutines.experimental.-deferred/as-single.html -[kotlinx.coroutines.experimental.channels.ReceiveChannel.asObservable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/kotlinx.coroutines.experimental.channels.-receive-channel/as-observable.html -[io.reactivex.Scheduler.asCoroutineDispatcher]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.experimental.rx2/io.reactivex.-scheduler/as-coroutine-dispatcher.html + + + +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[CoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/index.html + + + +[ProducerScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-producer-scope/index.html + + + +[Flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html + + + + +[rxCompletable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/rx-completable.html +[rxMaybe]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/rx-maybe.html +[rxSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/rx-single.html +[rxObservable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/rx-observable.html +[rxFlowable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/rx-flowable.html +[Flow.asFlowable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/as-flowable.html +[Flow.asObservable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/as-observable.html +[ObservableSource.asFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/as-flow.html +[io.reactivex.CompletableSource.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/await.html +[io.reactivex.MaybeSource.awaitSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/await-single.html +[io.reactivex.MaybeSource.awaitSingleOrNull]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/await-single-or-null.html +[io.reactivex.SingleSource.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/await.html +[io.reactivex.ObservableSource.awaitFirst]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/await-first.html +[io.reactivex.ObservableSource.awaitFirstOrDefault]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/await-first-or-default.html +[io.reactivex.ObservableSource.awaitFirstOrElse]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/await-first-or-else.html +[io.reactivex.ObservableSource.awaitFirstOrNull]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/await-first-or-null.html +[io.reactivex.ObservableSource.awaitSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/await-single.html +[kotlinx.coroutines.Job.asCompletable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/as-completable.html +[kotlinx.coroutines.Deferred.asSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/as-single.html +[io.reactivex.Scheduler.asCoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/kotlinx.coroutines.rx2/as-coroutine-dispatcher.html + -# Package kotlinx.coroutines.experimental.rx2 +# Package kotlinx.coroutines.rx2 Utilities for [RxJava 2.x](https://github.com/ReactiveX/RxJava). diff --git a/reactive/kotlinx-coroutines-rx2/api/kotlinx-coroutines-rx2.api b/reactive/kotlinx-coroutines-rx2/api/kotlinx-coroutines-rx2.api new file mode 100644 index 0000000000..803ac90564 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/api/kotlinx-coroutines-rx2.api @@ -0,0 +1,95 @@ +public final class kotlinx/coroutines/rx2/RxAwaitKt { + public static final fun await (Lio/reactivex/CompletableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun await (Lio/reactivex/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun await (Lio/reactivex/SingleSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirst (Lio/reactivex/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrDefault (Lio/reactivex/ObservableSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrElse (Lio/reactivex/ObservableSource;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrNull (Lio/reactivex/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitLast (Lio/reactivex/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitOrDefault (Lio/reactivex/MaybeSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingle (Lio/reactivex/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingle (Lio/reactivex/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingleOrNull (Lio/reactivex/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/rx2/RxChannelKt { + public static final fun collect (Lio/reactivex/MaybeSource;Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun collect (Lio/reactivex/ObservableSource;Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun openSubscription (Lio/reactivex/MaybeSource;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final synthetic fun openSubscription (Lio/reactivex/ObservableSource;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun toChannel (Lio/reactivex/MaybeSource;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun toChannel (Lio/reactivex/ObservableSource;)Lkotlinx/coroutines/channels/ReceiveChannel; +} + +public final class kotlinx/coroutines/rx2/RxCompletableKt { + public static final fun rxCompletable (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Completable; + public static final synthetic fun rxCompletable (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Completable; + public static synthetic fun rxCompletable$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Completable; + public static synthetic fun rxCompletable$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Completable; +} + +public final class kotlinx/coroutines/rx2/RxConvertKt { + public static final fun asCompletable (Lkotlinx/coroutines/Job;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Completable; + public static final fun asFlow (Lio/reactivex/ObservableSource;)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlowable (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Flowable; + public static synthetic fun asFlowable$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/Flowable; + public static final fun asMaybe (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Maybe; + public static final synthetic fun asObservable (Lkotlinx/coroutines/channels/ReceiveChannel;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Observable; + public static final fun asObservable (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Observable; + public static synthetic fun asObservable$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/Observable; + public static final fun asSingle (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Single; + public static final synthetic fun from (Lkotlinx/coroutines/flow/Flow;)Lio/reactivex/Flowable; + public static final synthetic fun from (Lkotlinx/coroutines/flow/Flow;)Lio/reactivex/Observable; + public static final synthetic fun from (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Flowable; + public static final synthetic fun from (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/Observable; + public static synthetic fun from$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/Flowable; + public static synthetic fun from$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/Observable; +} + +public final class kotlinx/coroutines/rx2/RxFlowableKt { + public static final fun rxFlowable (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Flowable; + public static final synthetic fun rxFlowable (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Flowable; + public static synthetic fun rxFlowable$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Flowable; + public static synthetic fun rxFlowable$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Flowable; +} + +public final class kotlinx/coroutines/rx2/RxMaybeKt { + public static final fun rxMaybe (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Maybe; + public static final synthetic fun rxMaybe (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Maybe; + public static synthetic fun rxMaybe$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Maybe; + public static synthetic fun rxMaybe$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Maybe; +} + +public final class kotlinx/coroutines/rx2/RxObservableKt { + public static final fun rxObservable (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Observable; + public static final synthetic fun rxObservable (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Observable; + public static synthetic fun rxObservable$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Observable; + public static synthetic fun rxObservable$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Observable; +} + +public final class kotlinx/coroutines/rx2/RxSchedulerKt { + public static final fun asCoroutineDispatcher (Lio/reactivex/Scheduler;)Lkotlinx/coroutines/CoroutineDispatcher; + public static final synthetic fun asCoroutineDispatcher (Lio/reactivex/Scheduler;)Lkotlinx/coroutines/rx2/SchedulerCoroutineDispatcher; + public static final fun asScheduler (Lkotlinx/coroutines/CoroutineDispatcher;)Lio/reactivex/Scheduler; +} + +public final class kotlinx/coroutines/rx2/RxSingleKt { + public static final fun rxSingle (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Single; + public static final synthetic fun rxSingle (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/Single; + public static synthetic fun rxSingle$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Single; + public static synthetic fun rxSingle$default (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/Single; +} + +public final class kotlinx/coroutines/rx2/SchedulerCoroutineDispatcher : kotlinx/coroutines/CoroutineDispatcher, kotlinx/coroutines/Delay { + public fun (Lio/reactivex/Scheduler;)V + public fun delay (JLkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun dispatch (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Runnable;)V + public fun equals (Ljava/lang/Object;)Z + public final fun getScheduler ()Lio/reactivex/Scheduler; + public fun hashCode ()I + public fun invokeOnTimeout (JLjava/lang/Runnable;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/DisposableHandle; + public fun scheduleResumeAfterDelay (JLkotlinx/coroutines/CancellableContinuation;)V + public fun toString ()Ljava/lang/String; +} + diff --git a/reactive/kotlinx-coroutines-rx2/build.gradle.kts b/reactive/kotlinx-coroutines-rx2/build.gradle.kts new file mode 100644 index 0000000000..bc7ac285a4 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/build.gradle.kts @@ -0,0 +1,36 @@ +import org.jetbrains.dokka.gradle.DokkaTaskPartial +import java.net.* + +dependencies { + api(project(":kotlinx-coroutines-reactive")) + testImplementation("org.reactivestreams:reactive-streams-tck:${version("reactive_streams")}") + api("io.reactivex.rxjava2:rxjava:${version("rxjava2")}") +} + +tasks.withType(DokkaTaskPartial::class) { + dokkaSourceSets.configureEach { + externalDocumentationLink { + url = URL("http://reactivex.io/RxJava/2.x/javadoc/") + packageListUrl = projectDir.toPath().resolve("package.list").toUri().toURL() + } + } +} + +val testNG by tasks.registering(Test::class) { + useTestNG() + reports.html.outputLocation = file("${layout.buildDirectory.get()}/reports/testng") + include("**/*ReactiveStreamTckTest.*") + // Skip testNG when tests are filtered with --tests, otherwise it simply fails + onlyIf { + filter.includePatterns.isEmpty() + } + doFirst { + // Classic gradle, nothing works without doFirst + println("TestNG tests: ($includes)") + } +} + +val test by tasks.getting(Test::class) { + dependsOn(testNG) + reports.html.outputLocation = file("${layout.buildDirectory.get()}/reports/junit") +} diff --git a/reactive/kotlinx-coroutines-rx2/package.list b/reactive/kotlinx-coroutines-rx2/package.list new file mode 100644 index 0000000000..5be2c9e9e3 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/package.list @@ -0,0 +1,14 @@ +io.reactivex +io.reactivex.annotations +io.reactivex.disposables +io.reactivex.exceptions +io.reactivex.flowables +io.reactivex.functions +io.reactivex.observables +io.reactivex.observers +io.reactivex.parallel +io.reactivex.plugins +io.reactivex.processors +io.reactivex.schedulers +io.reactivex.subjects +io.reactivex.subscribers diff --git a/reactive/kotlinx-coroutines-rx2/pom.xml b/reactive/kotlinx-coroutines-rx2/pom.xml deleted file mode 100644 index 85e195d731..0000000000 --- a/reactive/kotlinx-coroutines-rx2/pom.xml +++ /dev/null @@ -1,64 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - ../../pom.xml - - - kotlinx-coroutines-rx2 - jar - - - src/main/kotlin - src/test/kotlin - - - - - io.reactivex.rxjava2 - rxjava - 2.0.6 - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-reactive - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - tests - test - - - - diff --git a/reactive/kotlinx-coroutines-rx2/src/RxAwait.kt b/reactive/kotlinx-coroutines-rx2/src/RxAwait.kt new file mode 100644 index 0000000000..cfcc95db8f --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxAwait.kt @@ -0,0 +1,293 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import io.reactivex.disposables.Disposable +import kotlinx.coroutines.CancellableContinuation +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.Job +import kotlinx.coroutines.suspendCancellableCoroutine +import kotlin.coroutines.* + +// ------------------------ CompletableSource ------------------------ + +/** + * Awaits for completion of this completable without blocking the thread. + * Returns `Unit`, or throws the corresponding exception if this completable produces an error. + * + * This suspending function is cancellable. If the [Job] of the invoking coroutine is cancelled while this + * suspending function is suspended, this function immediately resumes with [CancellationException] and disposes of its + * subscription. + */ +public suspend fun CompletableSource.await(): Unit = suspendCancellableCoroutine { cont -> + subscribe(object : CompletableObserver { + override fun onSubscribe(d: Disposable) { + cont.disposeOnCancellation(d) + } + + override fun onComplete() { + cont.resume(Unit) + } + + override fun onError(e: Throwable) { + cont.resumeWithException(e) + } + }) +} + +// ------------------------ MaybeSource ------------------------ + +/** + * Awaits for completion of the [MaybeSource] without blocking the thread. + * Returns the resulting value, or `null` if no value is produced, or throws the corresponding exception if this + * [MaybeSource] produces an error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this + * function immediately resumes with [CancellationException] and disposes of its subscription. + */ +public suspend fun MaybeSource.awaitSingleOrNull(): T? = suspendCancellableCoroutine { cont -> + subscribe(object : MaybeObserver { + override fun onSubscribe(d: Disposable) { + cont.disposeOnCancellation(d) + } + + override fun onComplete() { + cont.resume(null) + } + + override fun onSuccess(t: T & Any) { + cont.resume(t) + } + + override fun onError(error: Throwable) { + cont.resumeWithException(error) + } + }) +} + +/** + * Awaits for completion of the [MaybeSource] without blocking the thread. + * Returns the resulting value, or throws if either no value is produced or this [MaybeSource] produces an error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this + * function immediately resumes with [CancellationException] and disposes of its subscription. + * + * @throws NoSuchElementException if no elements were produced by this [MaybeSource]. + */ +public suspend fun MaybeSource.awaitSingle(): T = awaitSingleOrNull() ?: throw NoSuchElementException() + +/** + * Awaits for completion of the maybe without blocking a thread. + * Returns the resulting value, null if no value was produced or throws the corresponding exception if this + * maybe had produced error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this function + * immediately resumes with [CancellationException]. + * + * ### Deprecation + * + * Deprecated in favor of [awaitSingleOrNull] in order to reflect that `null` can be returned to denote the absence of + * a value, as opposed to throwing in such case. + * @suppress + */ +@Deprecated( + message = "Deprecated in favor of awaitSingleOrNull()", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingleOrNull()") +) // Warning since 1.5, error in 1.6, hidden in 1.7 +public suspend fun MaybeSource.await(): T? = awaitSingleOrNull() + +/** + * Awaits for completion of the maybe without blocking a thread. + * Returns the resulting value, [default] if no value was produced or throws the corresponding exception if this + * maybe had produced error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this function + * immediately resumes with [CancellationException]. + * + * ### Deprecation + * + * Deprecated in favor of [awaitSingleOrNull] for naming consistency (see the deprecation of [MaybeSource.await] for + * details). + * @suppress + */ +@Deprecated( + message = "Deprecated in favor of awaitSingleOrNull()", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingleOrNull() ?: default") +) // Warning since 1.5, error in 1.6, hidden in 1.7 +public suspend fun MaybeSource.awaitOrDefault(default: T): T = awaitSingleOrNull() ?: default + +// ------------------------ SingleSource ------------------------ + +/** + * Awaits for completion of the single value response without blocking the thread. + * Returns the resulting value, or throws the corresponding exception if this response produces an error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + */ +public suspend fun SingleSource.await(): T = suspendCancellableCoroutine { cont -> + subscribe(object : SingleObserver { + override fun onSubscribe(d: Disposable) { + cont.disposeOnCancellation(d) + } + + override fun onSuccess(t: T & Any) { + cont.resume(t) + } + + override fun onError(error: Throwable) { + cont.resumeWithException(error) + } + }) +} + +// ------------------------ ObservableSource ------------------------ + +/** + * Awaits the first value from the given [Observable] without blocking the thread and returns the resulting value, or, + * if the observable has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the observable does not emit any value + */ +public suspend fun ObservableSource.awaitFirst(): T = awaitOne(Mode.FIRST) + +/** + * Awaits the first value from the given [Observable], or returns the [default] value if none is emitted, without + * blocking the thread, and returns the resulting value, or, if this observable has produced an error, throws the + * corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + */ +public suspend fun ObservableSource.awaitFirstOrDefault(default: T): T = awaitOne(Mode.FIRST_OR_DEFAULT, default) + +/** + * Awaits the first value from the given [Observable], or returns `null` if none is emitted, without blocking the + * thread, and returns the resulting value, or, if this observable has produced an error, throws the corresponding + * exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + */ +public suspend fun ObservableSource.awaitFirstOrNull(): T? = awaitOne(Mode.FIRST_OR_DEFAULT) + +/** + * Awaits the first value from the given [Observable], or calls [defaultValue] to get a value if none is emitted, + * without blocking the thread, and returns the resulting value, or, if this observable has produced an error, throws + * the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + */ +public suspend fun ObservableSource.awaitFirstOrElse(defaultValue: () -> T): T = + awaitOne(Mode.FIRST_OR_DEFAULT) ?: defaultValue() + +/** + * Awaits the last value from the given [Observable] without blocking the thread and + * returns the resulting value, or, if this observable has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the observable does not emit any value + */ +public suspend fun ObservableSource.awaitLast(): T = awaitOne(Mode.LAST) + +/** + * Awaits the single value from the given observable without blocking the thread and returns the resulting value, or, + * if this observable has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the observable does not emit any value + * @throws IllegalArgumentException if the observable emits more than one value + */ +public suspend fun ObservableSource.awaitSingle(): T = awaitOne(Mode.SINGLE) + +// ------------------------ private ------------------------ + +internal fun CancellableContinuation<*>.disposeOnCancellation(d: Disposable) = + invokeOnCancellation { d.dispose() } + +private enum class Mode(val s: String) { + FIRST("awaitFirst"), + FIRST_OR_DEFAULT("awaitFirstOrDefault"), + LAST("awaitLast"), + SINGLE("awaitSingle"); + override fun toString(): String = s +} + +private suspend fun ObservableSource.awaitOne( + mode: Mode, + default: T? = null +): T = suspendCancellableCoroutine { cont -> + subscribe(object : Observer { + private lateinit var subscription: Disposable + private var value: T? = null + private var seenValue = false + + override fun onSubscribe(sub: Disposable) { + subscription = sub + cont.invokeOnCancellation { sub.dispose() } + } + + override fun onNext(t: T & Any) { + when (mode) { + Mode.FIRST, Mode.FIRST_OR_DEFAULT -> { + if (!seenValue) { + seenValue = true + cont.resume(t) + subscription.dispose() + } + } + Mode.LAST, Mode.SINGLE -> { + if (mode == Mode.SINGLE && seenValue) { + if (cont.isActive) + cont.resumeWithException(IllegalArgumentException("More than one onNext value for $mode")) + subscription.dispose() + } else { + value = t + seenValue = true + } + } + } + } + + @Suppress("UNCHECKED_CAST") + override fun onComplete() { + if (seenValue) { + if (cont.isActive) cont.resume(value as T) + return + } + when { + mode == Mode.FIRST_OR_DEFAULT -> { + cont.resume(default as T) + } + cont.isActive -> { + cont.resumeWithException(NoSuchElementException("No value received via onNext for $mode")) + } + } + } + + override fun onError(e: Throwable) { + cont.resumeWithException(e) + } + }) +} + diff --git a/reactive/kotlinx-coroutines-rx2/src/RxCancellable.kt b/reactive/kotlinx-coroutines-rx2/src/RxCancellable.kt new file mode 100644 index 0000000000..b0154982dc --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxCancellable.kt @@ -0,0 +1,22 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.functions.* +import io.reactivex.plugins.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +internal class RxCancellable(private val job: Job) : Cancellable { + override fun cancel() { + job.cancel() + } +} + +internal fun handleUndeliverableException(cause: Throwable, context: CoroutineContext) { + if (cause is CancellationException) return // Async CE should be completely ignored + try { + RxJavaPlugins.onError(cause) + } catch (e: Throwable) { + cause.addSuppressed(e) + handleCoroutineException(context, cause) + } +} diff --git a/reactive/kotlinx-coroutines-rx2/src/RxChannel.kt b/reactive/kotlinx-coroutines-rx2/src/RxChannel.kt new file mode 100644 index 0000000000..0235468b94 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxChannel.kt @@ -0,0 +1,90 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import io.reactivex.disposables.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* + +/** + * Subscribes to this [MaybeSource] and performs the specified action for each received element. + * + * If [action] throws an exception at some point or if the [MaybeSource] raises an error, the exception is rethrown from + * [collect]. + */ +public suspend inline fun MaybeSource.collect(action: (T) -> Unit): Unit = + toChannel().consumeEach(action) + +/** + * Subscribes to this [ObservableSource] and performs the specified action for each received element. + * + * If [action] throws an exception at some point, the subscription is cancelled, and the exception is rethrown from + * [collect]. Also, if the [ObservableSource] signals an error, that error is rethrown from [collect]. + */ +public suspend inline fun ObservableSource.collect(action: (T) -> Unit): Unit = + toChannel().consumeEach(action) + +@PublishedApi +internal fun MaybeSource.toChannel(): ReceiveChannel { + val channel = SubscriptionChannel() + subscribe(channel) + return channel +} + +@PublishedApi +internal fun ObservableSource.toChannel(): ReceiveChannel { + val channel = SubscriptionChannel() + subscribe(channel) + return channel +} + +@Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") +private class SubscriptionChannel : + BufferedChannel(capacity = Channel.UNLIMITED), Observer, MaybeObserver +{ + private val _subscription = atomic(null) + + @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") + override fun onClosedIdempotent() { + _subscription.getAndSet(null)?.dispose() // dispose exactly once + } + + // Observer overrider + override fun onSubscribe(sub: Disposable) { + _subscription.value = sub + } + + override fun onSuccess(t: T & Any) { + trySend(t) + close(cause = null) + } + + override fun onNext(t: T & Any) { + trySend(t) // Safe to ignore return value here, expectedly racing with cancellation + } + + override fun onComplete() { + close(cause = null) + } + + override fun onError(e: Throwable) { + close(cause = e) + } +} + +/** @suppress */ +@Deprecated(message = "Deprecated in the favour of Flow", level = DeprecationLevel.HIDDEN) // ERROR in 1.4.0, HIDDEN in 1.6.0 +public fun ObservableSource.openSubscription(): ReceiveChannel { + val channel = SubscriptionChannel() + subscribe(channel) + return channel +} + +/** @suppress */ +@Deprecated(message = "Deprecated in the favour of Flow", level = DeprecationLevel.HIDDEN) // ERROR in 1.4.0, HIDDEN in 1.6.0 +public fun MaybeSource.openSubscription(): ReceiveChannel { + val channel = SubscriptionChannel() + subscribe(channel) + return channel +} diff --git a/reactive/kotlinx-coroutines-rx2/src/RxCompletable.kt b/reactive/kotlinx-coroutines-rx2/src/RxCompletable.kt new file mode 100644 index 0000000000..f1dc93d34f --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxCompletable.kt @@ -0,0 +1,70 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * Creates cold [Completable] that runs a given [block] in a coroutine and emits its result. + * Every time the returned completable is subscribed, it starts a new coroutine. + * Unsubscribing cancels running coroutine. + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + */ +public fun rxCompletable( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> Unit +): Completable { + require(context[Job] === null) { "Completable context cannot contain job in it. " + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return rxCompletableInternal(GlobalScope, context, block) +} + +private fun rxCompletableInternal( + scope: CoroutineScope, // support for legacy rxCompletable in scope + context: CoroutineContext, + block: suspend CoroutineScope.() -> Unit +): Completable = Completable.create { subscriber -> + val newContext = scope.newCoroutineContext(context) + val coroutine = RxCompletableCoroutine(newContext, subscriber) + subscriber.setCancellable(RxCancellable(coroutine)) + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private class RxCompletableCoroutine( + parentContext: CoroutineContext, + private val subscriber: CompletableEmitter +) : AbstractCoroutine(parentContext, false, true) { + override fun onCompleted(value: Unit) { + try { + subscriber.onComplete() + } catch (e: Throwable) { + handleUndeliverableException(e, context) + } + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + try { + if (subscriber.tryOnError(cause)) { + return + } + } catch (e: Throwable) { + cause.addSuppressed(e) + } + handleUndeliverableException(cause, context) + } +} + +/** + * @suppress + */ +@Deprecated( + message = "CoroutineScope.rxCompletable is deprecated in favour of top-level rxCompletable", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("rxCompletable(context, block)") +) // Since 1.3.0, will be error in 1.3.1 and hidden in 1.4.0 +public fun CoroutineScope.rxCompletable( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> Unit +): Completable = rxCompletableInternal(this, context, block) diff --git a/reactive/kotlinx-coroutines-rx2/src/RxConvert.kt b/reactive/kotlinx-coroutines-rx2/src/RxConvert.kt new file mode 100644 index 0000000000..980fb17693 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxConvert.kt @@ -0,0 +1,161 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import io.reactivex.disposables.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.reactivestreams.* +import java.util.concurrent.atomic.* +import kotlin.coroutines.* + +/** + * Converts this job to the hot reactive completable that signals + * with [onCompleted][CompletableObserver.onComplete] when the corresponding job completes. + * + * Every subscriber gets the signal at the same time. + * Unsubscribing from the resulting completable **does not** affect the original job in any way. + * + * **Note: This is an experimental api.** Conversion of coroutines primitives to reactive entities may change + * in the future to account for the concept of structured concurrency. + * + * @param context -- the coroutine context from which the resulting completable is going to be signalled + */ +public fun Job.asCompletable(context: CoroutineContext): Completable = rxCompletable(context) { + this@asCompletable.join() +} + +/** + * Converts this deferred value to the hot reactive maybe that signals + * [onComplete][MaybeEmitter.onComplete], [onSuccess][MaybeEmitter.onSuccess] or [onError][MaybeEmitter.onError]. + * + * Every subscriber gets the same completion value. + * Unsubscribing from the resulting maybe **does not** affect the original deferred value in any way. + * + * **Note: This is an experimental api.** Conversion of coroutines primitives to reactive entities may change + * in the future to account for the concept of structured concurrency. + * + * @param context -- the coroutine context from which the resulting maybe is going to be signalled + */ +public fun Deferred.asMaybe(context: CoroutineContext): Maybe = rxMaybe(context) { + this@asMaybe.await() +} + +/** + * Converts this deferred value to the hot reactive single that signals either + * [onSuccess][SingleObserver.onSuccess] or [onError][SingleObserver.onError]. + * + * Every subscriber gets the same completion value. + * Unsubscribing from the resulting single **does not** affect the original deferred value in any way. + * + * **Note: This is an experimental api.** Conversion of coroutines primitives to reactive entities may change + * in the future to account for the concept of structured concurrency. + * + * @param context -- the coroutine context from which the resulting single is going to be signalled + */ +public fun Deferred.asSingle(context: CoroutineContext): Single = rxSingle(context) { + this@asSingle.await() +} + +/** + * Transforms given cold [ObservableSource] into cold [Flow]. + * + * The resulting flow is _cold_, which means that [ObservableSource.subscribe] is called every time a terminal operator + * is applied to the resulting flow. + * + * A channel with the [default][Channel.BUFFERED] buffer size is used. Use the [buffer] operator on the + * resulting flow to specify a user-defined value and to control what happens when data is produced faster + * than consumed, i.e. to control the back-pressure behavior. Check [callbackFlow] for more details. + */ +public fun ObservableSource.asFlow(): Flow = callbackFlow { + val disposableRef = AtomicReference() + val observer = object : Observer { + override fun onComplete() { close() } + override fun onSubscribe(d: Disposable) { if (!disposableRef.compareAndSet(null, d)) d.dispose() } + override fun onNext(t: T) { + /* + * Channel was closed by the downstream, so the exception (if any) + * also was handled by the same downstream + */ + try { + trySendBlocking(t) + } catch (e: InterruptedException) { + // RxJava interrupts the source + } + } + override fun onError(e: Throwable) { close(e) } + } + + subscribe(observer) + awaitClose { disposableRef.getAndSet(Disposables.disposed())?.dispose() } +} + +/** + * Converts the given flow to a cold observable. + * The original flow is cancelled when the observable subscriber is disposed. + * + * An optional [context] can be specified to control the execution context of calls to [Observer] methods. + * You can set a [CoroutineDispatcher] to confine them to a specific thread and/or various [ThreadContextElement] to + * inject additional context into the caller thread. By default, the [Unconfined][Dispatchers.Unconfined] dispatcher + * is used, so calls are performed from an arbitrary thread. + */ +public fun Flow.asObservable(context: CoroutineContext = EmptyCoroutineContext) : Observable = Observable.create { emitter -> + /* + * ATOMIC is used here to provide stable behaviour of subscribe+dispose pair even if + * asObservable is already invoked from unconfined + */ + val job = GlobalScope.launch(Dispatchers.Unconfined + context, start = CoroutineStart.ATOMIC) { + try { + collect { value -> emitter.onNext(value) } + emitter.onComplete() + } catch (e: Throwable) { + // 'create' provides safe emitter, so we can unconditionally call on* here if exception occurs in `onComplete` + if (e !is CancellationException) { + if (!emitter.tryOnError(e)) { + handleUndeliverableException(e, coroutineContext) + } + } else { + emitter.onComplete() + } + } + } + emitter.setCancellable(RxCancellable(job)) +} + +/** + * Converts the given flow to a cold flowable. + * The original flow is cancelled when the flowable subscriber is disposed. + * + * An optional [context] can be specified to control the execution context of calls to [Subscriber] methods. + * You can set a [CoroutineDispatcher] to confine them to a specific thread and/or various [ThreadContextElement] to + * inject additional context into the caller thread. By default, the [Unconfined][Dispatchers.Unconfined] dispatcher + * is used, so calls are performed from an arbitrary thread. + */ +public fun Flow.asFlowable(context: CoroutineContext = EmptyCoroutineContext): Flowable = + Flowable.fromPublisher(asPublisher(context)) + +@Deprecated( + message = "Deprecated in the favour of Flow", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.consumeAsFlow().asObservable(context)", "kotlinx.coroutines.flow.consumeAsFlow") +) // Deprecated since 1.4.0 +public fun ReceiveChannel.asObservable(context: CoroutineContext): Observable = rxObservable(context) { + for (t in this@asObservable) + send(t) +} + +/** @suppress **/ +@Suppress("UNUSED") // KT-42513 +@JvmOverloads // binary compatibility +@JvmName("from") +@Deprecated(level = DeprecationLevel.HIDDEN, message = "") // Since 1.4, was experimental prior to that +public fun Flow._asFlowable(context: CoroutineContext = EmptyCoroutineContext): Flowable = + asFlowable(context) + +/** @suppress **/ +@Suppress("UNUSED") // KT-42513 +@JvmOverloads // binary compatibility +@JvmName("from") +@Deprecated(level = DeprecationLevel.HIDDEN, message = "") // Since 1.4, was experimental prior to that +public fun Flow._asObservable(context: CoroutineContext = EmptyCoroutineContext) : Observable = asObservable(context) diff --git a/reactive/kotlinx-coroutines-rx2/src/RxFlowable.kt b/reactive/kotlinx-coroutines-rx2/src/RxFlowable.kt new file mode 100644 index 0000000000..05cc0c653e --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxFlowable.kt @@ -0,0 +1,51 @@ +@file:Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") + +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.reactive.* +import kotlin.coroutines.* +import kotlin.internal.* + +/** + * Creates cold [flowable][Flowable] that will run a given [block] in a coroutine. + * Every time the returned flowable is subscribed, it starts a new coroutine. + * + * Coroutine emits ([ObservableEmitter.onNext]) values with `send`, completes ([ObservableEmitter.onComplete]) + * when the coroutine completes or channel is explicitly closed and emits error ([ObservableEmitter.onError]) + * if coroutine throws an exception or closes channel with a cause. + * Unsubscribing cancels running coroutine. + * + * Invocations of `send` are suspended appropriately when subscribers apply back-pressure and to ensure that + * `onNext` is not invoked concurrently. + * + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + * + * **Note: This is an experimental api.** Behaviour of publishers that work as children in a parent scope with respect + */ +public fun rxFlowable( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Flowable { + require(context[Job] === null) { "Flowable context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return Flowable.fromPublisher(publishInternal(GlobalScope, context, RX_HANDLER, block)) +} + +/** @suppress */ +@Deprecated( + message = "CoroutineScope.rxFlowable is deprecated in favour of top-level rxFlowable", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("rxFlowable(context, block)") +) // Since 1.3.0, will be error in 1.3.1 and hidden in 1.4.0 +@LowPriorityInOverloadResolution +public fun CoroutineScope.rxFlowable( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Flowable = Flowable.fromPublisher(publishInternal(this, context, RX_HANDLER, block)) + +private val RX_HANDLER: (Throwable, CoroutineContext) -> Unit = ::handleUndeliverableException diff --git a/reactive/kotlinx-coroutines-rx2/src/RxMaybe.kt b/reactive/kotlinx-coroutines-rx2/src/RxMaybe.kt new file mode 100644 index 0000000000..ad846efb15 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxMaybe.kt @@ -0,0 +1,69 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * Creates cold [maybe][Maybe] that will run a given [block] in a coroutine and emits its result. + * If [block] result is `null`, [onComplete][MaybeObserver.onComplete] is invoked without a value. + * Every time the returned observable is subscribed, it starts a new coroutine. + * Unsubscribing cancels running coroutine. + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + */ +public fun rxMaybe( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> T? +): Maybe { + require(context[Job] === null) { "Maybe context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return rxMaybeInternal(GlobalScope, context, block) +} + +private fun rxMaybeInternal( + scope: CoroutineScope, // support for legacy rxMaybe in scope + context: CoroutineContext, + block: suspend CoroutineScope.() -> T? +): Maybe = Maybe.create { subscriber -> + val newContext = scope.newCoroutineContext(context) + val coroutine = RxMaybeCoroutine(newContext, subscriber) + subscriber.setCancellable(RxCancellable(coroutine)) + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private class RxMaybeCoroutine( + parentContext: CoroutineContext, + private val subscriber: MaybeEmitter +) : AbstractCoroutine(parentContext, false, true) { + override fun onCompleted(value: T) { + try { + if (value == null) subscriber.onComplete() else subscriber.onSuccess(value) + } catch (e: Throwable) { + handleUndeliverableException(e, context) + } + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + try { + if (subscriber.tryOnError(cause)) { + return + } + } catch (e: Throwable) { + cause.addSuppressed(e) + } + handleUndeliverableException(cause, context) + } +} + +/** @suppress */ +@Deprecated( + message = "CoroutineScope.rxMaybe is deprecated in favour of top-level rxMaybe", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("rxMaybe(context, block)") +) // Since 1.3.0, will be error in 1.3.1 and hidden in 1.4.0 +public fun CoroutineScope.rxMaybe( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> T? +): Maybe = rxMaybeInternal(this, context, block) diff --git a/reactive/kotlinx-coroutines-rx2/src/RxObservable.kt b/reactive/kotlinx-coroutines-rx2/src/RxObservable.kt new file mode 100644 index 0000000000..d9c939236c --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxObservable.kt @@ -0,0 +1,216 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import io.reactivex.exceptions.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.internal.* +import kotlinx.coroutines.selects.* +import kotlinx.coroutines.sync.* +import kotlin.coroutines.* + +/** + * Creates cold [observable][Observable] that will run a given [block] in a coroutine. + * Every time the returned observable is subscribed, it starts a new coroutine. + * + * Coroutine emits ([ObservableEmitter.onNext]) values with `send`, completes ([ObservableEmitter.onComplete]) + * when the coroutine completes or channel is explicitly closed and emits error ([ObservableEmitter.onError]) + * if coroutine throws an exception or closes channel with a cause. + * Unsubscribing cancels running coroutine. + * + * Invocations of `send` are suspended appropriately to ensure that `onNext` is not invoked concurrently. + * Note that Rx 2.x [Observable] **does not support backpressure**. + * + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + */ +public fun rxObservable( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Observable { + require(context[Job] === null) { "Observable context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return rxObservableInternal(GlobalScope, context, block) +} + +private fun rxObservableInternal( + scope: CoroutineScope, // support for legacy rxObservable in scope + context: CoroutineContext, + block: suspend ProducerScope.() -> Unit +): Observable = Observable.create { subscriber -> + val newContext = scope.newCoroutineContext(context) + val coroutine = RxObservableCoroutine(newContext, subscriber) + subscriber.setCancellable(RxCancellable(coroutine)) // do it first (before starting coroutine), to await unnecessary suspensions + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private const val OPEN = 0 // open channel, still working +private const val CLOSED = -1 // closed, but have not signalled onCompleted/onError yet +private const val SIGNALLED = -2 // already signalled subscriber onCompleted/onError + +private class RxObservableCoroutine( + parentContext: CoroutineContext, + private val subscriber: ObservableEmitter +) : AbstractCoroutine(parentContext, false, true), ProducerScope { + override val channel: SendChannel get() = this + + private val _signal = atomic(OPEN) + + override val isClosedForSend: Boolean get() = !isActive + override fun close(cause: Throwable?): Boolean = cancelCoroutine(cause) + override fun invokeOnClose(handler: (Throwable?) -> Unit) = + throw UnsupportedOperationException("RxObservableCoroutine doesn't support invokeOnClose") + + // Mutex is locked when either nRequested == 0 or while subscriber.onXXX is being invoked + private val mutex: Mutex = Mutex() + + @Suppress("UNCHECKED_CAST", "INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + override val onSend: SelectClause2> get() = SelectClause2Impl( + clauseObject = this, + regFunc = RxObservableCoroutine<*>::registerSelectForSend as RegistrationFunction, + processResFunc = RxObservableCoroutine<*>::processResultSelectSend as ProcessResultFunction + ) + + @Suppress("UNUSED_PARAMETER") + private fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // Try to acquire the mutex and complete in the registration phase. + if (mutex.tryLock()) { + select.selectInRegistrationPhase(Unit) + return + } + // Start a new coroutine that waits for the mutex, invoking `trySelect(..)` after that. + // Please note that at the point of the `trySelect(..)` invocation the corresponding + // `select` can still be in the registration phase, making this `trySelect(..)` bound to fail. + // In this case, the `onSend` clause will be re-registered, which alongside with the mutex + // manipulation makes the resulting solution obstruction-free. + launch { + mutex.lock() + if (!select.trySelect(this@RxObservableCoroutine, Unit)) { + mutex.unlock() + } + } + } + + @Suppress("RedundantNullableReturnType", "UNUSED_PARAMETER", "UNCHECKED_CAST") + private fun processResultSelectSend(element: Any?, selectResult: Any?): Any? { + doLockedNext(element as T)?.let { throw it } + return this@RxObservableCoroutine + } + + override fun trySend(element: T): ChannelResult = + if (!mutex.tryLock()) { + ChannelResult.failure() + } else { + when (val throwable = doLockedNext(element)) { + null -> ChannelResult.success(Unit) + else -> ChannelResult.closed(throwable) + } + } + + override suspend fun send(element: T) { + mutex.lock() + doLockedNext(element)?.let { throw it } + } + + // assert: mutex.isLocked() + private fun doLockedNext(elem: T): Throwable? { + // check if already closed for send + if (!isActive) { + doLockedSignalCompleted(completionCause, completionCauseHandled) + return getCancellationException() + } + // notify subscriber + try { + subscriber.onNext(elem) + } catch (e: Throwable) { + val cause = UndeliverableException(e) + val causeDelivered = close(cause) + unlockAndCheckCompleted() + return if (causeDelivered) { + // `cause` is the reason this channel is closed + cause + } else { + // Someone else closed the channel during `onNext`. We report `cause` as an undeliverable exception. + handleUndeliverableException(cause, context) + getCancellationException() + } + } + /* + * There is no sense to check for `isActive` before doing `unlock`, because cancellation/completion might + * happen after this check and before `unlock` (see signalCompleted that does not do anything + * if it fails to acquire the lock that we are still holding). + * We have to recheck `isCompleted` after `unlock` anyway. + */ + unlockAndCheckCompleted() + return null + } + + private fun unlockAndCheckCompleted() { + mutex.unlock() + // recheck isActive + if (!isActive && mutex.tryLock()) + doLockedSignalCompleted(completionCause, completionCauseHandled) + } + + // assert: mutex.isLocked() + private fun doLockedSignalCompleted(cause: Throwable?, handled: Boolean) { + // cancellation failures + try { + if (_signal.value == SIGNALLED) + return + _signal.value = SIGNALLED // we'll signal onError/onCompleted (that the final state -- no CAS needed) + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + val unwrappedCause = cause?.let { unwrap(it) } + if (unwrappedCause == null) { + try { + subscriber.onComplete() + } catch (e: Exception) { + handleUndeliverableException(e, context) + } + } else if (unwrappedCause is UndeliverableException && !handled) { + /** Such exceptions are not reported to `onError`, as, according to the reactive specifications, + * exceptions thrown from the Subscriber methods must be treated as if the Subscriber was already + * cancelled. */ + handleUndeliverableException(cause, context) + } else if (unwrappedCause !== getCancellationException() || !subscriber.isDisposed) { + try { + /** If the subscriber is already in a terminal state, the error will be signalled to + * `RxJavaPlugins.onError`. */ + subscriber.onError(cause) + } catch (e: Exception) { + cause.addSuppressed(e) + handleUndeliverableException(cause, context) + } + } + } finally { + mutex.unlock() + } + } + + private fun signalCompleted(cause: Throwable?, handled: Boolean) { + if (!_signal.compareAndSet(OPEN, CLOSED)) return // abort, other thread invoked doLockedSignalCompleted + if (mutex.tryLock()) // if we can acquire the lock + doLockedSignalCompleted(cause, handled) + } + + override fun onCompleted(value: Unit) { + signalCompleted(null, false) + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + signalCompleted(cause, handled) + } +} + +/** @suppress */ +@Deprecated( + message = "CoroutineScope.rxObservable is deprecated in favour of top-level rxObservable", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("rxObservable(context, block)") +) // Since 1.3.0, will be error in 1.3.1 and hidden in 1.4.0 +public fun CoroutineScope.rxObservable( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Observable = rxObservableInternal(this, context, block) diff --git a/reactive/kotlinx-coroutines-rx2/src/RxScheduler.kt b/reactive/kotlinx-coroutines-rx2/src/RxScheduler.kt new file mode 100644 index 0000000000..bac20210f6 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxScheduler.kt @@ -0,0 +1,177 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import io.reactivex.disposables.* +import io.reactivex.plugins.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import java.util.concurrent.* +import kotlin.coroutines.* + +/** + * Converts an instance of [Scheduler] to an implementation of [CoroutineDispatcher] + * and provides native support of [delay] and [withTimeout]. + */ +public fun Scheduler.asCoroutineDispatcher(): CoroutineDispatcher = + if (this is DispatcherScheduler) { + dispatcher + } else { + SchedulerCoroutineDispatcher(this) + } + +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.4.2, binary compatibility with earlier versions") +@JvmName("asCoroutineDispatcher") +public fun Scheduler.asCoroutineDispatcher0(): SchedulerCoroutineDispatcher = + SchedulerCoroutineDispatcher(this) + +/** + * Converts an instance of [CoroutineDispatcher] to an implementation of [Scheduler]. + */ +public fun CoroutineDispatcher.asScheduler(): Scheduler = + if (this is SchedulerCoroutineDispatcher) { + scheduler + } else { + DispatcherScheduler(this) + } + +private class DispatcherScheduler(@JvmField val dispatcher: CoroutineDispatcher) : Scheduler() { + + private val schedulerJob = SupervisorJob() + + /** + * The scope for everything happening in this [DispatcherScheduler]. + * + * Running tasks, too, get launched under this scope, because [shutdown] should cancel the running tasks as well. + */ + private val scope = CoroutineScope(schedulerJob + dispatcher) + + /** + * The counter of created workers, for their pretty-printing. + */ + private val workerCounter = atomic(1L) + + override fun scheduleDirect(block: Runnable, delay: Long, unit: TimeUnit): Disposable = + scope.scheduleTask(block, unit.toMillis(delay)) { task -> + Runnable { scope.launch { task() } } + } + + override fun createWorker(): Worker = DispatcherWorker(workerCounter.getAndIncrement(), dispatcher, schedulerJob) + + override fun shutdown() { + schedulerJob.cancel() + } + + private class DispatcherWorker( + private val counter: Long, + private val dispatcher: CoroutineDispatcher, + parentJob: Job + ) : Worker() { + + private val workerJob = SupervisorJob(parentJob) + private val workerScope = CoroutineScope(workerJob + dispatcher) + private val blockChannel = Channel Unit>(Channel.UNLIMITED) + + init { + workerScope.launch { + blockChannel.consumeEach { + it() + } + } + } + + override fun schedule(block: Runnable, delay: Long, unit: TimeUnit): Disposable = + workerScope.scheduleTask(block, unit.toMillis(delay)) { task -> + Runnable { blockChannel.trySend(task) } + } + + override fun isDisposed(): Boolean = !workerScope.isActive + + override fun dispose() { + blockChannel.close() + workerJob.cancel() + } + + override fun toString(): String = "$dispatcher (worker $counter, ${if (isDisposed) "disposed" else "active"})" + } + + override fun toString(): String = dispatcher.toString() +} + +private typealias Task = suspend () -> Unit + +/** + * Schedule [block] so that an adapted version of it, wrapped in [adaptForScheduling], executes after [delayMillis] + * milliseconds. + */ +private fun CoroutineScope.scheduleTask( + block: Runnable, + delayMillis: Long, + adaptForScheduling: (Task) -> Runnable +): Disposable { + val ctx = coroutineContext + var handle: DisposableHandle? = null + val disposable = Disposables.fromRunnable { + // null if delay <= 0 + handle?.dispose() + } + val decoratedBlock = RxJavaPlugins.onSchedule(block) + suspend fun task() { + if (disposable.isDisposed) return + try { + runInterruptible { + decoratedBlock.run() + } + } catch (e: Throwable) { + handleUndeliverableException(e, ctx) + } + } + + val toSchedule = adaptForScheduling(::task) + if (!isActive) return Disposables.disposed() + if (delayMillis <= 0) { + toSchedule.run() + } else { + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + ctx.delay.invokeOnTimeout(delayMillis, toSchedule, ctx).let { handle = it } + } + return disposable +} + +/** + * Implements [CoroutineDispatcher] on top of an arbitrary [Scheduler]. + */ +public class SchedulerCoroutineDispatcher( + /** + * Underlying scheduler of current [CoroutineDispatcher]. + */ + public val scheduler: Scheduler +) : CoroutineDispatcher(), Delay { + /** @suppress */ + override fun dispatch(context: CoroutineContext, block: Runnable) { + scheduler.scheduleDirect(block) + } + + /** @suppress */ + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val disposable = scheduler.scheduleDirect({ + with(continuation) { resumeUndispatched(Unit) } + }, timeMillis, TimeUnit.MILLISECONDS) + continuation.disposeOnCancellation(disposable) + } + + /** @suppress */ + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { + val disposable = scheduler.scheduleDirect(block, timeMillis, TimeUnit.MILLISECONDS) + return DisposableHandle { disposable.dispose() } + } + + /** @suppress */ + override fun toString(): String = scheduler.toString() + + /** @suppress */ + override fun equals(other: Any?): Boolean = other is SchedulerCoroutineDispatcher && other.scheduler === scheduler + + /** @suppress */ + override fun hashCode(): Int = System.identityHashCode(scheduler) +} diff --git a/reactive/kotlinx-coroutines-rx2/src/RxSingle.kt b/reactive/kotlinx-coroutines-rx2/src/RxSingle.kt new file mode 100644 index 0000000000..1bdbaa5bcd --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/RxSingle.kt @@ -0,0 +1,68 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * Creates cold [single][Single] that will run a given [block] in a coroutine and emits its result. + * Every time the returned observable is subscribed, it starts a new coroutine. + * Unsubscribing cancels running coroutine. + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + */ +public fun rxSingle( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> T +): Single { + require(context[Job] === null) { "Single context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return rxSingleInternal(GlobalScope, context, block) +} + +private fun rxSingleInternal( + scope: CoroutineScope, // support for legacy rxSingle in scope + context: CoroutineContext, + block: suspend CoroutineScope.() -> T +): Single = Single.create { subscriber -> + val newContext = scope.newCoroutineContext(context) + val coroutine = RxSingleCoroutine(newContext, subscriber) + subscriber.setCancellable(RxCancellable(coroutine)) + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private class RxSingleCoroutine( + parentContext: CoroutineContext, + private val subscriber: SingleEmitter +) : AbstractCoroutine(parentContext, false, true) { + override fun onCompleted(value: T) { + try { + subscriber.onSuccess(value) + } catch (e: Throwable) { + handleUndeliverableException(e, context) + } + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + try { + if (subscriber.tryOnError(cause)) { + return + } + } catch (e: Throwable) { + cause.addSuppressed(e) + } + handleUndeliverableException(cause, context) + } +} + +/** @suppress */ +@Deprecated( + message = "CoroutineScope.rxSingle is deprecated in favour of top-level rxSingle", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("rxSingle(context, block)") +) // Since 1.3.0, will be error in 1.3.1 and hidden in 1.4.0 +public fun CoroutineScope.rxSingle( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> T +): Single = rxSingleInternal(this, context, block) diff --git a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxAwait.kt b/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxAwait.kt deleted file mode 100644 index 50594a3ad0..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxAwait.kt +++ /dev/null @@ -1,209 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.* -import io.reactivex.disposables.Disposable -import kotlinx.coroutines.experimental.CancellableContinuation -import kotlinx.coroutines.experimental.CancellationException -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.suspendCancellableCoroutine - -// ------------------------ CompletableSource ------------------------ - -/** - * Awaits for completion of this completable without blocking a thread. - * Returns `Unit` or throws the corresponding exception if this completable had produced error. - * - * This suspending function is cancellable. If the [Job] of the invoking coroutine is completed while this - * suspending function is suspended, this function immediately resumes with [CancellationException]. - */ -public suspend fun CompletableSource.await(): Unit = suspendCancellableCoroutine { cont -> - subscribe(object : CompletableObserver { - override fun onSubscribe(d: Disposable) { cont.disposeOnCompletion(d) } - override fun onComplete() { cont.resume(Unit) } - override fun onError(e: Throwable) { cont.resumeWithException(e) } - }) -} - -// ------------------------ MaybeSource ------------------------ - -/** - * Awaits for completion of the maybe without blocking a thread. - * Returns the resulting value, null if no value was produced or throws the corresponding exception if this - * maybe had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - */ -@Suppress("UNCHECKED_CAST") -public suspend fun MaybeSource.await(): T? = (this as MaybeSource).awaitOrDefault(null) - -/** - * Awaits for completion of the maybe without blocking a thread. - * Returns the resulting value, [default] if no value was produced or throws the corresponding exception if this - * maybe had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - */ -public suspend fun MaybeSource.awaitOrDefault(default: T): T = suspendCancellableCoroutine { cont -> - subscribe(object : MaybeObserver { - override fun onSubscribe(d: Disposable) { cont.disposeOnCompletion(d) } - override fun onComplete() { cont.resume(default) } - override fun onSuccess(t: T) { cont.resume(t) } - override fun onError(error: Throwable) { cont.resumeWithException(error) } - }) -} - -// ------------------------ SingleSource ------------------------ - -/** - * Awaits for completion of the single value without blocking a thread. - * Returns the resulting value or throws the corresponding exception if this single had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - */ -public suspend fun SingleSource.await(): T = suspendCancellableCoroutine { cont -> - subscribe(object : SingleObserver { - override fun onSubscribe(d: Disposable) { cont.disposeOnCompletion(d) } - override fun onSuccess(t: T) { cont.resume(t) } - override fun onError(error: Throwable) { cont.resumeWithException(error) } - }) -} - -// ------------------------ ObservableSource ------------------------ - -/** - * Awaits for the first value from the given observable without blocking a thread. - * Returns the resulting value or throws the corresponding exception if this observable had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * @throws NoSuchElementException if observable does not emit any value - */ -public suspend fun ObservableSource.awaitFirst(): T = awaitOne(Mode.FIRST) - -/** - * Awaits for the first value from the given observable or the [default] value if none is emitted without blocking a - * thread and returns the resulting value or throws the corresponding exception if this observable had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - */ -public suspend fun ObservableSource.awaitFirstOrDefault(default: T): T = awaitOne(Mode.FIRST_OR_DEFAULT, default) - -/** - * Awaits for the last value from the given observable without blocking a thread. - * Returns the resulting value or throws the corresponding exception if this observable had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * @throws NoSuchElementException if observable does not emit any value - */ -public suspend fun ObservableSource.awaitLast(): T = awaitOne(Mode.LAST) - -/** - * Awaits for the single value from the given observable without blocking a thread. - * Returns the resulting value or throws the corresponding exception if this observable had produced error. - * - * This suspending function is cancellable. - * If the [Job] of the current coroutine is completed while this suspending function is waiting, this function - * immediately resumes with [CancellationException]. - * - * @throws NoSuchElementException if observable does not emit any value - * @throws IllegalArgumentException if observable emits more than one value - */ -public suspend fun ObservableSource.awaitSingle(): T = awaitOne(Mode.SINGLE) - -// ------------------------ private ------------------------ - -internal fun CancellableContinuation<*>.disposeOnCompletion(d: Disposable) = - invokeOnCompletion { d.dispose() } - -private enum class Mode(val s: String) { - FIRST("awaitFirst"), - FIRST_OR_DEFAULT("awaitFirstOrDefault"), - LAST("awaitLast"), - SINGLE("awaitSingle"); - override fun toString(): String = s -} - -private suspend fun ObservableSource.awaitOne( - mode: Mode, - default: T? = null -): T = suspendCancellableCoroutine { cont -> - subscribe(object : Observer { - private lateinit var subscription: Disposable - private var value: T? = null - private var seenValue = false - - override fun onSubscribe(sub: Disposable) { - subscription = sub - cont.invokeOnCompletion { sub.dispose() } - } - - override fun onNext(t: T) { - when (mode) { - Mode.FIRST, Mode.FIRST_OR_DEFAULT -> { - seenValue = true - cont.resume(t) - subscription.dispose() - } - Mode.LAST, Mode.SINGLE -> { - if (mode == Mode.SINGLE && seenValue) { - if (cont.isActive) - cont.resumeWithException(IllegalArgumentException("More that one onNext value for $mode")) - subscription.dispose() - } else { - value = t - seenValue = true - } - } - } - } - - override fun onComplete() { - if (seenValue) { - if (cont.isActive) cont.resume(value as T) - return - } - when { - mode == Mode.FIRST_OR_DEFAULT -> { - cont.resume(default as T) - } - cont.isActive -> { - cont.resumeWithException(NoSuchElementException("No value received via onNext for $mode")) - } - } - } - - override fun onError(e: Throwable) { - cont.resumeWithException(e) - } - }) -} - diff --git a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxChannel.kt b/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxChannel.kt deleted file mode 100644 index 7f6d767f4d..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxChannel.kt +++ /dev/null @@ -1,117 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.MaybeObserver -import io.reactivex.MaybeSource -import io.reactivex.Observable -import io.reactivex.ObservableSource -import io.reactivex.Observer -import io.reactivex.disposables.Disposable -import kotlinx.coroutines.experimental.channels.LinkedListChannel -import kotlinx.coroutines.experimental.channels.ReceiveChannel -import kotlinx.coroutines.experimental.channels.SubscriptionReceiveChannel -import java.io.Closeable - -/** - * Subscribes to this [MaybeSource] and returns a channel to receive elements emitted by it. - * The resulting channel shall be [closed][SubscriptionReceiveChannel.close] to unsubscribe from this source. - */ -public fun MaybeSource.open(): SubscriptionReceiveChannel { - val channel = SubscriptionChannel() - subscribe(channel) - return channel -} - -/** - * Subscribes to this [ObservableSource] and returns a channel to receive elements emitted by it. - * The resulting channel shall be [closed][SubscriptionReceiveChannel.close] to unsubscribe from this source. - */ -public fun ObservableSource.open(): SubscriptionReceiveChannel { - val channel = SubscriptionChannel() - subscribe(channel) - return channel -} - -/** - * Subscribes to this [Observable] and returns an iterator to receive elements emitted by it. - * - * This is a shortcut for `open().iterator()`. See [open] if you need an ability to manually - * unsubscribe from the observable. - */ -@Suppress("DeprecatedCallableAddReplaceWith") -@Deprecated(message = -"This iteration operator for `for (x in source) { ... }` loop is deprecated, " + - "because it leaves code vulnerable to leaving unclosed subscriptions on exception. " + - "Use `source.consumeEach { x -> ... }`.") -public operator fun ObservableSource.iterator() = open().iterator() - -/** - * Subscribes to this [MaybeSource] and performs the specified action for each received element. - */ -// :todo: make it inline when this bug is fixed: https://youtrack.jetbrains.com/issue/KT-16448 -public suspend fun MaybeSource.consumeEach(action: suspend (T) -> Unit) { - open().use { channel -> - for (x in channel) action(x) - } -} - -/** - * Subscribes to this [ObservableSource] and performs the specified action for each received element. - */ -// :todo: make it inline when this bug is fixed: https://youtrack.jetbrains.com/issue/KT-16448 -public suspend fun ObservableSource.consumeEach(action: suspend (T) -> Unit) { - open().use { channel -> - for (x in channel) action(x) - } -} - -private class SubscriptionChannel : LinkedListChannel(), SubscriptionReceiveChannel, Observer, MaybeObserver { - @Volatile - var subscription: Disposable? = null - - // AbstractChannel overrides - override fun afterClose(cause: Throwable?) { - subscription?.dispose() - } - - // Subscription overrides - override fun close() { - close(cause = null) - } - - // Observer overrider - override fun onSubscribe(sub: Disposable) { - subscription = sub - } - - override fun onSuccess(t: T) { - offer(t) - } - - override fun onNext(t: T) { - offer(t) - } - - override fun onComplete() { - close(cause = null) - } - - override fun onError(e: Throwable) { - close(cause = e) - } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxCompletable.kt b/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxCompletable.kt deleted file mode 100644 index 71d1f9585c..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxCompletable.kt +++ /dev/null @@ -1,65 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Completable -import io.reactivex.CompletableEmitter -import io.reactivex.functions.Cancellable -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.CoroutineScope -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.newCoroutineContext -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Creates cold [Completable] that runs a given [block] in a coroutine. - * Every time the returned completable is subscribed, it starts a new coroutine in the specified [context]. - * Unsubscribing cancels running coroutine. - * - * | **Coroutine action** | **Signal to subscriber** - * | ------------------------------------- | ------------------------ - * | Completes successfully | `onCompleted` - * | Failure with exception or unsubscribe | `onError` - */ -public fun rxCompletable( - context: CoroutineContext, - block: suspend CoroutineScope.() -> Unit -): Completable = Completable.create { subscriber -> - val newContext = newCoroutineContext(context) - val coroutine = RxCompletableCoroutine(newContext, subscriber) - coroutine.initParentJob(context[Job]) - subscriber.setCancellable(coroutine) - block.startCoroutine(coroutine, coroutine) -} - -private class RxCompletableCoroutine( - override val parentContext: CoroutineContext, - private val subscriber: CompletableEmitter -) : AbstractCoroutine(true), Cancellable { - @Suppress("UNCHECKED_CAST") - override fun afterCompletion(state: Any?, mode: Int) { - if (subscriber.isDisposed) return - if (state is CompletedExceptionally) - subscriber.onError(state.exception) - else - subscriber.onComplete() - } - - // Cancellable impl - override fun cancel() { cancel(cause = null) } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxConvert.kt b/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxConvert.kt deleted file mode 100644 index 648cdad18c..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxConvert.kt +++ /dev/null @@ -1,99 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Completable -import io.reactivex.Maybe -import kotlinx.coroutines.experimental.Deferred -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.channels.ReceiveChannel -import io.reactivex.Observable -import io.reactivex.Single -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Converts this job to the hot reactive completable that signals - * with [onCompleted][CompletableSubscriber.onCompleted] when the corresponding job completes. - * - * Every subscriber gets the signal at the same time. - * Unsubscribing from the resulting completable **does not** affect the original job in any way. - * - * @param context -- the coroutine context from which the resulting completable is going to be signalled - */ -public fun Job.asCompletable(context: CoroutineContext): Completable = rxCompletable(context) { - this@asCompletable.join() -} - -/** - * Converts this deferred value to the hot reactive maybe that signals - * [onComplete][MaybeEmitter.onComplete], [onSuccess][MaybeEmitter.onSuccess] or [onError][MaybeEmitter.onError]. - * - * Every subscriber gets the same completion value. - * Unsubscribing from the resulting maybe **does not** affect the original deferred value in any way. - * - * @param context -- the coroutine context from which the resulting maybe is going to be signalled - */ -public fun Deferred.asMaybe(context: CoroutineContext): Maybe = rxMaybe(context) { - this@asMaybe.await() -} - -/** - * Converts this deferred value to the hot reactive single that signals either - * [onSuccess][SingleSubscriber.onSuccess] or [onError][SingleSubscriber.onError]. - * - * Every subscriber gets the same completion value. - * Unsubscribing from the resulting single **does not** affect the original deferred value in any way. - * - * @param context -- the coroutine context from which the resulting single is going to be signalled - */ -public fun Deferred.asSingle(context: CoroutineContext): Single = rxSingle(context) { - this@asSingle.await() -} - -/** - * Converts a stream of elements received from the channel to the hot reactive observable. - * - * Every subscriber receives values from this channel in **fan-out** fashion. If the are multiple subscribers, - * they'll receive values in round-robin way. - * - * @param context -- the coroutine context from which the resulting observable is going to be signalled - */ -public fun ReceiveChannel.asObservable(context: CoroutineContext): Observable = rxObservable(context) { - for (t in this@asObservable) - send(t) -} - -/** - * @suppress **Deprecated**: Renamed to [asCompletable] - */ -@Deprecated(message = "Renamed to `asCompletable`", - replaceWith = ReplaceWith("asCompletable(context)")) -public fun Job.toCompletable(context: CoroutineContext): Completable = asCompletable(context) - -/** - * @suppress **Deprecated**: Renamed to [asSingle] - */ -@Deprecated(message = "Renamed to `asSingle`", - replaceWith = ReplaceWith("asSingle(context)")) -public fun Deferred.toSingle(context: CoroutineContext): Single = asSingle(context) - -/** - * @suppress **Deprecated**: Renamed to [asObservable] - */ -@Deprecated(message = "Renamed to `asObservable`", - replaceWith = ReplaceWith("asObservable(context)")) -public fun ReceiveChannel.toObservable(context: CoroutineContext): Observable = asObservable(context) diff --git a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxFlowable.kt b/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxFlowable.kt deleted file mode 100644 index 1a6176ace8..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxFlowable.kt +++ /dev/null @@ -1,41 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Flowable -import kotlinx.coroutines.experimental.channels.ProducerScope -import kotlinx.coroutines.experimental.reactive.publish -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Creates cold [flowable][Flowable] that will run a given [block] in a coroutine. - * Every time the returned flowable is subscribed, it starts a new coroutine in the specified [context]. - * Coroutine emits items with `send`. Unsubscribing cancels running coroutine. - * - * Invocations of `send` are suspended appropriately when subscribers apply back-pressure and to ensure that - * `onNext` is not invoked concurrently. - * - * | **Coroutine action** | **Signal to subscriber** - * | -------------------------------------------- | ------------------------ - * | `send` | `onNext` - * | Normal completion or `close` without cause | `onComplete` - * | Failure with exception or `close` with cause | `onError` - */ -public fun rxFlowable( - context: CoroutineContext, - block: suspend ProducerScope.() -> Unit -): Flowable = Flowable.fromPublisher(publish(context, block)) diff --git a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxMaybe.kt b/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxMaybe.kt deleted file mode 100644 index 787a5ebabd..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxMaybe.kt +++ /dev/null @@ -1,67 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Maybe -import io.reactivex.MaybeEmitter -import io.reactivex.functions.Cancellable -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.CoroutineScope -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.newCoroutineContext -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Creates cold [maybe][Maybe] that will run a given [block] in a coroutine. - * Every time the returned observable is subscribed, it starts a new coroutine in the specified [context]. - * Coroutine returns a single, possibly null value. Unsubscribing cancels running coroutine. - * - * | **Coroutine action** | **Signal to subscriber** - * | ------------------------------------- | ------------------------ - * | Returns a non-null value | `onSuccess` - * | Returns a null | `onComplete` - * | Failure with exception or unsubscribe | `onError` - */ -public fun rxMaybe( - context: CoroutineContext, - block: suspend CoroutineScope.() -> T? -): Maybe = Maybe.create { subscriber -> - val newContext = newCoroutineContext(context) - val coroutine = RxMaybeCoroutine(newContext, subscriber) - coroutine.initParentJob(context[Job]) - subscriber.setCancellable(coroutine) - block.startCoroutine(coroutine, coroutine) -} - -private class RxMaybeCoroutine( - override val parentContext: CoroutineContext, - private val subscriber: MaybeEmitter -) : AbstractCoroutine(true), Cancellable { - @Suppress("UNCHECKED_CAST") - override fun afterCompletion(state: Any?, mode: Int) { - if (subscriber.isDisposed) return - when { - state is CompletedExceptionally -> subscriber.onError(state.exception) - state != null -> subscriber.onSuccess(state as T) - else -> subscriber.onComplete() - } - } - - // Cancellable impl - override fun cancel() { cancel(cause = null) } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxObservable.kt b/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxObservable.kt deleted file mode 100644 index 2b3e4e26a3..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxObservable.kt +++ /dev/null @@ -1,172 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Observable -import io.reactivex.ObservableEmitter -import io.reactivex.functions.Cancellable -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.channels.ClosedSendChannelException -import kotlinx.coroutines.experimental.channels.ProducerScope -import kotlinx.coroutines.experimental.channels.SendChannel -import kotlinx.coroutines.experimental.handleCoroutineException -import kotlinx.coroutines.experimental.newCoroutineContext -import kotlinx.coroutines.experimental.selects.SelectInstance -import kotlinx.coroutines.experimental.sync.Mutex -import java.util.concurrent.atomic.AtomicIntegerFieldUpdater -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Creates cold [observable][Observable] that will run a given [block] in a coroutine. - * Every time the returned observable is subscribed, it starts a new coroutine in the specified [context]. - * Coroutine emits items with `send`. Unsubscribing cancels running coroutine. - * - * Invocations of `send` are suspended appropriately to ensure that `onNext` is not invoked concurrently. - * Note, that Rx 2.x [Observable] **does not support backpressure**. Use [rxFlowable]. - * - * | **Coroutine action** | **Signal to subscriber** - * | -------------------------------------------- | ------------------------ - * | `send` | `onNext` - * | Normal completion or `close` without cause | `onComplete` - * | Failure with exception or `close` with cause | `onError` - */ -public fun rxObservable( - context: CoroutineContext, - block: suspend ProducerScope.() -> Unit -): Observable = Observable.create { subscriber -> - val newContext = newCoroutineContext(context) - val coroutine = RxObservableCoroutine(newContext, subscriber) - coroutine.initParentJob(context[Job]) - subscriber.setCancellable(coroutine) // do it first (before starting coroutine), to await unnecessary suspensions - block.startCoroutine(coroutine, coroutine) -} - -private class RxObservableCoroutine( - override val parentContext: CoroutineContext, - private val subscriber: ObservableEmitter -) : AbstractCoroutine(true), ProducerScope, Cancellable { - override val channel: SendChannel get() = this - - // Mutex is locked when while subscriber.onXXX is being invoked - private val mutex = Mutex() - - @Volatile - private var signal: Int = OPEN - - companion object { - private val SIGNAL = AtomicIntegerFieldUpdater - .newUpdater(RxObservableCoroutine::class.java, "signal") - - private const val CLOSED_MESSAGE = "This subscription had already closed (completed or failed)" - private const val OPEN = 0 // open channel, still working - private const val CLOSED = -1 // closed, but have not signalled onCompleted/onError yet - private const val SIGNALLED = -2 // already signalled subscriber onCompleted/onError - } - - override val isClosedForSend: Boolean get() = isCompleted - override val isFull: Boolean = mutex.isLocked - override fun close(cause: Throwable?): Boolean = cancel(cause) - - private fun sendException() = - (state as? CompletedExceptionally)?.cause ?: ClosedSendChannelException(CLOSED_MESSAGE) - - override fun offer(element: T): Boolean { - if (!mutex.tryLock()) return false - doLockedNext(element) - return true - } - - public suspend override fun send(element: T): Unit { - // fast-path -- try send without suspension - if (offer(element)) return - // slow-path does suspend - return sendSuspend(element) - } - - private suspend fun sendSuspend(element: T) { - mutex.lock() - doLockedNext(element) - } - - override fun registerSelectSend(select: SelectInstance, element: T, block: suspend () -> R) = - mutex.registerSelectLock(select, null) { - doLockedNext(element) - block() - } - - // assert: mutex.isLocked() - private fun doLockedNext(elem: T) { - // check if already closed for send - if (isCompleted) { - doLockedSignalCompleted() - throw sendException() - } - // notify subscriber - try { - subscriber.onNext(elem) - } catch (e: Throwable) { - try { - if (!cancel(e)) - handleCoroutineException(context, e) - } finally { - doLockedSignalCompleted() - } - throw sendException() - } - /* - There is no sense to check for `isCompleted` before doing `unlock`, because completion might - happen after this check and before `unlock` (see `afterCompleted` that does not do anything - if it fails to acquire the lock that we are still holding). - We have to recheck `isCompleted` after `unlock` anyway. - */ - mutex.unlock() - // recheck isCompleted - if (isCompleted && mutex.tryLock()) - doLockedSignalCompleted() - } - - // assert: mutex.isLocked() - private fun doLockedSignalCompleted() { - try { - if (signal >= CLOSED) { - signal = SIGNALLED // we'll signal onError/onCompleted (that the final state -- no CAS needed) - val state = this.state - try { - if (state is CompletedExceptionally && state.cause != null) - subscriber.onError(state.cause) - else - subscriber.onComplete() - } catch (e: Throwable) { - handleCoroutineException(context, e) - } - } - } finally { - mutex.unlock() - } - } - - override fun afterCompletion(state: Any?, mode: Int) { - if (!SIGNAL.compareAndSet(this, OPEN, CLOSED)) return // abort, other thread invoked doLockedSignalCompleted - if (mutex.tryLock()) // if we can acquire the lock - doLockedSignalCompleted() - } - - // Cancellable impl - override fun cancel() { cancel(cause = null) } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxScheduler.kt b/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxScheduler.kt deleted file mode 100644 index 5d5e754ab7..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxScheduler.kt +++ /dev/null @@ -1,61 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Scheduler -import kotlinx.coroutines.experimental.CancellableContinuation -import kotlinx.coroutines.experimental.CoroutineDispatcher -import kotlinx.coroutines.experimental.Delay -import kotlinx.coroutines.experimental.DisposableHandle -import java.util.concurrent.TimeUnit -import kotlin.coroutines.experimental.CoroutineContext - -/** - * Converts an instance of [Scheduler] to an implementation of [CoroutineDispatcher] - * and provides native [delay][Delay.delay] support. - */ -public fun Scheduler.asCoroutineDispatcher() = SchedulerCoroutineDispatcher(this) - -/** - * Implements [CoroutineDispatcher] on top of an arbitrary [Scheduler]. - * @param scheduler a scheduler. - */ -public class SchedulerCoroutineDispatcher(private val scheduler: Scheduler) : CoroutineDispatcher(), Delay { - override fun dispatch(context: CoroutineContext, block: Runnable) { - scheduler.scheduleDirect(block) - } - - override fun scheduleResumeAfterDelay(time: Long, unit: TimeUnit, continuation: CancellableContinuation) { - val disposable = scheduler.scheduleDirect({ - with(continuation) { resumeUndispatched(Unit) } - }, time, unit) - continuation.disposeOnCompletion(disposable) - } - - override fun invokeOnTimeout(time: Long, unit: TimeUnit, block: Runnable): DisposableHandle { - val disposable = scheduler.scheduleDirect(block, time, unit) - return object : DisposableHandle { - override fun dispose() { - disposable.dispose() - } - } - } - - override fun toString(): String = scheduler.toString() - override fun equals(other: Any?): Boolean = other is SchedulerCoroutineDispatcher && other.scheduler === scheduler - override fun hashCode(): Int = System.identityHashCode(scheduler) -} diff --git a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxSingle.kt b/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxSingle.kt deleted file mode 100644 index 1b5ff0c3db..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/main/kotlin/kotlinx/coroutines/experimental/rx2/RxSingle.kt +++ /dev/null @@ -1,65 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Single -import io.reactivex.SingleEmitter -import io.reactivex.functions.Cancellable -import kotlinx.coroutines.experimental.AbstractCoroutine -import kotlinx.coroutines.experimental.CoroutineScope -import kotlinx.coroutines.experimental.Job -import kotlinx.coroutines.experimental.newCoroutineContext -import kotlin.coroutines.experimental.CoroutineContext -import kotlin.coroutines.experimental.startCoroutine - -/** - * Creates cold [single][Single] that will run a given [block] in a coroutine. - * Every time the returned observable is subscribed, it starts a new coroutine in the specified [context]. - * Coroutine returns a single value. Unsubscribing cancels running coroutine. - * - * | **Coroutine action** | **Signal to subscriber** - * | ------------------------------------- | ------------------------ - * | Returns a value | `onSuccess` - * | Failure with exception or unsubscribe | `onError` - */ -public fun rxSingle( - context: CoroutineContext, - block: suspend CoroutineScope.() -> T -): Single = Single.create { subscriber -> - val newContext = newCoroutineContext(context) - val coroutine = RxSingleCoroutine(newContext, subscriber) - coroutine.initParentJob(context[Job]) - subscriber.setCancellable(coroutine) - block.startCoroutine(coroutine, coroutine) -} - -private class RxSingleCoroutine( - override val parentContext: CoroutineContext, - private val subscriber: SingleEmitter -) : AbstractCoroutine(true), Cancellable { - @Suppress("UNCHECKED_CAST") - override fun afterCompletion(state: Any?, mode: Int) { - if (subscriber.isDisposed) return - if (state is CompletedExceptionally) - subscriber.onError(state.exception) - else - subscriber.onSuccess(state as T) - } - - // Cancellable impl - override fun cancel() { cancel(cause = null) } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/module-info.java b/reactive/kotlinx-coroutines-rx2/src/module-info.java new file mode 100644 index 0000000000..539ea3ee05 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/src/module-info.java @@ -0,0 +1,10 @@ +@SuppressWarnings("JavaModuleNaming") +module kotlinx.coroutines.rx2 { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.coroutines.reactive; + requires kotlinx.atomicfu; + requires io.reactivex.rxjava2; + + exports kotlinx.coroutines.rx2; +} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-01.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-01.kt deleted file mode 100644 index d59ecb1c33..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-01.kt +++ /dev/null @@ -1,42 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.basic.example01 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.* - -fun main(args: Array) = runBlocking { - // create a channel that produces numbers from 1 to 3 with 200ms delays between them - val source = produce(context) { - println("Begin") // mark the beginning of this coroutine in output - for (x in 1..3) { - delay(200) // wait for 200ms - send(x) // send number x to the channel - } - } - // print elements from the source - println("Elements:") - source.consumeEach { // consume elements from it - println(it) - } - // print elements from the source AGAIN - println("Again:") - source.consumeEach { // consume elements from it - println(it) - } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-02.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-02.kt deleted file mode 100644 index 62ba11701e..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-02.kt +++ /dev/null @@ -1,43 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.basic.example02 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* - -fun main(args: Array) = runBlocking { - // create a publisher that produces numbers from 1 to 3 with 200ms delays between them - val source = publish(context) { - // ^^^^^^^ <--- Difference from the previous examples is here - println("Begin") // mark the beginning of this coroutine in output - for (x in 1..3) { - delay(200) // wait for 200ms - send(x) // send number x to the channel - } - } - // print elements from the source - println("Elements:") - source.consumeEach { // consume elements from it - println(it) - } - // print elements from the source AGAIN - println("Again:") - source.consumeEach { // consume elements from it - println(it) - } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-03.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-03.kt deleted file mode 100644 index 144b74ed3e..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-03.kt +++ /dev/null @@ -1,36 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.basic.example03 - -import io.reactivex.* -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* - -fun main(args: Array) = runBlocking { - val source = Flowable.range(1, 5) // a range of five numbers - .doOnSubscribe { println("OnSubscribe") } // provide some insight - .doFinally { println("Finally") } // ... into what's going on - var cnt = 0 - source.open().use { channel -> // open channel to the source - for (x in channel) { // iterate over the channel to receive elements from it - println(x) - if (++cnt >= 3) break // break when 3 elements are printed - } - // `use` will close the channel when this block of code is complete - } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-04.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-04.kt deleted file mode 100644 index f41348afd6..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-04.kt +++ /dev/null @@ -1,30 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.basic.example04 - -import io.reactivex.* -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* - -fun main(args: Array) = runBlocking { - val source = Flowable.range(1, 5) // a range of five numbers - .doOnSubscribe { println("OnSubscribe") } // provide some insight - .doFinally { println("Finally") } // ... into what's going on - // iterate over the source fully - source.consumeEach { println(it) } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-05.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-05.kt deleted file mode 100644 index adde0078af..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-05.kt +++ /dev/null @@ -1,41 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.basic.example05 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.rx2.rxFlowable -import io.reactivex.schedulers.Schedulers - -fun main(args: Array) = runBlocking { - // coroutine -- fast producer of elements in the context of the main thread - val source = rxFlowable(context) { - for (x in 1..3) { - send(x) // this is a suspending function - println("Sent $x") // print after successfully sent item - } - } - // subscribe on another thread with a slow subscriber using Rx - source - .observeOn(Schedulers.io(), false, 1) // specify buffer size of 1 item - .doOnComplete { println("Complete") } - .subscribe { x -> - Thread.sleep(500) // 500ms to process each item - println("Processed $x") - } - delay(2000) // suspend the main thread for a few seconds -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-06.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-06.kt deleted file mode 100644 index eb84eaa6d9..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-06.kt +++ /dev/null @@ -1,30 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.basic.example06 - -import io.reactivex.subjects.BehaviorSubject - -fun main(args: Array) { - val subject = BehaviorSubject.create() - subject.onNext("one") - subject.onNext("two") // updates the state of BehaviorSubject, "one" value is lost - // now subscribe to this subject and print everything - subject.subscribe(System.out::println) - subject.onNext("three") - subject.onNext("four") -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-07.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-07.kt deleted file mode 100644 index 75cc3a6904..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-07.kt +++ /dev/null @@ -1,35 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.basic.example07 - -import io.reactivex.subjects.BehaviorSubject -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.rx2.consumeEach - -fun main(args: Array) = runBlocking { - val subject = BehaviorSubject.create() - subject.onNext("one") - subject.onNext("two") - // now launch a coroutine to print everything - launch(context) { // use the context of the main thread for a coroutine - subject.consumeEach { println(it) } - } - subject.onNext("three") - subject.onNext("four") -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-08.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-08.kt deleted file mode 100644 index 293b151075..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-08.kt +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.basic.example08 - -import io.reactivex.subjects.BehaviorSubject -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.rx2.consumeEach -import kotlinx.coroutines.experimental.yield - -fun main(args: Array) = runBlocking { - val subject = BehaviorSubject.create() - subject.onNext("one") - subject.onNext("two") - // now launch a coroutine to print everything - launch(context) { // use the context of the main thread for a coroutine - subject.consumeEach { println(it) } - } - subject.onNext("three") - yield() // yield the main thread to the launched coroutine <--- HERE - subject.onNext("four") -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-09.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-09.kt deleted file mode 100644 index 2ad05dfde9..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-basic-09.kt +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.basic.example09 - -import kotlinx.coroutines.experimental.channels.ConflatedBroadcastChannel -import kotlinx.coroutines.experimental.channels.consumeEach -import kotlinx.coroutines.experimental.launch -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield - -fun main(args: Array) = runBlocking { - val broadcast = ConflatedBroadcastChannel() - broadcast.offer("one") - broadcast.offer("two") - // now launch a coroutine to print everything - launch(context) { // use the context of the main thread for a coroutine - broadcast.consumeEach { println(it) } - } - broadcast.offer("three") - yield() // yield the main thread to the launched coroutine - broadcast.offer("four") -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-01.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-01.kt deleted file mode 100644 index 1fdeaa9669..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-01.kt +++ /dev/null @@ -1,35 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.context.example01 - -import io.reactivex.* -import io.reactivex.functions.BiFunction -import io.reactivex.schedulers.Schedulers -import java.util.concurrent.TimeUnit - -fun rangeWithIntervalRx(scheduler: Scheduler, time: Long, start: Int, count: Int): Flowable = - Flowable.zip( - Flowable.range(start, count), - Flowable.interval(time, TimeUnit.MILLISECONDS, scheduler), - BiFunction { x, _ -> x }) - -fun main(args: Array) { - rangeWithIntervalRx(Schedulers.computation(), 100, 1, 3) - .subscribe { println("$it on thread ${Thread.currentThread().name}") } - Thread.sleep(1000) -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-02.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-02.kt deleted file mode 100644 index 96a62defd2..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-02.kt +++ /dev/null @@ -1,36 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.context.example02 - -import io.reactivex.* -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* -import kotlin.coroutines.experimental.CoroutineContext - -fun rangeWithInterval(context: CoroutineContext, time: Long, start: Int, count: Int) = publish(context) { - for (x in start until start + count) { - delay(time) // wait before sending each number - send(x) - } -} - -fun main(args: Array) { - Flowable.fromPublisher(rangeWithInterval(CommonPool, 100, 1, 3)) - .subscribe { println("$it on thread ${Thread.currentThread().name}") } - Thread.sleep(1000) -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-03.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-03.kt deleted file mode 100644 index b4b6255689..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-03.kt +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.context.example03 - -import io.reactivex.* -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* -import io.reactivex.schedulers.Schedulers -import kotlin.coroutines.experimental.CoroutineContext - -fun rangeWithInterval(context: CoroutineContext, time: Long, start: Int, count: Int) = publish(context) { - for (x in start until start + count) { - delay(time) // wait before sending each number - send(x) - } -} - -fun main(args: Array) { - Flowable.fromPublisher(rangeWithInterval(CommonPool, 100, 1, 3)) - .observeOn(Schedulers.computation()) // <-- THIS LINE IS ADDED - .subscribe { println("$it on thread ${Thread.currentThread().name}") } - Thread.sleep(1000) -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-04.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-04.kt deleted file mode 100644 index 5d0594081d..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-04.kt +++ /dev/null @@ -1,36 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.context.example04 - -import io.reactivex.* -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* -import io.reactivex.functions.BiFunction -import io.reactivex.schedulers.Schedulers -import java.util.concurrent.TimeUnit - -fun rangeWithIntervalRx(scheduler: Scheduler, time: Long, start: Int, count: Int): Flowable = - Flowable.zip( - Flowable.range(start, count), - Flowable.interval(time, TimeUnit.MILLISECONDS, scheduler), - BiFunction { x, _ -> x }) - -fun main(args: Array) = runBlocking { - rangeWithIntervalRx(Schedulers.computation(), 100, 1, 3) - .consumeEach { println("$it on thread ${Thread.currentThread().name}") } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-05.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-05.kt deleted file mode 100644 index 2d19c502c0..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-context-05.kt +++ /dev/null @@ -1,39 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.context.example05 - -import io.reactivex.* -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* -import io.reactivex.functions.BiFunction -import io.reactivex.schedulers.Schedulers -import java.util.concurrent.TimeUnit - -fun rangeWithIntervalRx(scheduler: Scheduler, time: Long, start: Int, count: Int): Flowable = - Flowable.zip( - Flowable.range(start, count), - Flowable.interval(time, TimeUnit.MILLISECONDS, scheduler), - BiFunction { x, _ -> x }) - -fun main(args: Array) = runBlocking { - val job = launch(Unconfined) { // launch new coroutine in Unconfined context (without its own thread pool) - rangeWithIntervalRx(Schedulers.computation(), 100, 1, 3) - .consumeEach { println("$it on thread ${Thread.currentThread().name}") } - } - job.join() // wait for our coroutine to complete -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-01.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-01.kt deleted file mode 100644 index cda91db461..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-01.kt +++ /dev/null @@ -1,30 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.operators.example01 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* -import kotlin.coroutines.experimental.CoroutineContext - -fun range(context: CoroutineContext, start: Int, count: Int) = publish(context) { - for (x in start until start + count) send(x) -} - -fun main(args: Array) = runBlocking { - range(CommonPool, 1, 5).consumeEach { println(it) } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-02.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-02.kt deleted file mode 100644 index e5a85f2427..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-02.kt +++ /dev/null @@ -1,44 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.operators.example02 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* -import org.reactivestreams.Publisher -import kotlin.coroutines.experimental.CoroutineContext - -fun Publisher.fusedFilterMap( - context: CoroutineContext, // the context to execute this coroutine in - predicate: (T) -> Boolean, // the filter predicate - mapper: (T) -> R // the mapper function -) = publish(context) { - consumeEach { // consume the source stream - if (predicate(it)) // filter part - send(mapper(it)) // map part - } -} - -fun range(context: CoroutineContext, start: Int, count: Int) = publish(context) { - for (x in start until start + count) send(x) -} - -fun main(args: Array) = runBlocking { - range(context, 1, 5) - .fusedFilterMap(context, { it % 2 == 0}, { "$it is even" }) - .consumeEach { println(it) } // print all the resulting strings -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-03.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-03.kt deleted file mode 100644 index cca8893cc1..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-03.kt +++ /dev/null @@ -1,48 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.operators.example03 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* -import org.reactivestreams.Publisher -import kotlin.coroutines.experimental.CoroutineContext -import kotlinx.coroutines.experimental.selects.whileSelect - -fun Publisher.takeUntil(context: CoroutineContext, other: Publisher) = publish(context) { - this@takeUntil.open().use { thisChannel -> // explicitly open channel to Publisher - other.open().use { otherChannel -> // explicitly open channel to Publisher - whileSelect { - otherChannel.onReceive { false } // bail out on any received element from `other` - thisChannel.onReceive { send(it); true } // resend element from this channel and continue - } - } - } -} - -fun rangeWithInterval(context: CoroutineContext, time: Long, start: Int, count: Int) = publish(context) { - for (x in start until start + count) { - delay(time) // wait before sending each number - send(x) - } -} - -fun main(args: Array) = runBlocking { - val slowNums = rangeWithInterval(context, 200, 1, 10) // numbers with 200ms interval - val stop = rangeWithInterval(context, 500, 1, 10) // the first one after 500ms - slowNums.takeUntil(context, stop).consumeEach { println(it) } // let's test it -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-04.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-04.kt deleted file mode 100644 index b5ce14367a..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/example-reactive-operators-04.kt +++ /dev/null @@ -1,49 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.reactive.operators.example04 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.reactive.* -import org.reactivestreams.Publisher -import kotlin.coroutines.experimental.CoroutineContext - -fun Publisher>.merge(context: CoroutineContext) = publish(context) { - consumeEach { pub -> // for each publisher received on the source channel - launch(this.context) { // launch a child coroutine - pub.consumeEach { send(it) } // resend all element from this publisher - } - } -} - -fun rangeWithInterval(context: CoroutineContext, time: Long, start: Int, count: Int) = publish(context) { - for (x in start until start + count) { - delay(time) // wait before sending each number - send(x) - } -} - -fun testPub(context: CoroutineContext) = publish>(context) { - send(rangeWithInterval(context, 250, 1, 4)) // number 1 at 250ms, 2 at 500ms, 3 at 750ms, 4 at 1000ms - delay(100) // wait for 100 ms - send(rangeWithInterval(context, 500, 11, 3)) // number 11 at 600ms, 12 at 1100ms, 13 at 1600ms - delay(1100) // wait for 1.1s - done in 1.2 sec after start -} - -fun main(args: Array) = runBlocking { - testPub(context).merge(context).consumeEach { println(it) } // print the whole stream -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/test/GuideReactiveTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/test/GuideReactiveTest.kt deleted file mode 100644 index de76106808..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/guide/test/GuideReactiveTest.kt +++ /dev/null @@ -1,188 +0,0 @@ -// This file was automatically generated from coroutines-guide-reactive.md by Knit tool. Do not edit. -package guide.test - -import org.junit.Test - -class GuideReactiveTest { - - @Test - fun testGuideReactiveBasicExample01() { - test { guide.reactive.basic.example01.main(emptyArray()) }.verifyLines( - "Elements:", - "Begin", - "1", - "2", - "3", - "Again:" - ) - } - - @Test - fun testGuideReactiveBasicExample02() { - test { guide.reactive.basic.example02.main(emptyArray()) }.verifyLines( - "Elements:", - "Begin", - "1", - "2", - "3", - "Again:", - "Begin", - "1", - "2", - "3" - ) - } - - @Test - fun testGuideReactiveBasicExample03() { - test { guide.reactive.basic.example03.main(emptyArray()) }.verifyLines( - "OnSubscribe", - "1", - "2", - "3", - "Finally" - ) - } - - @Test - fun testGuideReactiveBasicExample04() { - test { guide.reactive.basic.example04.main(emptyArray()) }.verifyLines( - "OnSubscribe", - "1", - "2", - "3", - "4", - "Finally", - "5" - ) - } - - @Test - fun testGuideReactiveBasicExample05() { - test { guide.reactive.basic.example05.main(emptyArray()) }.verifyLines( - "Sent 1", - "Processed 1", - "Sent 2", - "Processed 2", - "Sent 3", - "Processed 3", - "Complete" - ) - } - - @Test - fun testGuideReactiveBasicExample06() { - test { guide.reactive.basic.example06.main(emptyArray()) }.verifyLines( - "two", - "three", - "four" - ) - } - - @Test - fun testGuideReactiveBasicExample07() { - test { guide.reactive.basic.example07.main(emptyArray()) }.verifyLines( - "four" - ) - } - - @Test - fun testGuideReactiveBasicExample08() { - test { guide.reactive.basic.example08.main(emptyArray()) }.verifyLines( - "three", - "four" - ) - } - - @Test - fun testGuideReactiveBasicExample09() { - test { guide.reactive.basic.example09.main(emptyArray()) }.verifyLines( - "three", - "four" - ) - } - - @Test - fun testGuideReactiveOperatorsExample01() { - test { guide.reactive.operators.example01.main(emptyArray()) }.verifyLines( - "1", - "2", - "3", - "4", - "5" - ) - } - - @Test - fun testGuideReactiveOperatorsExample02() { - test { guide.reactive.operators.example02.main(emptyArray()) }.verifyLines( - "2 is even", - "4 is even" - ) - } - - @Test - fun testGuideReactiveOperatorsExample03() { - test { guide.reactive.operators.example03.main(emptyArray()) }.verifyLines( - "1", - "2" - ) - } - - @Test - fun testGuideReactiveOperatorsExample04() { - test { guide.reactive.operators.example04.main(emptyArray()) }.verifyLines( - "1", - "2", - "11", - "3", - "4", - "12" - ) - } - - @Test - fun testGuideReactiveContextExample01() { - test { guide.reactive.context.example01.main(emptyArray()) }.verifyLinesFlexibleThread( - "1 on thread RxComputationThreadPool-1", - "2 on thread RxComputationThreadPool-1", - "3 on thread RxComputationThreadPool-1" - ) - } - - @Test - fun testGuideReactiveContextExample02() { - test { guide.reactive.context.example02.main(emptyArray()) }.verifyLinesStart( - "1 on thread ForkJoinPool.commonPool-worker-1", - "2 on thread ForkJoinPool.commonPool-worker-1", - "3 on thread ForkJoinPool.commonPool-worker-1" - ) - } - - @Test - fun testGuideReactiveContextExample03() { - test { guide.reactive.context.example03.main(emptyArray()) }.verifyLinesFlexibleThread( - "1 on thread RxComputationThreadPool-1", - "2 on thread RxComputationThreadPool-1", - "3 on thread RxComputationThreadPool-1" - ) - } - - @Test - fun testGuideReactiveContextExample04() { - test { guide.reactive.context.example04.main(emptyArray()) }.verifyLinesStart( - "1 on thread main", - "2 on thread main", - "3 on thread main" - ) - } - - @Test - fun testGuideReactiveContextExample05() { - test { guide.reactive.context.example05.main(emptyArray()) }.verifyLinesStart( - "1 on thread RxComputationThreadPool-1", - "2 on thread RxComputationThreadPool-1", - "3 on thread RxComputationThreadPool-1" - ) - } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/Check.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/Check.kt deleted file mode 100644 index b6deb48e60..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/Check.kt +++ /dev/null @@ -1,78 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Maybe -import io.reactivex.Observable -import io.reactivex.Single - -fun checkSingleValue( - observable: Observable, - checker: (T) -> Unit -) { - val singleValue = observable.blockingSingle() - checker(singleValue) -} - -fun checkErroneous( - observable: Observable<*>, - checker: (Throwable) -> Unit -) { - val singleNotification = observable.materialize().blockingSingle() - checker(singleNotification.error) -} - -fun checkSingleValue( - single: Single, - checker: (T) -> Unit -) { - val singleValue = single.blockingGet() - checker(singleValue) -} - -fun checkErroneous( - single: Single<*>, - checker: (Throwable) -> Unit -) { - try { - single.blockingGet() - error("Should have failed") - } catch (e: Throwable) { - checker(e) - } -} - -fun checkMaybeValue( - maybe: Maybe, - checker: (T?) -> Unit -) { - val maybeValue = maybe.toFlowable().blockingIterable().firstOrNull() - checker(maybeValue) -} - -fun checkErroneous( - maybe: Maybe<*>, - checker: (Throwable) -> Unit -) { - try { - (maybe as Maybe).blockingGet() - error("Should have failed") - } catch (e: Throwable) { - checker(e) - } -} - diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/CompletableTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/CompletableTest.kt deleted file mode 100644 index a85af968d5..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/CompletableTest.kt +++ /dev/null @@ -1,113 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert.assertThat -import org.junit.Test - -class CompletableTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(4) - } - expect(2) - completable.subscribe { - expect(5) - } - expect(3) - yield() // to completable coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - completable.subscribe({ - expectUnreached() - }, { error -> - expect(5) - assertThat(error, IsInstanceOf(RuntimeException::class.java)) - assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to completable coroutine - finish(6) - } - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - // nothing is called on a disposed rx2 completable - val sub = completable.subscribe({ - expectUnreached() - }, { - expectUnreached() - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.dispose() // will cancel coroutine - yield() - finish(6) - } - - @Test - fun testAwaitSuccess() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(3) - } - expect(2) - completable.await() // shall launch coroutine - finish(4) - } - - @Test - fun testAwaitFailure() = runBlocking { - expect(1) - val completable = rxCompletable(context) { - expect(3) - throw RuntimeException("OK") - } - expect(2) - try { - completable.await() // shall launch coroutine and throw exception - expectUnreached() - } catch (e: RuntimeException) { - finish(4) - assertThat(e.message, IsEqual("OK")) - } - } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ConvertTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ConvertTest.kt deleted file mode 100644 index 7649391ba2..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ConvertTest.kt +++ /dev/null @@ -1,172 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import kotlinx.coroutines.experimental.* -import kotlinx.coroutines.experimental.channels.produce -import org.junit.Assert.assertEquals -import org.junit.Assert.assertNull -import org.junit.Test - -class ConvertTest : TestBase() { - class TestException(s: String): RuntimeException(s) - - @Test - fun testToCompletableSuccess() = runBlocking { - expect(1) - val job = launch(context) { - expect(3) - } - val completable = job.asCompletable(context) - completable.subscribe { - expect(4) - } - expect(2) - yield() - finish(5) - } - - @Test - fun testToCompletableFail() = runBlocking { - expect(1) - val job = async(context + NonCancellable) { // don't kill parent on exception - expect(3) - throw RuntimeException("OK") - } - val completable = job.asCompletable(context) - completable.subscribe { - expect(4) - } - expect(2) - yield() - finish(5) - } - - @Test - fun testToMaybe() { - val d = async(CommonPool) { - delay(50) - "OK" - } - val maybe1 = d.asMaybe(Unconfined) - checkMaybeValue(maybe1) { - assertEquals("OK", it) - } - val maybe2 = d.asMaybe(Unconfined) - checkMaybeValue(maybe2) { - assertEquals("OK", it) - } - } - - @Test - fun testToMaybeEmpty() { - val d = async(CommonPool) { - delay(50) - null - } - val maybe1 = d.asMaybe(Unconfined) - checkMaybeValue(maybe1, ::assertNull) - val maybe2 = d.asMaybe(Unconfined) - checkMaybeValue(maybe2, ::assertNull) - } - - @Test - fun testToMaybeFail() { - val d = async(CommonPool) { - delay(50) - throw TestException("OK") - } - val maybe1 = d.asMaybe(Unconfined) - checkErroneous(maybe1) { - check(it is TestException && it.message == "OK") { "$it" } - } - val maybe2 = d.asMaybe(Unconfined) - checkErroneous(maybe2) { - check(it is TestException && it.message == "OK") { "$it" } - } - } - - @Test - fun testToSingle() { - val d = async(CommonPool) { - delay(50) - "OK" - } - val single1 = d.asSingle(Unconfined) - checkSingleValue(single1) { - assertEquals("OK", it) - } - val single2 = d.asSingle(Unconfined) - checkSingleValue(single2) { - assertEquals("OK", it) - } - } - - @Test - fun testToSingleFail() { - val d = async(CommonPool) { - delay(50) - throw TestException("OK") - } - val single1 = d.asSingle(Unconfined) - checkErroneous(single1) { - check(it is TestException && it.message == "OK") { "$it" } - } - val single2 = d.asSingle(Unconfined) - checkErroneous(single2) { - check(it is TestException && it.message == "OK") { "$it" } - } - } - - @Test - fun testToObservable() { - val c = produce(CommonPool) { - delay(50) - send("O") - delay(50) - send("K") - } - val observable = c.asObservable(Unconfined) - checkSingleValue(observable.reduce { t1, t2 -> t1 + t2 }.toSingle()) { - assertEquals("OK", it) - } - } - - @Test - fun testToObservableFail() { - val c = produce(CommonPool) { - delay(50) - send("O") - delay(50) - throw TestException("K") - } - val observable = c.asObservable(Unconfined) - val single = rxSingle(Unconfined) { - var result = "" - try { - observable.consumeEach { result += it } - } catch(e: Throwable) { - check(e is TestException) - result += e.message - } - result - } - checkSingleValue(single) { - assertEquals("OK", it) - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/FlowableTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/FlowableTest.kt deleted file mode 100644 index aaee010fa8..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/FlowableTest.kt +++ /dev/null @@ -1,86 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert -import org.junit.Test - -class FlowableTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val observable = rxFlowable(context) { - expect(4) - send("OK") - } - expect(2) - observable.subscribe { value -> - expect(5) - Assert.assertThat(value, IsEqual("OK")) - } - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val observable = rxFlowable(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - observable.subscribe({ - expectUnreached() - }, { error -> - expect(5) - Assert.assertThat(error, IsInstanceOf(RuntimeException::class.java)) - Assert.assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val observable = rxFlowable(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - val sub = observable.subscribe({ - expectUnreached() - }, { - expectUnreached() - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.dispose() // will cancel coroutine - yield() - finish(6) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/IntegrationTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/IntegrationTest.kt deleted file mode 100644 index fdd10c1eb9..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/IntegrationTest.kt +++ /dev/null @@ -1,131 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Observable -import kotlinx.coroutines.experimental.* -import org.hamcrest.MatcherAssert.assertThat -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Test -import org.junit.runner.RunWith -import org.junit.runners.Parameterized -import kotlin.coroutines.experimental.CoroutineContext - -@RunWith(Parameterized::class) -class IntegrationTest( - val ctx: Ctx, - val delay: Boolean -) : TestBase() { - - enum class Ctx { - MAIN { override fun invoke(context: CoroutineContext): CoroutineContext = context }, - COMMON_POOL { override fun invoke(context: CoroutineContext): CoroutineContext = CommonPool }, - UNCONFINED { override fun invoke(context: CoroutineContext): CoroutineContext = Unconfined }; - - abstract operator fun invoke(context: CoroutineContext): CoroutineContext - } - - companion object { - @Parameterized.Parameters(name = "ctx={0}, delay={1}") - @JvmStatic - fun params(): Collection> = Ctx.values().flatMap { ctx -> - listOf(false, true).map { delay -> - arrayOf(ctx, delay) - } - } - } - - @Test - fun testEmpty(): Unit = runBlocking { - val observable = rxObservable(ctx(context)) { - if (delay) delay(1) - // does not send anything - } - assertNSE { observable.awaitFirst() } - assertThat(observable.awaitFirstOrDefault("OK"), IsEqual("OK")) - assertNSE { observable.awaitLast() } - assertNSE { observable.awaitSingle() } - var cnt = 0 - observable.consumeEach { - cnt++ - } - assertThat(cnt, IsEqual(0)) - } - - @Test - fun testSingle() = runBlocking { - val observable = rxObservable(ctx(context)) { - if (delay) delay(1) - send("OK") - } - assertThat(observable.awaitFirst(), IsEqual("OK")) - assertThat(observable.awaitLast(), IsEqual("OK")) - assertThat(observable.awaitSingle(), IsEqual("OK")) - var cnt = 0 - observable.consumeEach { - assertThat(it, IsEqual("OK")) - cnt++ - } - assertThat(cnt, IsEqual(1)) - } - - @Test - fun testNumbers() = runBlocking { - val n = 100 * stressTestMultiplier - val observable = rxObservable(ctx(context)) { - for (i in 1..n) { - send(i) - if (delay) delay(1) - } - } - assertThat(observable.awaitFirst(), IsEqual(1)) - assertThat(observable.awaitLast(), IsEqual(n)) - assertIAE { observable.awaitSingle() } - checkNumbers(n, observable) - val channel = observable.open() - checkNumbers(n, channel.asObservable(ctx(context))) - channel.close() - } - - private suspend fun checkNumbers(n: Int, observable: Observable) { - var last = 0 - observable.consumeEach { - assertThat(it, IsEqual(++last)) - } - assertThat(last, IsEqual(n)) - } - - - inline fun assertIAE(block: () -> Unit) { - try { - block() - expectUnreached() - } catch (e: Throwable) { - assertThat(e, IsInstanceOf(IllegalArgumentException::class.java)) - } - } - - inline fun assertNSE(block: () -> Unit) { - try { - block() - expectUnreached() - } catch (e: Throwable) { - assertThat(e, IsInstanceOf(NoSuchElementException::class.java)) - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/MaybeTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/MaybeTest.kt deleted file mode 100644 index 628441fa46..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/MaybeTest.kt +++ /dev/null @@ -1,216 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Maybe -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.runBlocking -import org.junit.Assert.assertEquals -import org.junit.Test -import io.reactivex.Observable -import io.reactivex.functions.Action -import io.reactivex.internal.functions.Functions.ON_ERROR_MISSING -import io.reactivex.internal.functions.Functions.emptyConsumer -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert -import org.junit.Assert.assertNull -import java.util.concurrent.TimeUnit - -class MaybeTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val maybe = rxMaybe(context) { - expect(4) - "OK" - } - expect(2) - maybe.subscribe { value -> - expect(5) - Assert.assertThat(value, IsEqual("OK")) - } - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicEmpty() = runBlocking { - expect(1) - val maybe = rxMaybe(context) { - expect(4) - null - } - expect(2) - maybe.subscribe (emptyConsumer(), ON_ERROR_MISSING, Action { - expect(5) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val maybe = rxMaybe(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - maybe.subscribe({ - expectUnreached() - }, { error -> - expect(5) - Assert.assertThat(error, IsInstanceOf(RuntimeException::class.java)) - Assert.assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val maybe = rxMaybe(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - // nothing is called on a disposed rx2 maybe - val sub = maybe.subscribe({ - expectUnreached() - }, { - expectUnreached() - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.dispose() // will cancel coroutine - yield() - finish(6) - } - - @Test - fun testMaybeNoWait() { - val maybe = rxMaybe(CommonPool) { - "OK" - } - - checkMaybeValue(maybe) { - assertEquals("OK", it) - } - } - - @Test - fun testMaybeAwait() = runBlocking { - assertEquals("OK", Maybe.just("O").await() + "K") - } - - @Test - fun testMaybeAwaitForNull() = runBlocking { - assertNull(Maybe.empty().await()) - } - - @Test - fun testMaybeEmitAndAwait() { - val maybe = rxMaybe(CommonPool) { - Maybe.just("O").await() + "K" - } - - checkMaybeValue(maybe) { - assertEquals("OK", it) - } - } - - @Test - fun testMaybeWithDelay() { - val maybe = rxMaybe(CommonPool) { - Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K" - } - - checkMaybeValue(maybe) { - assertEquals("OK", it) - } - } - - @Test - fun testMaybeException() { - val maybe = rxMaybe(CommonPool) { - Observable.just("O", "K").awaitSingle() + "K" - } - - checkErroneous(maybe) { - assert(it is IllegalArgumentException) - } - } - - @Test - fun testAwaitFirst() { - val maybe = rxMaybe(CommonPool) { - Observable.just("O", "#").awaitFirst() + "K" - } - - checkMaybeValue(maybe) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitLast() { - val maybe = rxMaybe(CommonPool) { - Observable.just("#", "O").awaitLast() + "K" - } - - checkMaybeValue(maybe) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromObservable() { - val maybe = rxMaybe(CommonPool) { - try { - Observable.error(RuntimeException("O")).awaitFirst() - } catch (e: RuntimeException) { - Observable.just(e.message!!).awaitLast() + "K" - } - } - - checkMaybeValue(maybe) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromCoroutine() { - val maybe = rxMaybe(CommonPool) { - throw IllegalStateException(Observable.just("O").awaitSingle() + "K") - } - - checkErroneous(maybe) { - assert(it is IllegalStateException) - assertEquals("OK", it.message) - } - } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableCompletionStressTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableCompletionStressTest.kt deleted file mode 100644 index bf61ba1fd0..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableCompletionStressTest.kt +++ /dev/null @@ -1,51 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.withTimeout -import org.junit.Test -import java.util.* -import kotlin.coroutines.experimental.CoroutineContext - -class ObservableCompletionStressTest : TestBase() { - val N_REPEATS = 10_000 * stressTestMultiplier - - fun range(context: CoroutineContext, start: Int, count: Int) = rxObservable(context) { - for (x in start until start + count) send(x) - } - - @Test - fun testCompletion() { - val rnd = Random() - repeat(N_REPEATS) { - val count = rnd.nextInt(5) - runBlocking { - withTimeout(5000) { - var received = 0 - range(CommonPool, 1, count).consumeEach { x -> - received++ - if (x != received) error("$x != $received") - } - if (received != count) error("$received != $count") - } - } - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableMultiTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableMultiTest.kt deleted file mode 100644 index 49a2abe6d5..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableMultiTest.kt +++ /dev/null @@ -1,103 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Observable -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.Unconfined -import kotlinx.coroutines.experimental.launch -import org.junit.Assert.assertEquals -import org.junit.Test -import java.io.IOException - -/** - * Test emitting multiple values with [rxObservable]. - */ -class ObservableMultiTest : TestBase() { - @Test - fun testNumbers() { - val n = 100 * stressTestMultiplier - val observable = rxObservable(CommonPool) { - repeat(n) { send(it) } - } - checkSingleValue(observable.toList()) { list -> - assertEquals((0..n - 1).toList(), list) - } - } - - @Test - fun testConcurrentStress() { - val n = 10_000 * stressTestMultiplier - val observable = rxObservable(CommonPool) { - // concurrent emitters (many coroutines) - val jobs = List(n) { - // launch - launch(CommonPool) { - send(it) - } - } - jobs.forEach { it.join() } - } - checkSingleValue(observable.toList()) { list -> - assertEquals(n, list.size) - assertEquals((0..n - 1).toList(), list.sorted()) - } - } - - @Test - fun testIteratorResendUnconfined() { - val n = 10_000 * stressTestMultiplier - val observable = rxObservable(Unconfined) { - Observable.range(0, n).consumeEach { send(it) } - } - checkSingleValue(observable.toList()) { list -> - assertEquals((0..n - 1).toList(), list) - } - } - - @Test - fun testIteratorResendPool() { - val n = 10_000 * stressTestMultiplier - val observable = rxObservable(CommonPool) { - Observable.range(0, n).consumeEach { send(it) } - } - checkSingleValue(observable.toList()) { list -> - assertEquals((0..n - 1).toList(), list) - } - } - - @Test - fun testSendAndCrash() { - val observable = rxObservable(CommonPool) { - send("O") - throw IOException("K") - } - val single = rxSingle(CommonPool) { - var result = "" - try { - observable.consumeEach { result += it } - } catch(e: IOException) { - result += e.message - } - result - } - checkSingleValue(single) { - assertEquals("OK", it) - } - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableSingleTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableSingleTest.kt deleted file mode 100644 index a20a6be786..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableSingleTest.kt +++ /dev/null @@ -1,179 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.Observable -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.runBlocking -import org.junit.Assert.assertEquals -import org.junit.Assert.fail -import org.junit.Test -import java.util.concurrent.TimeUnit - -/** - * Tests emitting single item with [rxObservable]. - */ -class ObservableSingleTest { - @Test - fun testSingleNoWait() { - val observable = rxObservable(CommonPool) { - send("OK") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleAwait() = runBlocking { - assertEquals("OK", Observable.just("O").awaitSingle() + "K") - } - - @Test - fun testSingleEmitAndAwait() { - val observable = rxObservable(CommonPool) { - send(Observable.just("O").awaitSingle() + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleWithDelay() { - val observable = rxObservable(CommonPool) { - send(Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleException() { - val observable = rxObservable(CommonPool) { - send(Observable.just("O", "K").awaitSingle() + "K") - } - - checkErroneous(observable) { - assert(it is IllegalArgumentException) - } - } - - @Test - fun testAwaitFirst() { - val observable = rxObservable(CommonPool) { - send(Observable.just("O", "#").awaitFirst() + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitFirstOrDefault() { - val observable = rxObservable(CommonPool) { - send(Observable.empty().awaitFirstOrDefault("O") + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitFirstOrDefaultWithValues() { - val observable = rxObservable(CommonPool) { - send(Observable.just("O", "#").awaitFirstOrDefault("!") + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitLast() { - val observable = rxObservable(CommonPool) { - send(Observable.just("#", "O").awaitLast() + "K") - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromObservable() { - val observable = rxObservable(CommonPool) { - try { - send(Observable.error(RuntimeException("O")).awaitFirst()) - } catch (e: RuntimeException) { - send(Observable.just(e.message!!).awaitLast() + "K") - } - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromCoroutine() { - val observable = rxObservable(CommonPool) { - error(Observable.just("O").awaitSingle() + "K") - } - - checkErroneous(observable) { - assert(it is IllegalStateException) - assertEquals("OK", it.message) - } - } - - @Test - fun testObservableIteration() { - val observable = rxObservable(CommonPool) { - var result = "" - Observable.just("O", "K").consumeEach { result += it } - send(result) - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } - - @Test - fun testObservableIterationFailure() { - val observable = rxObservable(CommonPool) { - try { - Observable.error(RuntimeException("OK")).consumeEach { fail("Should not be here") } - send("Fail") - } catch (e: RuntimeException) { - send(e.message!!) - } - } - - checkSingleValue(observable) { - assertEquals("OK", it) - } - } -} diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableTest.kt deleted file mode 100644 index 7bdfa69d4e..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/ObservableTest.kt +++ /dev/null @@ -1,86 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.runBlocking -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert -import org.junit.Test - -class ObservableTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val observable = rxObservable(context) { - expect(4) - send("OK") - } - expect(2) - observable.subscribe { value -> - expect(5) - Assert.assertThat(value, IsEqual("OK")) - } - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val observable = rxObservable(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - observable.subscribe({ - expectUnreached() - }, { error -> - expect(5) - Assert.assertThat(error, IsInstanceOf(RuntimeException::class.java)) - Assert.assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val observable = rxObservable(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - val sub = observable.subscribe({ - expectUnreached() - }, { - expectUnreached() - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.dispose() // will cancel coroutine - yield() - finish(6) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/SchedulerTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/SchedulerTest.kt deleted file mode 100644 index 24a6ee85e2..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/SchedulerTest.kt +++ /dev/null @@ -1,44 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import io.reactivex.schedulers.Schedulers -import kotlinx.coroutines.experimental.* -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsNot -import org.junit.Assert.assertThat -import org.junit.Test - -class SchedulerTest : TestBase() { - @Test - fun testIoScheduler(): Unit = runBlocking { - expect(1) - val mainThread = Thread.currentThread() - run(Schedulers.io().asCoroutineDispatcher()) { - val t1 = Thread.currentThread() - println(t1) - assertThat(t1, IsNot(IsEqual(mainThread))) - expect(2) - delay(100) - val t2 = Thread.currentThread() - println(t2) - assertThat(t2, IsNot(IsEqual(mainThread))) - expect(3) - } - finish(4) - } -} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/SingleTest.kt b/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/SingleTest.kt deleted file mode 100644 index 224737aa12..0000000000 --- a/reactive/kotlinx-coroutines-rx2/src/test/kotlin/kotlinx/coroutines/experimental/rx2/SingleTest.kt +++ /dev/null @@ -1,196 +0,0 @@ -/* - * Copyright 2016-2017 JetBrains s.r.o. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package kotlinx.coroutines.experimental.rx2 - -import kotlinx.coroutines.experimental.CommonPool -import kotlinx.coroutines.experimental.runBlocking -import org.junit.Assert.assertEquals -import org.junit.Assert.fail -import org.junit.Test -import io.reactivex.Observable -import io.reactivex.Single -import kotlinx.coroutines.experimental.TestBase -import kotlinx.coroutines.experimental.yield -import org.hamcrest.core.IsEqual -import org.hamcrest.core.IsInstanceOf -import org.junit.Assert -import java.util.concurrent.CancellationException -import java.util.concurrent.TimeUnit - -/** - * Tests emitting single item with [rxSingle]. - */ -class SingleTest : TestBase() { - @Test - fun testBasicSuccess() = runBlocking { - expect(1) - val single = rxSingle(context) { - expect(4) - "OK" - } - expect(2) - single.subscribe { value -> - expect(5) - Assert.assertThat(value, IsEqual("OK")) - } - expect(3) - yield() // to started coroutine - finish(6) - } - - @Test - fun testBasicFailure() = runBlocking { - expect(1) - val single = rxSingle(context) { - expect(4) - throw RuntimeException("OK") - } - expect(2) - single.subscribe({ - expectUnreached() - }, { error -> - expect(5) - Assert.assertThat(error, IsInstanceOf(RuntimeException::class.java)) - Assert.assertThat(error.message, IsEqual("OK")) - }) - expect(3) - yield() // to started coroutine - finish(6) - } - - - @Test - fun testBasicUnsubscribe() = runBlocking { - expect(1) - val single = rxSingle(context) { - expect(4) - yield() // back to main, will get cancelled - expectUnreached() - } - expect(2) - // nothing is called on a disposed rx2 single - val sub = single.subscribe({ - expectUnreached() - }, { - expectUnreached() - }) - expect(3) - yield() // to started coroutine - expect(5) - sub.dispose() // will cancel coroutine - yield() - finish(6) - } - - @Test - fun testSingleNoWait() { - val single = rxSingle(CommonPool) { - "OK" - } - - checkSingleValue(single) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleAwait() = runBlocking { - assertEquals("OK", Single.just("O").await() + "K") - } - - @Test - fun testSingleEmitAndAwait() { - val single = rxSingle(CommonPool) { - Single.just("O").await() + "K" - } - - checkSingleValue(single) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleWithDelay() { - val single = rxSingle(CommonPool) { - Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K" - } - - checkSingleValue(single) { - assertEquals("OK", it) - } - } - - @Test - fun testSingleException() { - val single = rxSingle(CommonPool) { - Observable.just("O", "K").awaitSingle() + "K" - } - - checkErroneous(single) { - assert(it is IllegalArgumentException) - } - } - - @Test - fun testAwaitFirst() { - val single = rxSingle(CommonPool) { - Observable.just("O", "#").awaitFirst() + "K" - } - - checkSingleValue(single) { - assertEquals("OK", it) - } - } - - @Test - fun testAwaitLast() { - val single = rxSingle(CommonPool) { - Observable.just("#", "O").awaitLast() + "K" - } - - checkSingleValue(single) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromObservable() { - val single = rxSingle(CommonPool) { - try { - Observable.error(RuntimeException("O")).awaitFirst() - } catch (e: RuntimeException) { - Observable.just(e.message!!).awaitLast() + "K" - } - } - - checkSingleValue(single) { - assertEquals("OK", it) - } - } - - @Test - fun testExceptionFromCoroutine() { - val single = rxSingle(CommonPool) { - throw IllegalStateException(Observable.just("O").awaitSingle() + "K") - } - - checkErroneous(single) { - assert(it is IllegalStateException) - assertEquals("OK", it.message) - } - } -} diff --git a/reactive/kotlinx-coroutines-rx2/test/BackpressureTest.kt b/reactive/kotlinx-coroutines-rx2/test/BackpressureTest.kt new file mode 100644 index 0000000000..033df37991 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/BackpressureTest.kt @@ -0,0 +1,36 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import kotlin.test.* + +class BackpressureTest : TestBase() { + @Test + fun testBackpressureDropDirect() = runTest { + expect(1) + Flowable.fromArray(1) + .onBackpressureDrop() + .collect { + assertEquals(1, it) + expect(2) + } + finish(3) + } + + @Test + fun testBackpressureDropFlow() = runTest { + expect(1) + Flowable.fromArray(1) + .onBackpressureDrop() + .asFlow() + .collect { + assertEquals(1, it) + expect(2) + } + finish(3) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/test/Check.kt b/reactive/kotlinx-coroutines-rx2/test/Check.kt new file mode 100644 index 0000000000..ded4b7aed5 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/Check.kt @@ -0,0 +1,73 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import io.reactivex.functions.Consumer +import io.reactivex.plugins.* + +fun checkSingleValue( + observable: Observable, + checker: (T) -> Unit +) { + val singleValue = observable.blockingSingle() + checker(singleValue) +} + +fun checkErroneous( + observable: Observable<*>, + checker: (Throwable) -> Unit +) { + val singleNotification = observable.materialize().blockingSingle() + val error = singleNotification.error ?: error("Excepted error") + checker(error) +} + +fun checkSingleValue( + single: Single, + checker: (T) -> Unit +) { + val singleValue = single.blockingGet() + checker(singleValue) +} + +fun checkErroneous( + single: Single<*>, + checker: (Throwable) -> Unit +) { + try { + single.blockingGet() + error("Should have failed") + } catch (e: Throwable) { + checker(e) + } +} + +fun checkMaybeValue( + maybe: Maybe, + checker: (T?) -> Unit +) { + val maybeValue = maybe.toFlowable().blockingIterable().firstOrNull() + checker(maybeValue) +} + +@Suppress("UNCHECKED_CAST") +fun checkErroneous( + maybe: Maybe<*>, + checker: (Throwable) -> Unit +) { + try { + (maybe as Maybe).blockingGet() + error("Should have failed") + } catch (e: Throwable) { + checker(e) + } +} + +inline fun withExceptionHandler(noinline handler: (Throwable) -> Unit, block: () -> Unit) { + val original = RxJavaPlugins.getErrorHandler() + RxJavaPlugins.setErrorHandler { handler(it) } + try { + block() + } finally { + RxJavaPlugins.setErrorHandler(original) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/test/CompletableTest.kt b/reactive/kotlinx-coroutines-rx2/test/CompletableTest.kt new file mode 100644 index 0000000000..c271a2fbd2 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/CompletableTest.kt @@ -0,0 +1,202 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import io.reactivex.disposables.* +import io.reactivex.exceptions.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class CompletableTest : TestBase() { + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(4) + } + expect(2) + completable.subscribe { + expect(5) + } + expect(3) + yield() // to completable coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + completable.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to completable coroutine + finish(6) + } + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + // nothing is called on a disposed rx2 completable + val sub = completable.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testAwaitSuccess() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(3) + } + expect(2) + completable.await() // shall launch coroutine + finish(4) + } + + @Test + fun testAwaitFailure() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(3) + throw RuntimeException("OK") + } + expect(2) + try { + completable.await() // shall launch coroutine and throw exception + expectUnreached() + } catch (e: RuntimeException) { + finish(4) + assertEquals("OK", e.message) + } + } + + /** Tests that calls to [await] throw [CancellationException] and dispose of the subscription when their [Job] is + * cancelled. */ + @Test + fun testAwaitCancellation() = runTest { + expect(1) + val completable = CompletableSource { s -> + s.onSubscribe(object: Disposable { + override fun dispose() { expect(4) } + override fun isDisposed(): Boolean { expectUnreached(); return false } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + completable.await() + } catch (e: CancellationException) { + expect(5) + throw e + } + } + expect(3) + job.cancelAndJoin() + finish(6) + } + + @Test + fun testSuppressedException() = runTest { + val completable = rxCompletable(currentDispatcher()) { + launch(start = CoroutineStart.ATOMIC) { + throw TestException() // child coroutine fails + } + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() // but parent throws another exception while cleaning up + } + } + try { + completable.await() + expectUnreached() + } catch (e: TestException) { + assertIs(e.suppressed[0]) + } + } + + @Test + fun testUnhandledException() = runTest { + expect(1) + var disposable: Disposable? = null + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is TestException) + expect(5) + } + val completable = rxCompletable(currentDispatcher()) { + expect(4) + disposable!!.dispose() // cancel our own subscription, so that delay will get cancelled + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException() // would not be able to handle it since mono is disposed + } + } + withExceptionHandler(handler) { + completable.subscribe(object : CompletableObserver { + override fun onSubscribe(d: Disposable) { + expect(2) + disposable = d + } + + override fun onComplete() { + expectUnreached() + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + expect(3) + yield() // run coroutine + finish(6) + } + } + + @Test + fun testFatalExceptionInSubscribe() = runTest { + val handler: (Throwable) -> Unit = { e -> + assertTrue(e is UndeliverableException && e.cause is LinkageError); expect(2) + } + + withExceptionHandler(handler) { + rxCompletable(Dispatchers.Unconfined) { + expect(1) + }.subscribe { throw LinkageError() } + finish(3) + } + } + + @Test + fun testFatalExceptionInSingle() = runTest { + rxCompletable(Dispatchers.Unconfined) { + throw LinkageError() + }.subscribe({ expectUnreached() }, { expect(1); assertIs(it) }) + finish(2) + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/ConvertTest.kt b/reactive/kotlinx-coroutines-rx2/test/ConvertTest.kt new file mode 100644 index 0000000000..d57f6fa010 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ConvertTest.kt @@ -0,0 +1,156 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import org.junit.Assert +import org.junit.Test +import kotlin.test.* + +class ConvertTest : TestBase() { + @Test + fun testToCompletableSuccess() = runBlocking { + expect(1) + val job = launch { + expect(3) + } + val completable = job.asCompletable(coroutineContext.minusKey(Job)) + completable.subscribe { + expect(4) + } + expect(2) + yield() + finish(5) + } + + @Test + fun testToCompletableFail() = runBlocking { + expect(1) + val job = async(NonCancellable) { // don't kill parent on exception + expect(3) + throw RuntimeException("OK") + } + val completable = job.asCompletable(coroutineContext.minusKey(Job)) + completable.subscribe { + expect(4) + } + expect(2) + yield() + finish(5) + } + + @Test + fun testToMaybe() { + val d = GlobalScope.async { + delay(50) + "OK" + } + val maybe1 = d.asMaybe(Dispatchers.Unconfined) + checkMaybeValue(maybe1) { + assertEquals("OK", it) + } + val maybe2 = d.asMaybe(Dispatchers.Unconfined) + checkMaybeValue(maybe2) { + assertEquals("OK", it) + } + } + + @Test + fun testToMaybeEmpty() { + val d = GlobalScope.async { + delay(50) + null + } + val maybe1 = d.asMaybe(Dispatchers.Unconfined) + checkMaybeValue(maybe1, Assert::assertNull) + val maybe2 = d.asMaybe(Dispatchers.Unconfined) + checkMaybeValue(maybe2, Assert::assertNull) + } + + @Test + fun testToMaybeFail() { + val d = GlobalScope.async { + delay(50) + throw TestRuntimeException("OK") + } + val maybe1 = d.asMaybe(Dispatchers.Unconfined) + checkErroneous(maybe1) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + val maybe2 = d.asMaybe(Dispatchers.Unconfined) + checkErroneous(maybe2) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + } + + @Test + fun testToSingle() { + val d = GlobalScope.async { + delay(50) + "OK" + } + val single1 = d.asSingle(Dispatchers.Unconfined) + checkSingleValue(single1) { + assertEquals("OK", it) + } + val single2 = d.asSingle(Dispatchers.Unconfined) + checkSingleValue(single2) { + assertEquals("OK", it) + } + } + + @Test + fun testToSingleFail() { + val d = GlobalScope.async { + delay(50) + throw TestRuntimeException("OK") + } + val single1 = d.asSingle(Dispatchers.Unconfined) + checkErroneous(single1) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + val single2 = d.asSingle(Dispatchers.Unconfined) + checkErroneous(single2) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + } + + @Test + fun testToObservable() { + val c = GlobalScope.produce { + delay(50) + send("O") + delay(50) + send("K") + } + val observable = c.consumeAsFlow().asObservable(Dispatchers.Unconfined) + checkSingleValue(observable.reduce { t1, t2 -> t1 + t2 }.toSingle()) { + assertEquals("OK", it) + } + } + + @Test + fun testToObservableFail() { + val c = GlobalScope.produce { + delay(50) + send("O") + delay(50) + throw TestException("K") + } + val observable = c.consumeAsFlow().asObservable(Dispatchers.Unconfined) + val single = rxSingle(Dispatchers.Unconfined) { + var result = "" + try { + observable.collect { result += it } + } catch(e: Throwable) { + check(e is TestException) + result += e.message + } + result + } + checkSingleValue(single) { + assertEquals("OK", it) + } + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/FlowAsFlowableTest.kt b/reactive/kotlinx-coroutines-rx2/test/FlowAsFlowableTest.kt new file mode 100644 index 0000000000..0bcacef06c --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/FlowAsFlowableTest.kt @@ -0,0 +1,86 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.junit.Test +import org.reactivestreams.* +import java.util.concurrent.* +import kotlin.test.* + +@Suppress("ReactiveStreamsSubscriberImplementation") +class FlowAsFlowableTest : TestBase() { + @Test + fun testUnconfinedDefaultContext() { + expect(1) + val thread = Thread.currentThread() + fun checkThread() { + assertSame(thread, Thread.currentThread()) + } + flowOf(42).asFlowable().subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onSubscribe(s: Subscription) { + expect(2) + subscription = s + subscription.request(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + finish(5) + } + + @Test + fun testConfinedContext() { + expect(1) + val threadName = "FlowAsFlowableTest.testConfinedContext" + fun checkThread() { + val currentThread = Thread.currentThread() + assertTrue(currentThread.name.startsWith(threadName), "Unexpected thread $currentThread") + } + val completed = CountDownLatch(1) + newSingleThreadContext(threadName).use { dispatcher -> + flowOf(42).asFlowable(dispatcher).subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onSubscribe(s: Subscription) { + expect(2) + subscription = s + subscription.request(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + completed.countDown() + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + completed.await() + } + finish(5) + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/FlowAsObservableTest.kt b/reactive/kotlinx-coroutines-rx2/test/FlowAsObservableTest.kt new file mode 100644 index 0000000000..c976d9b615 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/FlowAsObservableTest.kt @@ -0,0 +1,208 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import io.reactivex.disposables.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class FlowAsObservableTest : TestBase() { + @Test + fun testBasicSuccess() = runTest { + expect(1) + val observable = flow { + expect(3) + emit("OK") + }.asObservable() + + expect(2) + observable.subscribe { value -> + expect(4) + assertEquals("OK", value) + } + + finish(5) + } + + @Test + fun testBasicFailure() = runTest { + expect(1) + val observable = flow { + expect(3) + throw RuntimeException("OK") + }.asObservable() + + expect(2) + observable.subscribe({ expectUnreached() }, { error -> + expect(4) + assertIs(error) + assertEquals("OK", error.message) + }) + finish(5) + } + + @Test + fun testBasicUnsubscribe() = runTest { + expect(1) + val observable = flow { + expect(3) + hang { + expect(4) + } + }.asObservable() + + expect(2) + val sub = observable.subscribe({ expectUnreached() }, { expectUnreached() }) + sub.dispose() // will cancel coroutine + finish(5) + } + + @Test + fun testNotifyOnceOnCancellation() = runTest { + val observable = + flow { + expect(3) + emit("OK") + hang { + expect(7) + } + }.asObservable() + .doOnNext { + expect(4) + assertEquals("OK", it) + } + .doOnDispose { + expect(6) // notified once! + } + + expect(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + observable.collect { + expect(5) + assertEquals("OK", it) + } + } + + yield() + job.cancelAndJoin() + finish(8) + } + + @Test + fun testFailingConsumer() = runTest { + expect(1) + val observable = flow { + expect(2) + emit("OK") + hang { + expect(4) + } + + }.asObservable() + + try { + observable.collect { + expect(3) + throw TestException() + } + } catch (e: TestException) { + finish(5) + } + } + + @Test + fun testNonAtomicStart() = runTest { + withContext(Dispatchers.Unconfined) { + val observable = flow { + expect(1) + }.asObservable() + + val disposable = observable.subscribe({ expectUnreached() }, { expectUnreached() }, { expectUnreached() }) + disposable.dispose() + } + finish(2) + } + + @Test + fun testFlowCancelledFromWithin() = runTest { + val observable = flow { + expect(1) + emit(1) + kotlin.coroutines.coroutineContext.cancel() + kotlin.coroutines.coroutineContext.ensureActive() + expectUnreached() + }.asObservable() + + observable.subscribe({ expect(2) }, { expectUnreached() }, { finish(3) }) + } + + @Test + fun testUnconfinedDefaultContext() { + expect(1) + val thread = Thread.currentThread() + fun checkThread() { + assertSame(thread, Thread.currentThread()) + } + flowOf(42).asObservable().subscribe(object : Observer { + override fun onSubscribe(d: Disposable) { + expect(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + finish(5) + } + + @Test + fun testConfinedContext() { + expect(1) + val threadName = "FlowAsObservableTest.testConfinedContext" + fun checkThread() { + val currentThread = Thread.currentThread() + assertTrue(currentThread.name.startsWith(threadName), "Unexpected thread $currentThread") + } + val completed = CountDownLatch(1) + newSingleThreadContext(threadName).use { dispatcher -> + flowOf(42).asObservable(dispatcher).subscribe(object : Observer { + override fun onSubscribe(d: Disposable) { + expect(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + completed.countDown() + } + + override fun onError(e: Throwable) { + expectUnreached() + } + }) + completed.await() + } + finish(5) + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/FlowableContextTest.kt b/reactive/kotlinx-coroutines-rx2/test/FlowableContextTest.kt new file mode 100644 index 0000000000..295250e564 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/FlowableContextTest.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class FlowableContextTest : TestBase() { + private val dispatcher = newSingleThreadContext("FlowableContextTest") + + @After + fun tearDown() { + dispatcher.close() + } + + @Test + fun testFlowableCreateAsFlowThread() = runTest { + expect(1) + val mainThread = Thread.currentThread() + val dispatcherThread = withContext(dispatcher) { Thread.currentThread() } + assertTrue(dispatcherThread != mainThread) + Flowable.create({ + assertEquals(dispatcherThread, Thread.currentThread()) + it.onNext("OK") + it.onComplete() + }, BackpressureStrategy.BUFFER) + .asFlow() + .flowOn(dispatcher) + .collect { + expect(2) + assertEquals("OK", it) + assertEquals(mainThread, Thread.currentThread()) + } + finish(3) + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/test/FlowableExceptionHandlingTest.kt b/reactive/kotlinx-coroutines-rx2/test/FlowableExceptionHandlingTest.kt new file mode 100644 index 0000000000..bcfd32aaa5 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/FlowableExceptionHandlingTest.kt @@ -0,0 +1,131 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.exceptions.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class FlowableExceptionHandlingTest : TestBase() { + + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + private inline fun handler(expect: Int) = { t: Throwable -> + assertTrue(t is UndeliverableException && t.cause is T) + expect(expect) + } + + private fun cehUnreached() = CoroutineExceptionHandler { _, _ -> expectUnreached() } + + @Test + fun testException() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + throw TestException() + }.subscribe({ + expectUnreached() + }, { + expect(2) // Reported to onError + }) + finish(3) + } + + @Test + fun testFatalException() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined) { + expect(1) + throw LinkageError() + }.subscribe({ + expectUnreached() + }, { + expect(2) // Fatal exceptions are not treated as special + }) + finish(3) + } + + @Test + fun testExceptionAsynchronous() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + throw TestException() + }.publish() + .refCount() + .subscribe({ + expectUnreached() + }, { + expect(2) // Reported to onError + }) + finish(3) + } + + @Test + fun testFatalExceptionAsynchronous() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined) { + expect(1) + throw LinkageError() + }.publish() + .refCount() + .subscribe({ + expectUnreached() + }, { + expect(2) + }) + finish(3) + } + + @Test + fun testFatalExceptionFromSubscribe() = withExceptionHandler(handler(3)) { + rxFlowable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.subscribe({ + expect(2) + throw LinkageError() + }, { expectUnreached() }) // Fatal exception is rethrown from `onNext` => the subscription is thought to be cancelled + finish(4) + } + + @Test + fun testExceptionFromSubscribe() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + send(Unit) + }.subscribe({ + expect(2) + throw TestException() + }, { expect(3) }) // not reported to onError because came from the subscribe itself + finish(4) + } + + @Test + fun testAsynchronousExceptionFromSubscribe() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + send(Unit) + }.publish() + .refCount() + .subscribe({ + expect(2) + throw RuntimeException() + }, { expect(3) }) + finish(4) + } + + @Test + fun testAsynchronousFatalExceptionFromSubscribe() = withExceptionHandler(handler(3)) { + rxFlowable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.publish() + .refCount() + .subscribe({ + expect(2) + throw LinkageError() + }, { expectUnreached() }) + finish(4) + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/FlowableTest.kt b/reactive/kotlinx-coroutines-rx2/test/FlowableTest.kt new file mode 100644 index 0000000000..94fc7d6ff9 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/FlowableTest.kt @@ -0,0 +1,124 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.reactive.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class FlowableTest : TestBase() { + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val observable = rxFlowable(currentDispatcher()) { + expect(4) + send("OK") + } + expect(2) + observable.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val observable = rxFlowable(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + observable.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val observable = rxFlowable(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + val sub = observable.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testNotifyOnceOnCancellation() = runTest { + expect(1) + val observable = + rxFlowable(currentDispatcher()) { + expect(5) + send("OK") + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + expect(11) + } + } + .doOnNext { + expect(6) + assertEquals("OK", it) + } + .doOnCancel { + expect(10) // notified once! + } + expect(2) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(3) + observable.collect { + expect(8) + assertEquals("OK", it) + } + } + expect(4) + yield() // to observable code + expect(7) + yield() // to consuming coroutines + expect(9) + job.cancel() + job.join() + finish(12) + } + + @Test + fun testFailingConsumer() = runTest { + val pub = rxFlowable(currentDispatcher()) { + repeat(3) { + expect(it + 1) // expect(1), expect(2) *should* be invoked + send(it) + } + } + try { + pub.collect { + throw TestException() + } + } catch (e: TestException) { + finish(3) + } + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/IntegrationTest.kt b/reactive/kotlinx-coroutines-rx2/test/IntegrationTest.kt new file mode 100644 index 0000000000..8469af41dc --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/IntegrationTest.kt @@ -0,0 +1,148 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.coroutines.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class IntegrationTest( + private val ctx: Ctx, + private val delay: Boolean +) : TestBase() { + + enum class Ctx { + MAIN { override fun invoke(context: CoroutineContext): CoroutineContext = context.minusKey(Job) }, + DEFAULT { override fun invoke(context: CoroutineContext): CoroutineContext = Dispatchers.Default }, + UNCONFINED { override fun invoke(context: CoroutineContext): CoroutineContext = Dispatchers.Unconfined }; + + abstract operator fun invoke(context: CoroutineContext): CoroutineContext + } + + companion object { + @Parameterized.Parameters(name = "ctx={0}, delay={1}") + @JvmStatic + fun params(): Collection> = Ctx.values().flatMap { ctx -> + listOf(false, true).map { delay -> + arrayOf(ctx, delay) + } + } + } + + @Test + fun testEmpty(): Unit = runBlocking { + val observable = rxObservable(ctx(coroutineContext)) { + if (delay) delay(1) + // does not send anything + } + assertFailsWith { observable.awaitFirst() } + assertEquals("OK", observable.awaitFirstOrDefault("OK")) + assertNull(observable.awaitFirstOrNull()) + assertEquals("ELSE", observable.awaitFirstOrElse { "ELSE" }) + assertFailsWith { observable.awaitLast() } + assertFailsWith { observable.awaitSingle() } + var cnt = 0 + observable.collect { + cnt++ + } + assertEquals(0, cnt) + } + + @Test + fun testSingle() = runBlocking { + val observable = rxObservable(ctx(coroutineContext)) { + if (delay) delay(1) + send("OK") + } + assertEquals("OK", observable.awaitFirst()) + assertEquals("OK", observable.awaitFirstOrDefault("OK")) + assertEquals("OK", observable.awaitFirstOrNull()) + assertEquals("OK", observable.awaitFirstOrElse { "ELSE" }) + assertEquals("OK", observable.awaitLast()) + assertEquals("OK", observable.awaitSingle()) + var cnt = 0 + observable.collect { + assertEquals("OK", it) + cnt++ + } + assertEquals(1, cnt) + } + + @Test + fun testNumbers() = runBlocking { + val n = 100 * stressTestMultiplier + val observable = rxObservable(ctx(coroutineContext)) { + for (i in 1..n) { + send(i) + if (delay) delay(1) + } + } + assertEquals(1, observable.awaitFirst()) + assertEquals(1, observable.awaitFirstOrDefault(0)) + assertEquals(1, observable.awaitFirstOrNull()) + assertEquals(1, observable.awaitFirstOrElse { 0 }) + assertEquals(n, observable.awaitLast()) + assertFailsWith { observable.awaitSingle() } + checkNumbers(n, observable) + val channel = observable.toChannel() + checkNumbers(n, channel.consumeAsFlow().asObservable(ctx(coroutineContext))) + channel.cancel() + } + + @Test + fun testCancelWithoutValue() = runTest { + val job = launch(Job(), start = CoroutineStart.UNDISPATCHED) { + rxObservable { + hang { } + }.awaitFirst() + } + + job.cancel() + job.join() + } + + @Test + fun testEmptySingle() = runTest(unhandled = listOf({e -> e is NoSuchElementException})) { + expect(1) + val job = launch(Job(), start = CoroutineStart.UNDISPATCHED) { + rxObservable { + yield() + expect(2) + // Nothing to emit + }.awaitFirst() + } + + job.join() + finish(3) + } + + @Test + fun testObservableWithTimeout() = runTest { + val observable = rxObservable { + expect(2) + withTimeout(1) { delay(100) } + } + try { + expect(1) + observable.awaitFirstOrNull() + } catch (e: CancellationException) { + expect(3) + } + finish(4) + } + + private suspend fun checkNumbers(n: Int, observable: Observable) { + var last = 0 + observable.collect { + assertEquals(++last, it) + } + assertEquals(n, last) + } + +} diff --git a/reactive/kotlinx-coroutines-rx2/test/IterableFlowAsFlowableTckTest.kt b/reactive/kotlinx-coroutines-rx2/test/IterableFlowAsFlowableTckTest.kt new file mode 100644 index 0000000000..89423a8331 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/IterableFlowAsFlowableTckTest.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.rx2 + +import io.reactivex.* +import kotlinx.coroutines.flow.* +import org.junit.* +import org.reactivestreams.* +import org.reactivestreams.tck.* + +class IterableFlowAsFlowableTckTest : PublisherVerification(TestEnvironment()) { + + private fun generate(num: Long): Array { + return Array(if (num >= Integer.MAX_VALUE) 1000000 else num.toInt()) { it.toLong() } + } + + override fun createPublisher(elements: Long): Flowable { + return generate(elements).asIterable().asFlow().asFlowable() + } + + override fun createFailedPublisher(): Publisher? = null + + @Ignore + override fun required_spec309_requestZeroMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec309_requestNegativeNumberMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec312_cancelMustMakeThePublisherToEventuallyStopSignaling() { + // + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/LeakedExceptionTest.kt b/reactive/kotlinx-coroutines-rx2/test/LeakedExceptionTest.kt new file mode 100644 index 0000000000..4f16966422 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/LeakedExceptionTest.kt @@ -0,0 +1,104 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import io.reactivex.exceptions.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import java.util.concurrent.Executors +import java.util.concurrent.TimeUnit +import kotlin.test.* + +// Check that exception is not leaked to the global exception handler +class LeakedExceptionTest : TestBase() { + + private val handler: (Throwable) -> Unit = + { assertTrue { it is UndeliverableException && it.cause is TestException } } + + @Test + fun testSingle() = withExceptionHandler(handler) { + withFixedThreadPool(4) { dispatcher -> + val flow = rxSingle(dispatcher) { throw TestException() }.toFlowable().asFlow() + runBlocking { + repeat(10000) { + combine(flow, flow) { _, _ -> Unit } + .catch {} + .collect {} + } + } + } + } + + @Test + fun testObservable() = withExceptionHandler(handler) { + withFixedThreadPool(4) { dispatcher -> + val flow = rxObservable(dispatcher) { throw TestException() } + .toFlowable(BackpressureStrategy.BUFFER) + .asFlow() + runBlocking { + repeat(10000) { + combine(flow, flow) { _, _ -> Unit } + .catch {} + .collect {} + } + } + } + } + + @Test + fun testFlowable() = withExceptionHandler(handler) { + withFixedThreadPool(4) { dispatcher -> + val flow = rxFlowable(dispatcher) { throw TestException() }.asFlow() + runBlocking { + repeat(10000) { + combine(flow, flow) { _, _ -> Unit } + .catch {} + .collect {} + } + } + } + } + + /** + * This test doesn't test much and was added to display a problem with straighforward use of + * [withExceptionHandler]. + * + * If one was to remove `dispatcher` and launch `rxFlowable` with an empty coroutine context, + * this test would fail fairly often, while other tests were also vulnerable, but the problem is + * much more difficult to reproduce. Thus, this test is a justification for adding `dispatcher` + * to other tests. + * + * See the commit that introduced this test for a better explanation. + */ + @Test + fun testResettingExceptionHandler() = withExceptionHandler(handler) { + withFixedThreadPool(4) { dispatcher -> + val flow = rxFlowable(dispatcher) { + if ((0..1).random() == 0) { + Thread.sleep(100) + } + throw TestException() + }.asFlow() + runBlocking { + combine(flow, flow) { _, _ -> Unit } + .catch {} + .collect {} + } + } + } + + /** + * Run in a thread pool, then wait for all the tasks to finish. + */ + private fun withFixedThreadPool(numberOfThreads: Int, block: (CoroutineDispatcher) -> Unit) { + val pool = Executors.newFixedThreadPool(numberOfThreads) + val dispatcher = pool.asCoroutineDispatcher() + block(dispatcher) + pool.shutdown() + while (!pool.awaitTermination(10, TimeUnit.SECONDS)) { + /* deliberately empty */ + } + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/MaybeTest.kt b/reactive/kotlinx-coroutines-rx2/test/MaybeTest.kt new file mode 100644 index 0000000000..74bc8324e3 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/MaybeTest.kt @@ -0,0 +1,389 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import io.reactivex.disposables.* +import io.reactivex.exceptions.* +import io.reactivex.internal.functions.Functions.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class MaybeTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + "OK" + } + expect(2) + maybe.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicEmpty() = runBlocking { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + null + } + expect(2) + maybe.subscribe (emptyConsumer(), ON_ERROR_MISSING, { + expect(5) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + maybe.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + // nothing is called on a disposed rx2 maybe + val sub = maybe.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testMaybeNoWait() { + val maybe = rxMaybe { + "OK" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testMaybeAwait() = runBlocking { + assertEquals("OK", Maybe.just("O").awaitSingleOrNull() + "K") + assertEquals("OK", Maybe.just("O").awaitSingle() + "K") + } + + @Test + fun testMaybeAwaitForNull(): Unit = runBlocking { + assertNull(Maybe.empty().awaitSingleOrNull()) + assertFailsWith { Maybe.empty().awaitSingle() } + } + + /** Tests that calls to [awaitSingleOrNull] throw [CancellationException] and dispose of the subscription when their + * [Job] is cancelled. */ + @Test + fun testMaybeAwaitCancellation() = runTest { + expect(1) + val maybe = MaybeSource { s -> + s.onSubscribe(object: Disposable { + override fun dispose() { expect(4) } + override fun isDisposed(): Boolean { expectUnreached(); return false } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + maybe.awaitSingleOrNull() + } catch (e: CancellationException) { + expect(5) + throw e + } + } + expect(3) + job.cancelAndJoin() + finish(6) + } + + @Test + fun testMaybeEmitAndAwait() { + val maybe = rxMaybe { + Maybe.just("O").awaitSingleOrNull() + "K" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testMaybeWithDelay() { + val maybe = rxMaybe { + Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testMaybeException() { + val maybe = rxMaybe { + Observable.just("O", "K").awaitSingle() + "K" + } + + checkErroneous(maybe) { + assert(it is IllegalArgumentException) + } + } + + @Test + fun testAwaitFirst() { + val maybe = rxMaybe { + Observable.just("O", "#").awaitFirst() + "K" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitLast() { + val maybe = rxMaybe { + Observable.just("#", "O").awaitLast() + "K" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromObservable() { + val maybe = rxMaybe { + try { + Observable.error(RuntimeException("O")).awaitFirst() + } catch (e: RuntimeException) { + Observable.just(e.message!!).awaitLast() + "K" + } + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromCoroutine() { + val maybe = rxMaybe { + throw IllegalStateException(Observable.just("O").awaitSingle() + "K") + } + + checkErroneous(maybe) { + assert(it is IllegalStateException) + assertEquals("OK", it.message) + } + } + + @Test + fun testCancelledConsumer() = runTest { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + expect(6) + } + 42 + } + expect(2) + val timeout = withTimeoutOrNull(100) { + expect(3) + maybe.collect { + expectUnreached() + } + expectUnreached() + } + assertNull(timeout) + expect(5) + yield() // must cancel code inside maybe!!! + finish(7) + } + + /** Tests the simple scenario where the Maybe doesn't output a value. */ + @Test + fun testMaybeCollectEmpty() = runTest { + expect(1) + Maybe.empty().collect { + expectUnreached() + } + finish(2) + } + + /** Tests the simple scenario where the Maybe doesn't output a value. */ + @Test + fun testMaybeCollectSingle() = runTest { + expect(1) + Maybe.just("OK").collect { + assertEquals("OK", it) + expect(2) + } + finish(3) + } + + /** Tests the behavior of [collect] when the Maybe raises an error. */ + @Test + fun testMaybeCollectThrowingMaybe() = runTest { + expect(1) + try { + Maybe.error(TestException()).collect { + expectUnreached() + } + } catch (e: TestException) { + expect(2) + } + finish(3) + } + + /** Tests the behavior of [collect] when the action throws. */ + @Test + fun testMaybeCollectThrowingAction() = runTest { + expect(1) + try { + Maybe.just("OK").collect { + expect(2) + throw TestException() + } + } catch (e: TestException) { + expect(3) + } + finish(4) + } + + @Test + fun testSuppressedException() = runTest { + val maybe = rxMaybe(currentDispatcher()) { + launch(start = CoroutineStart.ATOMIC) { + throw TestException() // child coroutine fails + } + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() // but parent throws another exception while cleaning up + } + } + try { + maybe.awaitSingleOrNull() + expectUnreached() + } catch (e: TestException) { + assertIs(e.suppressed[0]) + } + } + + @Test + fun testUnhandledException() = runTest { + expect(1) + var disposable: Disposable? = null + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is TestException) + expect(5) + } + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + disposable!!.dispose() // cancel our own subscription, so that delay will get cancelled + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException() // would not be able to handle it since mono is disposed + } + } + withExceptionHandler(handler) { + maybe.subscribe(object : MaybeObserver { + override fun onSubscribe(d: Disposable) { + expect(2) + disposable = d + } + + override fun onComplete() { + expectUnreached() + } + + override fun onSuccess(t: Unit) { + expectUnreached() + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + expect(3) + yield() // run coroutine + finish(6) + } + } + + @Test + fun testFatalExceptionInSubscribe() = runTest { + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is LinkageError) + expect(2) + } + + withExceptionHandler(handler) { + rxMaybe(Dispatchers.Unconfined) { + expect(1) + 42 + }.subscribe { throw LinkageError() } + finish(3) + } + } + + @Test + fun testFatalExceptionInSingle() = runTest { + rxMaybe(Dispatchers.Unconfined) { + throw LinkageError() + }.subscribe({ expectUnreached() }, { expect(1); assertIs(it) }) + finish(2) + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/ObservableAsFlowTest.kt b/reactive/kotlinx-coroutines-rx2/test/ObservableAsFlowTest.kt new file mode 100644 index 0000000000..1391d7e810 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ObservableAsFlowTest.kt @@ -0,0 +1,183 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.Observable +import io.reactivex.ObservableSource +import io.reactivex.Observer +import io.reactivex.disposables.Disposables +import io.reactivex.subjects.PublishSubject +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.testing.flow.* +import kotlin.test.* + +class ObservableAsFlowTest : TestBase() { + @Test + fun testCancellation() = runTest { + var onNext = 0 + var onCancelled = 0 + var onError = 0 + + val source = rxObservable(currentDispatcher()) { + coroutineContext[Job]?.invokeOnCompletion { + if (it is CancellationException) ++onCancelled + } + + repeat(100) { + send(it) + } + } + + source.asFlow().launchIn(CoroutineScope(Dispatchers.Unconfined)) { + onEach { + ++onNext + throw RuntimeException() + } + catch { + ++onError + } + }.join() + + + assertEquals(1, onNext) + assertEquals(1, onError) + assertEquals(1, onCancelled) + } + + @Test + fun testImmediateCollection() { + val source = PublishSubject.create() + val flow = source.asFlow() + GlobalScope.launch(Dispatchers.Unconfined) { + expect(1) + flow.collect { expect(it) } + expect(6) + } + expect(2) + source.onNext(3) + expect(4) + source.onNext(5) + source.onComplete() + finish(7) + } + + @Test + fun testOnErrorCancellation() { + val source = PublishSubject.create() + val flow = source.asFlow() + val exception = RuntimeException() + GlobalScope.launch(Dispatchers.Unconfined) { + try { + expect(1) + flow.collect { expect(it) } + expectUnreached() + } + catch (e: Exception) { + assertSame(exception, e.cause) + expect(5) + } + expect(6) + } + expect(2) + source.onNext(3) + expect(4) + source.onError(exception) + finish(7) + } + + @Test + fun testUnsubscribeOnCollectionException() { + val source = PublishSubject.create() + val flow = source.asFlow() + val exception = RuntimeException() + GlobalScope.launch(Dispatchers.Unconfined) { + try { + expect(1) + flow.collect { + expect(it) + if (it == 3) throw exception + } + expectUnreached() + } + catch (e: Exception) { + assertSame(exception, e.cause) + expect(4) + } + expect(5) + } + expect(2) + assertTrue(source.hasObservers()) + source.onNext(3) + assertFalse(source.hasObservers()) + finish(6) + } + + @Test + fun testLateOnSubscribe() { + var observer: Observer? = null + val source = ObservableSource { observer = it } + val flow = source.asFlow() + assertNull(observer) + val job = GlobalScope.launch(Dispatchers.Unconfined) { + expect(1) + flow.collect { expectUnreached() } + expectUnreached() + } + expect(2) + assertNotNull(observer) + job.cancel() + val disposable = Disposables.empty() + observer!!.onSubscribe(disposable) + assertTrue(disposable.isDisposed) + finish(3) + } + + @Test + fun testBufferUnlimited() = runTest { + val source = rxObservable(currentDispatcher()) { + expect(1); send(10) + expect(2); send(11) + expect(3); send(12) + expect(4); send(13) + expect(5); send(14) + expect(6); send(15) + expect(7); send(16) + expect(8); send(17) + expect(9) + } + source.asFlow().buffer(Channel.UNLIMITED).collect { expect(it) } + finish(18) + } + + @Test + fun testConflated() = runTest { + val source = Observable.range(1, 5) + val list = source.asFlow().conflate().toList() + assertEquals(listOf(1, 5), list) + } + + @Test + fun testLongRange() = runTest { + val source = Observable.range(1, 10_000) + val count = source.asFlow().count() + assertEquals(10_000, count) + } + + @Test + fun testProduce() = runTest { + val source = Observable.range(0, 10) + val flow = source.asFlow() + check((0..9).toList(), flow.produceIn(this)) + check((0..9).toList(), flow.buffer(Channel.UNLIMITED).produceIn(this)) + check((0..9).toList(), flow.buffer(2).produceIn(this)) + check((0..9).toList(), flow.buffer(0).produceIn(this)) + check(listOf(0, 9), flow.conflate().produceIn(this)) + } + + private suspend fun check(expected: List, channel: ReceiveChannel) { + val result = ArrayList(10) + channel.consumeEach { result.add(it) } + assertEquals(expected, result) + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/ObservableCollectTest.kt b/reactive/kotlinx-coroutines-rx2/test/ObservableCollectTest.kt new file mode 100644 index 0000000000..59c886d9da --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ObservableCollectTest.kt @@ -0,0 +1,66 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import io.reactivex.disposables.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class ObservableCollectTest: TestBase() { + + /** Tests the behavior of [collect] when the publisher raises an error. */ + @Test + fun testObservableCollectThrowingObservable() = runTest { + expect(1) + var sum = 0 + try { + rxObservable { + for (i in 0..100) { + send(i) + } + throw TestException() + }.collect { + sum += it + } + } catch (e: TestException) { + assertTrue(sum > 0) + finish(2) + } + } + + /** Tests the behavior of [collect] when the action throws. */ + @Test + fun testObservableCollectThrowingAction() = runTest { + expect(1) + var sum = 0 + val expectedSum = 5 + try { + var disposed = false + ObservableSource { observer -> + launch(Dispatchers.Default) { + observer.onSubscribe(object : Disposable { + override fun dispose() { + disposed = true + expect(expectedSum + 2) + } + + override fun isDisposed(): Boolean = disposed + }) + while (!disposed) { + observer.onNext(1) + } + } + }.collect { + expect(sum + 2) + sum += it + if (sum == expectedSum) { + throw TestException() + } + } + } catch (e: TestException) { + assertEquals(expectedSum, sum) + finish(expectedSum + 3) + } + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/test/ObservableCompletionStressTest.kt b/reactive/kotlinx-coroutines-rx2/test/ObservableCompletionStressTest.kt new file mode 100644 index 0000000000..3d2546aff7 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ObservableCompletionStressTest.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.* +import kotlin.coroutines.* + +class ObservableCompletionStressTest : TestBase() { + private val N_REPEATS = 10_000 * stressTestMultiplier + + private fun range(context: CoroutineContext, start: Int, count: Int) = rxObservable(context) { + for (x in start until start + count) send(x) + } + + @Test + fun testCompletion() { + val rnd = Random() + repeat(N_REPEATS) { + val count = rnd.nextInt(5) + runBlocking { + withTimeout(5000) { + var received = 0 + range(Dispatchers.Default, 1, count).collect { x -> + received++ + if (x != received) error("$x != $received") + } + if (received != count) error("$received != $count") + } + } + } + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/ObservableExceptionHandlingTest.kt b/reactive/kotlinx-coroutines-rx2/test/ObservableExceptionHandlingTest.kt new file mode 100644 index 0000000000..0d34a4e5b0 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ObservableExceptionHandlingTest.kt @@ -0,0 +1,140 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.exceptions.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class ObservableExceptionHandlingTest : TestBase() { + + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + private inline fun handler(expect: Int) = { t: Throwable -> + assertTrue(t is UndeliverableException && t.cause is T, "$t") + expect(expect) + } + + private fun cehUnreached() = CoroutineExceptionHandler { _, _ -> expectUnreached() } + + @Test + fun testException() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + throw TestException() + }.subscribe({ + expectUnreached() + }, { + expect(2) // Reported to onError + }) + finish(3) + } + + @Test + fun testFatalException() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + throw LinkageError() + }.subscribe({ + expectUnreached() + }, { + expect(2) + }) + finish(3) + } + + @Test + fun testExceptionAsynchronous() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + throw TestException() + }.publish() + .refCount() + .subscribe({ + expectUnreached() + }, { + expect(2) // Reported to onError + }) + finish(3) + } + + @Test + fun testFatalExceptionAsynchronous() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + throw LinkageError() + }.publish() + .refCount() + .subscribe({ + expectUnreached() + }, { + expect(2) // Fatal exceptions are not treated in a special manner + }) + finish(3) + } + + @Test + fun testFatalExceptionFromSubscribe() = withExceptionHandler(handler(3)) { + val latch = CountDownLatch(1) + rxObservable(Dispatchers.Unconfined) { + expect(1) + val result = trySend(Unit) + val exception = result.exceptionOrNull() + assertIs(exception) + assertIs(exception.cause) + assertTrue(isClosedForSend) + expect(4) + latch.countDown() + }.subscribe({ + expect(2) + throw LinkageError() + }, { expectUnreached() }) // Unreached because RxJava bubbles up fatal exceptions, causing `onNext` to throw. + latch.await() + finish(5) + } + + @Test + fun testExceptionFromSubscribe() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.subscribe({ + expect(2) + throw TestException() + }, { expect(3) }) + finish(4) + } + + @Test + fun testAsynchronousExceptionFromSubscribe() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.publish() + .refCount() + .subscribe({ + expect(2) + throw RuntimeException() + }, { expect(3) }) + finish(4) + } + + @Test + fun testAsynchronousFatalExceptionFromSubscribe() = withExceptionHandler(handler(3)) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.publish() + .refCount() + .subscribe({ + expect(2) + throw LinkageError() + }, { expectUnreached() }) // Unreached because RxJava bubbles up fatal exceptions, causing `onNext` to throw. + finish(4) + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/ObservableMultiTest.kt b/reactive/kotlinx-coroutines-rx2/test/ObservableMultiTest.kt new file mode 100644 index 0000000000..09562fbb74 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ObservableMultiTest.kt @@ -0,0 +1,112 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.Test +import java.io.* +import kotlin.test.* + +/** + * Test emitting multiple values with [rxObservable]. + */ +class ObservableMultiTest : TestBase() { + @Test + fun testNumbers() { + val n = 100 * stressTestMultiplier + val observable = rxObservable { + repeat(n) { send(it) } + } + checkSingleValue(observable.toList()) { list -> + assertEquals((0 until n).toList(), list) + } + } + + + @Test + fun testConcurrentStress() { + val n = 10_000 * stressTestMultiplier + val observable = rxObservable { + newCoroutineContext(coroutineContext) + // concurrent emitters (many coroutines) + val jobs = List(n) { + // launch + launch { + val i = it + send(i) + } + } + jobs.forEach { it.join() } + } + checkSingleValue(observable.toList()) { list -> + assertEquals(n, list.size) + assertEquals((0 until n).toList(), list.sorted()) + } + } + + @Test + fun testConcurrentStressOnSend() { + val n = 10_000 * stressTestMultiplier + val observable = rxObservable { + newCoroutineContext(coroutineContext) + // concurrent emitters (many coroutines) + val jobs = List(n) { + // launch + launch(Dispatchers.Default) { + val i = it + select { + onSend(i) {} + } + } + } + jobs.forEach { it.join() } + } + checkSingleValue(observable.toList()) { list -> + assertEquals(n, list.size) + assertEquals((0 until n).toList(), list.sorted()) + } + } + + @Test + fun testIteratorResendUnconfined() { + val n = 10_000 * stressTestMultiplier + val observable = rxObservable(Dispatchers.Unconfined) { + Observable.range(0, n).collect { send(it) } + } + checkSingleValue(observable.toList()) { list -> + assertEquals((0 until n).toList(), list) + } + } + + @Test + fun testIteratorResendPool() { + val n = 10_000 * stressTestMultiplier + val observable = rxObservable { + Observable.range(0, n).collect { send(it) } + } + checkSingleValue(observable.toList()) { list -> + assertEquals((0 until n).toList(), list) + } + } + + @Test + fun testSendAndCrash() { + val observable = rxObservable { + send("O") + throw IOException("K") + } + val single = rxSingle { + var result = "" + try { + observable.collect { result += it } + } catch(e: IOException) { + result += e.message + } + result + } + checkSingleValue(single) { + assertEquals("OK", it) + } + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/ObservableSingleTest.kt b/reactive/kotlinx-coroutines-rx2/test/ObservableSingleTest.kt new file mode 100644 index 0000000000..5bd6d0d715 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ObservableSingleTest.kt @@ -0,0 +1,237 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import io.reactivex.disposables.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class ObservableSingleTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testSingleNoWait() { + val observable = rxObservable { + send("OK") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleAwait() = runBlocking { + assertEquals("OK", Observable.just("O").awaitSingle() + "K") + } + + @Test + fun testSingleEmitAndAwait() { + val observable = rxObservable { + send(Observable.just("O").awaitSingle() + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleWithDelay() { + val observable = rxObservable { + send(Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleException() { + val observable = rxObservable { + send(Observable.just("O", "K").awaitSingle() + "K") + } + + checkErroneous(observable) { + assertIs(it) + } + } + + @Test + fun testAwaitFirst() { + val observable = rxObservable { + send(Observable.just("O", "#").awaitFirst() + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrDefault() { + val observable = rxObservable { + send(Observable.empty().awaitFirstOrDefault("O") + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrDefaultWithValues() { + val observable = rxObservable { + send(Observable.just("O", "#").awaitFirstOrDefault("!") + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrNull() { + val observable = rxObservable { + send(Observable.empty().awaitFirstOrNull() ?: "OK") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrNullWithValues() { + val observable = rxObservable { + send((Observable.just("O", "#").awaitFirstOrNull() ?: "!") + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrElse() { + val observable = rxObservable { + send(Observable.empty().awaitFirstOrElse { "O" } + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrElseWithValues() { + val observable = rxObservable { + send(Observable.just("O", "#").awaitFirstOrElse { "!" } + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitLast() { + val observable = rxObservable { + send(Observable.just("#", "O").awaitLast() + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + /** Tests that calls to [awaitFirst] (and, thus, the other methods) throw [CancellationException] and dispose of + * the subscription when their [Job] is cancelled. */ + @Test + fun testAwaitCancellation() = runTest { + expect(1) + val observable = ObservableSource { s -> + s.onSubscribe(object: Disposable { + override fun dispose() { expect(4) } + override fun isDisposed(): Boolean { expectUnreached(); return false } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + observable.awaitFirst() + } catch (e: CancellationException) { + expect(5) + throw e + } + } + expect(3) + job.cancelAndJoin() + finish(6) + } + + + @Test + fun testExceptionFromObservable() { + val observable = rxObservable { + try { + send(Observable.error(RuntimeException("O")).awaitFirst()) + } catch (e: RuntimeException) { + send(Observable.just(e.message!!).awaitLast() + "K") + } + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromCoroutine() { + val observable = rxObservable { + throw IllegalStateException(Observable.just("O").awaitSingle() + "K") + } + + checkErroneous(observable) { + assertIs(it) + assertEquals("OK", it.message) + } + } + + @Test + fun testObservableIteration() { + val observable = rxObservable { + var result = "" + Observable.just("O", "K").collect { result += it } + send(result) + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testObservableIterationFailure() { + val observable = rxObservable { + try { + Observable.error(RuntimeException("OK")).collect { fail("Should not be here") } + send("Fail") + } catch (e: RuntimeException) { + send(e.message!!) + } + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/ObservableSourceAsFlowStressTest.kt b/reactive/kotlinx-coroutines-rx2/test/ObservableSourceAsFlowStressTest.kt new file mode 100644 index 0000000000..2e44de88d9 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ObservableSourceAsFlowStressTest.kt @@ -0,0 +1,32 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import org.junit.* +import java.util.concurrent.* + +class ObservableSourceAsFlowStressTest : TestBase() { + + private val iterations = 100 * stressTestMultiplierSqrt + + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testAsFlowCancellation() = runTest { + repeat(iterations) { + val latch = Channel(1) + var i = 0 + val observable = Observable.interval(100L, TimeUnit.MICROSECONDS) + .doOnNext { if (++i > 100) latch.trySend(Unit) } + val job = observable.asFlow().launchIn(CoroutineScope(Dispatchers.Default)) + latch.receive() + job.cancelAndJoin() + } + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/ObservableSubscriptionSelectTest.kt b/reactive/kotlinx-coroutines-rx2/test/ObservableSubscriptionSelectTest.kt new file mode 100644 index 0000000000..0c77a553ab --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ObservableSubscriptionSelectTest.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import org.junit.Test +import kotlin.onSuccess +import kotlin.test.* + +class ObservableSubscriptionSelectTest : TestBase() { + @Test + fun testSelect() = runTest { + // source with n ints + val n = 1000 * stressTestMultiplier + val source = rxObservable { repeat(n) { send(it) } } + var a = 0 + var b = 0 + // open two subs + val channelA = source.toChannel() + val channelB = source.toChannel() + loop@ while (true) { + val done: Int = select { + channelA.onReceiveCatching { result -> + result.onSuccess { assertEquals(a++, it) } + if (result.isSuccess) 1 else 0 + } + channelB.onReceiveCatching { result -> + result.onSuccess { assertEquals(b++, it) } + if (result.isSuccess) 2 else 0 + } + } + when (done) { + 0 -> break@loop + 1 -> { + val r = channelB.receiveCatching().getOrNull() + if (r != null) assertEquals(b++, r) + } + 2 -> { + val r = channelA.receiveCatching().getOrNull() + if (r != null) assertEquals(a++, r) + } + } + } + channelA.cancel() + channelB.cancel() + // should receive one of them fully + assertTrue(a == n || b == n) + } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/ObservableTest.kt b/reactive/kotlinx-coroutines-rx2/test/ObservableTest.kt new file mode 100644 index 0000000000..06ee05bd77 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/ObservableTest.kt @@ -0,0 +1,161 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import io.reactivex.plugins.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class ObservableTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val observable = rxObservable(currentDispatcher()) { + expect(4) + send("OK") + } + expect(2) + observable.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val observable = rxObservable(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + observable.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val observable = rxObservable(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + val sub = observable.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testNotifyOnceOnCancellation() = runTest { + expect(1) + val observable = + rxObservable(currentDispatcher()) { + expect(5) + send("OK") + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + expect(11) + } + } + .doOnNext { + expect(6) + assertEquals("OK", it) + } + .doOnDispose { + expect(10) // notified once! + } + expect(2) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(3) + observable.collect { + expect(8) + assertEquals("OK", it) + } + } + expect(4) + yield() // to observable code + expect(7) + yield() // to consuming coroutines + expect(9) + job.cancel() + job.join() + finish(12) + } + + @Test + fun testFailingConsumer() = runTest { + expect(1) + val pub = rxObservable(currentDispatcher()) { + expect(2) + send("OK") + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + finish(5) + } + } + try { + pub.collect { + expect(3) + throw TestException() + } + } catch (e: TestException) { + expect(4) + } + } + + @Test + fun testExceptionAfterCancellation() { + // Test that no exceptions were reported to the global EH (it will fail the test if so) + val handler = { e: Throwable -> + assertFalse(e is CancellationException) + } + withExceptionHandler(handler) { + RxJavaPlugins.setErrorHandler { + require(it !is CancellationException) + } + Observable + .interval(1, TimeUnit.MILLISECONDS) + .take(1000) + .switchMapSingle { + rxSingle { + timeBomb().await() + } + } + .blockingSubscribe({}, {}) + } + } + + private fun timeBomb() = Single.timer(1, TimeUnit.MILLISECONDS).doOnSuccess { throw TestException() } +} diff --git a/reactive/kotlinx-coroutines-rx2/test/SchedulerStressTest.kt b/reactive/kotlinx-coroutines-rx2/test/SchedulerStressTest.kt new file mode 100644 index 0000000000..5b77e6aa63 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/SchedulerStressTest.kt @@ -0,0 +1,84 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.concurrent.* + +class SchedulerStressTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxCachedThreadScheduler-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + /** + * Test that we don't get an OOM if we schedule many jobs at once. + * It's expected that if you don't dispose you'd see an OOM error. + */ + @Test + fun testSchedulerDisposed(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableDisposed(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerDisposed(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + val worker = scheduler.createWorker() + testRunnableDisposed(worker::schedule) + } + + private suspend fun testRunnableDisposed(block: RxSchedulerBlockNoDelay) { + val n = 2000 * stressTestMultiplier + repeat(n) { + val a = ByteArray(1000000) //1MB + val disposable = block(Runnable { + keepMe(a) + expectUnreached() + }) + disposable.dispose() + yield() // allow the scheduled task to observe that it was disposed + } + } + + /** + * Test function that holds a reference. Used for testing OOM situations + */ + private fun keepMe(a: ByteArray) { + Thread.sleep(a.size / (a.size + 1) + 10L) + } + + /** + * Test that we don't get an OOM if we schedule many delayed jobs at once. It's expected that if you don't dispose that you'd + * see a OOM error. + */ + @Test + fun testSchedulerDisposedDuringDelay(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableDisposedDuringDelay(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerDisposedDuringDelay(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + val worker = scheduler.createWorker() + testRunnableDisposedDuringDelay(worker::schedule) + } + + private fun testRunnableDisposedDuringDelay(block: RxSchedulerBlockWithDelay) { + val n = 2000 * stressTestMultiplier + repeat(n) { + val a = ByteArray(1000000) //1MB + val delayMillis: Long = 10 + val disposable = block(Runnable { + keepMe(a) + expectUnreached() + }, delayMillis, TimeUnit.MILLISECONDS) + disposable.dispose() + } + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/test/SchedulerTest.kt b/reactive/kotlinx-coroutines-rx2/test/SchedulerTest.kt new file mode 100644 index 0000000000..cf163b71de --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/SchedulerTest.kt @@ -0,0 +1,493 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import io.reactivex.disposables.* +import io.reactivex.plugins.* +import io.reactivex.schedulers.* +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import org.junit.* +import org.junit.Test +import java.lang.Runnable +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicReference +import kotlin.coroutines.* +import kotlin.test.* + +class SchedulerTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxCachedThreadScheduler-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testIoScheduler(): Unit = runTest { + expect(1) + val mainThread = Thread.currentThread() + withContext(Schedulers.io().asCoroutineDispatcher()) { + val t1 = Thread.currentThread() + assertNotSame(t1, mainThread) + expect(2) + delay(100) + val t2 = Thread.currentThread() + assertNotSame(t2, mainThread) + expect(3) + } + finish(4) + } + + /** Tests [toString] implementations of [CoroutineDispatcher.asScheduler] and its [Scheduler.Worker]. */ + @Test + fun testSchedulerToString() { + val name = "Dispatchers.Default" + val scheduler = Dispatchers.Default.asScheduler() + assertContains(scheduler.toString(), name) + val worker = scheduler.createWorker() + val activeWorkerName = worker.toString() + assertContains(worker.toString(), name) + worker.dispose() + val disposedWorkerName = worker.toString() + assertNotEquals(activeWorkerName, disposedWorkerName) + } + + private fun runSchedulerTest(nThreads: Int = 1, action: (Scheduler) -> Unit) { + val future = CompletableFuture() + try { + newFixedThreadPoolContext(nThreads, "test").use { dispatcher -> + RxJavaPlugins.setErrorHandler { + if (!future.completeExceptionally(it)) { + handleUndeliverableException(it, dispatcher) + } + } + action(dispatcher.asScheduler()) + } + } finally { + RxJavaPlugins.setErrorHandler(null) + } + future.complete(Unit) + future.getNow(Unit) // rethrow any encountered errors + } + + private fun ensureSeparateThread(schedule: (Runnable, Long, TimeUnit) -> Unit, scheduleNoDelay: (Runnable) -> Unit) { + val mainThread = Thread.currentThread() + val cdl1 = CountDownLatch(1) + val cdl2 = CountDownLatch(1) + expect(1) + val thread = AtomicReference(null) + fun checkThread() { + val current = Thread.currentThread() + thread.getAndSet(current)?.let { assertEquals(it, current) } + } + schedule({ + assertNotSame(mainThread, Thread.currentThread()) + checkThread() + cdl2.countDown() + }, 300, TimeUnit.MILLISECONDS) + scheduleNoDelay { + expect(2) + checkThread() + assertNotSame(mainThread, Thread.currentThread()) + cdl1.countDown() + } + cdl1.await() + cdl2.await() + finish(3) + } + + /** + * Tests [Scheduler.scheduleDirect] for [CoroutineDispatcher.asScheduler] on a single-threaded dispatcher. + */ + @Test + fun testSingleThreadedDispatcherDirect(): Unit = runSchedulerTest(1) { + ensureSeparateThread(it::scheduleDirect, it::scheduleDirect) + } + + /** + * Tests [Scheduler.Worker.schedule] for [CoroutineDispatcher.asScheduler] running its tasks on the correct thread. + */ + @Test + fun testSingleThreadedWorker(): Unit = runSchedulerTest(1) { + val worker = it.createWorker() + ensureSeparateThread(worker::schedule, worker::schedule) + } + + private fun checkCancelling(schedule: (Runnable, Long, TimeUnit) -> Disposable) { + // cancel the task before it has a chance to run. + val handle1 = schedule({ + throw IllegalStateException("should have been successfully cancelled") + }, 10_000, TimeUnit.MILLISECONDS) + handle1.dispose() + // cancel the task after it started running. + val cdl1 = CountDownLatch(1) + val cdl2 = CountDownLatch(1) + val handle2 = schedule({ + cdl1.countDown() + cdl2.await() + if (Thread.interrupted()) + throw IllegalStateException("cancelling the task should not interrupt the thread") + }, 100, TimeUnit.MILLISECONDS) + cdl1.await() + handle2.dispose() + cdl2.countDown() + } + + /** + * Test cancelling [Scheduler.scheduleDirect] for [CoroutineDispatcher.asScheduler]. + */ + @Test + fun testCancellingDirect(): Unit = runSchedulerTest { + checkCancelling(it::scheduleDirect) + } + + /** + * Test cancelling [Scheduler.Worker.schedule] for [CoroutineDispatcher.asScheduler]. + */ + @Test + fun testCancellingWorker(): Unit = runSchedulerTest { + val worker = it.createWorker() + checkCancelling(worker::schedule) + } + + /** + * Test shutting down [CoroutineDispatcher.asScheduler]. + */ + @Test + fun testShuttingDown() { + val n = 5 + runSchedulerTest(nThreads = n) { scheduler -> + val cdl1 = CountDownLatch(n) + val cdl2 = CountDownLatch(1) + val cdl3 = CountDownLatch(n) + repeat(n) { + scheduler.scheduleDirect { + cdl1.countDown() + try { + cdl2.await() + } catch (e: InterruptedException) { + // this is the expected outcome + cdl3.countDown() + } + } + } + cdl1.await() + scheduler.shutdown() + if (!cdl3.await(1, TimeUnit.SECONDS)) { + cdl2.countDown() + error("the tasks were not cancelled when the scheduler was shut down") + } + } + } + + /** Tests that there are no uncaught exceptions if [Disposable.dispose] on a worker happens when tasks are present. */ + @Test + fun testDisposingWorker() = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + val worker = scheduler.createWorker() + yield() // so that the worker starts waiting on the channel + assertFalse(worker.isDisposed) + worker.dispose() + assertTrue(worker.isDisposed) + } + + /** Tests trying to use a [Scheduler.Worker]/[Scheduler] after [Scheduler.Worker.dispose]/[Scheduler.shutdown]. */ + @Test + fun testSchedulingAfterDisposing() = runSchedulerTest { + expect(1) + val worker = it.createWorker() + // use CDL to ensure that the worker has properly initialized + val cdl1 = CountDownLatch(1) + setScheduler(2, 3) + val disposable1 = worker.schedule { + cdl1.countDown() + } + cdl1.await() + expect(4) + assertFalse(disposable1.isDisposed) + setScheduler(6, -1) + // check that the worker automatically disposes of the tasks after being disposed + assertFalse(worker.isDisposed) + worker.dispose() + assertTrue(worker.isDisposed) + expect(5) + val disposable2 = worker.schedule { + expectUnreached() + } + assertTrue(disposable2.isDisposed) + setScheduler(7, 8) + // ensure that the scheduler still works + val cdl2 = CountDownLatch(1) + val disposable3 = it.scheduleDirect { + cdl2.countDown() + } + cdl2.await() + expect(9) + assertFalse(disposable3.isDisposed) + // check that the scheduler automatically disposes of the tasks after being shut down + it.shutdown() + setScheduler(10, -1) + val disposable4 = it.scheduleDirect { + expectUnreached() + } + assertTrue(disposable4.isDisposed) + RxJavaPlugins.setScheduleHandler(null) + finish(11) + } + + @Test + fun testSchedulerWithNoDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithNoDelay(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerWithNoDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithNoDelay(scheduler.createWorker()::schedule) + } + + private suspend fun testRunnableWithNoDelay(block: RxSchedulerBlockNoDelay) { + expect(1) + suspendCancellableCoroutine { + block(Runnable { + expect(2) + it.resume(Unit) + }) + } + yield() + finish(3) + } + + @Test + fun testSchedulerWithDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler::scheduleDirect, 300) + } + + @Test + fun testSchedulerWorkerWithDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler.createWorker()::schedule, 300) + } + + @Test + fun testSchedulerWithZeroDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerWithZeroDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler.createWorker()::schedule) + } + + private suspend fun testRunnableWithDelay(block: RxSchedulerBlockWithDelay, delayMillis: Long = 0) { + expect(1) + suspendCancellableCoroutine { + block({ + expect(2) + it.resume(Unit) + }, delayMillis, TimeUnit.MILLISECONDS) + } + finish(3) + } + + @Test + fun testAsSchedulerWithNegativeDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler::scheduleDirect, -1) + } + + @Test + fun testAsSchedulerWorkerWithNegativeDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler.createWorker()::schedule, -1) + } + + @Test + fun testSchedulerImmediateDispose(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableImmediateDispose(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerImmediateDispose(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableImmediateDispose(scheduler.createWorker()::schedule) + } + + private fun testRunnableImmediateDispose(block: RxSchedulerBlockNoDelay) { + val disposable = block { + expectUnreached() + } + disposable.dispose() + } + + @Test + fun testConvertDispatcherToOriginalScheduler(): Unit = runTest { + val originalScheduler = Schedulers.io() + val dispatcher = originalScheduler.asCoroutineDispatcher() + val scheduler = dispatcher.asScheduler() + assertSame(originalScheduler, scheduler) + } + + @Test + fun testConvertSchedulerToOriginalDispatcher(): Unit = runTest { + val originalDispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = originalDispatcher.asScheduler() + val dispatcher = scheduler.asCoroutineDispatcher() + assertSame(originalDispatcher, dispatcher) + } + + @Test + fun testSchedulerExpectRxPluginsCall(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableExpectRxPluginsCall(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerExpectRxPluginsCall(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableExpectRxPluginsCall(scheduler.createWorker()::schedule) + } + + private suspend fun testRunnableExpectRxPluginsCall(block: RxSchedulerBlockNoDelay) { + expect(1) + setScheduler(2, 4) + suspendCancellableCoroutine { + block(Runnable { + expect(5) + it.resume(Unit) + }) + expect(3) + } + RxJavaPlugins.setScheduleHandler(null) + finish(6) + } + + @Test + fun testSchedulerExpectRxPluginsCallWithDelay(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableExpectRxPluginsCallDelay(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerExpectRxPluginsCallWithDelay(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + val worker = scheduler.createWorker() + testRunnableExpectRxPluginsCallDelay(worker::schedule) + } + + private suspend fun testRunnableExpectRxPluginsCallDelay(block: RxSchedulerBlockWithDelay) { + expect(1) + setScheduler(2, 4) + suspendCancellableCoroutine { + block({ + expect(5) + it.resume(Unit) + }, 10, TimeUnit.MILLISECONDS) + expect(3) + } + RxJavaPlugins.setScheduleHandler(null) + finish(6) + } + + private fun setScheduler(expectedCountOnSchedule: Int, expectCountOnRun: Int) { + RxJavaPlugins.setScheduleHandler { + expect(expectedCountOnSchedule) + Runnable { + expect(expectCountOnRun) + it.run() + } + } + } + + /** + * Tests that [Scheduler.Worker] runs all work sequentially. + */ + @Test + fun testWorkerSequentialOrdering() = runTest { + expect(1) + val scheduler = Dispatchers.Default.asScheduler() + val worker = scheduler.createWorker() + val iterations = 100 + for (i in 0..iterations) { + worker.schedule { + expect(2 + i) + } + } + suspendCoroutine { + worker.schedule { + it.resume(Unit) + } + } + finish((iterations + 2) + 1) + } + + /** + * Test that ensures that delays are actually respected (tasks scheduled sooner in the future run before tasks scheduled later, + * even when the later task is submitted before the earlier one) + */ + @Test + fun testSchedulerRespectsDelays(): Unit = runTest { + val scheduler = Dispatchers.Default.asScheduler() + testRunnableRespectsDelays(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerRespectsDelays(): Unit = runTest { + val scheduler = Dispatchers.Default.asScheduler() + testRunnableRespectsDelays(scheduler.createWorker()::schedule) + } + + private suspend fun testRunnableRespectsDelays(block: RxSchedulerBlockWithDelay) { + expect(1) + val semaphore = Semaphore(2, 2) + block({ + expect(3) + semaphore.release() + }, 100, TimeUnit.MILLISECONDS) + block({ + expect(2) + semaphore.release() + }, 1, TimeUnit.MILLISECONDS) + semaphore.acquire() + semaphore.acquire() + finish(4) + } + + /** + * Tests that cancelling a runnable in one worker doesn't affect work in another scheduler. + * + * This is part of expected behavior documented. + */ + @Test + fun testMultipleWorkerCancellation(): Unit = runTest { + expect(1) + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + suspendCancellableCoroutine { + val workerOne = scheduler.createWorker() + workerOne.schedule({ + expect(3) + it.resume(Unit) + }, 50, TimeUnit.MILLISECONDS) + val workerTwo = scheduler.createWorker() + workerTwo.schedule({ + expectUnreached() + }, 1000, TimeUnit.MILLISECONDS) + workerTwo.dispose() + expect(2) + } + finish(4) + } +} + +typealias RxSchedulerBlockNoDelay = (Runnable) -> Disposable +typealias RxSchedulerBlockWithDelay = (Runnable, Long, TimeUnit) -> Disposable \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx2/test/SingleTest.kt b/reactive/kotlinx-coroutines-rx2/test/SingleTest.kt new file mode 100644 index 0000000000..0d9f0f3a2f --- /dev/null +++ b/reactive/kotlinx-coroutines-rx2/test/SingleTest.kt @@ -0,0 +1,289 @@ +package kotlinx.coroutines.rx2 + +import kotlinx.coroutines.testing.* +import io.reactivex.* +import io.reactivex.disposables.* +import io.reactivex.exceptions.* +import io.reactivex.functions.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class SingleTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val single = rxSingle(currentDispatcher()) { + expect(4) + "OK" + } + expect(2) + single.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val single = rxSingle(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + single.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val single = rxSingle(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + + } + expect(2) + // nothing is called on a disposed rx2 single + val sub = single.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testSingleNoWait() { + val single = rxSingle { + "OK" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleAwait() = runBlocking { + assertEquals("OK", Single.just("O").await() + "K") + } + + /** Tests that calls to [await] throw [CancellationException] and dispose of the subscription when their + * [Job] is cancelled. */ + @Test + fun testSingleAwaitCancellation() = runTest { + expect(1) + val single = SingleSource { s -> + s.onSubscribe(object: Disposable { + override fun dispose() { expect(4) } + override fun isDisposed(): Boolean { expectUnreached(); return false } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + single.await() + } catch (e: CancellationException) { + expect(5) + throw e + } + } + expect(3) + job.cancelAndJoin() + finish(6) + } + + @Test + fun testSingleEmitAndAwait() { + val single = rxSingle { + Single.just("O").await() + "K" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleWithDelay() { + val single = rxSingle { + Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleException() { + val single = rxSingle { + Observable.just("O", "K").awaitSingle() + "K" + } + + checkErroneous(single) { + assert(it is IllegalArgumentException) + } + } + + @Test + fun testAwaitFirst() { + val single = rxSingle { + Observable.just("O", "#").awaitFirst() + "K" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitLast() { + val single = rxSingle { + Observable.just("#", "O").awaitLast() + "K" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromObservable() { + val single = rxSingle { + try { + Observable.error(RuntimeException("O")).awaitFirst() + } catch (e: RuntimeException) { + Observable.just(e.message!!).awaitLast() + "K" + } + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromCoroutine() { + val single = rxSingle { + throw IllegalStateException(Observable.just("O").awaitSingle() + "K") + } + + checkErroneous(single) { + assert(it is IllegalStateException) + assertEquals("OK", it.message) + } + } + + @Test + fun testSuppressedException() = runTest { + val single = rxSingle(currentDispatcher()) { + launch(start = CoroutineStart.ATOMIC) { + throw TestException() // child coroutine fails + } + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() // but parent throws another exception while cleaning up + } + } + try { + single.await() + expectUnreached() + } catch (e: TestException) { + assertIs(e.suppressed[0]) + } + } + + @Test + fun testFatalExceptionInSubscribe() = runTest { + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is LinkageError) + expect(2) + } + withExceptionHandler(handler) { + rxSingle(Dispatchers.Unconfined) { + expect(1) + 42 + }.subscribe(Consumer { + throw LinkageError() + }) + finish(3) + } + } + + @Test + fun testFatalExceptionInSingle() = runTest { + rxSingle(Dispatchers.Unconfined) { + throw LinkageError() + }.subscribe { _, e -> assertIs(e); expect(1) } + + finish(2) + } + + @Test + fun testUnhandledException() = runTest { + expect(1) + var disposable: Disposable? = null + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is TestException) + expect(5) + } + val single = rxSingle(currentDispatcher()) { + expect(4) + disposable!!.dispose() // cancel our own subscription, so that delay will get cancelled + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException() // would not be able to handle it since mono is disposed + } + } + withExceptionHandler(handler) { + single.subscribe(object : SingleObserver { + override fun onSubscribe(d: Disposable) { + expect(2) + disposable = d + } + + override fun onSuccess(t: Unit) { + expectUnreached() + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + expect(3) + yield() // run coroutine + finish(6) + } + } +} diff --git a/reactive/kotlinx-coroutines-rx3/README.md b/reactive/kotlinx-coroutines-rx3/README.md new file mode 100644 index 0000000000..753df277de --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/README.md @@ -0,0 +1,92 @@ +# Module kotlinx-coroutines-rx3 + +Utilities for [RxJava 3.x](https://github.com/ReactiveX/RxJava). + +Coroutine builders: + +| **Name** | **Result** | **Scope** | **Description** +| --------------- | --------------------------------------- | ---------------- | --------------- +| [rxCompletable] | `Completable` | [CoroutineScope] | Cold completable that starts coroutine on subscribe +| [rxMaybe] | `Maybe` | [CoroutineScope] | Cold maybe that starts coroutine on subscribe +| [rxSingle] | `Single` | [CoroutineScope] | Cold single that starts coroutine on subscribe +| [rxObservable] | `Observable` | [ProducerScope] | Cold observable that starts coroutine on subscribe +| [rxFlowable] | `Flowable` | [ProducerScope] | Cold observable that starts coroutine on subscribe with **backpressure** support + +Integration with [Flow]: + +| **Name** | **Result** | **Description** +| --------------- | -------------- | --------------- +| [Flow.asFlowable] | `Flowable` | Converts the given flow to a cold Flowable. +| [Flow.asObservable] | `Observable` | Converts the given flow to a cold Observable. +| [ObservableSource.asFlow] | `Flow` | Converts the given cold ObservableSource to flow + +Suspending extension functions and suspending iteration: + +| **Name** | **Description** +| -------- | --------------- +| [CompletableSource.await][io.reactivex.rxjava3.core.CompletableSource.await] | Awaits for completion of the completable value +| [MaybeSource.awaitSingle][io.reactivex.rxjava3.core.MaybeSource.awaitSingle] | Awaits for the value of the maybe and returns it or throws an exception +| [MaybeSource.awaitSingleOrNull][io.reactivex.rxjava3.core.MaybeSource.awaitSingleOrNull] | Awaits for the value of the maybe and returns it or null +| [SingleSource.await][io.reactivex.rxjava3.core.SingleSource.await] | Awaits for completion of the single value and returns it +| [ObservableSource.awaitFirst][io.reactivex.rxjava3.core.ObservableSource.awaitFirst] | Awaits for the first value from the given observable +| [ObservableSource.awaitFirstOrDefault][io.reactivex.rxjava3.core.ObservableSource.awaitFirstOrDefault] | Awaits for the first value from the given observable or default +| [ObservableSource.awaitFirstOrElse][io.reactivex.rxjava3.core.ObservableSource.awaitFirstOrElse] | Awaits for the first value from the given observable or default from a function +| [ObservableSource.awaitFirstOrNull][io.reactivex.rxjava3.core.ObservableSource.awaitFirstOrNull] | Awaits for the first value from the given observable or null +| [ObservableSource.awaitLast][io.reactivex.rxjava3.core.ObservableSource.awaitFirst] | Awaits for the last value from the given observable +| [ObservableSource.awaitSingle][io.reactivex.rxjava3.core.ObservableSource.awaitSingle] | Awaits for the single value from the given observable + +Note that `Flowable` is a subclass of [Reactive Streams](https://www.reactive-streams.org) +`Publisher` and extensions for it are covered by +[kotlinx-coroutines-reactive](../kotlinx-coroutines-reactive) module. + +Conversion functions: + +| **Name** | **Description** +| -------- | --------------- +| [Job.asCompletable][kotlinx.coroutines.Job.asCompletable] | Converts job to hot completable +| [Deferred.asSingle][kotlinx.coroutines.Deferred.asSingle] | Converts deferred value to hot single +| [Scheduler.asCoroutineDispatcher][io.reactivex.rxjava3.core.Scheduler.asCoroutineDispatcher] | Converts scheduler to [CoroutineDispatcher] + + + + +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[CoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/index.html + + + +[ProducerScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-producer-scope/index.html + + + +[Flow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/index.html + + + + +[rxCompletable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/rx-completable.html +[rxMaybe]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/rx-maybe.html +[rxSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/rx-single.html +[rxObservable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/rx-observable.html +[rxFlowable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/rx-flowable.html +[Flow.asFlowable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/as-flowable.html +[Flow.asObservable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/as-observable.html +[ObservableSource.asFlow]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/as-flow.html +[io.reactivex.rxjava3.core.CompletableSource.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/await.html +[io.reactivex.rxjava3.core.MaybeSource.awaitSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/await-single.html +[io.reactivex.rxjava3.core.MaybeSource.awaitSingleOrNull]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/await-single-or-null.html +[io.reactivex.rxjava3.core.SingleSource.await]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/await.html +[io.reactivex.rxjava3.core.ObservableSource.awaitFirst]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/await-first.html +[io.reactivex.rxjava3.core.ObservableSource.awaitFirstOrDefault]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/await-first-or-default.html +[io.reactivex.rxjava3.core.ObservableSource.awaitFirstOrElse]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/await-first-or-else.html +[io.reactivex.rxjava3.core.ObservableSource.awaitFirstOrNull]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/await-first-or-null.html +[io.reactivex.rxjava3.core.ObservableSource.awaitSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/await-single.html +[kotlinx.coroutines.Job.asCompletable]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/as-completable.html +[kotlinx.coroutines.Deferred.asSingle]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/as-single.html +[io.reactivex.rxjava3.core.Scheduler.asCoroutineDispatcher]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx3/kotlinx.coroutines.rx3/as-coroutine-dispatcher.html + + + +# Package kotlinx.coroutines.rx3 + +Utilities for [RxJava 3.x](https://github.com/ReactiveX/RxJava). diff --git a/reactive/kotlinx-coroutines-rx3/api/kotlinx-coroutines-rx3.api b/reactive/kotlinx-coroutines-rx3/api/kotlinx-coroutines-rx3.api new file mode 100644 index 0000000000..f86276e195 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/api/kotlinx-coroutines-rx3.api @@ -0,0 +1,82 @@ +public final class kotlinx/coroutines/rx3/RxAwaitKt { + public static final fun await (Lio/reactivex/rxjava3/core/CompletableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun await (Lio/reactivex/rxjava3/core/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun await (Lio/reactivex/rxjava3/core/SingleSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirst (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrDefault (Lio/reactivex/rxjava3/core/ObservableSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrElse (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/jvm/functions/Function0;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitFirstOrNull (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitLast (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final synthetic fun awaitOrDefault (Lio/reactivex/rxjava3/core/MaybeSource;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingle (Lio/reactivex/rxjava3/core/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingle (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun awaitSingleOrNull (Lio/reactivex/rxjava3/core/MaybeSource;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; +} + +public final class kotlinx/coroutines/rx3/RxChannelKt { + public static final fun collect (Lio/reactivex/rxjava3/core/MaybeSource;Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun collect (Lio/reactivex/rxjava3/core/ObservableSource;Lkotlin/jvm/functions/Function1;Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun openSubscription (Lio/reactivex/rxjava3/core/MaybeSource;)Lkotlinx/coroutines/channels/ReceiveChannel; + public static final fun openSubscription (Lio/reactivex/rxjava3/core/ObservableSource;)Lkotlinx/coroutines/channels/ReceiveChannel; +} + +public final class kotlinx/coroutines/rx3/RxCompletableKt { + public static final fun rxCompletable (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/rxjava3/core/Completable; + public static synthetic fun rxCompletable$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/rxjava3/core/Completable; +} + +public final class kotlinx/coroutines/rx3/RxConvertKt { + public static final fun asCompletable (Lkotlinx/coroutines/Job;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/rxjava3/core/Completable; + public static final fun asFlow (Lio/reactivex/rxjava3/core/ObservableSource;)Lkotlinx/coroutines/flow/Flow; + public static final fun asFlowable (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/rxjava3/core/Flowable; + public static synthetic fun asFlowable$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/rxjava3/core/Flowable; + public static final fun asMaybe (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/rxjava3/core/Maybe; + public static final fun asObservable (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/rxjava3/core/Observable; + public static synthetic fun asObservable$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/rxjava3/core/Observable; + public static final fun asSingle (Lkotlinx/coroutines/Deferred;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/rxjava3/core/Single; + public static final synthetic fun from (Lkotlinx/coroutines/flow/Flow;)Lio/reactivex/rxjava3/core/Flowable; + public static final synthetic fun from (Lkotlinx/coroutines/flow/Flow;)Lio/reactivex/rxjava3/core/Observable; + public static final synthetic fun from (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/rxjava3/core/Flowable; + public static final synthetic fun from (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;)Lio/reactivex/rxjava3/core/Observable; + public static synthetic fun from$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/rxjava3/core/Flowable; + public static synthetic fun from$default (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/CoroutineContext;ILjava/lang/Object;)Lio/reactivex/rxjava3/core/Observable; +} + +public final class kotlinx/coroutines/rx3/RxFlowableKt { + public static final fun rxFlowable (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/rxjava3/core/Flowable; + public static synthetic fun rxFlowable$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/rxjava3/core/Flowable; +} + +public final class kotlinx/coroutines/rx3/RxMaybeKt { + public static final fun rxMaybe (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/rxjava3/core/Maybe; + public static synthetic fun rxMaybe$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/rxjava3/core/Maybe; +} + +public final class kotlinx/coroutines/rx3/RxObservableKt { + public static final fun rxObservable (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/rxjava3/core/Observable; + public static synthetic fun rxObservable$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/rxjava3/core/Observable; +} + +public final class kotlinx/coroutines/rx3/RxSchedulerKt { + public static final fun asCoroutineDispatcher (Lio/reactivex/rxjava3/core/Scheduler;)Lkotlinx/coroutines/CoroutineDispatcher; + public static final synthetic fun asCoroutineDispatcher (Lio/reactivex/rxjava3/core/Scheduler;)Lkotlinx/coroutines/rx3/SchedulerCoroutineDispatcher; + public static final fun asScheduler (Lkotlinx/coroutines/CoroutineDispatcher;)Lio/reactivex/rxjava3/core/Scheduler; +} + +public final class kotlinx/coroutines/rx3/RxSingleKt { + public static final fun rxSingle (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;)Lio/reactivex/rxjava3/core/Single; + public static synthetic fun rxSingle$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)Lio/reactivex/rxjava3/core/Single; +} + +public final class kotlinx/coroutines/rx3/SchedulerCoroutineDispatcher : kotlinx/coroutines/CoroutineDispatcher, kotlinx/coroutines/Delay { + public fun (Lio/reactivex/rxjava3/core/Scheduler;)V + public fun delay (JLkotlin/coroutines/Continuation;)Ljava/lang/Object; + public fun dispatch (Lkotlin/coroutines/CoroutineContext;Ljava/lang/Runnable;)V + public fun equals (Ljava/lang/Object;)Z + public final fun getScheduler ()Lio/reactivex/rxjava3/core/Scheduler; + public fun hashCode ()I + public fun invokeOnTimeout (JLjava/lang/Runnable;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/DisposableHandle; + public fun scheduleResumeAfterDelay (JLkotlinx/coroutines/CancellableContinuation;)V + public fun toString ()Ljava/lang/String; +} + diff --git a/reactive/kotlinx-coroutines-rx3/build.gradle.kts b/reactive/kotlinx-coroutines-rx3/build.gradle.kts new file mode 100644 index 0000000000..f88d2ecb30 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/build.gradle.kts @@ -0,0 +1,36 @@ +import org.jetbrains.dokka.gradle.DokkaTaskPartial +import java.net.* + +dependencies { + api(project(":kotlinx-coroutines-reactive")) + testImplementation("org.reactivestreams:reactive-streams-tck:${version("reactive_streams")}") + api("io.reactivex.rxjava3:rxjava:${version("rxjava3")}") +} + +tasks.withType(DokkaTaskPartial::class) { + dokkaSourceSets.configureEach { + externalDocumentationLink { + url = URL("https://reactivex.io/RxJava/3.x/javadoc/") + packageListUrl = projectDir.toPath().resolve("package.list").toUri().toURL() + } + } +} + +val testNG by tasks.registering(Test::class) { + useTestNG() + reports.html.outputLocation = file("${layout.buildDirectory.get()}/reports/testng") + include("**/*ReactiveStreamTckTest.*") + // Skip testNG when tests are filtered with --tests, otherwise it simply fails + onlyIf { + filter.includePatterns.isEmpty() + } + doFirst { + // Classic gradle, nothing works without doFirst + println("TestNG tests: ($includes)") + } +} + +val test by tasks.getting(Test::class) { + dependsOn(testNG) + reports.html.outputLocation = file("${layout.buildDirectory.get()}/reports/junit") +} diff --git a/reactive/kotlinx-coroutines-rx3/package.list b/reactive/kotlinx-coroutines-rx3/package.list new file mode 100644 index 0000000000..889916d0db --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/package.list @@ -0,0 +1,14 @@ +io.reactivex.rxjava3.core +io.reactivex.rxjava3.annotations +io.reactivex.rxjava3.disposables +io.reactivex.rxjava3.exceptions +io.reactivex.rxjava3.flowables +io.reactivex.rxjava3.functions +io.reactivex.rxjava3.observables +io.reactivex.rxjava3.observers +io.reactivex.rxjava3.parallel +io.reactivex.rxjava3.plugins +io.reactivex.rxjava3.processors +io.reactivex.rxjava3.schedulers +io.reactivex.rxjava3.subjects +io.reactivex.rxjava3.subscribers diff --git a/reactive/kotlinx-coroutines-rx3/src/RxAwait.kt b/reactive/kotlinx-coroutines-rx3/src/RxAwait.kt new file mode 100644 index 0000000000..0353d77f12 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxAwait.kt @@ -0,0 +1,273 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.Disposable +import kotlinx.coroutines.CancellableContinuation +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.Job +import kotlinx.coroutines.suspendCancellableCoroutine +import kotlin.coroutines.* + +// ------------------------ CompletableSource ------------------------ + +/** + * Awaits for completion of this completable without blocking the thread. + * Returns `Unit`, or throws the corresponding exception if this completable produces an error. + * + * This suspending function is cancellable. If the [Job] of the invoking coroutine is cancelled while this + * suspending function is suspended, this function immediately resumes with [CancellationException] and disposes of its + * subscription. + */ +public suspend fun CompletableSource.await(): Unit = suspendCancellableCoroutine { cont -> + subscribe(object : CompletableObserver { + override fun onSubscribe(d: Disposable) { cont.disposeOnCancellation(d) } + override fun onComplete() { cont.resume(Unit) } + override fun onError(e: Throwable) { cont.resumeWithException(e) } + }) +} + +// ------------------------ MaybeSource ------------------------ + +/** + * Awaits for completion of the [MaybeSource] without blocking the thread. + * Returns the resulting value, or `null` if no value is produced, or throws the corresponding exception if this + * [MaybeSource] produces an error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this + * function immediately resumes with [CancellationException] and disposes of its subscription. + */ +public suspend fun MaybeSource.awaitSingleOrNull(): T? = suspendCancellableCoroutine { cont -> + subscribe(object : MaybeObserver { + override fun onSubscribe(d: Disposable) { cont.disposeOnCancellation(d) } + override fun onComplete() { cont.resume(null) } + override fun onSuccess(t: T & Any) { cont.resume(t) } + override fun onError(error: Throwable) { cont.resumeWithException(error) } + }) +} + +/** + * Awaits for completion of the [MaybeSource] without blocking the thread. + * Returns the resulting value, or throws if either no value is produced or this [MaybeSource] produces an error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this + * function immediately resumes with [CancellationException] and disposes of its subscription. + * + * @throws NoSuchElementException if no elements were produced by this [MaybeSource]. + */ +public suspend fun MaybeSource.awaitSingle(): T = awaitSingleOrNull() ?: throw NoSuchElementException() + +/** + * Awaits for completion of the maybe without blocking a thread. + * Returns the resulting value, null if no value was produced or throws the corresponding exception if this + * maybe had produced error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this function + * immediately resumes with [CancellationException]. + * + * ### Deprecation + * + * Deprecated in favor of [awaitSingleOrNull] in order to reflect that `null` can be returned to denote the absence of + * a value, as opposed to throwing in such case. + * + * @suppress + */ +@Deprecated( + message = "Deprecated in favor of awaitSingleOrNull()", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingleOrNull()") +) // Warning since 1.5, error in 1.6, hidden in 1.7 +public suspend fun MaybeSource.await(): T? = awaitSingleOrNull() + +/** + * Awaits for completion of the maybe without blocking a thread. + * Returns the resulting value, [default] if no value was produced or throws the corresponding exception if this + * maybe had produced error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while this suspending function is waiting, this function + * immediately resumes with [CancellationException]. + * + * ### Deprecation + * + * Deprecated in favor of [awaitSingleOrNull] for naming consistency (see the deprecation of [MaybeSource.await] for + * details). + * + * @suppress + */ +@Deprecated( + message = "Deprecated in favor of awaitSingleOrNull()", + level = DeprecationLevel.HIDDEN, + replaceWith = ReplaceWith("this.awaitSingleOrNull() ?: default") +) // Warning since 1.5, error in 1.6, hidden in 1.7 +public suspend fun MaybeSource.awaitOrDefault(default: T): T = awaitSingleOrNull() ?: default + +// ------------------------ SingleSource ------------------------ + +/** + * Awaits for completion of the single value response without blocking the thread. + * Returns the resulting value, or throws the corresponding exception if this response produces an error. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + */ +public suspend fun SingleSource.await(): T = suspendCancellableCoroutine { cont -> + subscribe(object : SingleObserver { + override fun onSubscribe(d: Disposable) { cont.disposeOnCancellation(d) } + override fun onSuccess(t: T & Any) { cont.resume(t) } + override fun onError(error: Throwable) { cont.resumeWithException(error) } + }) +} + +// ------------------------ ObservableSource ------------------------ + +/** + * Awaits the first value from the given [Observable] without blocking the thread and returns the resulting value, or, + * if the observable has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the observable does not emit any value + */ +@Suppress("UNCHECKED_CAST") +public suspend fun ObservableSource.awaitFirst(): T = awaitOne(Mode.FIRST) as T + +/** + * Awaits the first value from the given [Observable], or returns the [default] value if none is emitted, without + * blocking the thread, and returns the resulting value, or, if this observable has produced an error, throws the + * corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + */ +@Suppress("UNCHECKED_CAST") +public suspend fun ObservableSource.awaitFirstOrDefault(default: T): T = + awaitOne(Mode.FIRST_OR_DEFAULT, default) as T + +/** + * Awaits the first value from the given [Observable], or returns `null` if none is emitted, without blocking the + * thread, and returns the resulting value, or, if this observable has produced an error, throws the corresponding + * exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + */ +public suspend fun ObservableSource.awaitFirstOrNull(): T? = awaitOne(Mode.FIRST_OR_DEFAULT) + +/** + * Awaits the first value from the given [Observable], or calls [defaultValue] to get a value if none is emitted, + * without blocking the thread, and returns the resulting value, or, if this observable has produced an error, throws + * the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + */ +public suspend fun ObservableSource.awaitFirstOrElse(defaultValue: () -> T): T = + awaitOne(Mode.FIRST_OR_DEFAULT) ?: defaultValue() + +/** + * Awaits the last value from the given [Observable] without blocking the thread and + * returns the resulting value, or, if this observable has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the observable does not emit any value + */ +@Suppress("UNCHECKED_CAST") +public suspend fun ObservableSource.awaitLast(): T = awaitOne(Mode.LAST) as T + +/** + * Awaits the single value from the given observable without blocking the thread and returns the resulting value, or, + * if this observable has produced an error, throws the corresponding exception. + * + * This suspending function is cancellable. + * If the [Job] of the current coroutine is cancelled while the suspending function is waiting, this + * function immediately disposes of its subscription and resumes with [CancellationException]. + * + * @throws NoSuchElementException if the observable does not emit any value + * @throws IllegalArgumentException if the observable emits more than one value + */ +@Suppress("UNCHECKED_CAST") +public suspend fun ObservableSource.awaitSingle(): T = awaitOne(Mode.SINGLE) as T + +// ------------------------ private ------------------------ + +internal fun CancellableContinuation<*>.disposeOnCancellation(d: Disposable) = + invokeOnCancellation { d.dispose() } + +private enum class Mode(@JvmField val s: String) { + FIRST("awaitFirst"), + FIRST_OR_DEFAULT("awaitFirstOrDefault"), + LAST("awaitLast"), + SINGLE("awaitSingle"); + override fun toString(): String = s +} + +private suspend fun ObservableSource.awaitOne( + mode: Mode, + default: T? = null +): T? = suspendCancellableCoroutine { cont -> + subscribe(object : Observer { + private lateinit var subscription: Disposable + private var value: T? = null + private var seenValue = false + + override fun onSubscribe(sub: Disposable) { + subscription = sub + cont.invokeOnCancellation { sub.dispose() } + } + + override fun onNext(t: T & Any) { + when (mode) { + Mode.FIRST, Mode.FIRST_OR_DEFAULT -> { + if (!seenValue) { + seenValue = true + cont.resume(t) + subscription.dispose() + } + } + Mode.LAST, Mode.SINGLE -> { + if (mode == Mode.SINGLE && seenValue) { + if (cont.isActive) + cont.resumeWithException(IllegalArgumentException("More than one onNext value for $mode")) + subscription.dispose() + } else { + value = t + seenValue = true + } + } + } + } + + @Suppress("UNCHECKED_CAST") + override fun onComplete() { + if (seenValue) { + if (cont.isActive) cont.resume(value as T) + return + } + when { + mode == Mode.FIRST_OR_DEFAULT -> { + cont.resume(default as T) + } + cont.isActive -> { + cont.resumeWithException(NoSuchElementException("No value received via onNext for $mode")) + } + } + } + + override fun onError(e: Throwable) { + cont.resumeWithException(e) + } + }) +} + diff --git a/reactive/kotlinx-coroutines-rx3/src/RxCancellable.kt b/reactive/kotlinx-coroutines-rx3/src/RxCancellable.kt new file mode 100644 index 0000000000..041c5a2ded --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxCancellable.kt @@ -0,0 +1,22 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.functions.* +import io.reactivex.rxjava3.plugins.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +internal class RxCancellable(private val job: Job) : Cancellable { + override fun cancel() { + job.cancel() + } +} + +internal fun handleUndeliverableException(cause: Throwable, context: CoroutineContext) { + if (cause is CancellationException) return // Async CE should be completely ignored + try { + RxJavaPlugins.onError(cause) + } catch (e: Throwable) { + cause.addSuppressed(e) + handleCoroutineException(context, cause) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/src/RxChannel.kt b/reactive/kotlinx-coroutines-rx3/src/RxChannel.kt new file mode 100644 index 0000000000..d65dde0534 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxChannel.kt @@ -0,0 +1,86 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* + +/** + * Subscribes to this [MaybeSource] and returns a channel to receive elements emitted by it. + * The resulting channel shall be [cancelled][ReceiveChannel.cancel] to unsubscribe from this source. + * + * This API is internal in the favour of [Flow]. + * [MaybeSource] doesn't have a corresponding [Flow] adapter, so it should be transformed to [Observable] first. + */ +@PublishedApi +internal fun MaybeSource.openSubscription(): ReceiveChannel { + val channel = SubscriptionChannel() + subscribe(channel) + return channel +} + +/** + * Subscribes to this [ObservableSource] and returns a channel to receive elements emitted by it. + * The resulting channel shall be [cancelled][ReceiveChannel.cancel] to unsubscribe from this source. + * + * This API is internal in the favour of [Flow]. + * [ObservableSource] doesn't have a corresponding [Flow] adapter, so it should be transformed to [Observable] first. + */ +@PublishedApi +internal fun ObservableSource.openSubscription(): ReceiveChannel { + val channel = SubscriptionChannel() + subscribe(channel) + return channel +} + +/** + * Subscribes to this [MaybeSource] and performs the specified action for each received element. + * + * If [action] throws an exception at some point or if the [MaybeSource] raises an error, the exception is rethrown from + * [collect]. + */ +public suspend inline fun MaybeSource.collect(action: (T) -> Unit): Unit = + openSubscription().consumeEach(action) + +/** + * Subscribes to this [ObservableSource] and performs the specified action for each received element. + * + * If [action] throws an exception at some point, the subscription is cancelled, and the exception is rethrown from + * [collect]. Also, if the [ObservableSource] signals an error, that error is rethrown from [collect]. + */ +public suspend inline fun ObservableSource.collect(action: (T) -> Unit): Unit = openSubscription().consumeEach(action) + +@Suppress("INVISIBLE_REFERENCE", "INVISIBLE_MEMBER") +private class SubscriptionChannel : + BufferedChannel(capacity = Channel.UNLIMITED), Observer, MaybeObserver +{ + private val _subscription = atomic(null) + + @Suppress("CANNOT_OVERRIDE_INVISIBLE_MEMBER") + override fun onClosedIdempotent() { + _subscription.getAndSet(null)?.dispose() // dispose exactly once + } + + // Observer overrider + override fun onSubscribe(sub: Disposable) { + _subscription.value = sub + } + + override fun onSuccess(t: T & Any) { + trySend(t) + close(cause = null) + } + + override fun onNext(t: T & Any) { + trySend(t) // Safe to ignore return value here, expectedly racing with cancellation + } + + override fun onComplete() { + close(cause = null) + } + + override fun onError(e: Throwable) { + close(cause = e) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/src/RxCompletable.kt b/reactive/kotlinx-coroutines-rx3/src/RxCompletable.kt new file mode 100644 index 0000000000..87f302fe8b --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxCompletable.kt @@ -0,0 +1,57 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * Creates cold [Completable] that runs a given [block] in a coroutine and emits its result. + * Every time the returned completable is subscribed, it starts a new coroutine. + * Unsubscribing cancels running coroutine. + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + */ +public fun rxCompletable( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> Unit +): Completable { + require(context[Job] === null) { "Completable context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return rxCompletableInternal(GlobalScope, context, block) +} + +private fun rxCompletableInternal( + scope: CoroutineScope, // support for legacy rxCompletable in scope + context: CoroutineContext, + block: suspend CoroutineScope.() -> Unit +): Completable = Completable.create { subscriber -> + val newContext = scope.newCoroutineContext(context) + val coroutine = RxCompletableCoroutine(newContext, subscriber) + subscriber.setCancellable(RxCancellable(coroutine)) + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private class RxCompletableCoroutine( + parentContext: CoroutineContext, + private val subscriber: CompletableEmitter +) : AbstractCoroutine(parentContext, false, true) { + override fun onCompleted(value: Unit) { + try { + subscriber.onComplete() + } catch (e: Throwable) { + handleUndeliverableException(e, context) + } + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + try { + if (subscriber.tryOnError(cause)) { + return + } + } catch (e: Throwable) { + cause.addSuppressed(e) + } + handleUndeliverableException(cause, context) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/src/RxConvert.kt b/reactive/kotlinx-coroutines-rx3/src/RxConvert.kt new file mode 100644 index 0000000000..a3731f85ab --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxConvert.kt @@ -0,0 +1,151 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.reactivestreams.* +import java.util.concurrent.atomic.* +import kotlin.coroutines.* + +/** + * Converts this job to the hot reactive completable that signals + * with [onCompleted][CompletableObserver.onComplete] when the corresponding job completes. + * + * Every subscriber gets the signal at the same time. + * Unsubscribing from the resulting completable **does not** affect the original job in any way. + * + * **Note: This is an experimental api.** Conversion of coroutines primitives to reactive entities may change + * in the future to account for the concept of structured concurrency. + * + * @param context -- the coroutine context from which the resulting completable is going to be signalled + */ +public fun Job.asCompletable(context: CoroutineContext): Completable = rxCompletable(context) { + this@asCompletable.join() +} + +/** + * Converts this deferred value to the hot reactive maybe that signals + * [onComplete][MaybeEmitter.onComplete], [onSuccess][MaybeEmitter.onSuccess] or [onError][MaybeEmitter.onError]. + * + * Every subscriber gets the same completion value. + * Unsubscribing from the resulting maybe **does not** affect the original deferred value in any way. + * + * **Note: This is an experimental api.** Conversion of coroutines primitives to reactive entities may change + * in the future to account for the concept of structured concurrency. + * + * @param context -- the coroutine context from which the resulting maybe is going to be signalled + */ +public fun Deferred.asMaybe(context: CoroutineContext): Maybe = rxMaybe(context) { + this@asMaybe.await() +} + +/** + * Converts this deferred value to the hot reactive single that signals either + * [onSuccess][SingleObserver.onSuccess] or [onError][SingleObserver.onError]. + * + * Every subscriber gets the same completion value. + * Unsubscribing from the resulting single **does not** affect the original deferred value in any way. + * + * **Note: This is an experimental api.** Conversion of coroutines primitives to reactive entities may change + * in the future to account for the concept of structured concurrency. + * + * @param context -- the coroutine context from which the resulting single is going to be signalled + */ +public fun Deferred.asSingle(context: CoroutineContext): Single = rxSingle(context) { + this@asSingle.await() +} + +/** + * Transforms given cold [ObservableSource] into cold [Flow]. + * + * The resulting flow is _cold_, which means that [ObservableSource.subscribe] is called every time a terminal operator + * is applied to the resulting flow. + * + * A channel with the [default][Channel.BUFFERED] buffer size is used. Use the [buffer] operator on the + * resulting flow to specify a user-defined value and to control what happens when data is produced faster + * than consumed, i.e. to control the back-pressure behavior. Check [callbackFlow] for more details. + */ +public fun ObservableSource.asFlow(): Flow = callbackFlow { + val disposableRef = AtomicReference() + val observer = object : Observer { + override fun onComplete() { close() } + override fun onSubscribe(d: Disposable) { if (!disposableRef.compareAndSet(null, d)) d.dispose() } + override fun onNext(t: T) { + /* + * Channel was closed by the downstream, so the exception (if any) + * also was handled by the same downstream + */ + try { + trySendBlocking(t) + } catch (e: InterruptedException) { + // RxJava interrupts the source + } + } + override fun onError(e: Throwable) { close(e) } + } + + subscribe(observer) + awaitClose { disposableRef.getAndSet(Disposable.disposed())?.dispose() } +} + +/** + * Converts the given flow to a cold observable. + * The original flow is cancelled when the observable subscriber is disposed. + * + * An optional [context] can be specified to control the execution context of calls to [Observer] methods. + * You can set a [CoroutineDispatcher] to confine them to a specific thread and/or various [ThreadContextElement] to + * inject additional context into the caller thread. By default, the [Unconfined][Dispatchers.Unconfined] dispatcher + * is used, so calls are performed from an arbitrary thread. + */ +public fun Flow.asObservable(context: CoroutineContext = EmptyCoroutineContext) : Observable = Observable.create { emitter -> + /* + * ATOMIC is used here to provide stable behaviour of subscribe+dispose pair even if + * asObservable is already invoked from unconfined + */ + val job = GlobalScope.launch(Dispatchers.Unconfined + context, start = CoroutineStart.ATOMIC) { + try { + collect { value -> emitter.onNext(value) } + emitter.onComplete() + } catch (e: Throwable) { + // 'create' provides safe emitter, so we can unconditionally call on* here if exception occurs in `onComplete` + if (e !is CancellationException) { + if (!emitter.tryOnError(e)) { + handleUndeliverableException(e, coroutineContext) + } + } else { + emitter.onComplete() + } + } + } + emitter.setCancellable(RxCancellable(job)) +} + +/** + * Converts the given flow to a cold flowable. + * The original flow is cancelled when the flowable subscriber is disposed. + * + * An optional [context] can be specified to control the execution context of calls to [Subscriber] methods. + * You can set a [CoroutineDispatcher] to confine them to a specific thread and/or various [ThreadContextElement] to + * inject additional context into the caller thread. By default, the [Unconfined][Dispatchers.Unconfined] dispatcher + * is used, so calls are performed from an arbitrary thread. + */ +public fun Flow.asFlowable(context: CoroutineContext = EmptyCoroutineContext): Flowable = + Flowable.fromPublisher(asPublisher(context)) + +/** @suppress */ +@Suppress("UNUSED") // KT-42513 +@JvmOverloads // binary compatibility +@JvmName("from") +@Deprecated(level = DeprecationLevel.HIDDEN, message = "") // Since 1.4, was experimental prior to that +public fun Flow._asFlowable(context: CoroutineContext = EmptyCoroutineContext): Flowable = + asFlowable(context) + +/** @suppress */ +@Suppress("UNUSED") // KT-42513 +@JvmOverloads // binary compatibility +@JvmName("from") +@Deprecated(level = DeprecationLevel.HIDDEN, message = "") // Since 1.4, was experimental prior to that +public fun Flow._asObservable(context: CoroutineContext = EmptyCoroutineContext) : Observable = asObservable(context) diff --git a/reactive/kotlinx-coroutines-rx3/src/RxFlowable.kt b/reactive/kotlinx-coroutines-rx3/src/RxFlowable.kt new file mode 100644 index 0000000000..0dfd68c836 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxFlowable.kt @@ -0,0 +1,36 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.reactive.* +import kotlin.coroutines.* + +/** + * Creates cold [flowable][Flowable] that will run a given [block] in a coroutine. + * Every time the returned flowable is subscribed, it starts a new coroutine. + * + * Coroutine emits ([ObservableEmitter.onNext]) values with `send`, completes ([ObservableEmitter.onComplete]) + * when the coroutine completes or channel is explicitly closed and emits error ([ObservableEmitter.onError]) + * if coroutine throws an exception or closes channel with a cause. + * Unsubscribing cancels running coroutine. + * + * Invocations of `send` are suspended appropriately when subscribers apply back-pressure and to ensure that + * `onNext` is not invoked concurrently. + * + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + * + * **Note: This is an experimental api.** Behaviour of publishers that work as children in a parent scope with respect + */ +public fun rxFlowable( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Flowable { + require(context[Job] === null) { "Flowable context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return Flowable.fromPublisher(publishInternal(GlobalScope, context, RX_HANDLER, block)) +} + +private val RX_HANDLER: (Throwable, CoroutineContext) -> Unit = ::handleUndeliverableException diff --git a/reactive/kotlinx-coroutines-rx3/src/RxMaybe.kt b/reactive/kotlinx-coroutines-rx3/src/RxMaybe.kt new file mode 100644 index 0000000000..abc59a26bc --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxMaybe.kt @@ -0,0 +1,58 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * Creates cold [maybe][Maybe] that will run a given [block] in a coroutine and emits its result. + * If [block] result is `null`, [onComplete][MaybeObserver.onComplete] is invoked without a value. + * Every time the returned observable is subscribed, it starts a new coroutine. + * Unsubscribing cancels running coroutine. + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + */ +public fun rxMaybe( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> T? +): Maybe { + require(context[Job] === null) { "Maybe context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return rxMaybeInternal(GlobalScope, context, block) +} + +private fun rxMaybeInternal( + scope: CoroutineScope, // support for legacy rxMaybe in scope + context: CoroutineContext, + block: suspend CoroutineScope.() -> T? +): Maybe = Maybe.create { subscriber -> + val newContext = scope.newCoroutineContext(context) + val coroutine = RxMaybeCoroutine(newContext, subscriber) + subscriber.setCancellable(RxCancellable(coroutine)) + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private class RxMaybeCoroutine( + parentContext: CoroutineContext, + private val subscriber: MaybeEmitter +) : AbstractCoroutine(parentContext, false, true) { + override fun onCompleted(value: T?) { + try { + if (value == null) subscriber.onComplete() else subscriber.onSuccess(value) + } catch (e: Throwable) { + handleUndeliverableException(e, context) + } + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + try { + if (subscriber.tryOnError(cause)) { + return + } + } catch (e: Throwable) { + cause.addSuppressed(e) + } + handleUndeliverableException(cause, context) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/src/RxObservable.kt b/reactive/kotlinx-coroutines-rx3/src/RxObservable.kt new file mode 100644 index 0000000000..0b26f47e66 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxObservable.kt @@ -0,0 +1,206 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.exceptions.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import kotlinx.coroutines.sync.* +import kotlin.coroutines.* +import kotlinx.coroutines.internal.* + +/** + * Creates cold [observable][Observable] that will run a given [block] in a coroutine. + * Every time the returned observable is subscribed, it starts a new coroutine. + * + * Coroutine emits ([ObservableEmitter.onNext]) values with `send`, completes ([ObservableEmitter.onComplete]) + * when the coroutine completes or channel is explicitly closed and emits error ([ObservableEmitter.onError]) + * if coroutine throws an exception or closes channel with a cause. + * Unsubscribing cancels running coroutine. + * + * Invocations of `send` are suspended appropriately to ensure that `onNext` is not invoked concurrently. + * Note that Rx 2.x [Observable] **does not support backpressure**. + * + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + */ +public fun rxObservable( + context: CoroutineContext = EmptyCoroutineContext, + @BuilderInference block: suspend ProducerScope.() -> Unit +): Observable { + require(context[Job] === null) { "Observable context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return rxObservableInternal(GlobalScope, context, block) +} + +private fun rxObservableInternal( + scope: CoroutineScope, // support for legacy rxObservable in scope + context: CoroutineContext, + block: suspend ProducerScope.() -> Unit +): Observable = Observable.create { subscriber -> + val newContext = scope.newCoroutineContext(context) + val coroutine = RxObservableCoroutine(newContext, subscriber) + subscriber.setCancellable(RxCancellable(coroutine)) // do it first (before starting coroutine), to await unnecessary suspensions + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private const val OPEN = 0 // open channel, still working +private const val CLOSED = -1 // closed, but have not signalled onCompleted/onError yet +private const val SIGNALLED = -2 // already signalled subscriber onCompleted/onError + +private class RxObservableCoroutine( + parentContext: CoroutineContext, + private val subscriber: ObservableEmitter +) : AbstractCoroutine(parentContext, false, true), ProducerScope { + override val channel: SendChannel get() = this + + private val _signal = atomic(OPEN) + + override val isClosedForSend: Boolean get() = !isActive + override fun close(cause: Throwable?): Boolean = cancelCoroutine(cause) + override fun invokeOnClose(handler: (Throwable?) -> Unit) = + throw UnsupportedOperationException("RxObservableCoroutine doesn't support invokeOnClose") + + // Mutex is locked when either nRequested == 0 or while subscriber.onXXX is being invoked + private val mutex: Mutex = Mutex() + + @Suppress("UNCHECKED_CAST", "INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + override val onSend: SelectClause2> get() = SelectClause2Impl( + clauseObject = this, + regFunc = RxObservableCoroutine<*>::registerSelectForSend as RegistrationFunction, + processResFunc = RxObservableCoroutine<*>::processResultSelectSend as ProcessResultFunction + ) + + @Suppress("UNUSED_PARAMETER") + private fun registerSelectForSend(select: SelectInstance<*>, element: Any?) { + // Try to acquire the mutex and complete in the registration phase. + if (mutex.tryLock()) { + select.selectInRegistrationPhase(Unit) + return + } + // Start a new coroutine that waits for the mutex, invoking `trySelect(..)` after that. + // Please note that at the point of the `trySelect(..)` invocation the corresponding + // `select` can still be in the registration phase, making this `trySelect(..)` bound to fail. + // In this case, the `onSend` clause will be re-registered, which alongside with the mutex + // manipulation makes the resulting solution obstruction-free. + launch { + mutex.lock() + if (!select.trySelect(this@RxObservableCoroutine, Unit)) { + mutex.unlock() + } + } + } + + @Suppress("RedundantNullableReturnType", "UNUSED_PARAMETER", "UNCHECKED_CAST") + private fun processResultSelectSend(element: Any?, selectResult: Any?): Any? { + doLockedNext(element as T)?.let { throw it } + return this@RxObservableCoroutine + } + + override fun trySend(element: T): ChannelResult = + if (!mutex.tryLock()) { + ChannelResult.failure() + } else { + when (val throwable = doLockedNext(element)) { + null -> ChannelResult.success(Unit) + else -> ChannelResult.closed(throwable) + } + } + + override suspend fun send(element: T) { + mutex.lock() + doLockedNext(element)?.let { throw it } + } + + // assert: mutex.isLocked() + private fun doLockedNext(elem: T): Throwable? { + // check if already closed for send + if (!isActive) { + doLockedSignalCompleted(completionCause, completionCauseHandled) + return getCancellationException() + } + // notify subscriber + try { + subscriber.onNext(elem) + } catch (e: Throwable) { + val cause = UndeliverableException(e) + val causeDelivered = close(cause) + unlockAndCheckCompleted() + return if (causeDelivered) { + // `cause` is the reason this channel is closed + cause + } else { + // Someone else closed the channel during `onNext`. We report `cause` as an undeliverable exception. + handleUndeliverableException(cause, context) + getCancellationException() + } + } + /* + * There is no sense to check for `isActive` before doing `unlock`, because cancellation/completion might + * happen after this check and before `unlock` (see signalCompleted that does not do anything + * if it fails to acquire the lock that we are still holding). + * We have to recheck `isCompleted` after `unlock` anyway. + */ + unlockAndCheckCompleted() + return null + } + + private fun unlockAndCheckCompleted() { + mutex.unlock() + // recheck isActive + if (!isActive && mutex.tryLock()) + doLockedSignalCompleted(completionCause, completionCauseHandled) + } + + // assert: mutex.isLocked() + private fun doLockedSignalCompleted(cause: Throwable?, handled: Boolean) { + // cancellation failures + try { + if (_signal.value == SIGNALLED) + return + _signal.value = SIGNALLED // we'll signal onError/onCompleted (that the final state -- no CAS needed) + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + val unwrappedCause = cause?.let { unwrap(it) } + if (unwrappedCause == null) { + try { + subscriber.onComplete() + } catch (e: Exception) { + handleUndeliverableException(e, context) + } + } else if (unwrappedCause is UndeliverableException && !handled) { + /** Such exceptions are not reported to `onError`, as, according to the reactive specifications, + * exceptions thrown from the Subscriber methods must be treated as if the Subscriber was already + * cancelled. */ + handleUndeliverableException(cause, context) + } else if (unwrappedCause !== getCancellationException() || !subscriber.isDisposed) { + try { + /** If the subscriber is already in a terminal state, the error will be signalled to + * `RxJavaPlugins.onError`. */ + subscriber.onError(cause) + } catch (e: Exception) { + cause.addSuppressed(e) + handleUndeliverableException(cause, context) + } + } + } finally { + mutex.unlock() + } + } + + private fun signalCompleted(cause: Throwable?, handled: Boolean) { + if (!_signal.compareAndSet(OPEN, CLOSED)) return // abort, other thread invoked doLockedSignalCompleted + if (mutex.tryLock()) // if we can acquire the lock + doLockedSignalCompleted(cause, handled) + } + + override fun onCompleted(value: Unit) { + signalCompleted(null, false) + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + signalCompleted(cause, handled) + } +} + diff --git a/reactive/kotlinx-coroutines-rx3/src/RxScheduler.kt b/reactive/kotlinx-coroutines-rx3/src/RxScheduler.kt new file mode 100644 index 0000000000..8f77d4c867 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxScheduler.kt @@ -0,0 +1,177 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.* +import io.reactivex.rxjava3.plugins.* +import kotlinx.atomicfu.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import java.util.concurrent.* +import kotlin.coroutines.* + +/** + * Converts an instance of [Scheduler] to an implementation of [CoroutineDispatcher] + * and provides native support of [delay] and [withTimeout]. + */ +public fun Scheduler.asCoroutineDispatcher(): CoroutineDispatcher = + if (this is DispatcherScheduler) { + dispatcher + } else { + SchedulerCoroutineDispatcher(this) + } + +@Deprecated(level = DeprecationLevel.HIDDEN, message = "Since 1.4.2, binary compatibility with earlier versions") +@JvmName("asCoroutineDispatcher") +public fun Scheduler.asCoroutineDispatcher0(): SchedulerCoroutineDispatcher = + SchedulerCoroutineDispatcher(this) + +/** + * Converts an instance of [CoroutineDispatcher] to an implementation of [Scheduler]. + */ +public fun CoroutineDispatcher.asScheduler(): Scheduler = + if (this is SchedulerCoroutineDispatcher) { + scheduler + } else { + DispatcherScheduler(this) + } + +private class DispatcherScheduler(@JvmField val dispatcher: CoroutineDispatcher) : Scheduler() { + + private val schedulerJob = SupervisorJob() + + /** + * The scope for everything happening in this [DispatcherScheduler]. + * + * Running tasks, too, get launched under this scope, because [shutdown] should cancel the running tasks as well. + */ + private val scope = CoroutineScope(schedulerJob + dispatcher) + + /** + * The counter of created workers, for their pretty-printing. + */ + private val workerCounter = atomic(1L) + + override fun scheduleDirect(block: Runnable, delay: Long, unit: TimeUnit): Disposable = + scope.scheduleTask(block, unit.toMillis(delay)) { task -> + Runnable { scope.launch { task() } } + } + + override fun createWorker(): Worker = DispatcherWorker(workerCounter.getAndIncrement(), dispatcher, schedulerJob) + + override fun shutdown() { + schedulerJob.cancel() + } + + private class DispatcherWorker( + private val counter: Long, + private val dispatcher: CoroutineDispatcher, + parentJob: Job + ) : Worker() { + + private val workerJob = SupervisorJob(parentJob) + private val workerScope = CoroutineScope(workerJob + dispatcher) + private val blockChannel = Channel Unit>(Channel.UNLIMITED) + + init { + workerScope.launch { + blockChannel.consumeEach { + it() + } + } + } + + override fun schedule(block: Runnable, delay: Long, unit: TimeUnit): Disposable = + workerScope.scheduleTask(block, unit.toMillis(delay)) { task -> + Runnable { blockChannel.trySend(task) } + } + + override fun isDisposed(): Boolean = !workerScope.isActive + + override fun dispose() { + blockChannel.close() + workerJob.cancel() + } + + override fun toString(): String = "$dispatcher (worker $counter, ${if (isDisposed) "disposed" else "active"})" + } + + override fun toString(): String = dispatcher.toString() +} + +private typealias Task = suspend () -> Unit + +/** + * Schedule [block] so that an adapted version of it, wrapped in [adaptForScheduling], executes after [delayMillis] + * milliseconds. + */ +private fun CoroutineScope.scheduleTask( + block: Runnable, + delayMillis: Long, + adaptForScheduling: (Task) -> Runnable +): Disposable { + val ctx = coroutineContext + var handle: DisposableHandle? = null + val disposable = Disposable.fromRunnable { + // null if delay <= 0 + handle?.dispose() + } + val decoratedBlock = RxJavaPlugins.onSchedule(block) + suspend fun task() { + if (disposable.isDisposed) return + try { + runInterruptible { + decoratedBlock.run() + } + } catch (e: Throwable) { + handleUndeliverableException(e, ctx) + } + } + + val toSchedule = adaptForScheduling(::task) + if (!isActive) return Disposable.disposed() + if (delayMillis <= 0) { + toSchedule.run() + } else { + @Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") // do not remove the INVISIBLE_REFERENCE suppression: required in K2 + ctx.delay.invokeOnTimeout(delayMillis, toSchedule, ctx).let { handle = it } + } + return disposable +} + +/** + * Implements [CoroutineDispatcher] on top of an arbitrary [Scheduler]. + */ +public class SchedulerCoroutineDispatcher( + /** + * Underlying scheduler of current [CoroutineDispatcher]. + */ + public val scheduler: Scheduler +) : CoroutineDispatcher(), Delay { + /** @suppress */ + override fun dispatch(context: CoroutineContext, block: Runnable) { + scheduler.scheduleDirect(block) + } + + /** @suppress */ + override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation) { + val disposable = scheduler.scheduleDirect({ + with(continuation) { resumeUndispatched(Unit) } + }, timeMillis, TimeUnit.MILLISECONDS) + continuation.disposeOnCancellation(disposable) + } + + /** @suppress */ + override fun invokeOnTimeout(timeMillis: Long, block: Runnable, context: CoroutineContext): DisposableHandle { + val disposable = scheduler.scheduleDirect(block, timeMillis, TimeUnit.MILLISECONDS) + return DisposableHandle { disposable.dispose() } + } + + /** @suppress */ + override fun toString(): String = scheduler.toString() + + /** @suppress */ + override fun equals(other: Any?): Boolean = other is SchedulerCoroutineDispatcher && other.scheduler === scheduler + + /** @suppress */ + override fun hashCode(): Int = System.identityHashCode(scheduler) +} diff --git a/reactive/kotlinx-coroutines-rx3/src/RxSingle.kt b/reactive/kotlinx-coroutines-rx3/src/RxSingle.kt new file mode 100644 index 0000000000..b424a149bc --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/RxSingle.kt @@ -0,0 +1,57 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.* +import kotlin.coroutines.* + +/** + * Creates cold [single][Single] that will run a given [block] in a coroutine and emits its result. + * Every time the returned observable is subscribed, it starts a new coroutine. + * Unsubscribing cancels running coroutine. + * Coroutine context can be specified with [context] argument. + * If the context does not have any dispatcher nor any other [ContinuationInterceptor], then [Dispatchers.Default] is used. + * Method throws [IllegalArgumentException] if provided [context] contains a [Job] instance. + */ +public fun rxSingle( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> T +): Single { + require(context[Job] === null) { "Single context cannot contain job in it." + + "Its lifecycle should be managed via Disposable handle. Had $context" } + return rxSingleInternal(GlobalScope, context, block) +} + +private fun rxSingleInternal( + scope: CoroutineScope, // support for legacy rxSingle in scope + context: CoroutineContext, + block: suspend CoroutineScope.() -> T +): Single = Single.create { subscriber -> + val newContext = scope.newCoroutineContext(context) + val coroutine = RxSingleCoroutine(newContext, subscriber) + subscriber.setCancellable(RxCancellable(coroutine)) + coroutine.start(CoroutineStart.DEFAULT, coroutine, block) +} + +private class RxSingleCoroutine( + parentContext: CoroutineContext, + private val subscriber: SingleEmitter +) : AbstractCoroutine(parentContext, false, true) { + override fun onCompleted(value: T) { + try { + subscriber.onSuccess(value) + } catch (e: Throwable) { + handleUndeliverableException(e, context) + } + } + + override fun onCancelled(cause: Throwable, handled: Boolean) { + try { + if (subscriber.tryOnError(cause)) { + return + } + } catch (e: Throwable) { + cause.addSuppressed(e) + } + handleUndeliverableException(cause, context) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/src/module-info.java b/reactive/kotlinx-coroutines-rx3/src/module-info.java new file mode 100644 index 0000000000..d57d5279d8 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/src/module-info.java @@ -0,0 +1,10 @@ +@SuppressWarnings("JavaModuleNaming") +module kotlinx.coroutines.rx3 { + requires kotlin.stdlib; + requires kotlinx.coroutines.core; + requires kotlinx.coroutines.reactive; + requires kotlinx.atomicfu; + requires io.reactivex.rxjava3; + + exports kotlinx.coroutines.rx3; +} diff --git a/reactive/kotlinx-coroutines-rx3/test/BackpressureTest.kt b/reactive/kotlinx-coroutines-rx3/test/BackpressureTest.kt new file mode 100644 index 0000000000..bfffe15f84 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/BackpressureTest.kt @@ -0,0 +1,36 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import kotlin.test.* + +class BackpressureTest : TestBase() { + @Test + fun testBackpressureDropDirect() = runTest { + expect(1) + Flowable.fromArray(1) + .onBackpressureDrop() + .collect { + assertEquals(1, it) + expect(2) + } + finish(3) + } + + @Test + fun testBackpressureDropFlow() = runTest { + expect(1) + Flowable.fromArray(1) + .onBackpressureDrop() + .asFlow() + .collect { + assertEquals(1, it) + expect(2) + } + finish(3) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/Check.kt b/reactive/kotlinx-coroutines-rx3/test/Check.kt new file mode 100644 index 0000000000..c1e6d2ed00 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/Check.kt @@ -0,0 +1,72 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.plugins.* + +fun checkSingleValue( + observable: Observable, + checker: (T) -> Unit +) { + val singleValue = observable.blockingSingle() + checker(singleValue) +} + +fun checkErroneous( + observable: Observable<*>, + checker: (Throwable) -> Unit +) { + val singleNotification = observable.materialize().blockingSingle() + val error = singleNotification.error ?: error("Excepted error") + checker(error) +} + +fun checkSingleValue( + single: Single, + checker: (T) -> Unit +) { + val singleValue = single.blockingGet() + checker(singleValue) +} + +fun checkErroneous( + single: Single<*>, + checker: (Throwable) -> Unit +) { + try { + single.blockingGet() + error("Should have failed") + } catch (e: Throwable) { + checker(e) + } +} + +fun checkMaybeValue( + maybe: Maybe, + checker: (T?) -> Unit +) { + val maybeValue = maybe.toFlowable().blockingIterable().firstOrNull() + checker(maybeValue) +} + +@Suppress("UNCHECKED_CAST") +fun checkErroneous( + maybe: Maybe<*>, + checker: (Throwable) -> Unit +) { + try { + (maybe as Maybe).blockingGet() + error("Should have failed") + } catch (e: Throwable) { + checker(e) + } +} + +inline fun withExceptionHandler(noinline handler: (Throwable) -> Unit, block: () -> Unit) { + val original = RxJavaPlugins.getErrorHandler() + RxJavaPlugins.setErrorHandler { handler(it) } + try { + block() + } finally { + RxJavaPlugins.setErrorHandler(original) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/CompletableTest.kt b/reactive/kotlinx-coroutines-rx3/test/CompletableTest.kt new file mode 100644 index 0000000000..b9355cce10 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/CompletableTest.kt @@ -0,0 +1,202 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.* +import io.reactivex.rxjava3.exceptions.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class CompletableTest : TestBase() { + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(4) + } + expect(2) + completable.subscribe { + expect(5) + } + expect(3) + yield() // to completable coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + completable.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to completable coroutine + finish(6) + } + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + // nothing is called on a disposed rx3 completable + val sub = completable.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testAwaitSuccess() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(3) + } + expect(2) + completable.await() // shall launch coroutine + finish(4) + } + + @Test + fun testAwaitFailure() = runBlocking { + expect(1) + val completable = rxCompletable(currentDispatcher()) { + expect(3) + throw RuntimeException("OK") + } + expect(2) + try { + completable.await() // shall launch coroutine and throw exception + expectUnreached() + } catch (e: RuntimeException) { + finish(4) + assertEquals("OK", e.message) + } + } + + /** Tests that calls to [await] throw [CancellationException] and dispose of the subscription when their [Job] is + * cancelled. */ + @Test + fun testAwaitCancellation() = runTest { + expect(1) + val completable = CompletableSource { s -> + s.onSubscribe(object: Disposable { + override fun dispose() { expect(4) } + override fun isDisposed(): Boolean { expectUnreached(); return false } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + completable.await() + } catch (e: CancellationException) { + expect(5) + throw e + } + } + expect(3) + job.cancelAndJoin() + finish(6) + } + + @Test + fun testSuppressedException() = runTest { + val completable = rxCompletable(currentDispatcher()) { + launch(start = CoroutineStart.ATOMIC) { + throw TestException() // child coroutine fails + } + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() // but parent throws another exception while cleaning up + } + } + try { + completable.await() + expectUnreached() + } catch (e: TestException) { + assertIs(e.suppressed[0]) + } + } + + @Test + fun testUnhandledException() = runTest { + expect(1) + var disposable: Disposable? = null + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is TestException) + expect(5) + } + val completable = rxCompletable(currentDispatcher()) { + expect(4) + disposable!!.dispose() // cancel our own subscription, so that delay will get cancelled + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException() // would not be able to handle it since mono is disposed + } + } + withExceptionHandler(handler) { + completable.subscribe(object : CompletableObserver { + override fun onSubscribe(d: Disposable) { + expect(2) + disposable = d + } + + override fun onComplete() { + expectUnreached() + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + expect(3) + yield() // run coroutine + finish(6) + } + } + + @Test + fun testFatalExceptionInSubscribe() = runTest { + val handler: (Throwable) -> Unit = { e -> + assertTrue(e is UndeliverableException && e.cause is LinkageError); expect(2) + } + + withExceptionHandler(handler) { + rxCompletable(Dispatchers.Unconfined) { + expect(1) + }.subscribe { throw LinkageError() } + finish(3) + } + } + + @Test + fun testFatalExceptionInSingle() = runTest { + rxCompletable(Dispatchers.Unconfined) { + throw LinkageError() + }.subscribe({ expectUnreached() }, { expect(1); assertIs(it) }) + finish(2) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/ConvertTest.kt b/reactive/kotlinx-coroutines-rx3/test/ConvertTest.kt new file mode 100644 index 0000000000..5013bb9da4 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ConvertTest.kt @@ -0,0 +1,156 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.consumeAsFlow +import org.junit.Assert +import org.junit.Test +import kotlin.test.* + +class ConvertTest : TestBase() { + @Test + fun testToCompletableSuccess() = runBlocking { + expect(1) + val job = launch { + expect(3) + } + val completable = job.asCompletable(coroutineContext.minusKey(Job)) + completable.subscribe { + expect(4) + } + expect(2) + yield() + finish(5) + } + + @Test + fun testToCompletableFail() = runBlocking { + expect(1) + val job = async(NonCancellable) { // don't kill parent on exception + expect(3) + throw RuntimeException("OK") + } + val completable = job.asCompletable(coroutineContext.minusKey(Job)) + completable.subscribe { + expect(4) + } + expect(2) + yield() + finish(5) + } + + @Test + fun testToMaybe() { + val d = GlobalScope.async { + delay(50) + "OK" + } + val maybe1 = d.asMaybe(Dispatchers.Unconfined) + checkMaybeValue(maybe1) { + assertEquals("OK", it) + } + val maybe2 = d.asMaybe(Dispatchers.Unconfined) + checkMaybeValue(maybe2) { + assertEquals("OK", it) + } + } + + @Test + fun testToMaybeEmpty() { + val d = GlobalScope.async { + delay(50) + null + } + val maybe1 = d.asMaybe(Dispatchers.Unconfined) + checkMaybeValue(maybe1, Assert::assertNull) + val maybe2 = d.asMaybe(Dispatchers.Unconfined) + checkMaybeValue(maybe2, Assert::assertNull) + } + + @Test + fun testToMaybeFail() { + val d = GlobalScope.async { + delay(50) + throw TestRuntimeException("OK") + } + val maybe1 = d.asMaybe(Dispatchers.Unconfined) + checkErroneous(maybe1) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + val maybe2 = d.asMaybe(Dispatchers.Unconfined) + checkErroneous(maybe2) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + } + + @Test + fun testToSingle() { + val d = GlobalScope.async { + delay(50) + "OK" + } + val single1 = d.asSingle(Dispatchers.Unconfined) + checkSingleValue(single1) { + assertEquals("OK", it) + } + val single2 = d.asSingle(Dispatchers.Unconfined) + checkSingleValue(single2) { + assertEquals("OK", it) + } + } + + @Test + fun testToSingleFail() { + val d = GlobalScope.async { + delay(50) + throw TestRuntimeException("OK") + } + val single1 = d.asSingle(Dispatchers.Unconfined) + checkErroneous(single1) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + val single2 = d.asSingle(Dispatchers.Unconfined) + checkErroneous(single2) { + check(it is TestRuntimeException && it.message == "OK") { "$it" } + } + } + + @Test + fun testToObservable() { + val c = GlobalScope.produce { + delay(50) + send("O") + delay(50) + send("K") + } + val observable = c.consumeAsFlow().asObservable() + checkSingleValue(observable.reduce { t1, t2 -> t1 + t2 }.toSingle()) { + assertEquals("OK", it) + } + } + + @Test + fun testToObservableFail() { + val c = GlobalScope.produce { + delay(50) + send("O") + delay(50) + throw TestException("K") + } + val observable = c.consumeAsFlow().asObservable() + val single = rxSingle(Dispatchers.Unconfined) { + var result = "" + try { + observable.collect { result += it } + } catch(e: Throwable) { + check(e is TestException) + result += e.message + } + result + } + checkSingleValue(single) { + assertEquals("OK", it) + } + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/FlowAsFlowableTest.kt b/reactive/kotlinx-coroutines-rx3/test/FlowAsFlowableTest.kt new file mode 100644 index 0000000000..cc565c832a --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/FlowAsFlowableTest.kt @@ -0,0 +1,86 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.junit.Test +import org.reactivestreams.* +import java.util.concurrent.* +import kotlin.test.* + +@Suppress("ReactiveStreamsSubscriberImplementation") +class FlowAsFlowableTest : TestBase() { + @Test + fun testUnconfinedDefaultContext() { + expect(1) + val thread = Thread.currentThread() + fun checkThread() { + assertSame(thread, Thread.currentThread()) + } + flowOf(42).asFlowable().subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onSubscribe(s: Subscription) { + expect(2) + subscription = s + subscription.request(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + finish(5) + } + + @Test + fun testConfinedContext() { + expect(1) + val threadName = "FlowAsFlowableTest.testConfinedContext" + fun checkThread() { + val currentThread = Thread.currentThread() + assertTrue(currentThread.name.startsWith(threadName), "Unexpected thread $currentThread") + } + val completed = CountDownLatch(1) + newSingleThreadContext(threadName).use { dispatcher -> + flowOf(42).asFlowable(dispatcher).subscribe(object : Subscriber { + private lateinit var subscription: Subscription + + override fun onSubscribe(s: Subscription) { + expect(2) + subscription = s + subscription.request(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + completed.countDown() + } + + override fun onError(t: Throwable?) { + expectUnreached() + } + }) + completed.await() + } + finish(5) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/FlowAsObservableTest.kt b/reactive/kotlinx-coroutines-rx3/test/FlowAsObservableTest.kt new file mode 100644 index 0000000000..8bacb2f249 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/FlowAsObservableTest.kt @@ -0,0 +1,208 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class FlowAsObservableTest : TestBase() { + @Test + fun testBasicSuccess() = runTest { + expect(1) + val observable = flow { + expect(3) + emit("OK") + }.asObservable() + + expect(2) + observable.subscribe { value -> + expect(4) + assertEquals("OK", value) + } + + finish(5) + } + + @Test + fun testBasicFailure() = runTest { + expect(1) + val observable = flow { + expect(3) + throw RuntimeException("OK") + }.asObservable() + + expect(2) + observable.subscribe({ expectUnreached() }, { error -> + expect(4) + assertIs(error) + assertEquals("OK", error.message) + }) + finish(5) + } + + @Test + fun testBasicUnsubscribe() = runTest { + expect(1) + val observable = flow { + expect(3) + hang { + expect(4) + } + }.asObservable() + + expect(2) + val sub = observable.subscribe({ expectUnreached() }, { expectUnreached() }) + sub.dispose() // will cancel coroutine + finish(5) + } + + @Test + fun testNotifyOnceOnCancellation() = runTest { + val observable = + flow { + expect(3) + emit("OK") + hang { + expect(7) + } + }.asObservable() + .doOnNext { + expect(4) + assertEquals("OK", it) + } + .doOnDispose { + expect(6) // notified once! + } + + expect(1) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(2) + observable.collect { + expect(5) + assertEquals("OK", it) + } + } + + yield() + job.cancelAndJoin() + finish(8) + } + + @Test + fun testFailingConsumer() = runTest { + expect(1) + val observable = flow { + expect(2) + emit("OK") + hang { + expect(4) + } + + }.asObservable() + + try { + observable.collect { + expect(3) + throw TestException() + } + } catch (e: TestException) { + finish(5) + } + } + + @Test + fun testNonAtomicStart() = runTest { + withContext(Dispatchers.Unconfined) { + val observable = flow { + expect(1) + }.asObservable() + + val disposable = observable.subscribe({ expectUnreached() }, { expectUnreached() }, { expectUnreached() }) + disposable.dispose() + } + finish(2) + } + + @Test + fun testFlowCancelledFromWithin() = runTest { + val observable = flow { + expect(1) + emit(1) + kotlin.coroutines.coroutineContext.cancel() + kotlin.coroutines.coroutineContext.ensureActive() + expectUnreached() + }.asObservable() + + observable.subscribe({ expect(2) }, { expectUnreached() }, { finish(3) }) + } + + @Test + fun testUnconfinedDefaultContext() { + expect(1) + val thread = Thread.currentThread() + fun checkThread() { + assertSame(thread, Thread.currentThread()) + } + flowOf(42).asObservable().subscribe(object : Observer { + override fun onSubscribe(d: Disposable) { + expect(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + finish(5) + } + + @Test + fun testConfinedContext() { + expect(1) + val threadName = "FlowAsObservableTest.testConfinedContext" + fun checkThread() { + val currentThread = Thread.currentThread() + assertTrue(currentThread.name.startsWith(threadName), "Unexpected thread $currentThread") + } + val completed = CountDownLatch(1) + newSingleThreadContext(threadName).use { dispatcher -> + flowOf(42).asObservable(dispatcher).subscribe(object : Observer { + override fun onSubscribe(d: Disposable) { + expect(2) + } + + override fun onNext(t: Int) { + checkThread() + expect(3) + assertEquals(42, t) + } + + override fun onComplete() { + checkThread() + expect(4) + completed.countDown() + } + + override fun onError(e: Throwable) { + expectUnreached() + } + }) + completed.await() + } + finish(5) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/FlowableContextTest.kt b/reactive/kotlinx-coroutines-rx3/test/FlowableContextTest.kt new file mode 100644 index 0000000000..5e87d71bb9 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/FlowableContextTest.kt @@ -0,0 +1,40 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class FlowableContextTest : TestBase() { + private val dispatcher = newSingleThreadContext("FlowableContextTest") + + @After + fun tearDown() { + dispatcher.close() + } + + @Test + fun testFlowableCreateAsFlowThread() = runTest { + expect(1) + val mainThread = Thread.currentThread() + val dispatcherThread = withContext(dispatcher) { Thread.currentThread() } + assertTrue(dispatcherThread != mainThread) + Flowable.create({ + assertEquals(dispatcherThread, Thread.currentThread()) + it.onNext("OK") + it.onComplete() + }, BackpressureStrategy.BUFFER) + .asFlow() + .flowOn(dispatcher) + .collect { + expect(2) + assertEquals("OK", it) + assertEquals(mainThread, Thread.currentThread()) + } + finish(3) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/FlowableExceptionHandlingTest.kt b/reactive/kotlinx-coroutines-rx3/test/FlowableExceptionHandlingTest.kt new file mode 100644 index 0000000000..d250de9166 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/FlowableExceptionHandlingTest.kt @@ -0,0 +1,131 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.exceptions.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import kotlin.test.* + +class FlowableExceptionHandlingTest : TestBase() { + + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + private inline fun handler(expect: Int) = { t: Throwable -> + assertTrue(t is UndeliverableException && t.cause is T) + expect(expect) + } + + private fun cehUnreached() = CoroutineExceptionHandler { _, _ -> expectUnreached() } + + @Test + fun testException() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + throw TestException() + }.subscribe({ + expectUnreached() + }, { + expect(2) // Reported to onError + }) + finish(3) + } + + @Test + fun testFatalException() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined) { + expect(1) + throw LinkageError() + }.subscribe({ + expectUnreached() + }, { + expect(2) // Fatal exceptions are not treated as special + }) + finish(3) + } + + @Test + fun testExceptionAsynchronous() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + throw TestException() + }.publish() + .refCount() + .subscribe({ + expectUnreached() + }, { + expect(2) // Reported to onError + }) + finish(3) + } + + @Test + fun testFatalExceptionAsynchronous() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined) { + expect(1) + throw LinkageError() + }.publish() + .refCount() + .subscribe({ + expectUnreached() + }, { + expect(2) + }) + finish(3) + } + + @Test + fun testFatalExceptionFromSubscribe() = withExceptionHandler(handler(3)) { + rxFlowable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.subscribe({ + expect(2) + throw LinkageError() + }, { expectUnreached() }) // Fatal exception is rethrown from `onNext` => the subscription is thought to be cancelled + finish(4) + } + + @Test + fun testExceptionFromSubscribe() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + send(Unit) + }.subscribe({ + expect(2) + throw TestException() + }, { expect(3) }) // not reported to onError because came from the subscribe itself + finish(4) + } + + @Test + fun testAsynchronousExceptionFromSubscribe() = withExceptionHandler({ expectUnreached() }) { + rxFlowable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + send(Unit) + }.publish() + .refCount() + .subscribe({ + expect(2) + throw RuntimeException() + }, { expect(3) }) + finish(4) + } + + @Test + fun testAsynchronousFatalExceptionFromSubscribe() = withExceptionHandler(handler(3)) { + rxFlowable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.publish() + .refCount() + .subscribe({ + expect(2) + throw LinkageError() + }, { expectUnreached() }) + finish(4) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/FlowableTest.kt b/reactive/kotlinx-coroutines-rx3/test/FlowableTest.kt new file mode 100644 index 0000000000..2460bd6472 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/FlowableTest.kt @@ -0,0 +1,123 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import kotlin.test.* + +class FlowableTest : TestBase() { + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val observable = rxFlowable(currentDispatcher()) { + expect(4) + send("OK") + } + expect(2) + observable.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val observable = rxFlowable(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + observable.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val observable = rxFlowable(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + val sub = observable.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testNotifyOnceOnCancellation() = runTest { + expect(1) + val observable = + rxFlowable(currentDispatcher()) { + expect(5) + send("OK") + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + expect(11) + } + } + .doOnNext { + expect(6) + assertEquals("OK", it) + } + .doOnCancel { + expect(10) // notified once! + } + expect(2) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(3) + observable.collect { + expect(8) + assertEquals("OK", it) + } + } + expect(4) + yield() // to observable code + expect(7) + yield() // to consuming coroutines + expect(9) + job.cancel() + job.join() + finish(12) + } + + @Test + fun testFailingConsumer() = runTest { + val pub = rxFlowable(currentDispatcher()) { + repeat(3) { + expect(it + 1) // expect(1), expect(2) *should* be invoked + send(it) + } + } + try { + pub.collect { + throw TestException() + } + } catch (e: TestException) { + finish(3) + } + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/IntegrationTest.kt b/reactive/kotlinx-coroutines-rx3/test/IntegrationTest.kt new file mode 100644 index 0000000000..f51d7cd628 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/IntegrationTest.kt @@ -0,0 +1,148 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.consumeAsFlow +import org.junit.Test +import org.junit.runner.* +import org.junit.runners.* +import kotlin.coroutines.* +import kotlin.test.* + +@RunWith(Parameterized::class) +class IntegrationTest( + private val ctx: Ctx, + private val delay: Boolean +) : TestBase() { + + enum class Ctx { + MAIN { override fun invoke(context: CoroutineContext): CoroutineContext = context.minusKey(Job) }, + DEFAULT { override fun invoke(context: CoroutineContext): CoroutineContext = Dispatchers.Default }, + UNCONFINED { override fun invoke(context: CoroutineContext): CoroutineContext = Dispatchers.Unconfined }; + + abstract operator fun invoke(context: CoroutineContext): CoroutineContext + } + + companion object { + @Parameterized.Parameters(name = "ctx={0}, delay={1}") + @JvmStatic + fun params(): Collection> = Ctx.values().flatMap { ctx -> + listOf(false, true).map { delay -> + arrayOf(ctx, delay) + } + } + } + + @Test + fun testEmpty(): Unit = runBlocking { + val observable = rxObservable(ctx(coroutineContext)) { + if (delay) delay(1) + // does not send anything + } + assertFailsWith { observable.awaitFirst() } + assertEquals("OK", observable.awaitFirstOrDefault("OK")) + assertNull(observable.awaitFirstOrNull()) + assertEquals("ELSE", observable.awaitFirstOrElse { "ELSE" }) + assertFailsWith { observable.awaitLast() } + assertFailsWith { observable.awaitSingle() } + var cnt = 0 + observable.collect { + cnt++ + } + assertEquals(0, cnt) + } + + @Test + fun testSingle() = runBlocking { + val observable = rxObservable(ctx(coroutineContext)) { + if (delay) delay(1) + send("OK") + } + assertEquals("OK", observable.awaitFirst()) + assertEquals("OK", observable.awaitFirstOrDefault("OK")) + assertEquals("OK", observable.awaitFirstOrNull()) + assertEquals("OK", observable.awaitFirstOrElse { "ELSE" }) + assertEquals("OK", observable.awaitLast()) + assertEquals("OK", observable.awaitSingle()) + var cnt = 0 + observable.collect { + assertEquals("OK", it) + cnt++ + } + assertEquals(1, cnt) + } + + @Test + fun testNumbers() = runBlocking { + val n = 100 * stressTestMultiplier + val observable = rxObservable(ctx(coroutineContext)) { + for (i in 1..n) { + send(i) + if (delay) delay(1) + } + } + assertEquals(1, observable.awaitFirst()) + assertEquals(1, observable.awaitFirstOrDefault(0)) + assertEquals(1, observable.awaitFirstOrNull()) + assertEquals(1, observable.awaitFirstOrElse { 0 }) + assertEquals(n, observable.awaitLast()) + assertFailsWith { observable.awaitSingle() } + checkNumbers(n, observable) + val channel = observable.openSubscription() + ctx(coroutineContext) + checkNumbers(n, channel.consumeAsFlow().asObservable()) + channel.cancel() + } + + @Test + fun testCancelWithoutValue() = runTest { + val job = launch(Job(), start = CoroutineStart.UNDISPATCHED) { + rxObservable { + hang { } + }.awaitFirst() + } + + job.cancel() + job.join() + } + + @Test + fun testEmptySingle() = runTest(unhandled = listOf({e -> e is NoSuchElementException})) { + expect(1) + val job = launch(Job(), start = CoroutineStart.UNDISPATCHED) { + rxObservable { + yield() + expect(2) + // Nothing to emit + }.awaitFirst() + } + + job.join() + finish(3) + } + + @Test + fun testObservableWithTimeout() = runTest { + val observable = rxObservable { + expect(2) + withTimeout(1) { delay(100) } + } + try { + expect(1) + observable.awaitFirstOrNull() + } catch (e: CancellationException) { + expect(3) + } + finish(4) + } + + private suspend fun checkNumbers(n: Int, observable: Observable) { + var last = 0 + observable.collect { + assertEquals(++last, it) + } + assertEquals(n, last) + } + +} diff --git a/reactive/kotlinx-coroutines-rx3/test/IterableFlowAsFlowableTckTest.kt b/reactive/kotlinx-coroutines-rx3/test/IterableFlowAsFlowableTckTest.kt new file mode 100644 index 0000000000..295f055ee7 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/IterableFlowAsFlowableTckTest.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.rx3 + +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.flow.* +import org.junit.* +import org.reactivestreams.* +import org.reactivestreams.tck.* + +class IterableFlowAsFlowableTckTest : PublisherVerification(TestEnvironment()) { + + private fun generate(num: Long): Array { + return Array(if (num >= Integer.MAX_VALUE) 1000000 else num.toInt()) { it.toLong() } + } + + override fun createPublisher(elements: Long): Flowable { + return generate(elements).asIterable().asFlow().asFlowable() + } + + override fun createFailedPublisher(): Publisher? = null + + @Ignore + override fun required_spec309_requestZeroMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec309_requestNegativeNumberMustSignalIllegalArgumentException() { + } + + @Ignore + override fun required_spec312_cancelMustMakeThePublisherToEventuallyStopSignaling() { + // + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/LeakedExceptionTest.kt b/reactive/kotlinx-coroutines-rx3/test/LeakedExceptionTest.kt new file mode 100644 index 0000000000..5f9caf5c45 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/LeakedExceptionTest.kt @@ -0,0 +1,104 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.exceptions.* +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.reactive.* +import org.junit.Test +import java.util.concurrent.Executors +import java.util.concurrent.TimeUnit +import kotlin.test.* + +// Check that exception is not leaked to the global exception handler +class LeakedExceptionTest : TestBase() { + + private val handler: (Throwable) -> Unit = + { assertTrue { it is UndeliverableException && it.cause is TestException } } + + @Test + fun testSingle() = withExceptionHandler(handler) { + withFixedThreadPool(4) { dispatcher -> + val flow = rxSingle(dispatcher) { throw TestException() }.toFlowable().asFlow() + runBlocking { + repeat(10000) { + combine(flow, flow) { _, _ -> Unit } + .catch {} + .collect {} + } + } + } + } + + @Test + fun testObservable() = withExceptionHandler(handler) { + withFixedThreadPool(4) { dispatcher -> + val flow = rxObservable(dispatcher) { throw TestException() } + .toFlowable(BackpressureStrategy.BUFFER) + .asFlow() + runBlocking { + repeat(10000) { + combine(flow, flow) { _, _ -> Unit } + .catch {} + .collect {} + } + } + } + } + + @Test + fun testFlowable() = withExceptionHandler(handler) { + withFixedThreadPool(4) { dispatcher -> + val flow = rxFlowable(dispatcher) { throw TestException() }.asFlow() + runBlocking { + repeat(10000) { + combine(flow, flow) { _, _ -> Unit } + .catch {} + .collect {} + } + } + } + } + + /** + * This test doesn't test much and was added to display a problem with straighforward use of + * [withExceptionHandler]. + * + * If one was to remove `dispatcher` and launch `rxFlowable` with an empty coroutine context, + * this test would fail fairly often, while other tests were also vulnerable, but the problem is + * much more difficult to reproduce. Thus, this test is a justification for adding `dispatcher` + * to other tests. + * + * See the commit that introduced this test for a better explanation. + */ + @Test + fun testResettingExceptionHandler() = withExceptionHandler(handler) { + withFixedThreadPool(4) { dispatcher -> + val flow = rxFlowable(dispatcher) { + if ((0..1).random() == 0) { + Thread.sleep(100) + } + throw TestException() + }.asFlow() + runBlocking { + combine(flow, flow) { _, _ -> Unit } + .catch {} + .collect {} + } + } + } + + /** + * Run in a thread pool, then wait for all the tasks to finish. + */ + private fun withFixedThreadPool(numberOfThreads: Int, block: (CoroutineDispatcher) -> Unit) { + val pool = Executors.newFixedThreadPool(numberOfThreads) + val dispatcher = pool.asCoroutineDispatcher() + block(dispatcher) + pool.shutdown() + while (!pool.awaitTermination(10, TimeUnit.SECONDS)) { + /* deliberately empty */ + } + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/MaybeTest.kt b/reactive/kotlinx-coroutines-rx3/test/MaybeTest.kt new file mode 100644 index 0000000000..3746d43d87 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/MaybeTest.kt @@ -0,0 +1,389 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.* +import io.reactivex.rxjava3.exceptions.* +import io.reactivex.rxjava3.internal.functions.Functions.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class MaybeTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + "OK" + } + expect(2) + maybe.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicEmpty() = runBlocking { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + null + } + expect(2) + maybe.subscribe (emptyConsumer(), ON_ERROR_MISSING, { + expect(5) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + maybe.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + // nothing is called on a disposed rx2 maybe + val sub = maybe.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testMaybeNoWait() { + val maybe = rxMaybe { + "OK" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testMaybeAwait() = runBlocking { + assertEquals("OK", Maybe.just("O").awaitSingleOrNull() + "K") + assertEquals("OK", Maybe.just("O").awaitSingle() + "K") + } + + @Test + fun testMaybeAwaitForNull(): Unit = runBlocking { + assertNull(Maybe.empty().awaitSingleOrNull()) + assertFailsWith { Maybe.empty().awaitSingle() } + } + + /** Tests that calls to [awaitSingleOrNull] throw [CancellationException] and dispose of the subscription when their + * [Job] is cancelled. */ + @Test + fun testMaybeAwaitCancellation() = runTest { + expect(1) + val maybe = MaybeSource { s -> + s.onSubscribe(object: Disposable { + override fun dispose() { expect(4) } + override fun isDisposed(): Boolean { expectUnreached(); return false } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + maybe.awaitSingleOrNull() + } catch (e: CancellationException) { + expect(5) + throw e + } + } + expect(3) + job.cancelAndJoin() + finish(6) + } + + @Test + fun testMaybeEmitAndAwait() { + val maybe = rxMaybe { + Maybe.just("O").awaitSingleOrNull() + "K" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testMaybeWithDelay() { + val maybe = rxMaybe { + Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testMaybeException() { + val maybe = rxMaybe { + Observable.just("O", "K").awaitSingle() + "K" + } + + checkErroneous(maybe) { + assert(it is IllegalArgumentException) + } + } + + @Test + fun testAwaitFirst() { + val maybe = rxMaybe { + Observable.just("O", "#").awaitFirst() + "K" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitLast() { + val maybe = rxMaybe { + Observable.just("#", "O").awaitLast() + "K" + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromObservable() { + val maybe = rxMaybe { + try { + Observable.error(RuntimeException("O")).awaitFirst() + } catch (e: RuntimeException) { + Observable.just(e.message!!).awaitLast() + "K" + } + } + + checkMaybeValue(maybe) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromCoroutine() { + val maybe = rxMaybe { + throw IllegalStateException(Observable.just("O").awaitSingle() + "K") + } + + checkErroneous(maybe) { + assert(it is IllegalStateException) + assertEquals("OK", it.message) + } + } + + @Test + fun testCancelledConsumer() = runTest { + expect(1) + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + expect(6) + } + 42 + } + expect(2) + val timeout = withTimeoutOrNull(100) { + expect(3) + maybe.collect { + expectUnreached() + } + expectUnreached() + } + assertNull(timeout) + expect(5) + yield() // must cancel code inside maybe!!! + finish(7) + } + + /** Tests the simple scenario where the Maybe doesn't output a value. */ + @Test + fun testMaybeCollectEmpty() = runTest { + expect(1) + Maybe.empty().collect { + expectUnreached() + } + finish(2) + } + + /** Tests the simple scenario where the Maybe doesn't output a value. */ + @Test + fun testMaybeCollectSingle() = runTest { + expect(1) + Maybe.just("OK").collect { + assertEquals("OK", it) + expect(2) + } + finish(3) + } + + /** Tests the behavior of [collect] when the Maybe raises an error. */ + @Test + fun testMaybeCollectThrowingMaybe() = runTest { + expect(1) + try { + Maybe.error(TestException()).collect { + expectUnreached() + } + } catch (e: TestException) { + expect(2) + } + finish(3) + } + + /** Tests the behavior of [collect] when the action throws. */ + @Test + fun testMaybeCollectThrowingAction() = runTest { + expect(1) + try { + Maybe.just("OK").collect { + expect(2) + throw TestException() + } + } catch (e: TestException) { + expect(3) + } + finish(4) + } + + @Test + fun testSuppressedException() = runTest { + val maybe = rxMaybe(currentDispatcher()) { + launch(start = CoroutineStart.ATOMIC) { + throw TestException() // child coroutine fails + } + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() // but parent throws another exception while cleaning up + } + } + try { + maybe.awaitSingleOrNull() + expectUnreached() + } catch (e: TestException) { + assertIs(e.suppressed[0]) + } + } + + @Test + fun testUnhandledException() = runTest { + expect(1) + var disposable: Disposable? = null + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is TestException) + expect(5) + } + val maybe = rxMaybe(currentDispatcher()) { + expect(4) + disposable!!.dispose() // cancel our own subscription, so that delay will get cancelled + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException() // would not be able to handle it since mono is disposed + } + } + withExceptionHandler(handler) { + maybe.subscribe(object : MaybeObserver { + override fun onSubscribe(d: Disposable) { + expect(2) + disposable = d + } + + override fun onComplete() { + expectUnreached() + } + + override fun onSuccess(t: Unit) { + expectUnreached() + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + expect(3) + yield() // run coroutine + finish(6) + } + } + + @Test + fun testFatalExceptionInSubscribe() = runTest { + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is LinkageError) + expect(2) + } + + withExceptionHandler(handler) { + rxMaybe(Dispatchers.Unconfined) { + expect(1) + 42 + }.subscribe { throw LinkageError() } + finish(3) + } + } + + @Test + fun testFatalExceptionInSingle() = runTest { + rxMaybe(Dispatchers.Unconfined) { + throw LinkageError() + }.subscribe({ expectUnreached() }, { expect(1); assertIs(it) }) + finish(2) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/ObservableAsFlowTest.kt b/reactive/kotlinx-coroutines-rx3/test/ObservableAsFlowTest.kt new file mode 100644 index 0000000000..79d7052553 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ObservableAsFlowTest.kt @@ -0,0 +1,183 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.Observable +import io.reactivex.rxjava3.core.ObservableSource +import io.reactivex.rxjava3.core.Observer +import io.reactivex.rxjava3.disposables.Disposable +import io.reactivex.rxjava3.subjects.PublishSubject +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.testing.flow.* +import kotlin.test.* + +class ObservableAsFlowTest : TestBase() { + @Test + fun testCancellation() = runTest { + var onNext = 0 + var onCancelled = 0 + var onError = 0 + + val source = rxObservable(currentDispatcher()) { + coroutineContext[Job]?.invokeOnCompletion { + if (it is CancellationException) ++onCancelled + } + + repeat(100) { + send(it) + } + } + + source.asFlow().launchIn(CoroutineScope(Dispatchers.Unconfined)) { + onEach { + ++onNext + throw RuntimeException() + } + catch { + ++onError + } + }.join() + + + assertEquals(1, onNext) + assertEquals(1, onError) + assertEquals(1, onCancelled) + } + + @Test + fun testImmediateCollection() { + val source = PublishSubject.create() + val flow = source.asFlow() + GlobalScope.launch(Dispatchers.Unconfined) { + expect(1) + flow.collect { expect(it) } + expect(6) + } + expect(2) + source.onNext(3) + expect(4) + source.onNext(5) + source.onComplete() + finish(7) + } + + @Test + fun testOnErrorCancellation() { + val source = PublishSubject.create() + val flow = source.asFlow() + val exception = RuntimeException() + GlobalScope.launch(Dispatchers.Unconfined) { + try { + expect(1) + flow.collect { expect(it) } + expectUnreached() + } + catch (e: Exception) { + assertSame(exception, e.cause) + expect(5) + } + expect(6) + } + expect(2) + source.onNext(3) + expect(4) + source.onError(exception) + finish(7) + } + + @Test + fun testUnsubscribeOnCollectionException() { + val source = PublishSubject.create() + val flow = source.asFlow() + val exception = RuntimeException() + GlobalScope.launch(Dispatchers.Unconfined) { + try { + expect(1) + flow.collect { + expect(it) + if (it == 3) throw exception + } + expectUnreached() + } + catch (e: Exception) { + assertSame(exception, e.cause) + expect(4) + } + expect(5) + } + expect(2) + assertTrue(source.hasObservers()) + source.onNext(3) + assertFalse(source.hasObservers()) + finish(6) + } + + @Test + fun testLateOnSubscribe() { + var observer: Observer? = null + val source = ObservableSource { observer = it } + val flow = source.asFlow() + assertNull(observer) + val job = GlobalScope.launch(Dispatchers.Unconfined) { + expect(1) + flow.collect { expectUnreached() } + expectUnreached() + } + expect(2) + assertNotNull(observer) + job.cancel() + val disposable = Disposable.empty() + observer!!.onSubscribe(disposable) + assertTrue(disposable.isDisposed) + finish(3) + } + + @Test + fun testBufferUnlimited() = runTest { + val source = rxObservable(currentDispatcher()) { + expect(1); send(10) + expect(2); send(11) + expect(3); send(12) + expect(4); send(13) + expect(5); send(14) + expect(6); send(15) + expect(7); send(16) + expect(8); send(17) + expect(9) + } + source.asFlow().buffer(Channel.UNLIMITED).collect { expect(it) } + finish(18) + } + + @Test + fun testConflated() = runTest { + val source = Observable.range(1, 5) + val list = source.asFlow().conflate().toList() + assertEquals(listOf(1, 5), list) + } + + @Test + fun testLongRange() = runTest { + val source = Observable.range(1, 10_000) + val count = source.asFlow().count() + assertEquals(10_000, count) + } + + @Test + fun testProduce() = runTest { + val source = Observable.range(0, 10) + val flow = source.asFlow() + check((0..9).toList(), flow.produceIn(this)) + check((0..9).toList(), flow.buffer(Channel.UNLIMITED).produceIn(this)) + check((0..9).toList(), flow.buffer(2).produceIn(this)) + check((0..9).toList(), flow.buffer(0).produceIn(this)) + check(listOf(0, 9), flow.conflate().produceIn(this)) + } + + private suspend fun check(expected: List, channel: ReceiveChannel) { + val result = ArrayList(10) + channel.consumeEach { result.add(it) } + assertEquals(expected, result) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/ObservableCollectTest.kt b/reactive/kotlinx-coroutines-rx3/test/ObservableCollectTest.kt new file mode 100644 index 0000000000..ebfb8e6c92 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ObservableCollectTest.kt @@ -0,0 +1,65 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.ObservableSource +import io.reactivex.rxjava3.disposables.* +import kotlinx.coroutines.* +import org.junit.Test +import kotlin.test.* + +class ObservableCollectTest: TestBase() { + + /** Tests the behavior of [collect] when the publisher raises an error. */ + @Test + fun testObservableCollectThrowingObservable() = runTest { + expect(1) + var sum = 0 + try { + rxObservable { + for (i in 0..100) { + send(i) + } + throw TestException() + }.collect { + sum += it + } + } catch (e: TestException) { + assertTrue(sum > 0) + finish(2) + } + } + + @Test + fun testObservableCollectThrowingAction() = runTest { + expect(1) + var sum = 0 + val expectedSum = 5 + try { + var disposed = false + ObservableSource { observer -> + launch(Dispatchers.Default) { + observer.onSubscribe(object : Disposable { + override fun dispose() { + disposed = true + expect(expectedSum + 2) + } + + override fun isDisposed(): Boolean = disposed + }) + while (!disposed) { + observer.onNext(1) + } + } + }.collect { + expect(sum + 2) + sum += it + if (sum == expectedSum) { + throw TestException() + } + } + } catch (e: TestException) { + assertEquals(expectedSum, sum) + finish(expectedSum + 3) + } + } +} \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx3/test/ObservableCompletionStressTest.kt b/reactive/kotlinx-coroutines-rx3/test/ObservableCompletionStressTest.kt new file mode 100644 index 0000000000..c0acd07d8f --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ObservableCompletionStressTest.kt @@ -0,0 +1,33 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.* +import kotlin.coroutines.* + +class ObservableCompletionStressTest : TestBase() { + private val N_REPEATS = 10_000 * stressTestMultiplier + + private fun CoroutineScope.range(context: CoroutineContext, start: Int, count: Int) = rxObservable(context) { + for (x in start until start + count) send(x) + } + + @Test + fun testCompletion() { + val rnd = Random() + repeat(N_REPEATS) { + val count = rnd.nextInt(5) + runBlocking { + withTimeout(5000) { + var received = 0 + range(Dispatchers.Default, 1, count).collect { x -> + received++ + if (x != received) error("$x != $received") + } + if (received != count) error("$received != $count") + } + } + } + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/ObservableExceptionHandlingTest.kt b/reactive/kotlinx-coroutines-rx3/test/ObservableExceptionHandlingTest.kt new file mode 100644 index 0000000000..63fb73bf1e --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ObservableExceptionHandlingTest.kt @@ -0,0 +1,140 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.exceptions.* +import kotlinx.coroutines.* +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class ObservableExceptionHandlingTest : TestBase() { + + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + private inline fun handler(expect: Int) = { t: Throwable -> + assertTrue(t is UndeliverableException && t.cause is T, "$t") + expect(expect) + } + + private fun cehUnreached() = CoroutineExceptionHandler { _, _ -> expectUnreached() } + + @Test + fun testException() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + throw TestException() + }.subscribe({ + expectUnreached() + }, { + expect(2) // Reported to onError + }) + finish(3) + } + + @Test + fun testFatalException() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined + cehUnreached()) { + expect(1) + throw LinkageError() + }.subscribe({ + expectUnreached() + }, { + expect(2) + }) + finish(3) + } + + @Test + fun testExceptionAsynchronous() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + throw TestException() + }.publish() + .refCount() + .subscribe({ + expectUnreached() + }, { + expect(2) // Reported to onError + }) + finish(3) + } + + @Test + fun testFatalExceptionAsynchronous() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + throw LinkageError() + }.publish() + .refCount() + .subscribe({ + expectUnreached() + }, { + expect(2) // Fatal exceptions are not treated in a special manner + }) + finish(3) + } + + @Test + fun testFatalExceptionFromSubscribe() = withExceptionHandler(handler(3)) { + val latch = CountDownLatch(1) + rxObservable(Dispatchers.Unconfined) { + expect(1) + val result = trySend(Unit) + val exception = result.exceptionOrNull() + assertIs(exception) + assertIs(exception.cause) + assertTrue(isClosedForSend) + expect(4) + latch.countDown() + }.subscribe({ + expect(2) + throw LinkageError() + }, { expectUnreached() }) // Unreached because RxJava bubbles up fatal exceptions, causing `onNext` to throw. + latch.await() + finish(5) + } + + @Test + fun testExceptionFromSubscribe() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.subscribe({ + expect(2) + throw TestException() + }, { expect(3) }) + finish(4) + } + + @Test + fun testAsynchronousExceptionFromSubscribe() = withExceptionHandler({ expectUnreached() }) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.publish() + .refCount() + .subscribe({ + expect(2) + throw RuntimeException() + }, { expect(3) }) + finish(4) + } + + @Test + fun testAsynchronousFatalExceptionFromSubscribe() = withExceptionHandler(handler(3)) { + rxObservable(Dispatchers.Unconfined) { + expect(1) + send(Unit) + }.publish() + .refCount() + .subscribe({ + expect(2) + throw LinkageError() + }, { expectUnreached() }) // Unreached because RxJava bubbles up fatal exceptions, causing `onNext` to throw. + finish(4) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/ObservableMultiTest.kt b/reactive/kotlinx-coroutines-rx3/test/ObservableMultiTest.kt new file mode 100644 index 0000000000..83005e95f7 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ObservableMultiTest.kt @@ -0,0 +1,112 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.* +import kotlinx.coroutines.selects.* +import org.junit.Test +import java.io.* +import kotlin.test.* + +/** + * Test emitting multiple values with [rxObservable]. + */ +class ObservableMultiTest : TestBase() { + @Test + fun testNumbers() { + val n = 100 * stressTestMultiplier + val observable = rxObservable { + repeat(n) { send(it) } + } + checkSingleValue(observable.toList()) { list -> + assertEquals((0 until n).toList(), list) + } + } + + + @Test + fun testConcurrentStress() { + val n = 10_000 * stressTestMultiplier + val observable = rxObservable { + newCoroutineContext(coroutineContext) + // concurrent emitters (many coroutines) + val jobs = List(n) { + // launch + launch(Dispatchers.Default) { + val i = it + send(i) + } + } + jobs.forEach { it.join() } + } + checkSingleValue(observable.toList()) { list -> + assertEquals(n, list.size) + assertEquals((0 until n).toList(), list.sorted()) + } + } + + @Test + fun testConcurrentStressOnSend() { + val n = 10_000 * stressTestMultiplier + val observable = rxObservable { + newCoroutineContext(coroutineContext) + // concurrent emitters (many coroutines) + val jobs = List(n) { + // launch + launch(Dispatchers.Default) { + val i = it + select { + onSend(i) {} + } + } + } + jobs.forEach { it.join() } + } + checkSingleValue(observable.toList()) { list -> + assertEquals(n, list.size) + assertEquals((0 until n).toList(), list.sorted()) + } + } + + @Test + fun testIteratorResendUnconfined() { + val n = 10_000 * stressTestMultiplier + val observable = rxObservable(Dispatchers.Unconfined) { + Observable.range(0, n).collect { send(it) } + } + checkSingleValue(observable.toList()) { list -> + assertEquals((0 until n).toList(), list) + } + } + + @Test + fun testIteratorResendPool() { + val n = 10_000 * stressTestMultiplier + val observable = rxObservable { + Observable.range(0, n).collect { send(it) } + } + checkSingleValue(observable.toList()) { list -> + assertEquals((0 until n).toList(), list) + } + } + + @Test + fun testSendAndCrash() { + val observable = rxObservable { + send("O") + throw IOException("K") + } + val single = rxSingle { + var result = "" + try { + observable.collect { result += it } + } catch(e: IOException) { + result += e.message + } + result + } + checkSingleValue(single) { + assertEquals("OK", it) + } + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/ObservableSingleTest.kt b/reactive/kotlinx-coroutines-rx3/test/ObservableSingleTest.kt new file mode 100644 index 0000000000..ea09a2289b --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ObservableSingleTest.kt @@ -0,0 +1,237 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class ObservableSingleTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testSingleNoWait() { + val observable = rxObservable { + send("OK") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleAwait() = runBlocking { + assertEquals("OK", Observable.just("O").awaitSingle() + "K") + } + + @Test + fun testSingleEmitAndAwait() { + val observable = rxObservable { + send(Observable.just("O").awaitSingle() + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleWithDelay() { + val observable = rxObservable { + send(Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleException() { + val observable = rxObservable { + send(Observable.just("O", "K").awaitSingle() + "K") + } + + checkErroneous(observable) { + assertIs(it) + } + } + + @Test + fun testAwaitFirst() { + val observable = rxObservable { + send(Observable.just("O", "#").awaitFirst() + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrDefault() { + val observable = rxObservable { + send(Observable.empty().awaitFirstOrDefault("O") + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrDefaultWithValues() { + val observable = rxObservable { + send(Observable.just("O", "#").awaitFirstOrDefault("!") + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrNull() { + val observable = rxObservable { + send(Observable.empty().awaitFirstOrNull() ?: "OK") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrNullWithValues() { + val observable = rxObservable { + send((Observable.just("O", "#").awaitFirstOrNull() ?: "!") + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrElse() { + val observable = rxObservable { + send(Observable.empty().awaitFirstOrElse { "O" } + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitFirstOrElseWithValues() { + val observable = rxObservable { + send(Observable.just("O", "#").awaitFirstOrElse { "!" } + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitLast() { + val observable = rxObservable { + send(Observable.just("#", "O").awaitLast() + "K") + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + /** Tests that calls to [awaitFirst] (and, thus, the other methods) throw [CancellationException] and dispose of + * the subscription when their [Job] is cancelled. */ + @Test + fun testAwaitCancellation() = runTest { + expect(1) + val observable = ObservableSource { s -> + s.onSubscribe(object: Disposable { + override fun dispose() { expect(4) } + override fun isDisposed(): Boolean { expectUnreached(); return false } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + observable.awaitFirst() + } catch (e: CancellationException) { + expect(5) + throw e + } + } + expect(3) + job.cancelAndJoin() + finish(6) + } + + + @Test + fun testExceptionFromObservable() { + val observable = rxObservable { + try { + send(Observable.error(RuntimeException("O")).awaitFirst()) + } catch (e: RuntimeException) { + send(Observable.just(e.message!!).awaitLast() + "K") + } + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromCoroutine() { + val observable = rxObservable { + throw IllegalStateException(Observable.just("O").awaitSingle() + "K") + } + + checkErroneous(observable) { + assertIs(it) + assertEquals("OK", it.message) + } + } + + @Test + fun testObservableIteration() { + val observable = rxObservable { + var result = "" + Observable.just("O", "K").collect { result += it } + send(result) + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } + + @Test + fun testObservableIterationFailure() { + val observable = rxObservable { + try { + Observable.error(RuntimeException("OK")).collect { fail("Should not be here") } + send("Fail") + } catch (e: RuntimeException) { + send(e.message!!) + } + } + + checkSingleValue(observable) { + assertEquals("OK", it) + } + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/ObservableSourceAsFlowStressTest.kt b/reactive/kotlinx-coroutines-rx3/test/ObservableSourceAsFlowStressTest.kt new file mode 100644 index 0000000000..13b6a84e0a --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ObservableSourceAsFlowStressTest.kt @@ -0,0 +1,32 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.flow.* +import org.junit.* +import java.util.concurrent.* + +class ObservableSourceAsFlowStressTest : TestBase() { + + private val iterations = 100 * stressTestMultiplierSqrt + + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testAsFlowCancellation() = runTest { + repeat(iterations) { + val latch = Channel(1) + var i = 0 + val observable = Observable.interval(100L, TimeUnit.MICROSECONDS) + .doOnNext { if (++i > 100) latch.trySend(Unit) } + val job = observable.asFlow().launchIn(CoroutineScope(Dispatchers.Default)) + latch.receive() + job.cancelAndJoin() + } + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/ObservableSubscriptionSelectTest.kt b/reactive/kotlinx-coroutines-rx3/test/ObservableSubscriptionSelectTest.kt new file mode 100644 index 0000000000..7e21c5c0df --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ObservableSubscriptionSelectTest.kt @@ -0,0 +1,50 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.selects.* +import org.junit.Test +import kotlin.onSuccess +import kotlin.test.* + +class ObservableSubscriptionSelectTest : TestBase() { + @Test + fun testSelect() = runTest { + // source with n ints + val n = 1000 * stressTestMultiplier + val source = rxObservable { repeat(n) { send(it) } } + var a = 0 + var b = 0 + // open two subs + val channelA = source.openSubscription() + val channelB = source.openSubscription() + loop@ while (true) { + val done: Int = select { + channelA.onReceiveCatching { result -> + result.onSuccess { assertEquals(a++, it) } + if (result.isSuccess) 1 else 0 + } + channelB.onReceiveCatching { result -> + result.onSuccess { assertEquals(b++, it) } + if (result.isSuccess) 2 else 0 + } + } + when (done) { + 0 -> break@loop + 1 -> { + val r = channelB.receiveCatching().getOrNull() + if (r != null) assertEquals(b++, r) + } + 2 -> { + val r = channelA.receiveCatching().getOrNull() + if (r != null) assertEquals(a++, r) + } + } + } + channelA.cancel() + channelB.cancel() + // should receive one of them fully + assertTrue(a == n || b == n) + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/ObservableTest.kt b/reactive/kotlinx-coroutines-rx3/test/ObservableTest.kt new file mode 100644 index 0000000000..59031e32f9 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/ObservableTest.kt @@ -0,0 +1,161 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.plugins.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class ObservableTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val observable = rxObservable(currentDispatcher()) { + expect(4) + send("OK") + } + expect(2) + observable.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val observable = rxObservable(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + observable.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val observable = rxObservable(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + } + expect(2) + val sub = observable.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testNotifyOnceOnCancellation() = runTest { + expect(1) + val observable = + rxObservable(currentDispatcher()) { + expect(5) + send("OK") + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + expect(11) + } + } + .doOnNext { + expect(6) + assertEquals("OK", it) + } + .doOnDispose { + expect(10) // notified once! + } + expect(2) + val job = launch(start = CoroutineStart.UNDISPATCHED) { + expect(3) + observable.collect { + expect(8) + assertEquals("OK", it) + } + } + expect(4) + yield() // to observable code + expect(7) + yield() // to consuming coroutines + expect(9) + job.cancel() + job.join() + finish(12) + } + + @Test + fun testFailingConsumer() = runTest { + expect(1) + val pub = rxObservable(currentDispatcher()) { + expect(2) + send("OK") + try { + delay(Long.MAX_VALUE) + } catch (e: CancellationException) { + finish(5) + } + } + try { + pub.collect { + expect(3) + throw TestException() + } + } catch (e: TestException) { + expect(4) + } + } + + @Test + fun testExceptionAfterCancellation() { + // Test that no exceptions were reported to the global EH (it will fail the test if so) + val handler = { e: Throwable -> + assertFalse(e is CancellationException) + } + withExceptionHandler(handler) { + RxJavaPlugins.setErrorHandler { + require(it !is CancellationException) + } + Observable + .interval(1, TimeUnit.MILLISECONDS) + .take(1000) + .switchMapSingle { + rxSingle { + timeBomb().await() + } + } + .blockingSubscribe({}, {}) + } + } + + private fun timeBomb() = Single.timer(1, TimeUnit.MILLISECONDS).doOnSuccess { throw TestException() } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/SchedulerStressTest.kt b/reactive/kotlinx-coroutines-rx3/test/SchedulerStressTest.kt new file mode 100644 index 0000000000..96e2141672 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/SchedulerStressTest.kt @@ -0,0 +1,84 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import org.junit.* +import java.util.concurrent.* + +class SchedulerStressTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxCachedThreadScheduler-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + /** + * Test that we don't get an OOM if we schedule many jobs at once. + * It's expected that if you don't dispose you'd see an OOM error. + */ + @Test + fun testSchedulerDisposed(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableDisposed(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerDisposed(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + val worker = scheduler.createWorker() + testRunnableDisposed(worker::schedule) + } + + private suspend fun testRunnableDisposed(block: RxSchedulerBlockNoDelay) { + val n = 2000 * stressTestMultiplier + repeat(n) { + val a = ByteArray(1000000) //1MB + val disposable = block(Runnable { + keepMe(a) + expectUnreached() + }) + disposable.dispose() + yield() // allow the scheduled task to observe that it was disposed + } + } + + /** + * Test function that holds a reference. Used for testing OOM situations + */ + private fun keepMe(a: ByteArray) { + Thread.sleep(a.size / (a.size + 1) + 10L) + } + + /** + * Test that we don't get an OOM if we schedule many delayed jobs at once. It's expected that if you don't dispose that you'd + * see a OOM error. + */ + @Test + fun testSchedulerDisposedDuringDelay(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableDisposedDuringDelay(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerDisposedDuringDelay(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + val worker = scheduler.createWorker() + testRunnableDisposedDuringDelay(worker::schedule) + } + + private fun testRunnableDisposedDuringDelay(block: RxSchedulerBlockWithDelay) { + val n = 2000 * stressTestMultiplier + repeat(n) { + val a = ByteArray(1000000) //1MB + val delayMillis: Long = 10 + val disposable = block(Runnable { + keepMe(a) + expectUnreached() + }, delayMillis, TimeUnit.MILLISECONDS) + disposable.dispose() + } + } +} diff --git a/reactive/kotlinx-coroutines-rx3/test/SchedulerTest.kt b/reactive/kotlinx-coroutines-rx3/test/SchedulerTest.kt new file mode 100644 index 0000000000..fd59503722 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/SchedulerTest.kt @@ -0,0 +1,493 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.* +import io.reactivex.rxjava3.plugins.* +import io.reactivex.rxjava3.schedulers.* +import kotlinx.coroutines.* +import kotlinx.coroutines.sync.* +import org.junit.* +import org.junit.Test +import java.lang.Runnable +import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicReference +import kotlin.coroutines.* +import kotlin.test.* + +class SchedulerTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxCachedThreadScheduler-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testIoScheduler(): Unit = runTest { + expect(1) + val mainThread = Thread.currentThread() + withContext(Schedulers.io().asCoroutineDispatcher()) { + val t1 = Thread.currentThread() + assertNotSame(t1, mainThread) + expect(2) + delay(100) + val t2 = Thread.currentThread() + assertNotSame(t2, mainThread) + expect(3) + } + finish(4) + } + + /** Tests [toString] implementations of [CoroutineDispatcher.asScheduler] and its [Scheduler.Worker]. */ + @Test + fun testSchedulerToString() { + val name = "Dispatchers.Default" + val scheduler = Dispatchers.Default.asScheduler() + assertContains(scheduler.toString(), name) + val worker = scheduler.createWorker() + val activeWorkerName = worker.toString() + assertContains(worker.toString(), name) + worker.dispose() + val disposedWorkerName = worker.toString() + assertNotEquals(activeWorkerName, disposedWorkerName) + } + + private fun runSchedulerTest(nThreads: Int = 1, action: (Scheduler) -> Unit) { + val future = CompletableFuture() + try { + newFixedThreadPoolContext(nThreads, "test").use { dispatcher -> + RxJavaPlugins.setErrorHandler { + if (!future.completeExceptionally(it)) { + handleUndeliverableException(it, dispatcher) + } + } + action(dispatcher.asScheduler()) + } + } finally { + RxJavaPlugins.setErrorHandler(null) + } + future.complete(Unit) + future.getNow(Unit) // rethrow any encountered errors + } + + private fun ensureSeparateThread(schedule: (Runnable, Long, TimeUnit) -> Unit, scheduleNoDelay: (Runnable) -> Unit) { + val mainThread = Thread.currentThread() + val cdl1 = CountDownLatch(1) + val cdl2 = CountDownLatch(1) + expect(1) + val thread = AtomicReference(null) + fun checkThread() { + val current = Thread.currentThread() + thread.getAndSet(current)?.let { assertEquals(it, current) } + } + schedule({ + assertNotSame(mainThread, Thread.currentThread()) + checkThread() + cdl2.countDown() + }, 300, TimeUnit.MILLISECONDS) + scheduleNoDelay { + expect(2) + checkThread() + assertNotSame(mainThread, Thread.currentThread()) + cdl1.countDown() + } + cdl1.await() + cdl2.await() + finish(3) + } + + /** + * Tests [Scheduler.scheduleDirect] for [CoroutineDispatcher.asScheduler] on a single-threaded dispatcher. + */ + @Test + fun testSingleThreadedDispatcherDirect(): Unit = runSchedulerTest(1) { + ensureSeparateThread(it::scheduleDirect, it::scheduleDirect) + } + + /** + * Tests [Scheduler.Worker.schedule] for [CoroutineDispatcher.asScheduler] running its tasks on the correct thread. + */ + @Test + fun testSingleThreadedWorker(): Unit = runSchedulerTest(1) { + val worker = it.createWorker() + ensureSeparateThread(worker::schedule, worker::schedule) + } + + private fun checkCancelling(schedule: (Runnable, Long, TimeUnit) -> Disposable) { + // cancel the task before it has a chance to run. + val handle1 = schedule({ + throw IllegalStateException("should have been successfully cancelled") + }, 10_000, TimeUnit.MILLISECONDS) + handle1.dispose() + // cancel the task after it started running. + val cdl1 = CountDownLatch(1) + val cdl2 = CountDownLatch(1) + val handle2 = schedule({ + cdl1.countDown() + cdl2.await() + if (Thread.interrupted()) + throw IllegalStateException("cancelling the task should not interrupt the thread") + }, 100, TimeUnit.MILLISECONDS) + cdl1.await() + handle2.dispose() + cdl2.countDown() + } + + /** + * Test cancelling [Scheduler.scheduleDirect] for [CoroutineDispatcher.asScheduler]. + */ + @Test + fun testCancellingDirect(): Unit = runSchedulerTest { + checkCancelling(it::scheduleDirect) + } + + /** + * Test cancelling [Scheduler.Worker.schedule] for [CoroutineDispatcher.asScheduler]. + */ + @Test + fun testCancellingWorker(): Unit = runSchedulerTest { + val worker = it.createWorker() + checkCancelling(worker::schedule) + } + + /** + * Test shutting down [CoroutineDispatcher.asScheduler]. + */ + @Test + fun testShuttingDown() { + val n = 5 + runSchedulerTest(nThreads = n) { scheduler -> + val cdl1 = CountDownLatch(n) + val cdl2 = CountDownLatch(1) + val cdl3 = CountDownLatch(n) + repeat(n) { + scheduler.scheduleDirect { + cdl1.countDown() + try { + cdl2.await() + } catch (e: InterruptedException) { + // this is the expected outcome + cdl3.countDown() + } + } + } + cdl1.await() + scheduler.shutdown() + if (!cdl3.await(1, TimeUnit.SECONDS)) { + cdl2.countDown() + error("the tasks were not cancelled when the scheduler was shut down") + } + } + } + + /** Tests that there are no uncaught exceptions if [Disposable.dispose] on a worker happens when tasks are present. */ + @Test + fun testDisposingWorker() = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + val worker = scheduler.createWorker() + yield() // so that the worker starts waiting on the channel + assertFalse(worker.isDisposed) + worker.dispose() + assertTrue(worker.isDisposed) + } + + /** Tests trying to use a [Scheduler.Worker]/[Scheduler] after [Scheduler.Worker.dispose]/[Scheduler.shutdown]. */ + @Test + fun testSchedulingAfterDisposing() = runSchedulerTest { + expect(1) + val worker = it.createWorker() + // use CDL to ensure that the worker has properly initialized + val cdl1 = CountDownLatch(1) + setScheduler(2, 3) + val disposable1 = worker.schedule { + cdl1.countDown() + } + cdl1.await() + expect(4) + assertFalse(disposable1.isDisposed) + setScheduler(6, -1) + // check that the worker automatically disposes of the tasks after being disposed + assertFalse(worker.isDisposed) + worker.dispose() + assertTrue(worker.isDisposed) + expect(5) + val disposable2 = worker.schedule { + expectUnreached() + } + assertTrue(disposable2.isDisposed) + setScheduler(7, 8) + // ensure that the scheduler still works + val cdl2 = CountDownLatch(1) + val disposable3 = it.scheduleDirect { + cdl2.countDown() + } + cdl2.await() + expect(9) + assertFalse(disposable3.isDisposed) + // check that the scheduler automatically disposes of the tasks after being shut down + it.shutdown() + setScheduler(10, -1) + val disposable4 = it.scheduleDirect { + expectUnreached() + } + assertTrue(disposable4.isDisposed) + RxJavaPlugins.setScheduleHandler(null) + finish(11) + } + + @Test + fun testSchedulerWithNoDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithNoDelay(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerWithNoDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithNoDelay(scheduler.createWorker()::schedule) + } + + private suspend fun testRunnableWithNoDelay(block: RxSchedulerBlockNoDelay) { + expect(1) + suspendCancellableCoroutine { + block(Runnable { + expect(2) + it.resume(Unit) + }) + } + yield() + finish(3) + } + + @Test + fun testSchedulerWithDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler::scheduleDirect, 300) + } + + @Test + fun testSchedulerWorkerWithDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler.createWorker()::schedule, 300) + } + + @Test + fun testSchedulerWithZeroDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerWithZeroDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler.createWorker()::schedule) + } + + private suspend fun testRunnableWithDelay(block: RxSchedulerBlockWithDelay, delayMillis: Long = 0) { + expect(1) + suspendCancellableCoroutine { + block({ + expect(2) + it.resume(Unit) + }, delayMillis, TimeUnit.MILLISECONDS) + } + finish(3) + } + + @Test + fun testAsSchedulerWithNegativeDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler::scheduleDirect, -1) + } + + @Test + fun testAsSchedulerWorkerWithNegativeDelay(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableWithDelay(scheduler.createWorker()::schedule, -1) + } + + @Test + fun testSchedulerImmediateDispose(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableImmediateDispose(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerImmediateDispose(): Unit = runTest { + val scheduler = (currentDispatcher() as CoroutineDispatcher).asScheduler() + testRunnableImmediateDispose(scheduler.createWorker()::schedule) + } + + private fun testRunnableImmediateDispose(block: RxSchedulerBlockNoDelay) { + val disposable = block { + expectUnreached() + } + disposable.dispose() + } + + @Test + fun testConvertDispatcherToOriginalScheduler(): Unit = runTest { + val originalScheduler = Schedulers.io() + val dispatcher = originalScheduler.asCoroutineDispatcher() + val scheduler = dispatcher.asScheduler() + assertSame(originalScheduler, scheduler) + } + + @Test + fun testConvertSchedulerToOriginalDispatcher(): Unit = runTest { + val originalDispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = originalDispatcher.asScheduler() + val dispatcher = scheduler.asCoroutineDispatcher() + assertSame(originalDispatcher, dispatcher) + } + + @Test + fun testSchedulerExpectRxPluginsCall(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableExpectRxPluginsCall(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerExpectRxPluginsCall(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableExpectRxPluginsCall(scheduler.createWorker()::schedule) + } + + private suspend fun testRunnableExpectRxPluginsCall(block: RxSchedulerBlockNoDelay) { + expect(1) + setScheduler(2, 4) + suspendCancellableCoroutine { + block(Runnable { + expect(5) + it.resume(Unit) + }) + expect(3) + } + RxJavaPlugins.setScheduleHandler(null) + finish(6) + } + + @Test + fun testSchedulerExpectRxPluginsCallWithDelay(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + testRunnableExpectRxPluginsCallDelay(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerExpectRxPluginsCallWithDelay(): Unit = runTest { + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + val worker = scheduler.createWorker() + testRunnableExpectRxPluginsCallDelay(worker::schedule) + } + + private suspend fun testRunnableExpectRxPluginsCallDelay(block: RxSchedulerBlockWithDelay) { + expect(1) + setScheduler(2, 4) + suspendCancellableCoroutine { + block({ + expect(5) + it.resume(Unit) + }, 10, TimeUnit.MILLISECONDS) + expect(3) + } + RxJavaPlugins.setScheduleHandler(null) + finish(6) + } + + private fun setScheduler(expectedCountOnSchedule: Int, expectCountOnRun: Int) { + RxJavaPlugins.setScheduleHandler { + expect(expectedCountOnSchedule) + Runnable { + expect(expectCountOnRun) + it.run() + } + } + } + + /** + * Tests that [Scheduler.Worker] runs all work sequentially. + */ + @Test + fun testWorkerSequentialOrdering() = runTest { + expect(1) + val scheduler = Dispatchers.Default.asScheduler() + val worker = scheduler.createWorker() + val iterations = 100 + for (i in 0..iterations) { + worker.schedule { + expect(2 + i) + } + } + suspendCoroutine { + worker.schedule { + it.resume(Unit) + } + } + finish((iterations + 2) + 1) + } + + /** + * Test that ensures that delays are actually respected (tasks scheduled sooner in the future run before tasks scheduled later, + * even when the later task is submitted before the earlier one) + */ + @Test + fun testSchedulerRespectsDelays(): Unit = runTest { + val scheduler = Dispatchers.Default.asScheduler() + testRunnableRespectsDelays(scheduler::scheduleDirect) + } + + @Test + fun testSchedulerWorkerRespectsDelays(): Unit = runTest { + val scheduler = Dispatchers.Default.asScheduler() + testRunnableRespectsDelays(scheduler.createWorker()::schedule) + } + + private suspend fun testRunnableRespectsDelays(block: RxSchedulerBlockWithDelay) { + expect(1) + val semaphore = Semaphore(2, 2) + block({ + expect(3) + semaphore.release() + }, 100, TimeUnit.MILLISECONDS) + block({ + expect(2) + semaphore.release() + }, 1, TimeUnit.MILLISECONDS) + semaphore.acquire() + semaphore.acquire() + finish(4) + } + + /** + * Tests that cancelling a runnable in one worker doesn't affect work in another scheduler. + * + * This is part of expected behavior documented. + */ + @Test + fun testMultipleWorkerCancellation(): Unit = runTest { + expect(1) + val dispatcher = currentDispatcher() as CoroutineDispatcher + val scheduler = dispatcher.asScheduler() + suspendCancellableCoroutine { + val workerOne = scheduler.createWorker() + workerOne.schedule({ + expect(3) + it.resume(Unit) + }, 50, TimeUnit.MILLISECONDS) + val workerTwo = scheduler.createWorker() + workerTwo.schedule({ + expectUnreached() + }, 1000, TimeUnit.MILLISECONDS) + workerTwo.dispose() + expect(2) + } + finish(4) + } +} + +typealias RxSchedulerBlockNoDelay = (Runnable) -> Disposable +typealias RxSchedulerBlockWithDelay = (Runnable, Long, TimeUnit) -> Disposable \ No newline at end of file diff --git a/reactive/kotlinx-coroutines-rx3/test/SingleTest.kt b/reactive/kotlinx-coroutines-rx3/test/SingleTest.kt new file mode 100644 index 0000000000..381a9dfe15 --- /dev/null +++ b/reactive/kotlinx-coroutines-rx3/test/SingleTest.kt @@ -0,0 +1,289 @@ +package kotlinx.coroutines.rx3 + +import kotlinx.coroutines.testing.* +import io.reactivex.rxjava3.core.* +import io.reactivex.rxjava3.disposables.* +import io.reactivex.rxjava3.exceptions.* +import io.reactivex.rxjava3.functions.* +import kotlinx.coroutines.* +import kotlinx.coroutines.CancellationException +import org.junit.* +import org.junit.Test +import java.util.concurrent.* +import kotlin.test.* + +class SingleTest : TestBase() { + @Before + fun setup() { + ignoreLostThreads("RxComputationThreadPool-", "RxCachedWorkerPoolEvictor-", "RxSchedulerPurge-") + } + + @Test + fun testBasicSuccess() = runBlocking { + expect(1) + val single = rxSingle(currentDispatcher()) { + expect(4) + "OK" + } + expect(2) + single.subscribe { value -> + expect(5) + assertEquals("OK", value) + } + expect(3) + yield() // to started coroutine + finish(6) + } + + @Test + fun testBasicFailure() = runBlocking { + expect(1) + val single = rxSingle(currentDispatcher()) { + expect(4) + throw RuntimeException("OK") + } + expect(2) + single.subscribe({ + expectUnreached() + }, { error -> + expect(5) + assertIs(error) + assertEquals("OK", error.message) + }) + expect(3) + yield() // to started coroutine + finish(6) + } + + + @Test + fun testBasicUnsubscribe() = runBlocking { + expect(1) + val single = rxSingle(currentDispatcher()) { + expect(4) + yield() // back to main, will get cancelled + expectUnreached() + + } + expect(2) + // nothing is called on a disposed rx3 single + val sub = single.subscribe({ + expectUnreached() + }, { + expectUnreached() + }) + expect(3) + yield() // to started coroutine + expect(5) + sub.dispose() // will cancel coroutine + yield() + finish(6) + } + + @Test + fun testSingleNoWait() { + val single = rxSingle { + "OK" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleAwait() = runBlocking { + assertEquals("OK", Single.just("O").await() + "K") + } + + /** Tests that calls to [await] throw [CancellationException] and dispose of the subscription when their + * [Job] is cancelled. */ + @Test + fun testSingleAwaitCancellation() = runTest { + expect(1) + val single = SingleSource { s -> + s.onSubscribe(object: Disposable { + override fun dispose() { expect(4) } + override fun isDisposed(): Boolean { expectUnreached(); return false } + }) + } + val job = launch(start = CoroutineStart.UNDISPATCHED) { + try { + expect(2) + single.await() + } catch (e: CancellationException) { + expect(5) + throw e + } + } + expect(3) + job.cancelAndJoin() + finish(6) + } + + @Test + fun testSingleEmitAndAwait() { + val single = rxSingle { + Single.just("O").await() + "K" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleWithDelay() { + val single = rxSingle { + Observable.timer(50, TimeUnit.MILLISECONDS).map { "O" }.awaitSingle() + "K" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testSingleException() { + val single = rxSingle { + Observable.just("O", "K").awaitSingle() + "K" + } + + checkErroneous(single) { + assert(it is IllegalArgumentException) + } + } + + @Test + fun testAwaitFirst() { + val single = rxSingle { + Observable.just("O", "#").awaitFirst() + "K" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testAwaitLast() { + val single = rxSingle { + Observable.just("#", "O").awaitLast() + "K" + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromObservable() { + val single = rxSingle { + try { + Observable.error(RuntimeException("O")).awaitFirst() + } catch (e: RuntimeException) { + Observable.just(e.message!!).awaitLast() + "K" + } + } + + checkSingleValue(single) { + assertEquals("OK", it) + } + } + + @Test + fun testExceptionFromCoroutine() { + val single = rxSingle { + throw IllegalStateException(Observable.just("O").awaitSingle() + "K") + } + + checkErroneous(single) { + assert(it is IllegalStateException) + assertEquals("OK", it.message) + } + } + + @Test + fun testSuppressedException() = runTest { + val single = rxSingle(currentDispatcher()) { + launch(start = CoroutineStart.ATOMIC) { + throw TestException() // child coroutine fails + } + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException2() // but parent throws another exception while cleaning up + } + } + try { + single.await() + expectUnreached() + } catch (e: TestException) { + assertIs(e.suppressed[0]) + } + } + + @Test + fun testFatalExceptionInSubscribe() = runTest { + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is LinkageError) + expect(2) + } + withExceptionHandler(handler) { + rxSingle(Dispatchers.Unconfined) { + expect(1) + 42 + }.subscribe(Consumer { + throw LinkageError() + }) + finish(3) + } + } + + @Test + fun testFatalExceptionInSingle() = runTest { + rxSingle(Dispatchers.Unconfined) { + throw LinkageError() + }.subscribe { _, e -> assertIs(e); expect(1) } + + finish(2) + } + + @Test + fun testUnhandledException() = runTest { + expect(1) + var disposable: Disposable? = null + val handler = { e: Throwable -> + assertTrue(e is UndeliverableException && e.cause is TestException) + expect(5) + } + val single = rxSingle(currentDispatcher()) { + expect(4) + disposable!!.dispose() // cancel our own subscription, so that delay will get cancelled + try { + delay(Long.MAX_VALUE) + } finally { + throw TestException() // would not be able to handle it since mono is disposed + } + } + withExceptionHandler(handler) { + single.subscribe(object : SingleObserver { + override fun onSubscribe(d: Disposable) { + expect(2) + disposable = d + } + + override fun onSuccess(t: Unit) { + expectUnreached() + } + + override fun onError(t: Throwable) { + expectUnreached() + } + }) + expect(3) + yield() // run coroutine + finish(6) + } + } +} diff --git a/settings.gradle.kts b/settings.gradle.kts new file mode 100644 index 0000000000..423c613f3c --- /dev/null +++ b/settings.gradle.kts @@ -0,0 +1,52 @@ +pluginManagement { + val javafx_plugin_version: String by settings + plugins { + id("org.openjfx.javafxplugin") version javafx_plugin_version + id("me.champeau.jmh") version "0.7.2" + } + + repositories { + maven(url = "https://maven.pkg.jetbrains.space/kotlin/p/dokka/dev/") + gradlePluginPortal() + } +} + +rootProject.name = "kotlinx.coroutines" + +fun module(path: String) { + val i = path.lastIndexOf("/") + val name = path.substring(i + 1) + include(name) + project(":$name").projectDir = file(path) +} +val prop = System.getProperty("build_snapshot_train") +var build_snapshot_train: String by extra +build_snapshot_train = if (prop != null && prop != "") "true" else "false" +// --------------------------- + +include("benchmarks") +module("test-utils") + +include("kotlinx-coroutines-core") + +module("kotlinx-coroutines-test") +module("kotlinx-coroutines-debug") +module("kotlinx-coroutines-bom") + + +module("integration/kotlinx-coroutines-guava") +module("integration/kotlinx-coroutines-jdk8") +module("integration/kotlinx-coroutines-slf4j") +module("integration/kotlinx-coroutines-play-services") + +module("reactive/kotlinx-coroutines-reactive") +module("reactive/kotlinx-coroutines-reactor") +module("reactive/kotlinx-coroutines-jdk9") +module("reactive/kotlinx-coroutines-rx2") +module("reactive/kotlinx-coroutines-rx3") +module("ui/kotlinx-coroutines-android") +module("ui/kotlinx-coroutines-android/android-unit-tests") +if (JavaVersion.current().isJava11Compatible()) { + module("ui/kotlinx-coroutines-javafx") +} +module("ui/kotlinx-coroutines-swing") diff --git a/site/README.md b/site/README.md deleted file mode 100644 index cf3019426a..0000000000 --- a/site/README.md +++ /dev/null @@ -1,13 +0,0 @@ -# Reference documentation site - -This module builds references documentation. - -## Building - -* Install [Jekyll](http://jekyllrb.com) -* In project root directory do: - * Run `mvn clean` - * Run `mvn compile` - * Run `mvn site` -* The result is in `target/_site` -* Upload it to github pages (`gh-pages` branch) diff --git a/site/build.xml b/site/build.xml deleted file mode 100644 index d31a34e75d..0000000000 --- a/site/build.xml +++ /dev/null @@ -1,56 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/site/docs/Gemfile b/site/docs/Gemfile deleted file mode 100644 index 69628b20d0..0000000000 --- a/site/docs/Gemfile +++ /dev/null @@ -1,16 +0,0 @@ -source "https://rubygems.org/" -ruby RUBY_VERSION - -# Hello! This is where you manage which Jekyll version is used to run. -# When you want to use a different version, change it below, save the -# file and run `bundle install`. Run Jekyll with `bundle exec`, like so: -# -# bundle exec jekyll serve -# -# This will help ensure the proper Jekyll version is running. -# Happy Jekylling! - -gem "jekyll", "3.4.0" - -# Windows does not include zoneinfo files, so bundle the tzinfo-data gem -gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby] diff --git a/site/docs/Gemfile.lock b/site/docs/Gemfile.lock deleted file mode 100644 index d7e03ec7cf..0000000000 --- a/site/docs/Gemfile.lock +++ /dev/null @@ -1,58 +0,0 @@ -GEM - remote: https://rubygems.org/ - specs: - addressable (2.5.0) - public_suffix (~> 2.0, >= 2.0.2) - colorator (1.1.0) - ffi (1.9.17) - ffi (1.9.17-x64-mingw32) - forwardable-extended (2.6.0) - jekyll (3.4.0) - addressable (~> 2.4) - colorator (~> 1.0) - jekyll-sass-converter (~> 1.0) - jekyll-watch (~> 1.1) - kramdown (~> 1.3) - liquid (~> 3.0) - mercenary (~> 0.3.3) - pathutil (~> 0.9) - rouge (~> 1.7) - safe_yaml (~> 1.0) - jekyll-sass-converter (1.5.0) - sass (~> 3.4) - jekyll-watch (1.5.0) - listen (~> 3.0, < 3.1) - kramdown (1.13.2) - liquid (3.0.6) - listen (3.0.8) - rb-fsevent (~> 0.9, >= 0.9.4) - rb-inotify (~> 0.9, >= 0.9.7) - mercenary (0.3.6) - pathutil (0.14.0) - forwardable-extended (~> 2.6) - public_suffix (2.0.5) - rb-fsevent (0.9.8) - rb-inotify (0.9.8) - ffi (>= 0.5.0) - rouge (1.11.1) - safe_yaml (1.0.4) - sass (3.4.23) - thread_safe (0.3.5) - tzinfo (1.2.2) - thread_safe (~> 0.1) - tzinfo-data (1.2016.10) - tzinfo (>= 1.0.0) - -PLATFORMS - ruby - x64-mingw32 - -DEPENDENCIES - jekyll (= 3.4.0) - tzinfo-data - -RUBY VERSION - ruby 2.2.6p396 - -BUNDLED WITH - 1.14.3 diff --git a/site/docs/_config.yml b/site/docs/_config.yml deleted file mode 100644 index fb44615774..0000000000 --- a/site/docs/_config.yml +++ /dev/null @@ -1,11 +0,0 @@ -# Jekyll configuration file -title: kotlinx.coroutines -description: Library support for kotlin coroutines -baseurl: "/kotlinx.coroutines" -url: "https://kotlin.github.io/" - -# Build settings -markdown: kramdown -exclude: - - Gemfile - - Gemfile.lock diff --git a/site/docs/_includes/footer.html b/site/docs/_includes/footer.html deleted file mode 100644 index b115703c01..0000000000 --- a/site/docs/_includes/footer.html +++ /dev/null @@ -1,3 +0,0 @@ -
- -
diff --git a/site/docs/_includes/head.html b/site/docs/_includes/head.html deleted file mode 100644 index ed5c8e7d8a..0000000000 --- a/site/docs/_includes/head.html +++ /dev/null @@ -1,17 +0,0 @@ - - - - - {% if page.title %}{{ page.title | escape }}{% else %}{{ site.title | escape }}{% endif %} - - - - - diff --git a/site/docs/_includes/header.html b/site/docs/_includes/header.html deleted file mode 100644 index b250a17723..0000000000 --- a/site/docs/_includes/header.html +++ /dev/null @@ -1,5 +0,0 @@ - diff --git a/site/docs/_layouts/api.html b/site/docs/_layouts/api.html deleted file mode 100644 index 0f93a6b213..0000000000 --- a/site/docs/_layouts/api.html +++ /dev/null @@ -1,14 +0,0 @@ - - - {% include head.html %} - - {% include header.html %} -
-
- {{ content }} -
-
- {% include footer.html %} - - - diff --git a/site/docs/_sass/_api.scss b/site/docs/_sass/_api.scss deleted file mode 100644 index 2eca71be28..0000000000 --- a/site/docs/_sass/_api.scss +++ /dev/null @@ -1,202 +0,0 @@ - -// ----------------- Bits and pieces from kotlinlang.org reference ----------------- - -body { - -webkit-font-smoothing: antialiased; - font-smoothing: antialiased; - text-rendering: optimizeLegibility; -} - -a { - text-decoration: underline; -} - -$vertical-rhythm-unit: 15px !global; - -h1 { - margin-top: $vertical-rhythm-unit * 2; - margin-bottom: $vertical-rhythm-unit; - font-size: 30px; - line-height: 33px; - - &:first-of-type { - margin-top: 0; - margin-bottom: $vertical-rhythm-unit * 2; - } - - @media print { - page-break-before: always; - page-break-after: avoid; - } - - &%_section-title { - padding-top: 140px; - margin-bottom: 45px; - font-size: 55px; - line-height: 65px; - font-weight: bold; - } -} - -h2 { - margin-top: $vertical-rhythm-unit * 2; - margin-bottom: $vertical-rhythm-unit; - font-size: 24px; - line-height: 27px; - - &:first-of-type { - margin-top: 0; - } - - @media print { - page-break-after: avoid; - } -} - -h3 { - margin-top: $vertical-rhythm-unit * 2; - margin-bottom: $vertical-rhythm-unit; - font-size: 19px; - line-height: 22px; - - @media print { - page-break-after: avoid; - } -} - -h4 { - margin-top: $vertical-rhythm-unit * 2; - margin-bottom: $vertical-rhythm-unit; - font-size: 16px; - line-height: 20px; - font-weight: bold; - - @media print { - page-break-after: avoid; - } -} - -h5 { - margin-top: $vertical-rhythm-unit * 2; - margin-bottom: $vertical-rhythm-unit; - font-size: 16px; - line-height: 20px; - font-weight: normal; - - @media print { - page-break-after: avoid; - } -} - -caption {margin: 0;} - -$vertical-rhythm-unit: 15px !global; - -// tables - -table { - margin-bottom: $vertical-rhythm-unit*2; - line-height: inherit; - font-size: inherit; - border: 1px solid #dcdcdc; - - // Remove most spacing between table cells - border-collapse: collapse; - border-spacing: 0; - - &.zebra { - tbody tr:nth-child(odd) { - background-color: #f5f5f5; - } - } - - &.wide { - min-width: 100%; - } - - // Table header - thead { - background-color: #F7F7F7; - border-bottom-width: 2px; - } - - // Table footer - tfoot { - color: #ccc; - - tr {border-bottom: none;} - } - - // Row - tr { - border-bottom: 1px solid #dcdcdc; - } - - // Header cell - th { - padding-top: 10px; - padding-bottom: 6px; - text-align: left; - font-weight: bold; - } - - // Cell - th, - td { - padding: 6px 10px; - vertical-align: top; - - &:first-child { - padding-left: 12px; - } - - &:last-child { - padding-right: 12px; - } - } - - // ??? - p:last-child, - pre:last-child { - margin-bottom: 0; - } -} - -.api-docs-breadcrumbs { - margin-bottom: 25px; -} - -// code - -$font-family-mono: 'Liberation Mono', Consolas, Menlo, Courier, monospace !global; -$code-background: #efefef; - -code { - font-family: $font-family-mono; - font-style: normal; - background-color: $code-background; -} - -code :target { - background-color: #FFFFCC; -} - -// kotlin syntax highlight - -.signature { - background-color: $code-background; - padding: 4px; -} - -.keyword { - color: #0000C0; -} - -.summarizedTypeName { - background-color: lightcyan; - font-style: italic; -} - -.parameterName { - font-weight: bold; -} diff --git a/site/docs/_sass/_base.scss b/site/docs/_sass/_base.scss deleted file mode 100644 index b8d70d526b..0000000000 --- a/site/docs/_sass/_base.scss +++ /dev/null @@ -1,97 +0,0 @@ -// Bits and pieces from Minima Jekyll Layout -// The MIT License (MIT) Copyright (c) 2016 Parker Moor - -// Reset some basic elements -body, h1, h2, h3, h4, h5, h6, -p, blockquote, pre, hr, -dl, dd, ol, ul, figure { - margin: 0; - padding: 0; -} - -// Basic styling -body { - font: $base-font-weight #{$base-font-size}/#{$base-line-height} $base-font-family; - color: $text-color; - background-color: $background-color; - -webkit-text-size-adjust: 100%; - -webkit-font-feature-settings: "kern" 1; - -moz-font-feature-settings: "kern" 1; - -o-font-feature-settings: "kern" 1; - font-feature-settings: "kern" 1; - font-kerning: normal; -} - -// Set `margin-bottom` to maintain vertical rhythm -h1, h2, h3, h4, h5, h6, -p, blockquote, pre, -ul, ol, dl, figure, -%vertical-rhythm { - margin-bottom: $spacing-unit / 2; -} - -// Lists -ul, ol { - margin-left: $spacing-unit; -} - -li { - > ul, - > ol { - margin-bottom: 0; - } -} - -// Links -a { - color: $brand-color; - text-decoration: none; - - &:visited { - color: darken($brand-color, 15%); - } - - &:hover { - color: $text-color; - text-decoration: underline; - } -} - -// Blockquotes -blockquote { - color: $grey-color; - border-left: 4px solid $grey-color-light; - padding-left: $spacing-unit / 2; - font-size: 18px; - letter-spacing: -1px; - font-style: italic; - - > :last-child { - margin-bottom: 0; - } -} - -// Wrapper -.wrapper { - max-width: -webkit-calc(#{$content-width} - (#{$spacing-unit} * 2)); - max-width: calc(#{$content-width} - (#{$spacing-unit} * 2)); - margin-right: auto; - margin-left: auto; - padding-right: $spacing-unit; - padding-left: $spacing-unit; - @extend %clearfix; - - @include media-query($on-laptop) { - max-width: -webkit-calc(#{$content-width} - (#{$spacing-unit})); - max-width: calc(#{$content-width} - (#{$spacing-unit})); - padding-right: $spacing-unit / 2; - padding-left: $spacing-unit / 2; - } -} - -// Clearfix -%clearfix:after { - content: ""; - display: table; - clear: both; -} diff --git a/site/docs/_sass/_layout.scss b/site/docs/_sass/_layout.scss deleted file mode 100644 index d85d0592ef..0000000000 --- a/site/docs/_sass/_layout.scss +++ /dev/null @@ -1,37 +0,0 @@ -// Bits and pieces from Minima Jekyll Layout -// The MIT License (MIT) Copyright (c) 2016 Parker Moor - -// Site header -.site-header { - border-top: 5px solid $grey-color-dark; - border-bottom: 1px solid $grey-color-light; - min-height: 56px; - - // Positioning context for the mobile navigation icon - position: relative; -} - -.site-title { - font-size: 26px; - font-weight: 300; - line-height: 56px; - letter-spacing: -1px; - margin-bottom: 0; - float: left; - - &, - &:visited { - color: $grey-color-dark; - } -} -// Site footer -.site-footer { - border-top: 1px solid $grey-color-light; - padding: $spacing-unit 0; -} - -// Page content -.page-content { - padding: $spacing-unit 0; -} - diff --git a/site/docs/_sass/_minima.scss b/site/docs/_sass/_minima.scss deleted file mode 100644 index 86079f8529..0000000000 --- a/site/docs/_sass/_minima.scss +++ /dev/null @@ -1,35 +0,0 @@ -// Bits and pieces from Minima Jekyll Layout -// The MIT License (MIT) Copyright (c) 2016 Parker Moor - -// Define defaults for each variable. - -$base-font-family: "Helvetica Neue", Helvetica, Arial, sans-serif !default; -$base-font-size: 16px !default; -$base-font-weight: 400 !default; -$small-font-size: $base-font-size * 0.875 !default; -$base-line-height: 1.5 !default; - -$spacing-unit: 30px !default; - -$text-color: #111 !default; -$background-color: #fdfdfd !default; -$brand-color: #2a7ae2 !default; - -$grey-color: #828282 !default; -$grey-color-light: lighten($grey-color, 40%) !default; -$grey-color-dark: darken($grey-color, 25%) !default; - -// Width of the content area -$content-width: 800px !default; - -$on-palm: 600px !default; -$on-laptop: 800px !default; - -@mixin media-query($device) { - @media screen and (max-width: $device) { - @content; - } -} - -// Import partials. -@import "https://github.com/base", "layout"; diff --git a/site/docs/assets/main.scss b/site/docs/assets/main.scss deleted file mode 100644 index 1be8487fb4..0000000000 --- a/site/docs/assets/main.scss +++ /dev/null @@ -1,36 +0,0 @@ ---- -# Only the main Sass file needs front matter (the dashes are enough) ---- -@charset "utf-8"; - -// Sans Serif -@import url('https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400italic,700italic,400,700'); - -// Our variables -$base-font-family: "Open Sans", Helvetica, Arial, sans-serif; -$base-font-size: 14px; -$base-font-weight: 400; -$small-font-size: $base-font-size * 0.875; -$base-line-height: 20px; - -$spacing-unit: 30px; - -$text-color: #111; -$background-color: #fdfdfd; -$brand-color: #2a7ae2; - -$grey-color: #828282; -$grey-color-light: lighten($grey-color, 40%); -$grey-color-dark: darken($grey-color, 25%); - -// Width of the content area -$content-width: 800px; - -$on-palm: 600px; -$on-laptop: 800px; - -// Import partials from the `minima` theme. -@import "https://github.com/minima"; - -// Import api reference styles -@import "https://github.com/api"; diff --git a/site/docs/index.md b/site/docs/index.md deleted file mode 100644 index 8f56f2df49..0000000000 --- a/site/docs/index.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -title: kotlinx-coroutines -layout: api ---- - -# kotlinx.coroutines reference documentation - -Library support for Kotlin coroutines. This reference is a companion to -[Guide to kotlinx.coroutines by example](https://github.com/Kotlin/kotlinx.coroutines/blob/master/coroutines-guide.md). - -## Modules - -[kotlinx-coroutines-core](kotlinx-coroutines-core) | Core primitives to work with coroutines -[kotlinx-coroutines-jdk8](kotlinx-coroutines-jdk8) | Additional libraries for JDK8 (or Android API level 24) -[kotlinx-coroutines-nio](kotlinx-coroutines-nio) | Extensions for asynchronous IO on JDK7+ -[kotlinx-coroutines-reactive](kotlinx-coroutines-reactive) | Utilities for [Reactive Streams](http://www.reactive-streams.org) -[kotlinx-coroutines-rx1](kotlinx-coroutines-rx1) | Utilities for [RxJava 1.x](https://github.com/ReactiveX/RxJava/tree/1.x) -[kotlinx-coroutines-rx2](kotlinx-coroutines-rx2) | Utilities for [RxJava 2.x](https://github.com/ReactiveX/RxJava) -[kotlinx-coroutines-android](kotlinx-coroutines-android) | `UI` context for Android applications -[kotlinx-coroutines-javafx](kotlinx-coroutines-javafx) | `JavaFx` context for JavaFX UI applications -[kotlinx-coroutines-swing](kotlinx-coroutines-swing) | `Swing` context for Swing UI applications diff --git a/site/pom.xml b/site/pom.xml deleted file mode 100644 index 11955fad68..0000000000 --- a/site/pom.xml +++ /dev/null @@ -1,116 +0,0 @@ - - - - - - 4.0.0 - - - org.jetbrains.kotlinx - kotlinx-coroutines - 0.15-SNAPSHOT - - - kotlinx-coroutines-site - pom - - - - org.jetbrains.kotlinx - kotlinx-coroutines-core - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-jdk8 - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-nio - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-swing - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-javafx - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-reactive - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-rx1 - ${project.version} - compile - - - org.jetbrains.kotlinx - kotlinx-coroutines-rx2 - ${project.version} - compile - - - - - - - maven-antrun-plugin - - - site - - - - - - - run - - - - - - org.jetbrains.dokka - dokka-maven-plugin - ${dokka.version} - - true - - - - maven-deploy-plugin - - true - - - - - diff --git a/site/stdlib.package.list b/site/stdlib.package.list new file mode 100644 index 0000000000..3108a5c074 --- /dev/null +++ b/site/stdlib.package.list @@ -0,0 +1,229 @@ +$dokka.format:kotlin-website-html +$dokka.linkExtension:html +$dokka.location:kotlin$and(java.math.BigInteger, java.math.BigInteger)kotlin/java.math.-big-integer/and.html +$dokka.location:kotlin$dec(java.math.BigDecimal)kotlin/java.math.-big-decimal/dec.html +$dokka.location:kotlin$dec(java.math.BigInteger)kotlin/java.math.-big-integer/dec.html +$dokka.location:kotlin$div(java.math.BigDecimal, java.math.BigDecimal)kotlin/java.math.-big-decimal/div.html +$dokka.location:kotlin$div(java.math.BigInteger, java.math.BigInteger)kotlin/java.math.-big-integer/div.html +$dokka.location:kotlin$inc(java.math.BigDecimal)kotlin/java.math.-big-decimal/inc.html +$dokka.location:kotlin$inc(java.math.BigInteger)kotlin/java.math.-big-integer/inc.html +$dokka.location:kotlin$inv(java.math.BigInteger)kotlin/java.math.-big-integer/inv.html +$dokka.location:kotlin$minus(java.math.BigDecimal, java.math.BigDecimal)kotlin/java.math.-big-decimal/minus.html +$dokka.location:kotlin$minus(java.math.BigInteger, java.math.BigInteger)kotlin/java.math.-big-integer/minus.html +$dokka.location:kotlin$mod(java.math.BigDecimal, java.math.BigDecimal)kotlin/java.math.-big-decimal/mod.html +$dokka.location:kotlin$or(java.math.BigInteger, java.math.BigInteger)kotlin/java.math.-big-integer/or.html +$dokka.location:kotlin$plus(java.math.BigDecimal, java.math.BigDecimal)kotlin/java.math.-big-decimal/plus.html +$dokka.location:kotlin$plus(java.math.BigInteger, java.math.BigInteger)kotlin/java.math.-big-integer/plus.html +$dokka.location:kotlin$rem(java.math.BigDecimal, java.math.BigDecimal)kotlin/java.math.-big-decimal/rem.html +$dokka.location:kotlin$rem(java.math.BigInteger, java.math.BigInteger)kotlin/java.math.-big-integer/rem.html +$dokka.location:kotlin$shl(java.math.BigInteger, kotlin.Int)kotlin/java.math.-big-integer/shl.html +$dokka.location:kotlin$shr(java.math.BigInteger, kotlin.Int)kotlin/java.math.-big-integer/shr.html +$dokka.location:kotlin$times(java.math.BigDecimal, java.math.BigDecimal)kotlin/java.math.-big-decimal/times.html +$dokka.location:kotlin$times(java.math.BigInteger, java.math.BigInteger)kotlin/java.math.-big-integer/times.html +$dokka.location:kotlin$toBigDecimal(java.math.BigInteger)kotlin/java.math.-big-integer/to-big-decimal.html +$dokka.location:kotlin$toBigDecimal(java.math.BigInteger, kotlin.Int, java.math.MathContext)kotlin/java.math.-big-integer/to-big-decimal.html +$dokka.location:kotlin$unaryMinus(java.math.BigDecimal)kotlin/java.math.-big-decimal/unary-minus.html +$dokka.location:kotlin$unaryMinus(java.math.BigInteger)kotlin/java.math.-big-integer/unary-minus.html +$dokka.location:kotlin$xor(java.math.BigInteger, java.math.BigInteger)kotlin/java.math.-big-integer/xor.html +$dokka.location:kotlin.ArithmeticExceptionkotlin/-arithmetic-exception/index.html +$dokka.location:kotlin.AssertionErrorkotlin/-assertion-error/index.html +$dokka.location:kotlin.ClassCastExceptionkotlin/-class-cast-exception/index.html +$dokka.location:kotlin.Comparatorkotlin/-comparator/index.html +$dokka.location:kotlin.ConcurrentModificationExceptionkotlin/-concurrent-modification-exception/index.html +$dokka.location:kotlin.Errorkotlin/-error/index.html +$dokka.location:kotlin.Exceptionkotlin/-exception/index.html +$dokka.location:kotlin.IllegalArgumentExceptionkotlin/-illegal-argument-exception/index.html +$dokka.location:kotlin.IllegalStateExceptionkotlin/-illegal-state-exception/index.html +$dokka.location:kotlin.IndexOutOfBoundsExceptionkotlin/-index-out-of-bounds-exception/index.html +$dokka.location:kotlin.NoSuchElementExceptionkotlin/-no-such-element-exception/index.html +$dokka.location:kotlin.NullPointerExceptionkotlin/-null-pointer-exception/index.html +$dokka.location:kotlin.NumberFormatExceptionkotlin/-number-format-exception/index.html +$dokka.location:kotlin.RuntimeExceptionkotlin/-runtime-exception/index.html +$dokka.location:kotlin.Synchronizedkotlin/-synchronized/index.html +$dokka.location:kotlin.UnsupportedOperationExceptionkotlin/-unsupported-operation-exception/index.html +$dokka.location:kotlin.Volatilekotlin/-volatile/index.html +$dokka.location:kotlin.collections$getOrPut(java.util.concurrent.ConcurrentMap((kotlin.collections.getOrPut.K, kotlin.collections.getOrPut.V)), kotlin.collections.getOrPut.K, kotlin.Function0((kotlin.collections.getOrPut.V)))kotlin.collections/java.util.concurrent.-concurrent-map/get-or-put.html +$dokka.location:kotlin.collections$iterator(java.util.Enumeration((kotlin.collections.iterator.T)))kotlin.collections/java.util.-enumeration/iterator.html +$dokka.location:kotlin.collections$toList(java.util.Enumeration((kotlin.collections.toList.T)))kotlin.collections/java.util.-enumeration/to-list.html +$dokka.location:kotlin.collections.ArrayListkotlin.collections/-array-list/index.html +$dokka.location:kotlin.collections.HashMapkotlin.collections/-hash-map/index.html +$dokka.location:kotlin.collections.HashSetkotlin.collections/-hash-set/index.html +$dokka.location:kotlin.collections.LinkedHashMapkotlin.collections/-linked-hash-map/index.html +$dokka.location:kotlin.collections.LinkedHashSetkotlin.collections/-linked-hash-set/index.html +$dokka.location:kotlin.concurrent$getOrSet(java.lang.ThreadLocal((kotlin.concurrent.getOrSet.T)), kotlin.Function0((kotlin.concurrent.getOrSet.T)))kotlin.concurrent/java.lang.-thread-local/get-or-set.html +$dokka.location:kotlin.concurrent$read(java.util.concurrent.locks.ReentrantReadWriteLock, kotlin.Function0((kotlin.concurrent.read.T)))kotlin.concurrent/java.util.concurrent.locks.-reentrant-read-write-lock/read.html +$dokka.location:kotlin.concurrent$schedule(java.util.Timer, java.util.Date, kotlin.Function1((java.util.TimerTask, kotlin.Unit)))kotlin.concurrent/java.util.-timer/schedule.html +$dokka.location:kotlin.concurrent$schedule(java.util.Timer, java.util.Date, kotlin.Long, kotlin.Function1((java.util.TimerTask, kotlin.Unit)))kotlin.concurrent/java.util.-timer/schedule.html +$dokka.location:kotlin.concurrent$schedule(java.util.Timer, kotlin.Long, kotlin.Function1((java.util.TimerTask, kotlin.Unit)))kotlin.concurrent/java.util.-timer/schedule.html +$dokka.location:kotlin.concurrent$schedule(java.util.Timer, kotlin.Long, kotlin.Long, kotlin.Function1((java.util.TimerTask, kotlin.Unit)))kotlin.concurrent/java.util.-timer/schedule.html +$dokka.location:kotlin.concurrent$scheduleAtFixedRate(java.util.Timer, java.util.Date, kotlin.Long, kotlin.Function1((java.util.TimerTask, kotlin.Unit)))kotlin.concurrent/java.util.-timer/schedule-at-fixed-rate.html +$dokka.location:kotlin.concurrent$scheduleAtFixedRate(java.util.Timer, kotlin.Long, kotlin.Long, kotlin.Function1((java.util.TimerTask, kotlin.Unit)))kotlin.concurrent/java.util.-timer/schedule-at-fixed-rate.html +$dokka.location:kotlin.concurrent$withLock(java.util.concurrent.locks.Lock, kotlin.Function0((kotlin.concurrent.withLock.T)))kotlin.concurrent/java.util.concurrent.locks.-lock/with-lock.html +$dokka.location:kotlin.concurrent$write(java.util.concurrent.locks.ReentrantReadWriteLock, kotlin.Function0((kotlin.concurrent.write.T)))kotlin.concurrent/java.util.concurrent.locks.-reentrant-read-write-lock/write.html +$dokka.location:kotlin.io$appendBytes(java.io.File, kotlin.ByteArray)kotlin.io/java.io.-file/append-bytes.html +$dokka.location:kotlin.io$appendText(java.io.File, kotlin.String, java.nio.charset.Charset)kotlin.io/java.io.-file/append-text.html +$dokka.location:kotlin.io$buffered(java.io.InputStream, kotlin.Int)kotlin.io/java.io.-input-stream/buffered.html +$dokka.location:kotlin.io$buffered(java.io.OutputStream, kotlin.Int)kotlin.io/java.io.-output-stream/buffered.html +$dokka.location:kotlin.io$buffered(java.io.Reader, kotlin.Int)kotlin.io/java.io.-reader/buffered.html +$dokka.location:kotlin.io$buffered(java.io.Writer, kotlin.Int)kotlin.io/java.io.-writer/buffered.html +$dokka.location:kotlin.io$bufferedReader(java.io.File, java.nio.charset.Charset, kotlin.Int)kotlin.io/java.io.-file/buffered-reader.html +$dokka.location:kotlin.io$bufferedReader(java.io.InputStream, java.nio.charset.Charset)kotlin.io/java.io.-input-stream/buffered-reader.html +$dokka.location:kotlin.io$bufferedWriter(java.io.File, java.nio.charset.Charset, kotlin.Int)kotlin.io/java.io.-file/buffered-writer.html +$dokka.location:kotlin.io$bufferedWriter(java.io.OutputStream, java.nio.charset.Charset)kotlin.io/java.io.-output-stream/buffered-writer.html +$dokka.location:kotlin.io$copyRecursively(java.io.File, java.io.File, kotlin.Boolean, kotlin.Function2((java.io.File, java.io.IOException, kotlin.io.OnErrorAction)))kotlin.io/java.io.-file/copy-recursively.html +$dokka.location:kotlin.io$copyTo(java.io.File, java.io.File, kotlin.Boolean, kotlin.Int)kotlin.io/java.io.-file/copy-to.html +$dokka.location:kotlin.io$copyTo(java.io.InputStream, java.io.OutputStream, kotlin.Int)kotlin.io/java.io.-input-stream/copy-to.html +$dokka.location:kotlin.io$copyTo(java.io.Reader, java.io.Writer, kotlin.Int)kotlin.io/java.io.-reader/copy-to.html +$dokka.location:kotlin.io$deleteRecursively(java.io.File)kotlin.io/java.io.-file/delete-recursively.html +$dokka.location:kotlin.io$endsWith(java.io.File, java.io.File)kotlin.io/java.io.-file/ends-with.html +$dokka.location:kotlin.io$endsWith(java.io.File, kotlin.String)kotlin.io/java.io.-file/ends-with.html +$dokka.location:kotlin.io$extension#java.io.Filekotlin.io/java.io.-file/extension.html +$dokka.location:kotlin.io$forEachBlock(java.io.File, kotlin.Function2((kotlin.ByteArray, kotlin.Int, kotlin.Unit)))kotlin.io/java.io.-file/for-each-block.html +$dokka.location:kotlin.io$forEachBlock(java.io.File, kotlin.Int, kotlin.Function2((kotlin.ByteArray, kotlin.Int, kotlin.Unit)))kotlin.io/java.io.-file/for-each-block.html +$dokka.location:kotlin.io$forEachLine(java.io.File, java.nio.charset.Charset, kotlin.Function1((kotlin.String, kotlin.Unit)))kotlin.io/java.io.-file/for-each-line.html +$dokka.location:kotlin.io$forEachLine(java.io.Reader, kotlin.Function1((kotlin.String, kotlin.Unit)))kotlin.io/java.io.-reader/for-each-line.html +$dokka.location:kotlin.io$inputStream(java.io.File)kotlin.io/java.io.-file/input-stream.html +$dokka.location:kotlin.io$invariantSeparatorsPath#java.io.Filekotlin.io/java.io.-file/invariant-separators-path.html +$dokka.location:kotlin.io$isRooted#java.io.Filekotlin.io/java.io.-file/is-rooted.html +$dokka.location:kotlin.io$iterator(java.io.BufferedInputStream)kotlin.io/java.io.-buffered-input-stream/iterator.html +$dokka.location:kotlin.io$lineSequence(java.io.BufferedReader)kotlin.io/java.io.-buffered-reader/line-sequence.html +$dokka.location:kotlin.io$nameWithoutExtension#java.io.Filekotlin.io/java.io.-file/name-without-extension.html +$dokka.location:kotlin.io$normalize(java.io.File)kotlin.io/java.io.-file/normalize.html +$dokka.location:kotlin.io$outputStream(java.io.File)kotlin.io/java.io.-file/output-stream.html +$dokka.location:kotlin.io$printWriter(java.io.File, java.nio.charset.Charset)kotlin.io/java.io.-file/print-writer.html +$dokka.location:kotlin.io$readBytes(java.io.File)kotlin.io/java.io.-file/read-bytes.html +$dokka.location:kotlin.io$readBytes(java.io.InputStream)kotlin.io/java.io.-input-stream/read-bytes.html +$dokka.location:kotlin.io$readBytes(java.io.InputStream, kotlin.Int)kotlin.io/java.io.-input-stream/read-bytes.html +$dokka.location:kotlin.io$readBytes(java.net.URL)kotlin.io/java.net.-u-r-l/read-bytes.html +$dokka.location:kotlin.io$readLines(java.io.File, java.nio.charset.Charset)kotlin.io/java.io.-file/read-lines.html +$dokka.location:kotlin.io$readLines(java.io.Reader)kotlin.io/java.io.-reader/read-lines.html +$dokka.location:kotlin.io$readText(java.io.File, java.nio.charset.Charset)kotlin.io/java.io.-file/read-text.html +$dokka.location:kotlin.io$readText(java.io.Reader)kotlin.io/java.io.-reader/read-text.html +$dokka.location:kotlin.io$readText(java.net.URL, java.nio.charset.Charset)kotlin.io/java.net.-u-r-l/read-text.html +$dokka.location:kotlin.io$reader(java.io.File, java.nio.charset.Charset)kotlin.io/java.io.-file/reader.html +$dokka.location:kotlin.io$reader(java.io.InputStream, java.nio.charset.Charset)kotlin.io/java.io.-input-stream/reader.html +$dokka.location:kotlin.io$relativeTo(java.io.File, java.io.File)kotlin.io/java.io.-file/relative-to.html +$dokka.location:kotlin.io$relativeToOrNull(java.io.File, java.io.File)kotlin.io/java.io.-file/relative-to-or-null.html +$dokka.location:kotlin.io$relativeToOrSelf(java.io.File, java.io.File)kotlin.io/java.io.-file/relative-to-or-self.html +$dokka.location:kotlin.io$resolve(java.io.File, java.io.File)kotlin.io/java.io.-file/resolve.html +$dokka.location:kotlin.io$resolve(java.io.File, kotlin.String)kotlin.io/java.io.-file/resolve.html +$dokka.location:kotlin.io$resolveSibling(java.io.File, java.io.File)kotlin.io/java.io.-file/resolve-sibling.html +$dokka.location:kotlin.io$resolveSibling(java.io.File, kotlin.String)kotlin.io/java.io.-file/resolve-sibling.html +$dokka.location:kotlin.io$startsWith(java.io.File, java.io.File)kotlin.io/java.io.-file/starts-with.html +$dokka.location:kotlin.io$startsWith(java.io.File, kotlin.String)kotlin.io/java.io.-file/starts-with.html +$dokka.location:kotlin.io$toRelativeString(java.io.File, java.io.File)kotlin.io/java.io.-file/to-relative-string.html +$dokka.location:kotlin.io$useLines(java.io.File, java.nio.charset.Charset, kotlin.Function1((kotlin.sequences.Sequence((kotlin.String)), kotlin.io.useLines.T)))kotlin.io/java.io.-file/use-lines.html +$dokka.location:kotlin.io$useLines(java.io.Reader, kotlin.Function1((kotlin.sequences.Sequence((kotlin.String)), kotlin.io.useLines.T)))kotlin.io/java.io.-reader/use-lines.html +$dokka.location:kotlin.io$walk(java.io.File, kotlin.io.FileWalkDirection)kotlin.io/java.io.-file/walk.html +$dokka.location:kotlin.io$walkBottomUp(java.io.File)kotlin.io/java.io.-file/walk-bottom-up.html +$dokka.location:kotlin.io$walkTopDown(java.io.File)kotlin.io/java.io.-file/walk-top-down.html +$dokka.location:kotlin.io$writeBytes(java.io.File, kotlin.ByteArray)kotlin.io/java.io.-file/write-bytes.html +$dokka.location:kotlin.io$writeText(java.io.File, kotlin.String, java.nio.charset.Charset)kotlin.io/java.io.-file/write-text.html +$dokka.location:kotlin.io$writer(java.io.File, java.nio.charset.Charset)kotlin.io/java.io.-file/writer.html +$dokka.location:kotlin.io$writer(java.io.OutputStream, java.nio.charset.Charset)kotlin.io/java.io.-output-stream/writer.html +$dokka.location:kotlin.jvm$kotlin#java.lang.Class((kotlin.jvm.kotlin.T))kotlin.jvm/java.lang.-class/kotlin.html +$dokka.location:kotlin.random$asKotlinRandom(java.util.Random)kotlin.random/java.util.-random/as-kotlin-random.html +$dokka.location:kotlin.reflect.KAnnotatedElementkotlin.reflect/-k-annotated-element/index.html +$dokka.location:kotlin.reflect.KDeclarationContainerkotlin.reflect/-k-declaration-container/index.html +$dokka.location:kotlin.reflect.KFunctionkotlin.reflect/-k-function/index.html +$dokka.location:kotlin.reflect.KMutablePropertykotlin.reflect/-k-mutable-property/index.html +$dokka.location:kotlin.reflect.KPropertykotlin.reflect/-k-property/index.html +$dokka.location:kotlin.reflect.jvm$kotlinFunction#java.lang.reflect.Constructor((kotlin.reflect.jvm.kotlinFunction.T))kotlin.reflect.jvm/java.lang.reflect.-constructor/kotlin-function.html +$dokka.location:kotlin.reflect.jvm$kotlinFunction#java.lang.reflect.Methodkotlin.reflect.jvm/java.lang.reflect.-method/kotlin-function.html +$dokka.location:kotlin.reflect.jvm$kotlinProperty#java.lang.reflect.Fieldkotlin.reflect.jvm/java.lang.reflect.-field/kotlin-property.html +$dokka.location:kotlin.sequences$asSequence(java.util.Enumeration((kotlin.sequences.asSequence.T)))kotlin.sequences/java.util.-enumeration/as-sequence.html +$dokka.location:kotlin.streams$asSequence(java.util.stream.DoubleStream)kotlin.streams/java.util.stream.-double-stream/as-sequence.html +$dokka.location:kotlin.streams$asSequence(java.util.stream.IntStream)kotlin.streams/java.util.stream.-int-stream/as-sequence.html +$dokka.location:kotlin.streams$asSequence(java.util.stream.LongStream)kotlin.streams/java.util.stream.-long-stream/as-sequence.html +$dokka.location:kotlin.streams$asSequence(java.util.stream.Stream((kotlin.streams.asSequence.T)))kotlin.streams/java.util.stream.-stream/as-sequence.html +$dokka.location:kotlin.streams$asStream(kotlin.sequences.Sequence((kotlin.streams.asStream.T)))kotlin.streams/kotlin.sequences.-sequence/as-stream.html +$dokka.location:kotlin.streams$toList(java.util.stream.DoubleStream)kotlin.streams/java.util.stream.-double-stream/to-list.html +$dokka.location:kotlin.streams$toList(java.util.stream.IntStream)kotlin.streams/java.util.stream.-int-stream/to-list.html +$dokka.location:kotlin.streams$toList(java.util.stream.LongStream)kotlin.streams/java.util.stream.-long-stream/to-list.html +$dokka.location:kotlin.streams$toList(java.util.stream.Stream((kotlin.streams.toList.T)))kotlin.streams/java.util.stream.-stream/to-list.html +$dokka.location:kotlin.text$appendRange(java.lang.StringBuilder, kotlin.CharArray, kotlin.Int, kotlin.Int)kotlin.text/java.lang.-string-builder/append-range.html +$dokka.location:kotlin.text$appendRange(java.lang.StringBuilder, kotlin.CharSequence, kotlin.Int, kotlin.Int)kotlin.text/java.lang.-string-builder/append-range.html +$dokka.location:kotlin.text$appendln(java.lang.Appendable)kotlin.text/java.lang.-appendable/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.Appendable, kotlin.Char)kotlin.text/java.lang.-appendable/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.Appendable, kotlin.CharSequence)kotlin.text/java.lang.-appendable/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, java.lang.StringBuffer)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, java.lang.StringBuilder)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.Any)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.Boolean)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.Byte)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.Char)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.CharArray)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.CharSequence)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.Double)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.Float)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.Int)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.Long)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.Short)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$appendln(java.lang.StringBuilder, kotlin.String)kotlin.text/java.lang.-string-builder/appendln.html +$dokka.location:kotlin.text$clear(java.lang.StringBuilder)kotlin.text/java.lang.-string-builder/clear.html +$dokka.location:kotlin.text$deleteAt(java.lang.StringBuilder, kotlin.Int)kotlin.text/java.lang.-string-builder/delete-at.html +$dokka.location:kotlin.text$deleteRange(java.lang.StringBuilder, kotlin.Int, kotlin.Int)kotlin.text/java.lang.-string-builder/delete-range.html +$dokka.location:kotlin.text$insertRange(java.lang.StringBuilder, kotlin.Int, kotlin.CharArray, kotlin.Int, kotlin.Int)kotlin.text/java.lang.-string-builder/insert-range.html +$dokka.location:kotlin.text$insertRange(java.lang.StringBuilder, kotlin.Int, kotlin.CharSequence, kotlin.Int, kotlin.Int)kotlin.text/java.lang.-string-builder/insert-range.html +$dokka.location:kotlin.text$set(java.lang.StringBuilder, kotlin.Int, kotlin.Char)kotlin.text/java.lang.-string-builder/set.html +$dokka.location:kotlin.text$setRange(java.lang.StringBuilder, kotlin.Int, kotlin.Int, kotlin.String)kotlin.text/java.lang.-string-builder/set-range.html +$dokka.location:kotlin.text$toCharArray(java.lang.StringBuilder, kotlin.CharArray, kotlin.Int, kotlin.Int, kotlin.Int)kotlin.text/java.lang.-string-builder/to-char-array.html +$dokka.location:kotlin.text$toRegex(java.util.regex.Pattern)kotlin.text/java.util.regex.-pattern/to-regex.html +$dokka.location:kotlin.text.Appendablekotlin.text/-appendable/index.html +$dokka.location:kotlin.text.CharacterCodingExceptionkotlin.text/-character-coding-exception/index.html +$dokka.location:kotlin.text.StringBuilderkotlin.text/-string-builder/index.html +$dokka.location:kotlin.time$toKotlinDuration(java.time.Duration)kotlin.time/java.time.-duration/to-kotlin-duration.html +$dokka.location:kotlin.time.DurationUnitkotlin.time/-duration-unit/index.html +$dokka.location:kotlin.time.MonoClockkotlin.time/-mono-clock/index.html +kotlin +kotlin.annotation +kotlin.browser +kotlin.collections +kotlin.comparisons +kotlin.concurrent +kotlin.contracts +kotlin.coroutines +kotlin.coroutines.experimental +kotlin.coroutines.experimental.intrinsics +kotlin.coroutines.intrinsics +kotlin.dom +kotlin.experimental +kotlin.io +kotlin.js +kotlin.jvm +kotlin.math +kotlin.native +kotlin.native.concurrent +kotlin.native.ref +kotlin.properties +kotlin.random +kotlin.ranges +kotlin.reflect +kotlin.reflect.full +kotlin.reflect.jvm +kotlin.sequences +kotlin.streams +kotlin.system +kotlin.text +kotlin.time +kotlinx.cinterop +kotlinx.cinterop.internal +kotlinx.wasm.jsinterop +org.khronos.webgl +org.w3c.css.masking +org.w3c.dom +org.w3c.dom.clipboard +org.w3c.dom.css +org.w3c.dom.events +org.w3c.dom.mediacapture +org.w3c.dom.parsing +org.w3c.dom.pointerevents +org.w3c.dom.svg +org.w3c.dom.url +org.w3c.fetch +org.w3c.files +org.w3c.notifications +org.w3c.performance +org.w3c.workers +org.w3c.xhr diff --git a/test-utils/build.gradle.kts b/test-utils/build.gradle.kts new file mode 100644 index 0000000000..3461d5292b --- /dev/null +++ b/test-utils/build.gradle.kts @@ -0,0 +1,33 @@ +/* + * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. + */ + +kotlin { + sourceSets { + commonMain.dependencies { + api("org.jetbrains.kotlin:kotlin-test-common:${version("kotlin")}") + api("org.jetbrains.kotlin:kotlin-test-annotations-common:${version("kotlin")}") + } + jvmMain.dependencies { + api("org.jetbrains.kotlin:kotlin-test:${version("kotlin")}") + // Workaround to make addSuppressed work in tests + api("org.jetbrains.kotlin:kotlin-reflect:${version("kotlin")}") + api("org.jetbrains.kotlin:kotlin-stdlib-jdk7:${version("kotlin")}") + api("org.jetbrains.kotlin:kotlin-test-junit:${version("kotlin")}") + api("junit:junit:${version("junit")}") + } + jsMain.dependencies { + api("org.jetbrains.kotlin:kotlin-test-js:${version("kotlin")}") + } + val wasmJsMain by getting { + dependencies { + api("org.jetbrains.kotlin:kotlin-test-wasm-js:${version("kotlin")}") + } + } + val wasmWasiMain by getting { + dependencies { + api("org.jetbrains.kotlin:kotlin-test-wasm-wasi:${version("kotlin")}") + } + } + } +} diff --git a/test-utils/common/src/LaunchFlow.kt b/test-utils/common/src/LaunchFlow.kt new file mode 100644 index 0000000000..c13e07f505 --- /dev/null +++ b/test-utils/common/src/LaunchFlow.kt @@ -0,0 +1,95 @@ +package kotlinx.coroutines.testing.flow + +import kotlinx.coroutines.* +import kotlinx.coroutines.flow.* +import kotlin.jvm.* +import kotlin.reflect.* + +public typealias Handler = suspend CoroutineScope.(T) -> Unit + +/* + * Design of this builder is not yet stable, so leaving it as is. + */ +public class LaunchFlowBuilder { + /* + * NB: this implementation is a temporary ad-hoc (and slightly incorrect) + * solution until coroutine-builders are ready + * + * NB 2: this internal stuff is required to workaround KT-30795 + */ + @PublishedApi + internal var onEach: Handler? = null + @PublishedApi + internal var finally: Handler? = null + @PublishedApi + internal var exceptionHandlers = LinkedHashMap, Handler>() + + public fun onEach(action: suspend CoroutineScope.(value: T) -> Unit) { + check(onEach == null) { "onEach block is already registered" } + check(exceptionHandlers.isEmpty()) { "onEach block should be registered before exceptionHandlers block" } + check(finally == null) { "onEach block should be registered before finally block" } + onEach = action + } + + public inline fun catch(noinline action: suspend CoroutineScope.(T) -> Unit) { + check(onEach != null) { "onEach block should be registered first" } + check(finally == null) { "exceptionHandlers block should be registered before finally block" } + @Suppress("UNCHECKED_CAST") + exceptionHandlers[T::class] = action as Handler + } + + public fun finally(action: suspend CoroutineScope.(cause: Throwable?) -> Unit) { + check(finally == null) { "Finally block is already registered" } + check(onEach != null) { "onEach block should be registered before finally block" } + if (finally == null) finally = action + } + + internal fun build(): Handlers = + Handlers(onEach ?: error("onEach is not registered"), exceptionHandlers, finally) +} + +internal class Handlers( + @JvmField + internal var onEach: Handler, + @JvmField + internal var exceptionHandlers: Map, Handler>, + @JvmField + internal var finally: Handler? +) + +private fun CoroutineScope.launchFlow( + flow: Flow, + builder: LaunchFlowBuilder.() -> Unit +): Job { + val handlers = LaunchFlowBuilder().apply(builder).build() + return launch { + var caught: Throwable? = null + try { + coroutineScope { + flow.collect { value -> + handlers.onEach(this, value) + } + } + } catch (e: Throwable) { + handlers.exceptionHandlers.forEach { (key, value) -> + if (key.isInstance(e)) { + caught = e + value.invoke(this, e) + return@forEach + } + } + if (caught == null) { + caught = e + throw e + } + } finally { + cancel() // TODO discuss + handlers.finally?.invoke(CoroutineScope(coroutineContext + NonCancellable), caught) + } + } +} + +public fun Flow.launchIn( + scope: CoroutineScope, + builder: LaunchFlowBuilder.() -> Unit +): Job = scope.launchFlow(this, builder) diff --git a/test-utils/common/src/MainDispatcherTestBase.kt b/test-utils/common/src/MainDispatcherTestBase.kt new file mode 100644 index 0000000000..dd867a2d64 --- /dev/null +++ b/test-utils/common/src/MainDispatcherTestBase.kt @@ -0,0 +1,270 @@ +package kotlinx.coroutines.testing + +import kotlinx.coroutines.* +import kotlin.test.* + +abstract class MainDispatcherTestBase: TestBase() { + + open fun shouldSkipTesting(): Boolean = false + + open suspend fun spinTest(testBody: Job) { + testBody.join() + } + + abstract fun isMainThread(): Boolean? + + /** Runs the given block as a test, unless [shouldSkipTesting] indicates that the environment is not suitable. */ + fun runTestOrSkip(block: suspend CoroutineScope.() -> Unit): TestResult { + // written as a block body to make the need to return `TestResult` explicit + return runTest { + if (shouldSkipTesting()) return@runTest + val testBody = launch(Dispatchers.Default) { + block() + } + spinTest(testBody) + } + } + + /** Tests the [toString] behavior of [Dispatchers.Main] and [MainCoroutineDispatcher.immediate] */ + @Test + fun testMainDispatcherToString() { + assertEquals("Dispatchers.Main", Dispatchers.Main.toString()) + assertEquals("Dispatchers.Main.immediate", Dispatchers.Main.immediate.toString()) + } + + /** Tests that the tasks scheduled earlier from [MainCoroutineDispatcher.immediate] will be executed earlier, + * even if the immediate dispatcher was entered from the main thread. */ + @Test + fun testMainDispatcherOrderingInMainThread() = runTestOrSkip { + withContext(Dispatchers.Main) { + testMainDispatcherOrdering() + } + } + + /** Tests that the tasks scheduled earlier from [MainCoroutineDispatcher.immediate] will be executed earlier + * if the immediate dispatcher was entered from outside the main thread. */ + @Test + fun testMainDispatcherOrderingOutsideMainThread() = runTestOrSkip { + testMainDispatcherOrdering() + } + + /** Tests that [Dispatchers.Main] and its [MainCoroutineDispatcher.immediate] are treated as different values. */ + @Test + fun testHandlerDispatcherNotEqualToImmediate() { + assertNotEquals(Dispatchers.Main, Dispatchers.Main.immediate) + } + + /** Tests that [Dispatchers.Main] shares its queue with [MainCoroutineDispatcher.immediate]. */ + @Test + fun testImmediateDispatcherYield() = runTestOrSkip { + withContext(Dispatchers.Main) { + expect(1) + checkIsMainThread() + // launch in the immediate dispatcher + launch(Dispatchers.Main.immediate) { + expect(2) + yield() + expect(4) + } + expect(3) // after yield + yield() // yield back + expect(5) + } + finish(6) + } + + /** Tests that entering [MainCoroutineDispatcher.immediate] from [Dispatchers.Main] happens immediately. */ + @Test + fun testEnteringImmediateFromMain() = runTestOrSkip { + withContext(Dispatchers.Main) { + expect(1) + val job = launch { expect(3) } + withContext(Dispatchers.Main.immediate) { + expect(2) + } + job.join() + } + finish(4) + } + + /** Tests that dispatching to [MainCoroutineDispatcher.immediate] is required from and only from dispatchers + * other than the main dispatchers and that it's always required for [Dispatchers.Main] itself. */ + @Test + fun testDispatchRequirements() = runTestOrSkip { + checkDispatchRequirements() + withContext(Dispatchers.Main) { + checkDispatchRequirements() + withContext(Dispatchers.Main.immediate) { + checkDispatchRequirements() + } + checkDispatchRequirements() + } + checkDispatchRequirements() + } + + private suspend fun checkDispatchRequirements() { + isMainThread()?.let { + assertNotEquals( + it, + Dispatchers.Main.immediate.isDispatchNeeded(currentCoroutineContext()) + ) + } + assertTrue(Dispatchers.Main.isDispatchNeeded(currentCoroutineContext())) + assertTrue(Dispatchers.Default.isDispatchNeeded(currentCoroutineContext())) + } + + /** Tests that launching a coroutine in [MainScope] will execute it in the main thread. */ + @Test + fun testLaunchInMainScope() = runTestOrSkip { + var executed = false + withMainScope { + launch { + checkIsMainThread() + executed = true + }.join() + if (!executed) throw AssertionError("Should be executed") + } + } + + /** Tests that a failure in [MainScope] will not propagate upwards. */ + @Test + fun testFailureInMainScope() = runTestOrSkip { + var exception: Throwable? = null + withMainScope { + launch(CoroutineExceptionHandler { ctx, e -> exception = e }) { + checkIsMainThread() + throw TestException() + }.join() + } + if (exception!! !is TestException) throw AssertionError("Expected TestException, but had $exception") + } + + /** Tests cancellation in [MainScope]. */ + @Test + fun testCancellationInMainScope() = runTestOrSkip { + withMainScope { + cancel() + launch(start = CoroutineStart.ATOMIC) { + checkIsMainThread() + delay(Long.MAX_VALUE) + }.join() + } + } + + private suspend fun withMainScope(block: suspend CoroutineScope.() -> R): R { + MainScope().apply { + return block().also { coroutineContext[Job]!!.cancelAndJoin() } + } + } + + private suspend fun testMainDispatcherOrdering() { + withContext(Dispatchers.Main.immediate) { + expect(1) + launch(Dispatchers.Main) { + expect(2) + } + withContext(Dispatchers.Main) { + finish(3) + } + } + } + + abstract class WithRealTimeDelay : MainDispatcherTestBase() { + abstract fun scheduleOnMainQueue(block: () -> Unit) + + /** Tests that after a delay, the execution gets back to the main thread. */ + @Test + fun testDelay() = runTestOrSkip { + expect(1) + checkNotMainThread() + scheduleOnMainQueue { expect(2) } + withContext(Dispatchers.Main) { + checkIsMainThread() + expect(3) + scheduleOnMainQueue { expect(4) } + delay(100) + checkIsMainThread() + expect(5) + } + checkNotMainThread() + finish(6) + } + + /** Tests that [Dispatchers.Main] is in agreement with the default time source: it's not much slower. */ + @Test + fun testWithTimeoutContextDelayNoTimeout() = runTestOrSkip { + expect(1) + withTimeout(1000) { + withContext(Dispatchers.Main) { + checkIsMainThread() + expect(2) + delay(100) + checkIsMainThread() + expect(3) + } + } + checkNotMainThread() + finish(4) + } + + /** Tests that [Dispatchers.Main] is in agreement with the default time source: it's not much faster. */ + @Test + fun testWithTimeoutContextDelayTimeout() = runTestOrSkip { + expect(1) + assertFailsWith { + withTimeout(300) { + // A substitute for withContext(Dispatcher.Main) that is started even if the 300ms + // timeout happens fsater then dispatch + launch(Dispatchers.Main, start = CoroutineStart.ATOMIC) { + checkIsMainThread() + expect(2) + delay(1000) + expectUnreached() + }.join() + } + expectUnreached() + } + checkNotMainThread() + finish(3) + } + + /** Tests that the timeout of [Dispatchers.Main] is in agreement with its [delay]: it's not much faster. */ + @Test + fun testWithContextTimeoutDelayNoTimeout() = runTestOrSkip { + expect(1) + withContext(Dispatchers.Main) { + withTimeout(1000) { + checkIsMainThread() + expect(2) + delay(100) + checkIsMainThread() + expect(3) + } + } + checkNotMainThread() + finish(4) + } + + /** Tests that the timeout of [Dispatchers.Main] is in agreement with its [delay]: it's not much slower. */ + @Test + fun testWithContextTimeoutDelayTimeout() = runTestOrSkip { + expect(1) + assertFailsWith { + withContext(Dispatchers.Main) { + withTimeout(100) { + checkIsMainThread() + expect(2) + delay(1000) + expectUnreached() + } + } + expectUnreached() + } + checkNotMainThread() + finish(3) + } + } + + fun checkIsMainThread() { isMainThread()?.let { check(it) } } + fun checkNotMainThread() { isMainThread()?.let { check(!it) } } +} diff --git a/test-utils/common/src/TestBase.common.kt b/test-utils/common/src/TestBase.common.kt new file mode 100644 index 0000000000..100addd2e5 --- /dev/null +++ b/test-utils/common/src/TestBase.common.kt @@ -0,0 +1,301 @@ +@file:Suppress("unused") +package kotlinx.coroutines.testing + +import kotlinx.atomicfu.* +import kotlinx.coroutines.flow.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* +import kotlin.coroutines.* +import kotlin.test.* +import kotlin.time.* +import kotlin.time.Duration.Companion.seconds + +/** + * The number of milliseconds that is sure not to pass [assertRunsFast]. + */ +const val SLOW = 100_000L + +/** + * Asserts that a block completed within [timeout]. + */ +inline fun assertRunsFast(timeout: Duration, block: () -> T): T { + val result: T + val elapsed = TimeSource.Monotonic.measureTime { result = block() } + assertTrue("Should complete in $timeout, but took $elapsed") { elapsed < timeout } + return result +} + +/** + * Asserts that a block completed within two seconds. + */ +inline fun assertRunsFast(block: () -> T): T = assertRunsFast(2.seconds, block) + +/** + * Whether the tests should trace their calls to `expect` and `finish` with `println`. + * `false` by default. On the JVM, can be set to `true` by setting the `test.verbose` system property. + */ +expect val VERBOSE: Boolean + +interface OrderedExecution { + /** Expect the next action to be [index] in order. */ + fun expect(index: Int) + + /** Expect this action to be final, with the given [index]. */ + fun finish(index: Int) + + /** * Asserts that this line is never executed. */ + fun expectUnreached() + + /** + * Checks that [finish] was called. + * + * By default, it is allowed to not call [finish] if [expect] was not called. + * This is useful for tests that don't check the ordering of events. + * When [allowNotUsingExpect] is set to `false`, it is an error to not call [finish] in any case. + */ + fun checkFinishCall(allowNotUsingExpect: Boolean = true) + + class Impl : OrderedExecution { + private val actionIndex = atomic(0) + + override fun expect(index: Int) { + val wasIndex = actionIndex.incrementAndGet() + if (VERBOSE) println("expect($index), wasIndex=$wasIndex") + check(index == wasIndex) { + if (wasIndex < 0) "Expecting action index $index but it is actually finished" + else "Expecting action index $index but it is actually $wasIndex" + } + } + + override fun finish(index: Int) { + val wasIndex = actionIndex.getAndSet(Int.MIN_VALUE) + 1 + if (VERBOSE) println("finish($index), wasIndex=${if (wasIndex < 0) "finished" else wasIndex}") + check(index == wasIndex) { + if (wasIndex < 0) "Finished more than once" + else "Finishing with action index $index but it is actually $wasIndex" + } + } + + override fun expectUnreached() { + error("Should not be reached, ${ + actionIndex.value.let { + when { + it < 0 -> "already finished" + it == 0 -> "'expect' was not called yet" + else -> "the last executed action was $it" + } + } + }") + } + + override fun checkFinishCall(allowNotUsingExpect: Boolean) { + actionIndex.value.let { + assertTrue( + it < 0 || allowNotUsingExpect && it == 0, + "Expected `finish(${actionIndex.value + 1})` to be called, but the test finished" + ) + } + } + } +} + +interface ErrorCatching { + /** + * Returns `true` if errors were logged in the test. + */ + fun hasError(): Boolean + + /** + * Directly reports an error to the test catching facilities. + */ + fun reportError(error: Throwable) + + class Impl : ErrorCatching { + + private val errors = mutableListOf() + private val lock = SynchronizedObject() + private var closed = false + + override fun hasError(): Boolean = synchronized(lock) { + errors.isNotEmpty() + } + + override fun reportError(error: Throwable) { + synchronized(lock) { + if (closed) { + lastResortReportException(error) + } else { + errors.add(error) + } + } + } + + fun close() { + synchronized(lock) { + if (closed) { + val error = IllegalStateException("ErrorCatching closed more than once") + lastResortReportException(error) + errors.add(error) + } + closed = true + errors.firstOrNull()?.let { + for (error in errors.drop(1)) + it.addSuppressed(error) + throw it + } + } + } + } +} + +/** + * Reports an error *somehow* so that it doesn't get completely forgotten. + */ +internal expect fun lastResortReportException(error: Throwable) + +/** + * Throws [IllegalStateException] when `value` is false, like `check` in stdlib, but also ensures that the + * test will not complete successfully even if this exception is consumed somewhere in the test. + */ +public inline fun ErrorCatching.check(value: Boolean, lazyMessage: () -> Any) { + if (!value) error(lazyMessage()) +} + +/** + * Throws [IllegalStateException], like `error` in stdlib, but also ensures that the test will not + * complete successfully even if this exception is consumed somewhere in the test. + */ +fun ErrorCatching.error(message: Any, cause: Throwable? = null): Nothing { + throw IllegalStateException(message.toString(), cause).also { + reportError(it) + } +} + +/** + * A class inheriting from which allows to check the execution order inside tests. + * + * @see TestBase + */ +open class OrderedExecutionTestBase : OrderedExecution +{ + // TODO: move to by-delegation when [reset] is no longer needed. + private var orderedExecutionDelegate = OrderedExecution.Impl() + + @AfterTest + fun checkFinished() { orderedExecutionDelegate.checkFinishCall() } + + /** Resets counter and finish flag. Workaround for parametrized tests absence in common */ + public fun reset() { + orderedExecutionDelegate.checkFinishCall() + orderedExecutionDelegate = OrderedExecution.Impl() + } + + override fun expect(index: Int) = orderedExecutionDelegate.expect(index) + + override fun finish(index: Int) = orderedExecutionDelegate.finish(index) + + override fun expectUnreached() = orderedExecutionDelegate.expectUnreached() + + override fun checkFinishCall(allowNotUsingExpect: Boolean) = + orderedExecutionDelegate.checkFinishCall(allowNotUsingExpect) +} + +fun T.void() {} + +@OptionalExpectation +expect annotation class NoJs() + +@OptionalExpectation +expect annotation class NoNative() + +@OptionalExpectation +expect annotation class NoWasmJs() + +@OptionalExpectation +expect annotation class NoWasmWasi() + +expect val isStressTest: Boolean +expect val stressTestMultiplier: Int +expect val stressTestMultiplierSqrt: Int + +/** + * The result of a multiplatform asynchronous test. + * Aliases into Unit on K/JVM and K/N, and into Promise on K/JS. + */ +@Suppress("NO_ACTUAL_FOR_EXPECT") +public expect class TestResult + +public expect open class TestBase(): OrderedExecutionTestBase, ErrorCatching { + public fun println(message: Any?) + + public fun runTest( + expected: ((Throwable) -> Boolean)? = null, + unhandled: List<(Throwable) -> Boolean> = emptyList(), + block: suspend CoroutineScope.() -> Unit + ): TestResult + + override fun hasError(): Boolean + override fun reportError(error: Throwable) +} + +public suspend inline fun hang(onCancellation: () -> Unit) { + try { + suspendCancellableCoroutine { } + } finally { + onCancellation() + } +} + +suspend inline fun assertFailsWith(flow: Flow<*>) = assertFailsWith { flow.collect() } + +public suspend fun Flow.sum() = fold(0) { acc, value -> acc + value } +public suspend fun Flow.longSum() = fold(0L) { acc, value -> acc + value } + +// data is added to avoid stacktrace recovery because CopyableThrowable is not accessible from common modules +public class TestException(message: String? = null, private val data: Any? = null) : Throwable(message) +public class TestException1(message: String? = null, private val data: Any? = null) : Throwable(message) +public class TestException2(message: String? = null, private val data: Any? = null) : Throwable(message) +public class TestException3(message: String? = null, private val data: Any? = null) : Throwable(message) +public class TestCancellationException(message: String? = null, private val data: Any? = null) : + CancellationException(message) + +public class TestRuntimeException(message: String? = null, private val data: Any? = null) : RuntimeException(message) +public class RecoverableTestException(message: String? = null) : RuntimeException(message) +public class RecoverableTestCancellationException(message: String? = null) : CancellationException(message) + +// Erases identity and equality checks for tests +public fun wrapperDispatcher(context: CoroutineContext): CoroutineContext { + val dispatcher = context[ContinuationInterceptor] as CoroutineDispatcher + return object : CoroutineDispatcher() { + override fun isDispatchNeeded(context: CoroutineContext): Boolean = + dispatcher.isDispatchNeeded(context) + + override fun dispatch(context: CoroutineContext, block: Runnable) = + dispatcher.dispatch(context, block) + } +} + +public suspend fun wrapperDispatcher(): CoroutineContext = wrapperDispatcher(coroutineContext) +class BadClass { + override fun equals(other: Any?): Boolean = error("equals") + override fun hashCode(): Int = error("hashCode") + override fun toString(): String = error("toString") +} + +public expect val isJavaAndWindows: Boolean + +public expect val isNative: Boolean + +/* + * In common tests we emulate parameterized tests + * by iterating over parameters space in the single @Test method. + * This kind of tests is too slow for JS and does not fit into + * the default Mocha timeout, so we're using this flag to bail-out + * and run such tests only on JVM and K/N. + */ +public expect val isBoundByJsTestTimeout: Boolean + +/** + * `true` if this platform has the same event loop for `DefaultExecutor` and [Dispatchers.Unconfined] + */ +public expect val usesSharedEventLoop: Boolean diff --git a/test-utils/js/src/TestBase.kt b/test-utils/js/src/TestBase.kt new file mode 100644 index 0000000000..c6223e28e8 --- /dev/null +++ b/test-utils/js/src/TestBase.kt @@ -0,0 +1,106 @@ +package kotlinx.coroutines.testing + +import kotlinx.coroutines.* +import kotlin.test.* +import kotlin.js.* + +actual typealias NoJs = Ignore + +actual val VERBOSE = false + +actual val isStressTest: Boolean = false +actual val stressTestMultiplier: Int = 1 +actual val stressTestMultiplierSqrt: Int = 1 + +@JsName("Promise") +external class MyPromise { + fun then(onFulfilled: ((Unit) -> Unit), onRejected: ((Throwable) -> Unit)): MyPromise + fun then(onFulfilled: ((Unit) -> Unit)): MyPromise +} + +/** Always a `Promise` */ +public actual typealias TestResult = MyPromise + +internal actual fun lastResortReportException(error: Throwable) { + println(error) + console.log(error) +} + +actual open class TestBase( + private val errorCatching: ErrorCatching.Impl +): OrderedExecutionTestBase(), ErrorCatching by errorCatching { + private var lastTestPromise: Promise<*>? = null + + actual constructor(): this(errorCatching = ErrorCatching.Impl()) + + actual fun println(message: Any?) { + kotlin.io.println(message) + } + + actual fun runTest( + expected: ((Throwable) -> Boolean)?, + unhandled: List<(Throwable) -> Boolean>, + block: suspend CoroutineScope.() -> Unit + ): TestResult { + var exCount = 0 + var ex: Throwable? = null + /* + * This is an additional sanity check against `runTest` mis-usage on JS. + * The only way to write an async test on JS is to return Promise from the test function. + * _Just_ launching promise and returning `Unit` won't suffice as the underlying test framework + * won't be able to detect an asynchronous failure in a timely manner. + * We cannot detect such situations, but we can detect the most common erroneous pattern + * in our code base, an attempt to use multiple `runTest` in the same `@Test` method, + * which typically is a premise to the same error: + * ``` + * @Test + * fun incorrectTestForJs() { // <- promise is not returned + * for (parameter in parameters) { + * runTest { + * runTestForParameter(parameter) + * } + * } + * } + * ``` + */ + if (lastTestPromise != null) { + error("Attempt to run multiple asynchronous test within one @Test method") + } + val result = GlobalScope.promise(block = block, context = CoroutineExceptionHandler { _, e -> + if (e is CancellationException) return@CoroutineExceptionHandler // are ignored + exCount++ + when { + exCount > unhandled.size -> + error("Too many unhandled exceptions $exCount, expected ${unhandled.size}, got: $e", e) + !unhandled[exCount - 1](e) -> + error("Unhandled exception was unexpected: $e", e) + } + }).catch { e -> + ex = e + if (expected != null) { + if (!expected(e)) { + console.log(e) + error("Unexpected exception $e", e) + } + } else + throw e + }.finally { + if (ex == null && expected != null) error("Exception was expected but none produced") + if (exCount < unhandled.size) + error("Too few unhandled exceptions $exCount, expected ${unhandled.size}") + errorCatching.close() + checkFinishCall() + } + lastTestPromise = result + @Suppress("CAST_NEVER_SUCCEEDS") + return result as MyPromise + } +} + +actual val isNative = false + +actual val isBoundByJsTestTimeout = true + +actual val isJavaAndWindows: Boolean get() = false + +actual val usesSharedEventLoop: Boolean = false diff --git a/test-utils/jvm/src/Exceptions.kt b/test-utils/jvm/src/Exceptions.kt new file mode 100644 index 0000000000..a06958b90b --- /dev/null +++ b/test-utils/jvm/src/Exceptions.kt @@ -0,0 +1,58 @@ +package kotlinx.coroutines.testing.exceptions + +import kotlinx.coroutines.* +import java.io.* +import java.util.* +import kotlin.contracts.* +import kotlin.coroutines.* +import kotlin.test.* + +inline fun checkException(exception: Throwable) { + assertIs(exception) + assertTrue(exception.suppressed.isEmpty()) + assertNull(exception.cause) +} + +fun checkCycles(t: Throwable) { + val sw = StringWriter() + t.printStackTrace(PrintWriter(sw)) + assertFalse(sw.toString().contains("CIRCULAR REFERENCE")) +} + +class CapturingHandler : AbstractCoroutineContextElement(CoroutineExceptionHandler), + CoroutineExceptionHandler +{ + private var unhandled: ArrayList? = ArrayList() + + override fun handleException(context: CoroutineContext, exception: Throwable) = synchronized(this) { + unhandled!!.add(exception) + } + + fun getException(): Throwable = synchronized(this) { + val size = unhandled!!.size + assert(size == 1) { "Expected one unhandled exception, but have $size: $unhandled" } + return unhandled!![0].also { unhandled = null } + } +} + +fun captureExceptionsRun( + context: CoroutineContext = EmptyCoroutineContext, + block: suspend CoroutineScope.() -> Unit +): Throwable { + val handler = CapturingHandler() + runBlocking(context + handler, block = block) + return handler.getException() +} + +@OptIn(ExperimentalContracts::class) +suspend inline fun assertCallsExceptionHandlerWith( + crossinline operation: suspend (CoroutineExceptionHandler) -> Unit): E { + contract { + callsInPlace(operation, InvocationKind.EXACTLY_ONCE) + } + val handler = CapturingHandler() + return withContext(handler) { + operation(handler) + assertIs(handler.getException()) + } +} diff --git a/test-utils/jvm/src/ExecutorRule.kt b/test-utils/jvm/src/ExecutorRule.kt new file mode 100644 index 0000000000..ed0c87f6b9 --- /dev/null +++ b/test-utils/jvm/src/ExecutorRule.kt @@ -0,0 +1,43 @@ +package kotlinx.coroutines.testing + +import kotlinx.coroutines.* +import org.junit.rules.* +import org.junit.runner.* +import org.junit.runners.model.* +import java.lang.Runnable +import java.util.concurrent.* +import kotlin.coroutines.* + +class ExecutorRule(private val numberOfThreads: Int) : TestRule, ExecutorCoroutineDispatcher() { + + private var _executor: ExecutorCoroutineDispatcher? = null + override val executor: Executor + get() = _executor?.executor ?: error("Executor is not initialized") + + override fun apply(base: Statement, description: Description): Statement { + return object : Statement() { + override fun evaluate() { + val threadPrefix = description.className.substringAfterLast(".") + "#" + description.methodName + _executor = newFixedThreadPoolContext(numberOfThreads, threadPrefix) + ignoreLostThreads(threadPrefix) + try { + return base.evaluate() + } finally { + val service = executor as ExecutorService + service.shutdown() + if (!service.awaitTermination(10, TimeUnit.SECONDS)) { + error("Test $description timed out") + } + } + } + } + } + + override fun dispatch(context: CoroutineContext, block: Runnable) { + _executor?.dispatch(context, block) ?: error("Executor is not initialized") + } + + override fun close() { + error("Cannot be closed manually") + } +} diff --git a/test-utils/jvm/src/FieldWalker.kt b/test-utils/jvm/src/FieldWalker.kt new file mode 100644 index 0000000000..e7303199aa --- /dev/null +++ b/test-utils/jvm/src/FieldWalker.kt @@ -0,0 +1,183 @@ +package kotlinx.coroutines.testing + +import java.lang.ref.* +import java.lang.reflect.* +import java.text.* +import java.util.* +import java.util.Collections.* +import java.util.concurrent.* +import java.util.concurrent.atomic.* +import java.util.concurrent.locks.* +import kotlin.test.* + +object FieldWalker { + sealed class Ref { + object RootRef : Ref() + class FieldRef(val parent: Any, val name: String) : Ref() + class ArrayRef(val parent: Any, val index: Int) : Ref() + } + + private val fieldsCache = HashMap, List>() + + init { + // excluded/terminal classes (don't walk them) + fieldsCache += listOf( + Any::class, String::class, Thread::class, Throwable::class, StackTraceElement::class, + WeakReference::class, ReferenceQueue::class, AbstractMap::class, Enum::class, + ReentrantLock::class, ReentrantReadWriteLock::class, SimpleDateFormat::class, ThreadPoolExecutor::class, + CountDownLatch::class, + ) + .map { it.java } + .associateWith { emptyList() } + } + + /* + * Reflectively starts to walk through object graph and returns identity set of all reachable objects. + * Use [walkRefs] if you need a path from root for debugging. + */ + public fun walk(root: Any?): Set = walkRefs(root, false).keys + + public fun assertReachableCount(expected: Int, root: Any?, rootStatics: Boolean = false, predicate: (Any) -> Boolean) { + val visited = walkRefs(root, rootStatics) + val actual = visited.keys.filter(predicate) + if (actual.size != expected) { + val textDump = actual.joinToString("") { "\n\t" + showPath(it, visited) } + assertEquals( + expected, actual.size, + "Unexpected number objects. Expected $expected, found ${actual.size}$textDump" + ) + } + } + + /* + * Reflectively starts to walk through object graph and map to all the reached object to their path + * in from root. Use [showPath] do display a path if needed. + */ + private fun walkRefs(root: Any?, rootStatics: Boolean): IdentityHashMap { + val visited = IdentityHashMap() + if (root == null) return visited + visited[root] = Ref.RootRef + val stack = ArrayDeque() + stack.addLast(root) + var statics = rootStatics + while (stack.isNotEmpty()) { + val element = stack.removeLast() + try { + visit(element, visited, stack, statics) + statics = false // only scan root static when asked + } catch (e: Exception) { + error("Failed to visit element ${showPath(element, visited)}: $e") + } + } + return visited + } + + private fun showPath(element: Any, visited: Map): String { + val path = ArrayList() + var cur = element + while (true) { + when (val ref = visited.getValue(cur)) { + Ref.RootRef -> break + is Ref.FieldRef -> { + cur = ref.parent + path += "|${ref.parent.javaClass.simpleName}::${ref.name}" + } + is Ref.ArrayRef -> { + cur = ref.parent + path += "[${ref.index}]" + } + else -> { + // Nothing, kludge for IDE + } + } + } + path.reverse() + return path.joinToString("") + } + + private fun visit(element: Any, visited: IdentityHashMap, stack: ArrayDeque, statics: Boolean) { + val type = element.javaClass + when { + // Special code for arrays + type.isArray && !type.componentType.isPrimitive -> { + @Suppress("UNCHECKED_CAST") + val array = element as Array + array.forEachIndexed { index, value -> + push(value, visited, stack) { Ref.ArrayRef(element, index) } + } + } + // Special code for platform types that cannot be reflectively accessed on modern JDKs + type.name.startsWith("java.") && element is Collection<*> -> { + element.forEachIndexed { index, value -> + push(value, visited, stack) { Ref.ArrayRef(element, index) } + } + } + type.name.startsWith("java.") && element is Map<*, *> -> { + push(element.keys, visited, stack) { Ref.FieldRef(element, "keys") } + push(element.values, visited, stack) { Ref.FieldRef(element, "values") } + } + element is AtomicReference<*> -> { + push(element.get(), visited, stack) { Ref.FieldRef(element, "value") } + } + element is AtomicReferenceArray<*> -> { + for (index in 0 until element.length()) { + push(element[index], visited, stack) { Ref.ArrayRef(element, index) } + } + } + element is AtomicLongFieldUpdater<*> -> { + /* filter it out here to suppress its subclasses too */ + } + element is ExecutorService && type.name == "java.util.concurrent.Executors\$DelegatedExecutorService" -> { + /* can't access anything in the executor */ + } + // All the other classes are reflectively scanned + else -> fields(type, statics).forEach { field -> + push(field.get(element), visited, stack) { Ref.FieldRef(element, field.name) } + // special case to scan Throwable cause (cannot get it reflectively) + if (element is Throwable) { + push(element.cause, visited, stack) { Ref.FieldRef(element, "cause") } + } + } + } + } + + private inline fun push(value: Any?, visited: IdentityHashMap, stack: ArrayDeque, ref: () -> Ref) { + if (value != null && !visited.containsKey(value)) { + visited[value] = ref() + stack.addLast(value) + } + } + + private fun fields(type0: Class<*>, rootStatics: Boolean): List { + fieldsCache[type0]?.let { return it } + val result = ArrayList() + var type = type0 + var statics = rootStatics + while (true) { + val fields = type.declaredFields.filter { + !it.type.isPrimitive + && (statics || !Modifier.isStatic(it.modifiers)) + && !(it.type.isArray && it.type.componentType.isPrimitive) + && it.name != "previousOut" // System.out from TestBase that we store in a field to restore later + } + check(fields.isEmpty() || !type.name.startsWith("java.")) { + """ + Trying to walk through JDK's '$type' will get into illegal reflective access on JDK 9+. + Either modify your test to avoid usage of this class or update FieldWalker code to retrieve + the captured state of this class without going through reflection (see how collections are handled). + """.trimIndent() + } + fields.forEach { it.isAccessible = true } // make them all accessible + result.addAll(fields) + type = type.superclass + statics = false + val superFields = fieldsCache[type] // will stop at Any anyway + if (superFields != null) { + result.addAll(superFields) + break + } + } + fieldsCache[type0] = result + return result + } +} diff --git a/test-utils/jvm/src/TestBase.kt b/test-utils/jvm/src/TestBase.kt new file mode 100644 index 0000000000..194e71b474 --- /dev/null +++ b/test-utils/jvm/src/TestBase.kt @@ -0,0 +1,251 @@ +package kotlinx.coroutines.testing + +import kotlinx.coroutines.scheduling.* +import java.io.* +import java.util.* +import kotlin.coroutines.* +import kotlinx.coroutines.* +import java.util.concurrent.atomic.AtomicReference +import kotlin.test.* + +actual val VERBOSE = try { + System.getProperty("test.verbose")?.toBoolean() ?: false +} catch (e: SecurityException) { + false +} + +/** + * Is `true` when running in a nightly stress test mode. + */ +actual val isStressTest = System.getProperty("stressTest")?.toBoolean() ?: false + +actual val stressTestMultiplierSqrt = if (isStressTest) 5 else 1 + +private const val SHUTDOWN_TIMEOUT = 1_000L // 1s at most to wait per thread + +/** + * Multiply various constants in stress tests by this factor, so that they run longer during nightly stress test. + */ +actual val stressTestMultiplier = stressTestMultiplierSqrt * stressTestMultiplierSqrt + + +@Suppress("ACTUAL_WITHOUT_EXPECT") +actual typealias TestResult = Unit + +internal actual fun lastResortReportException(error: Throwable) { + System.err.println("${error.message}${error.cause?.let { ": $it" } ?: ""}") + error.cause?.printStackTrace(System.err) + System.err.println("--- Detected at ---") + Throwable().printStackTrace(System.err) +} + +/** + * Base class for tests, so that tests for predictable scheduling of actions in multiple coroutines sharing a single + * thread can be written. Use it like this: + * + * ``` + * class MyTest : TestBase() { + * @Test + * fun testSomething() = runBlocking { // run in the context of the main thread + * expect(1) // initiate action counter + * launch { // use the context of the main thread + * expect(3) // the body of this coroutine in going to be executed in the 3rd step + * } + * expect(2) // launch just scheduled coroutine for execution later, so this line is executed second + * yield() // yield main thread to the launched job + * finish(4) // fourth step is the last one. `finish` must be invoked or test fails + * } + * } + * ``` + */ +actual open class TestBase( + private var disableOutCheck: Boolean, + private val errorCatching: ErrorCatching.Impl = ErrorCatching.Impl() +): OrderedExecutionTestBase(), ErrorCatching by errorCatching { + + actual constructor(): this(false) + + // Shutdown sequence + private lateinit var threadsBefore: Set + private val uncaughtExceptions = Collections.synchronizedList(ArrayList()) + private var originalUncaughtExceptionHandler: Thread.UncaughtExceptionHandler? = null + + actual fun println(message: Any?) { + PrintlnStrategy.actualSystemOut.println(message) + } + + @BeforeTest + fun before() { + initPoolsBeforeTest() + threadsBefore = currentThreads() + originalUncaughtExceptionHandler = Thread.getDefaultUncaughtExceptionHandler() + Thread.setDefaultUncaughtExceptionHandler { t, e -> + println("Exception in thread $t: $e") // The same message as in default handler + e.printStackTrace() + uncaughtExceptions.add(e) + } + PrintlnStrategy.configure(disableOutCheck) + } + + @AfterTest + fun onCompletion() { + // onCompletion should not throw exceptions before it finishes all cleanup, so that other tests always + // start in a clear, restored state, so we postpone throwing the observed errors. + fun cleanupStep(block: () -> Unit) { + try { + block() + } catch (e: Throwable) { + reportError(e) + } + } + cleanupStep { checkFinishCall() } + // Reset the output stream first + cleanupStep { PrintlnStrategy.reset() } + // Shutdown all thread pools + cleanupStep { shutdownPoolsAfterTest() } + // Check that are now leftover threads + cleanupStep { checkTestThreads(threadsBefore) } + // Restore original uncaught exception handler after the main shutdown sequence + Thread.setDefaultUncaughtExceptionHandler(originalUncaughtExceptionHandler) + if (uncaughtExceptions.isNotEmpty()) { + reportError(IllegalStateException("Expected no uncaught exceptions, but got $uncaughtExceptions")) + } + // The very last action -- throw all the detected errors + errorCatching.close() + } + + actual fun runTest( + expected: ((Throwable) -> Boolean)?, + unhandled: List<(Throwable) -> Boolean>, + block: suspend CoroutineScope.() -> Unit + ): TestResult { + var exCount = 0 + var ex: Throwable? = null + try { + runBlocking(block = block, context = CoroutineExceptionHandler { _, e -> + if (e is CancellationException) return@CoroutineExceptionHandler // are ignored + exCount++ + when { + exCount > unhandled.size -> + error("Too many unhandled exceptions $exCount, expected ${unhandled.size}, got: $e", e) + !unhandled[exCount - 1](e) -> + error("Unhandled exception was unexpected: $e", e) + } + }) + } catch (e: Throwable) { + ex = e + if (expected != null) { + if (!expected(e)) + error("Unexpected exception: $e", e) + } else { + throw e + } + } finally { + if (ex == null && expected != null) error("Exception was expected but none produced") + } + if (exCount < unhandled.size) + error("Too few unhandled exceptions $exCount, expected ${unhandled.size}") + } + + protected suspend fun currentDispatcher() = coroutineContext[ContinuationInterceptor]!! +} + +private object PrintlnStrategy { + /** + * Installs a custom [PrintStream] instead of [System.out] to capture all the output and throw an exception if + * any was detected. + * + * Removes the previously set println handler and throws the exceptions detected by it. + * If [disableOutCheck] is set, this is the only effect. + */ + fun configure(disableOutCheck: Boolean) { + val systemOut = System.out + if (systemOut is TestOutputStream) { + try { + systemOut.remove() + } catch (e: AssertionError) { + throw AssertionError("The previous TestOutputStream contained ", e) + } + } + if (!disableOutCheck) { + // Invariant: at most one indirection level in `TestOutputStream`. + System.setOut(TestOutputStream(actualSystemOut)) + } + } + + /** + * Removes the custom [PrintStream] and throws an exception if any output was detected. + */ + fun reset() { + (System.out as? TestOutputStream)?.remove() + } + + /** + * The [PrintStream] representing the actual stdout, ignoring the replacement [TestOutputStream]. + */ + val actualSystemOut: PrintStream get() = when (val out = System.out) { + is TestOutputStream -> out.previousOut + else -> out + } + + private class TestOutputStream( + /* + * System.out that we redefine in order to catch any debugging/diagnostics + * 'println' from main source set. + * NB: We do rely on the name 'previousOut' in the FieldWalker in order to skip its + * processing + */ + val previousOut: PrintStream, + private val myOutputStream: MyOutputStream = MyOutputStream(), + ) : PrintStream(myOutputStream) { + + fun remove() { + System.setOut(previousOut) + if (myOutputStream.firstPrintStacktace.get() != null) { + throw AssertionError( + "Detected a println. The captured output is: <<<${myOutputStream.capturedOutput}>>>", + myOutputStream.firstPrintStacktace.get() + ) + } + } + + private class MyOutputStream(): OutputStream() { + val capturedOutput = ByteArrayOutputStream() + + val firstPrintStacktace = AtomicReference(null) + + override fun write(b: Int) { + if (firstPrintStacktace.get() == null) { + firstPrintStacktace.compareAndSet(null, IllegalStateException()) + } + capturedOutput.write(b) + } + } + + } +} + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +fun initPoolsBeforeTest() { + DefaultScheduler.usePrivateScheduler() +} + +@Suppress("INVISIBLE_MEMBER", "INVISIBLE_REFERENCE") +fun shutdownPoolsAfterTest() { + DefaultScheduler.shutdown(SHUTDOWN_TIMEOUT) + DefaultExecutor.shutdownForTests(SHUTDOWN_TIMEOUT) + DefaultScheduler.restore() +} + +actual val isNative = false + +actual val isBoundByJsTestTimeout = false + +/* + * We ignore tests that test **real** non-virtualized tests with time on Windows, because + * our CI Windows is virtualized itself (oh, the irony) and its clock resolution is dozens of ms, + * which makes such tests flaky. + */ +actual val isJavaAndWindows: Boolean = System.getProperty("os.name")!!.contains("Windows") + +actual val usesSharedEventLoop: Boolean = false diff --git a/test-utils/jvm/src/Threads.kt b/test-utils/jvm/src/Threads.kt new file mode 100644 index 0000000000..956b3348aa --- /dev/null +++ b/test-utils/jvm/src/Threads.kt @@ -0,0 +1,68 @@ +package kotlinx.coroutines.testing + +import kotlinx.coroutines.* +import java.lang.Runnable + +private const val WAIT_LOST_THREADS = 10_000L // 10s +private val ignoreLostThreads = mutableSetOf() + +fun ignoreLostThreads(vararg s: String) { ignoreLostThreads += s } + +fun currentThreads(): Set { + var estimate = 0 + while (true) { + estimate = estimate.coerceAtLeast(Thread.activeCount() + 1) + val arrayOfThreads = Array(estimate) { null } + val n = Thread.enumerate(arrayOfThreads) + if (n >= estimate) { + estimate = n + 1 + continue // retry with a better size estimate + } + val threads = hashSetOf() + for (i in 0 until n) + threads.add(arrayOfThreads[i]!!) + return threads + } +} + +fun List.dumpThreads(header: String) { + println("=== $header") + forEach { thread -> + println("Thread \"${thread.name}\" ${thread.state}") + val trace = thread.stackTrace + for (t in trace) println("\tat ${t.className}.${t.methodName}(${t.fileName}:${t.lineNumber})") + println() + } + println("===") +} + +class PoolThread( + @JvmField val dispatcher: ExecutorCoroutineDispatcher, // for debugging & tests + target: Runnable, name: String +) : Thread(target, name) { + init { + isDaemon = true + } +} + +fun ExecutorCoroutineDispatcher.dumpThreads(header: String) = + currentThreads().filter { it is PoolThread && it.dispatcher == this@dumpThreads }.dumpThreads(header) + +fun checkTestThreads(threadsBefore: Set) { + // give threads some time to shutdown + val waitTill = System.currentTimeMillis() + WAIT_LOST_THREADS + var diff: List + do { + val threadsAfter = currentThreads() + diff = (threadsAfter - threadsBefore).filter { thread -> + ignoreLostThreads.none { prefix -> thread.name.startsWith(prefix) } + } + if (diff.isEmpty()) break + } while (System.currentTimeMillis() <= waitTill) + ignoreLostThreads.clear() + if (diff.isEmpty()) return + val message = "Lost threads ${diff.map { it.name }}" + println("!!! $message") + diff.dumpThreads("Dumping lost thread stack traces") + error(message) +} diff --git a/test-utils/native/src/TestBase.kt b/test-utils/native/src/TestBase.kt new file mode 100644 index 0000000000..eac3dde9f0 --- /dev/null +++ b/test-utils/native/src/TestBase.kt @@ -0,0 +1,65 @@ +package kotlinx.coroutines.testing + +import kotlin.test.* +import kotlinx.coroutines.* + +actual val VERBOSE = false + +actual typealias NoNative = Ignore + +public actual val isStressTest: Boolean = false +public actual val stressTestMultiplier: Int = 1 +public actual val stressTestMultiplierSqrt: Int = 1 + +@Suppress("ACTUAL_WITHOUT_EXPECT") +public actual typealias TestResult = Unit + +internal actual fun lastResortReportException(error: Throwable) { + println(error) +} + +public actual open class TestBase actual constructor(): OrderedExecutionTestBase(), ErrorCatching by ErrorCatching.Impl() { + actual fun println(message: Any?) { + kotlin.io.println(message) + } + + public actual fun runTest( + expected: ((Throwable) -> Boolean)?, + unhandled: List<(Throwable) -> Boolean>, + block: suspend CoroutineScope.() -> Unit + ): TestResult { + var exCount = 0 + var ex: Throwable? = null + try { + runBlocking(block = block, context = CoroutineExceptionHandler { _, e -> + if (e is CancellationException) return@CoroutineExceptionHandler // are ignored + exCount++ + when { + exCount > unhandled.size -> + error("Too many unhandled exceptions $exCount, expected ${unhandled.size}, got: $e", e) + !unhandled[exCount - 1](e) -> + error("Unhandled exception was unexpected: $e", e) + } + }) + } catch (e: Throwable) { + ex = e + if (expected != null) { + if (!expected(e)) + error("Unexpected exception: $e", e) + } else + throw e + } finally { + if (ex == null && expected != null) error("Exception was expected but none produced") + } + if (exCount < unhandled.size) + error("Too few unhandled exceptions $exCount, expected ${unhandled.size}") + } +} + +public actual val isNative = true + +public actual val isBoundByJsTestTimeout = false + +public actual val isJavaAndWindows: Boolean get() = false + +actual val usesSharedEventLoop: Boolean = false diff --git a/test-utils/wasmJs/src/TestBase.kt b/test-utils/wasmJs/src/TestBase.kt new file mode 100644 index 0000000000..021dc5ee89 --- /dev/null +++ b/test-utils/wasmJs/src/TestBase.kt @@ -0,0 +1,106 @@ +package kotlinx.coroutines.testing + +import kotlin.test.* +import kotlin.js.* +import kotlinx.coroutines.* + +actual val VERBOSE = false + +actual typealias NoWasmJs = Ignore + +actual val isStressTest: Boolean = false +actual val stressTestMultiplier: Int = 1 +actual val stressTestMultiplierSqrt: Int = 1 + +@JsName("Promise") +external class MyPromise : JsAny { + fun then(onFulfilled: ((JsAny?) -> Unit), onRejected: ((JsAny) -> Unit)): MyPromise + fun then(onFulfilled: ((JsAny?) -> Unit)): MyPromise +} + +/** Always a `Promise` */ +public actual typealias TestResult = MyPromise + +internal actual fun lastResortReportException(error: Throwable) { + println(error) +} + +actual open class TestBase( + private val errorCatching: ErrorCatching.Impl +): OrderedExecutionTestBase(), ErrorCatching by errorCatching { + private var lastTestPromise: Promise? = null + + actual constructor(): this(errorCatching = ErrorCatching.Impl()) + + actual fun println(message: Any?) { + kotlin.io.println(message) + } + + actual fun runTest( + expected: ((Throwable) -> Boolean)?, + unhandled: List<(Throwable) -> Boolean>, + block: suspend CoroutineScope.() -> Unit + ): TestResult { + var exCount = 0 + var ex: Throwable? = null + /* + * This is an additional sanity check against `runTest` mis-usage on JS. + * The only way to write an async test on JS is to return Promise from the test function. + * _Just_ launching promise and returning `Unit` won't suffice as the underlying test framework + * won't be able to detect an asynchronous failure in a timely manner. + * We cannot detect such situations, but we can detect the most common erroneous pattern + * in our code base, an attempt to use multiple `runTest` in the same `@Test` method, + * which typically is a premise to the same error: + * ``` + * @Test + * fun incorrectTestForJs() { // <- promise is not returned + * for (parameter in parameters) { + * runTest { + * runTestForParameter(parameter) + * } + * } + * } + * ``` + */ + if (lastTestPromise != null) { + error("Attempt to run multiple asynchronous test within one @Test method") + } + val result = GlobalScope.promise(block = block, context = CoroutineExceptionHandler { _, e -> + if (e is CancellationException) return@CoroutineExceptionHandler // are ignored + exCount++ + when { + exCount > unhandled.size -> + error("Too many unhandled exceptions $exCount, expected ${unhandled.size}, got: $e", e) + !unhandled[exCount - 1](e) -> + error("Unhandled exception was unexpected: $e", e) + } + }).catch { jsE -> + val e = jsE.toThrowableOrNull() ?: error("Unexpected non-Kotlin exception $jsE") + ex = e + if (expected != null) { + if (!expected(e)) { + println(e) + error("Unexpected exception $e", e) + } + } else + throw e + null + }.finally { + if (ex == null && expected != null) error("Exception was expected but none produced") + if (exCount < unhandled.size) + error("Too few unhandled exceptions $exCount, expected ${unhandled.size}") + errorCatching.close() + checkFinishCall() + } + lastTestPromise = result + return result.unsafeCast() + } +} + +actual val isNative = false + +actual val isBoundByJsTestTimeout = true + +actual val isJavaAndWindows: Boolean get() = false + +actual val usesSharedEventLoop: Boolean = false diff --git a/test-utils/wasmWasi/src/TestBase.kt b/test-utils/wasmWasi/src/TestBase.kt new file mode 100644 index 0000000000..795ed375fd --- /dev/null +++ b/test-utils/wasmWasi/src/TestBase.kt @@ -0,0 +1,70 @@ +package kotlinx.coroutines.testing + +import kotlin.test.* +import kotlinx.coroutines.* +import kotlinx.coroutines.internal.* + +actual val VERBOSE = false + +actual typealias NoWasmWasi = Ignore + +actual val isStressTest: Boolean = false +actual val stressTestMultiplier: Int = 1 +actual val stressTestMultiplierSqrt: Int = 1 + +actual typealias TestResult = Unit + +internal actual fun lastResortReportException(error: Throwable) { + println(error) +} + +actual open class TestBase( + private val errorCatching: ErrorCatching.Impl +): OrderedExecutionTestBase(), ErrorCatching by errorCatching { + + actual constructor(): this(errorCatching = ErrorCatching.Impl()) + + actual fun println(message: Any?) { + kotlin.io.println(message) + } + + public actual fun runTest( + expected: ((Throwable) -> Boolean)?, + unhandled: List<(Throwable) -> Boolean>, + block: suspend CoroutineScope.() -> Unit + ): TestResult { + var exCount = 0 + var ex: Throwable? = null + try { + runTestCoroutine(block = block, context = CoroutineExceptionHandler { _, e -> + if (e is CancellationException) return@CoroutineExceptionHandler // are ignored + exCount++ + when { + exCount > unhandled.size -> + error("Too many unhandled exceptions $exCount, expected ${unhandled.size}, got: $e", e) + !unhandled[exCount - 1](e) -> + error("Unhandled exception was unexpected: $e", e) + } + }) + } catch (e: Throwable) { + ex = e + if (expected != null) { + if (!expected(e)) + error("Unexpected exception: $e", e) + } else + throw e + } finally { + if (ex == null && expected != null) kotlin.error("Exception was expected but none produced") + } + if (exCount < unhandled.size) + kotlin.error("Too few unhandled exceptions $exCount, expected ${unhandled.size}") + } +} + +actual val isNative = false + +actual val isBoundByJsTestTimeout = true + +actual val isJavaAndWindows: Boolean get() = false + +actual val usesSharedEventLoop: Boolean = true diff --git a/ui/README.md b/ui/README.md index 3c57817cb2..0417ff92e5 100644 --- a/ui/README.md +++ b/ui/README.md @@ -1,9 +1,11 @@ # Coroutines for UI -This directory contains modules for coroutine programming with various single-threaded UI libraries: +This directory contains modules for coroutine programming with various single-threaded UI libraries. +After adding dependency to the UI library, corresponding UI dispatcher will be available via `Dispatchers.Main`. +Module name below corresponds to the artifact name in Maven/Gradle. ## Modules -* [kotlinx-coroutines-android](kotlinx-coroutines-android) -- `UI` context for Android applications. -* [kotlinx-coroutines-javafx](kotlinx-coroutines-javafx) -- `JavaFx` context for JavaFX UI applications. -* [kotlinx-coroutines-swing](kotlinx-coroutines-swing) -- `Swing` context for Swing UI applications. +* [kotlinx-coroutines-android](kotlinx-coroutines-android/README.md) -- `Dispatchers.Main` context for Android applications. +* [kotlinx-coroutines-javafx](kotlinx-coroutines-javafx/README.md) -- `Dispatchers.JavaFx` context for JavaFX UI applications. +* [kotlinx-coroutines-swing](kotlinx-coroutines-swing/README.md) -- `Dispatchers.Swing` context for Swing UI applications. diff --git a/ui/coroutines-guide-ui.md b/ui/coroutines-guide-ui.md index ee1888342b..4dcbfba7c7 100644 --- a/ui/coroutines-guide-ui.md +++ b/ui/coroutines-guide-ui.md @@ -1,87 +1,24 @@ - - - # Guide to UI programming with coroutines This guide assumes familiarity with basic coroutine concepts that are -covered in [Guide to kotlinx.coroutines](../coroutines-guide.md) and gives specific +covered in [Guide to kotlinx.coroutines](../docs/topics/coroutines-guide.md) and gives specific examples on how to use coroutines in UI applications. -All UI application libraries have one thing in common. They have the single thread where all state of the UI +All UI application libraries have one thing in common. They have the single main thread where all state of the UI is confined, and all updates to the UI has to happen in this particular thread. With respect to coroutines, it means that you need an appropriate _coroutine dispatcher context_ that confines the coroutine -execution to this UI thread. +execution to this main UI thread. In particular, `kotlinx.coroutines` has three modules that provide coroutine context for different UI application libraries: -* [kotlinx-coroutines-android](kotlinx-coroutines-android) -- `UI` context for Android applications. -* [kotlinx-coroutines-javafx](kotlinx-coroutines-javafx) -- `JavaFx` context for JavaFX UI applications. -* [kotlinx-coroutines-swing](kotlinx-coroutines-swing) -- `Swing` context for Swing UI applications. +* [kotlinx-coroutines-android](kotlinx-coroutines-android) -- `Dispatchers.Main` context for Android applications. +* [kotlinx-coroutines-javafx](kotlinx-coroutines-javafx) -- `Dispatchers.JavaFx` context for JavaFX UI applications. +* [kotlinx-coroutines-swing](kotlinx-coroutines-swing) -- `Dispatchers.Swing` context for Swing UI applications. + +Also, UI dispatcher is available via `Dispatchers.Main` from `kotlinx-coroutines-core` and corresponding +implementation (Android, JavaFx or Swing) is discovered by [`ServiceLoader`](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) API. +For example, if you are writing JavaFx application, you can use either `Dispatchers.Main` or `Dispachers.JavaFx` extension, it will be the same object. This guide covers all UI libraries simultaneously, because each of these modules consists of just one object definition that is a couple of pages long. You can use any of them as an example to write the corresponding @@ -103,12 +40,12 @@ context object for your favourite UI library, even if it is not included out of * [Event conflation](#event-conflation) * [Blocking operations](#blocking-operations) * [The problem of UI freezes](#the-problem-of-ui-freezes) + * [Structured concurrency, lifecycle and coroutine parent-child hierarchy](#structured-concurrency-lifecycle-and-coroutine-parent-child-hierarchy) * [Blocking operations](#blocking-operations) * [Advanced topics](#advanced-topics) - * [Lifecycle and coroutine parent-child hierarchy](#lifecycle-and-coroutine-parent-child-hierarchy) * [Starting coroutine in UI event handlers without dispatch](#starting-coroutine-in-ui-event-handlers-without-dispatch) - + ## Setup @@ -133,7 +70,7 @@ fun setup(hello: Text, fab: Circle) { } ``` -> You can get full code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-basic-01.kt) +> You can get the full code [here](kotlinx-coroutines-javafx/test/guide/example-ui-basic-01.kt). You can clone [kotlinx.coroutines](https://github.com/Kotlin/kotlinx.coroutines) project from GitHub onto your workstation and open the project in IDE. All the examples from this guide are in the test folder of @@ -173,14 +110,7 @@ Add dependencies on `kotlinx-coroutines-android` module to the `dependencies { . `app/build.gradle` file: ```groovy -compile "org.jetbrains.kotlinx:kotlinx-coroutines-android:0.15" -``` - -Coroutines are experimental feature in Kotlin. -You need to enable coroutines in Kotlin compiler by adding the following line to `gradle.properties` file: - -```properties -kotlin.coroutines=enable +implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:1.10.2" ``` You can clone [kotlinx.coroutines](https://github.com/Kotlin/kotlinx.coroutines) project from GitHub onto your @@ -194,23 +124,25 @@ This section shows basic usage of coroutines in UI applications. ### Launch UI coroutine -The `kotlinx-coroutines-javafx` module contains [JavaFx] context that dispatches coroutine execution to -the JavaFx application thread. We import it as `UI` to make all the presented examples +The `kotlinx-coroutines-javafx` module contains +[Dispatchers.JavaFx][kotlinx.coroutines.Dispatchers.JavaFx] +dispatcher that dispatches coroutine execution to +the JavaFx application thread. We import it as `Main` to make all the presented examples easily portable to Android: ```kotlin -import kotlinx.coroutines.experimental.javafx.JavaFx as UI +import kotlinx.coroutines.javafx.JavaFx as Main ``` -Coroutines confined to the UI thread can freely update anything in UI and suspend without blocking the UI thread. +Coroutines confined to the main UI thread can freely update anything in UI and suspend without blocking the main thread. For example, we can perform animations by coding them in imperative style. The following code updates the text with a 10 to 1 countdown twice a second, using [launch] coroutine builder: ```kotlin fun setup(hello: Text, fab: Circle) { - launch(UI) { // launch coroutine in UI context + GlobalScope.launch(Dispatchers.Main) { // launch coroutine in the main thread for (i in 10 downTo 1) { // countdown from 10 to 1 hello.text = "Countdown $i ..." // update text delay(500) // wait half a second @@ -220,9 +152,9 @@ fun setup(hello: Text, fab: Circle) { } ``` -> You can get full code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-basic-02.kt) +> You can get the full code [here](kotlinx-coroutines-javafx/test/guide/example-ui-basic-02.kt). -So, what happens here? Because we are launching coroutine in UI context, we can freely update UI from +So, what happens here? Because we are launching coroutine in the main UI context, we can freely update UI from inside this coroutine and invoke _suspending functions_ like [delay] at the same time. UI is not frozen while `delay` waits, because it does not block the UI thread -- it just suspends the coroutine. @@ -236,7 +168,7 @@ coroutine when we want to stop it. Let us cancel the coroutine when pinkish circ ```kotlin fun setup(hello: Text, fab: Circle) { - val job = launch(UI) { // launch coroutine in UI context + val job = GlobalScope.launch(Dispatchers.Main) { // launch coroutine in the main thread for (i in 10 downTo 1) { // countdown from 10 to 1 hello.text = "Countdown $i ..." // update text delay(500) // wait half a second @@ -247,10 +179,10 @@ fun setup(hello: Text, fab: Circle) { } ``` -> You can get full code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-basic-03.kt) +> You can get the full code [here](kotlinx-coroutines-javafx/test/guide/example-ui-basic-03.kt). Now, if the circle is clicked while countdown is still running, the countdown stops. -Note, that [Job.cancel] is completely thread-safe and non-blocking. It just signals the coroutine to cancel +Note that [Job.cancel] is completely thread-safe and non-blocking. It just signals the coroutine to cancel its job, without waiting for it to actually terminate. It can be invoked from anywhere. Invoking it on a coroutine that was already cancelled or has completed does nothing. @@ -292,26 +224,26 @@ passes the corresponding mouse event into the supplied action (just in case we n ```kotlin fun Node.onClick(action: suspend (MouseEvent) -> Unit) { onMouseClicked = EventHandler { event -> - launch(UI) { + GlobalScope.launch(Dispatchers.Main) { action(event) } } } ``` -> You can get full code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-actor-01.kt) +> You can get the full code [here](kotlinx-coroutines-javafx/test/guide/example-ui-actor-01.kt). -Note, that each time the circle is clicked, it starts a new coroutine and they all compete to +Note that each time the circle is clicked, it starts a new coroutine and they all compete to update the text. Try it. It does not look very good. We'll fix it later. > On Android, the corresponding extension can be written for `View` class, so that the code in `setup` function that is shown above can be used without changes. There is no `MouseEvent` - on Android, so it is omitted. + used in OnClickListener on Android, so it is omitted. ```kotlin fun View.onClick(action: suspend () -> Unit) { setOnClickListener { - launch(UI) { + GlobalScope.launch(Dispatchers.Main) { action() } } @@ -331,39 +263,39 @@ not be performed concurrently. Let us change `onClick` extension implementation: ```kotlin fun Node.onClick(action: suspend (MouseEvent) -> Unit) { // launch one actor to handle all events on this node - val eventActor = actor(UI) { + val eventActor = GlobalScope.actor(Dispatchers.Main) { for (event in channel) action(event) // pass event to action } // install a listener to offer events to this actor onMouseClicked = EventHandler { event -> - eventActor.offer(event) + eventActor.trySend(event) } } ``` -> You can get full code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-actor-02.kt) +> You can get the full code [here](kotlinx-coroutines-javafx/test/guide/example-ui-actor-02.kt). The key idea that underlies an integration of an actor coroutine and a regular event handler is that -there is an [offer][SendChannel.offer] function on [SendChannel] that does not wait. It sends an element to the actor immediately, -if it is possible, or discards an element otherwise. An `offer` actually returns a `Boolean` result which we ignore here. +there is an [trySend][SendChannel.trySend] function on [SendChannel] that does not wait. It sends an element to the actor immediately, +if it is possible, or discards an element otherwise. A `trySend` actually returns a `ChanneResult` instance which we ignore here. Try clicking repeatedly on a circle in this version of the code. The clicks are just ignored while the countdown animation is running. This happens because the actor is busy with an animation and does not receive from its channel. -By default, an actor's mailbox is backed by [RendezvousChannel], whose `offer` operation succeeds only when +By default, an actor's mailbox is backed by `RendezvousChannel`, whose `trySend` operation succeeds only when the `receive` is active. -> On Android, there is no `MouseEvent`, so we just send a `Unit` to the actor as a signal. +> On Android, there is `View` sent in OnClickListener, so we send the `View` to the actor as a signal. The corresponding extension for `View` class looks like this: ```kotlin -fun View.onClick(action: suspend () -> Unit) { +fun View.onClick(action: suspend (View) -> Unit) { // launch one actor - val eventActor = actor(UI) { - for (event in channel) action() + val eventActor = GlobalScope.actor(Dispatchers.Main) { + for (event in channel) action(event) } // install a listener to activate this actor setOnClickListener { - eventActor.offer(Unit) + eventActor.trySend(it) } } ``` @@ -376,25 +308,25 @@ fun View.onClick(action: suspend () -> Unit) { Sometimes it is more appropriate to process the most recent event, instead of just ignoring events while we were busy processing the previous one. The [actor] coroutine builder accepts an optional `capacity` parameter that controls the implementation of the channel that this actor is using for its mailbox. The description of all -the available choices is given in documentation of the [Channel()][Channel.invoke] factory function. +the available choices is given in documentation of the [`Channel()`][Channel] factory function. -Let us change the code to use [ConflatedChannel] by passing [Channel.CONFLATED] capacity value. The +Let us change the code to use a conflated channel by passing [Channel.CONFLATED][Channel.Factory.CONFLATED] capacity value. The change is only to the line that creates an actor: ```kotlin fun Node.onClick(action: suspend (MouseEvent) -> Unit) { // launch one actor to handle all events on this node - val eventActor = actor(UI, capacity = Channel.CONFLATED) { // <--- Changed here + val eventActor = GlobalScope.actor(Dispatchers.Main, capacity = Channel.CONFLATED) { // <--- Changed here for (event in channel) action(event) // pass event to action } - // install a listener to offer events to this actor + // install a listener to send events to this actor onMouseClicked = EventHandler { event -> - eventActor.offer(event) + eventActor.trySend(event) } } ``` -> You can get full JavaFx code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-actor-03.kt). +> You can get full JavaFx code [here](kotlinx-coroutines-javafx/test/guide/example-ui-actor-03.kt). On Android you need to update `val eventActor = ...` line from the previous example. Now, if a circle is clicked while the animation is running, it restarts animation after the end of it. Just once. @@ -403,10 +335,12 @@ processed. This is also a desired behaviour for UI applications that have to react to incoming high-frequency event streams by updating their UI based on the most recently received update. A coroutine that is using -[ConflatedChannel] avoids delays that are usually introduced by buffering of events. +a conflated channel (`capacity = Channel.CONFLATED`, or a buffered channel with +`onBufferOverflow = DROP_OLDEST` or `onBufferOverflow = DROP_LATEST`) avoids delays +that are usually introduced by buffering of events. You can experiment with `capacity` parameter in the above line to see how it affects the behaviour of the code. -Setting `capacity = Channel.UNLIMITED` creates a coroutine with [LinkedListChannel] mailbox that buffers all +Setting `capacity = Channel.UNLIMITED` creates a coroutine with an unbounded mailbox that buffers all events. In this case, the animation runs as many times as the circle is clicked. ## Blocking operations @@ -418,40 +352,39 @@ This section explains how to use UI coroutines with thread-blocking operations. It would have been great if all APIs out there were written as suspending functions that never blocks an execution thread. However, it is quite often not the case. Sometimes you need to do a CPU-consuming computation or just need to invoke some 3rd party APIs for network access, for example, that blocks the invoker thread. -You cannot do that from the UI thread nor from the UI-confined coroutine directly, because that would -block the UI thread and cause the freeze up of the UI. +You cannot do that from the main UI thread nor from the UI-confined coroutine directly, because that would +block the main UI thread and cause the freeze up of the UI. The following example illustrates the problem. We are going to use `onClick` extension with UI-confined -event-conflating actor from the last section to process the last click in the UI thread. +event-conflating actor from the last section to process the last click in the main UI thread. For this example, we are going to perform naive computation of [Fibonacci numbers](https://en.wikipedia.org/wiki/Fibonacci_number): ```kotlin fun fib(x: Int): Int = - if (x <= 1) 1 else fib(x - 1) + fib(x - 2) + if (x <= 1) x else fib(x - 1) + fib(x - 2) ``` We'll be computing larger and larger Fibonacci number each time the circle is clicked. To make the UI freeze more obvious, there is also a fast counting animation that is always running -and is constantly updating the text in the UI context: +and is constantly updating the text in the main UI dispatcher: ```kotlin fun setup(hello: Text, fab: Circle) { var result = "none" // the last result // counting animation - launch(UI) { + GlobalScope.launch(Dispatchers.Main) { var counter = 0 while (true) { hello.text = "${++counter}: $result" @@ -467,19 +400,75 @@ fun setup(hello: Text, fab: Circle) { } ``` -> You can get full JavaFx code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-blocking-01.kt). +> You can get full JavaFx code [here](kotlinx-coroutines-javafx/test/guide/example-ui-blocking-01.kt). You can just copy the `fib` function and the body of the `setup` function to your Android project. - + Try clicking on the circle in this example. After around 30-40th click our naive computation is going to become -quite slow and you would immediately see how the UI thread freezes, because the animation stops running +quite slow and you would immediately see how the main UI thread freezes, because the animation stops running during UI freeze. +### Structured concurrency, lifecycle and coroutine parent-child hierarchy + +A typical UI application has a number of elements with a lifecycle. Windows, UI controls, activities, views, fragments +and other visual elements are created and destroyed. A long-running coroutine, performing some IO or a background +computation, can retain references to the corresponding UI elements for longer than it is needed, preventing garbage +collection of the whole trees of UI objects that were already destroyed and will not be displayed anymore. + +The natural solution to this problem is to associate a [CoroutineScope] object with each UI object that has a +lifecycle and create all the coroutines in the context of this scope. +For the sake of simplicity, [MainScope()] factory can be used. It automatically provides `Dispatchers.Main` and +a parent job for all the children coroutines. + +For example, in Android application an `Activity` is initially _created_ and is _destroyed_ when it is no longer +needed and when its memory must be released. A natural solution is to attach an +instance of a `CoroutineScope` to an instance of an `Activity`: + + + +```kotlin +class MainActivity : AppCompatActivity() { + private val scope = MainScope() + + override fun onDestroy() { + super.onDestroy() + scope.cancel() + } + + fun asyncShowData() = scope.launch { // Is invoked in UI context with Activity's scope as a parent + // actual implementation + } + + suspend fun showIOData() { + val data = withContext(Dispatchers.IO) { + // compute data in background thread + } + withContext(Dispatchers.Main) { + // Show data in UI + } + } +} +``` + +Every coroutine launched from within a `MainActivity` has its job as a parent and is immediately cancelled when +activity is destroyed. + +> Note, that Android has first-party support for coroutine scope in all entities with the lifecycle. +See [the corresponding documentation](https://developer.android.com/topic/libraries/architecture/coroutines#lifecyclescope). + +Parent-child relation between jobs forms a hierarchy. A coroutine that performs some background job on behalf of +the activity can create further children coroutines. The whole tree of coroutines gets cancelled +when the parent job is cancelled. An example of that is shown in the +["Children of a coroutine"](../docs/topics/coroutine-context-and-dispatchers.md#children-of-a-coroutine) section of the guide to coroutines. + + + ### Blocking operations -The fix for the blocking operations on the UI thread is quite straightforward with coroutines. We'll +The fix for the blocking operations on the main UI thread is quite straightforward with coroutines. We'll convert our "blocking" `fib` function to a non-blocking suspending function that runs the computation in -the background thread by using [run] function to change its execution context to [CommonPool] of background -threads. Notice, that `fib` function is now marked with `suspend` modifier. It does not block the coroutine that +the background thread by using [withContext] function to change its execution context to [Dispatchers.Default] that is +backed by the background pool of threads. +Notice, that `fib` function is now marked with `suspend` modifier. It does not block the coroutine that it is invoked from anymore, but suspends its execution when the computation in the background thread is working: ```kotlin -suspend fun fib(x: Int): Int = run(CommonPool) { - if (x <= 1) 1 else fib(x - 1) + fib(x - 2) +suspend fun fib(x: Int): Int = withContext(Dispatchers.Default) { + if (x <= 1) x else fib(x - 1) + fib(x - 2) } ``` -> You can get full code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-blocking-02.kt). +> You can get the full code [here](kotlinx-coroutines-javafx/test/guide/example-ui-blocking-02.kt). You can run this code and verify that UI is not frozen while large Fibonacci numbers are being computed. -However, this code computes `fib` somewhat slower, because every recursive call to `fib` goes via `run`. This is -not a big problem in practice, because `run` is smart enough to check that the coroutine is already running +However, this code computes `fib` somewhat slower, because every recursive call to `fib` goes via `withContext`. This is +not a big problem in practice, because `withContext` is smart enough to check that the coroutine is already running in the required context and avoids overhead of dispatching coroutine to a different thread again. It is an overhead nonetheless, which is visible on this primitive code that does nothing else, but only adds integers -in between invocations to `run`. For some more substantial code, the overhead of an extra `run` invocation is +in between invocations to `withContext`. For some more substantial code, the overhead of an extra `withContext` invocation is not going to be significant. Still, this particular `fib` implementation can be made to run as fast as before, but in the background thread, by renaming -the original `fib` function to `fibBlocking` and defining `fib` with `run` wrapper on top of `fibBlocking`: +the original `fib` function to `fibBlocking` and defining `fib` with `withContext` wrapper on top of `fibBlocking`: ```kotlin -suspend fun fib(x: Int): Int = run(CommonPool) { +suspend fun fib(x: Int): Int = withContext(Dispatchers.Default) { fibBlocking(x) } fun fibBlocking(x: Int): Int = - if (x <= 1) 1 else fibBlocking(x - 1) + fibBlocking(x - 2) + if (x <= 1) x else fibBlocking(x - 1) + fibBlocking(x - 2) ``` -> You can get full code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-blocking-03.kt). +> You can get the full code [here](kotlinx-coroutines-javafx/test/guide/example-ui-blocking-03.kt). -You can now enjoy full-speed naive Fibonacci computation without blocking the UI thread. All we need is `run(CommonPool)`. +You can now enjoy full-speed naive Fibonacci computation without blocking the main UI thread. +All we need is `withContext(Dispatchers.Default)`. -Note, that because the `fib` function is invoked from the single actor in our code, there is at most one concurrent +Note that because the `fib` function is invoked from the single actor in our code, there is at most one concurrent computation of it at any given time, so this code has a natural limit on the resource utilization. It can saturate at most one CPU core. ## Advanced topics This section covers various advanced topics. - -### Lifecycle and coroutine parent-child hierarchy - -A typical UI application has a number of elements with a lifecycle. Windows, UI controls, activities, views, fragments -and other visual elements are created and destroyed. A long-running coroutine, performing some IO or a background -computation, can retain references to the corresponding UI elements for longer than it is needed, preventing garbage -collection of the whole trees of UI objects that were already destroyed and will not be displayed anymore. - -The natural solution to this problem is to associate a [Job] object with each UI object that has a lifecycle and create -all the coroutines in the context of this job. - -For example, in Android application an `Activity` is initially _created_ and is _destroyed_ when it is no longer -needed and when its memory must be released. A natural solution is to attach an -instance of a `Job` to an instance of an `Activity`. We can create a mini-framework for that, -by defining the following `JobHolder` interface: - -```kotlin -interface JobHolder { - val job: Job -} -``` - -Now, an activity that is associated with a job needs to implement this `JobHolder` interface and define -its `onDestroy` function to cancel the corresponding job: - -```kotlin -class MainActivity : AppCompatActivity(), JobHolder { - override val job: Job = Job() // the instance of a Job for this activity - - override fun onDestroy() { - super.onDestroy() - job.cancel() // cancel the job when activity is destroyed - } - - // the rest of code -} -``` - -We also need a convenient way to retrieve a job for any view in the application. This is straightforward, because -an activity is an Android `Context` of the views in it, so we can define the following `View.contextJob` extension property: - -```kotlin -val View.contextJob: Job - get() = (context as? JobHolder)?.job ?: NonCancellable -``` - -Here we use [NonCancellable] implementation of the `Job` as a null-object for the case where our `contextJob` -extension property is invoked in a context that does not have an attached job. - -A convenience of having a `contextJob` available is that we can simply use it to start all the coroutines -without having to worry about explicitly maintaining a list of the coroutines we had started. -All the life-cycle management will be taken care of by the mechanics of parent-child relations between jobs. - -For example, `View.onClick` extension from the previous section can now be defined using `contextJob`: - -```kotlin -fun View.onClick(action: suspend () -> Unit) { - // launch one actor as a parent of the context job - val eventActor = actor(contextJob + UI, capacity = Channel.CONFLATED) { - for (event in channel) action() - } - // install a listener to activate this actor - setOnClickListener { - eventActor.offer(Unit) - } -} -``` - -Notice how `contextJob + UI` expression is used to start an actor in the above code. It defines a coroutine context -for our new actor that includes the job and the `UI` dispatcher. The coroutine that is started by this -`actor(contextJob + UI)` expression is going to become a child of the job of the corresponding context. When the -activity is destroyed and its job is cancelled all its children coroutines are cancelled, too. - -Parent-child relation between jobs forms a hierarchy. A coroutine that performs some background job on behalf of -the view and in its context can create further children coroutines. The whole tree of coroutines gets cancelled -when the parent job is cancelled. An example of that is shown in -["Children of a coroutine"](../coroutines-guide.md#children-of-a-coroutine) section of the guide to coroutines. ### Starting coroutine in UI event handlers without dispatch @@ -631,7 +544,7 @@ from the UI thread: fun setup(hello: Text, fab: Circle) { fab.onMouseClicked = EventHandler { println("Before launch") - launch(UI) { + GlobalScope.launch(Dispatchers.Main) { println("Inside coroutine") delay(100) println("After delay") @@ -641,7 +554,7 @@ fun setup(hello: Text, fab: Circle) { } ``` -> You can get full JavaFx code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-advanced-01.kt). +> You can get full JavaFx code [here](kotlinx-coroutines-javafx/test/guide/example-ui-advanced-01.kt). When we start this code and click on a pinkish circle, the following messages are printed to the console: @@ -652,11 +565,11 @@ Inside coroutine After delay ``` -As you can see, execution immediately continues after [launch], while the coroutine gets posted onto UI thread +As you can see, execution immediately continues after [launch], while the coroutine gets posted onto the main UI thread for execution later. All UI dispatchers in `kotlinx.coroutines` are implemented this way. Why so? Basically, the choice here is between "JS-style" asynchronous approach (async actions -are always postponed to be executed later in the even dispatch thread) and "C#-style" approach +are always postponed to be executed later in the event dispatch thread) and "C#-style" approach (async actions are executed in the invoker thread until the first suspension point). While, C# approach seems to be more efficient, it ends up with recommendations like "use `yield` if you need to ....". This is error-prone. JS-style approach is more consistent @@ -673,7 +586,7 @@ coroutine immediately until its first suspension point as the following example fun setup(hello: Text, fab: Circle) { fab.onMouseClicked = EventHandler { println("Before launch") - launch(UI, CoroutineStart.UNDISPATCHED) { // <--- Notice this change + GlobalScope.launch(Dispatchers.Main, CoroutineStart.UNDISPATCHED) { // <--- Notice this change println("Inside coroutine") delay(100) // <--- And this is where coroutine suspends println("After delay") @@ -683,7 +596,7 @@ fun setup(hello: Text, fab: Circle) { } ``` -> You can get full JavaFx code [here](kotlinx-coroutines-javafx/src/test/kotlin/guide/example-ui-advanced-02.kt). +> You can get full JavaFx code [here](kotlinx-coroutines-javafx/test/guide/example-ui-advanced-02.kt). It prints the following messages on click, confirming that code in the coroutine starts to execute immediately: @@ -694,32 +607,34 @@ After launch After delay ``` - - - -[launch]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/launch.html -[delay]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/delay.html -[Job]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/index.html -[Job.cancel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-job/cancel.html -[run]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/run.html -[CommonPool]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-common-pool/index.html -[NonCancellable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-non-cancellable/index.html -[CoroutineStart]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-start/index.html -[async]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/async.html -[CoroutineStart.UNDISPATCHED]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental/-coroutine-start/-u-n-d-i-s-p-a-t-c-h-e-d.html - -[actor]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/actor.html -[SendChannel.offer]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-send-channel/offer.html -[SendChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-send-channel/index.html -[RendezvousChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-rendezvous-channel/index.html -[Channel.invoke]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel/invoke.html -[ConflatedChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-conflated-channel/index.html -[Channel.CONFLATED]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-channel/-c-o-n-f-l-a-t-e-d.html -[LinkedListChannel]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.experimental.channels/-linked-list-channel/index.html - - - -[JavaFx]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-javafx/kotlinx.coroutines.experimental.javafx/-java-fx/index.html - - + + + +[launch]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html +[delay]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/delay.html +[Job]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html +[Job.cancel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/cancel.html +[CoroutineScope]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html +[MainScope()]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-main-scope.html +[withContext]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/with-context.html +[Dispatchers.Default]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-default.html +[CoroutineStart]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-start/index.html +[async]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html +[CoroutineStart.UNDISPATCHED]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-start/-u-n-d-i-s-p-a-t-c-h-e-d/index.html + + + +[actor]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/actor.html +[SendChannel.trySend]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/try-send.html +[SendChannel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-send-channel/index.html +[Channel]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-channel/index.html +[Channel.Factory.CONFLATED]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.channels/-channel/-factory/-c-o-n-f-l-a-t-e-d.html + + + + +[kotlinx.coroutines.Dispatchers.JavaFx]: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-javafx/kotlinx.coroutines.javafx/-java-fx.html + + + diff --git a/ui/knit.code.include b/ui/knit.code.include new file mode 100644 index 0000000000..c5a3e5700a --- /dev/null +++ b/ui/knit.code.include @@ -0,0 +1,47 @@ +// This file was automatically generated from ${file.name} by Knit tool. Do not edit. +package ${knit.package}.${knit.name} + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.* +import kotlinx.coroutines.javafx.JavaFx as Main +import javafx.application.Application +import javafx.event.EventHandler +import javafx.geometry.* +import javafx.scene.* +import javafx.scene.input.MouseEvent +import javafx.scene.layout.StackPane +import javafx.scene.paint.Color +import javafx.scene.shape.Circle +import javafx.scene.text.Text +import javafx.stage.Stage + +fun main(args: Array) { + Application.launch(ExampleApp::class.java, *args) +} + +class ExampleApp : Application() { + val hello = Text("Hello World!").apply { + fill = Color.valueOf("#C0C0C0") + } + + val fab = Circle(20.0, Color.valueOf("#FF4081")) + + val root = StackPane().apply { + children += hello + children += fab + StackPane.setAlignment(hello, Pos.CENTER) + StackPane.setAlignment(fab, Pos.BOTTOM_RIGHT) + StackPane.setMargin(fab, Insets(15.0)) + } + + val scene = Scene(root, 240.0, 380.0).apply { + fill = Color.valueOf("#303030") + } + + override fun start(stage: Stage) { + stage.title = "Example" + stage.scene = scene + stage.show() + setup(hello, fab) + } +} \ No newline at end of file diff --git a/ui/knit.properties b/ui/knit.properties new file mode 100644 index 0000000000..5bb873ac4c --- /dev/null +++ b/ui/knit.properties @@ -0,0 +1,3 @@ +knit.dir=kotlinx-coroutines-javafx/test/guide/ +knit.package=kotlinx.coroutines.javafx.guide +knit.include=knit.code.include diff --git a/ui/kotlinx-coroutines-android/README.md b/ui/kotlinx-coroutines-android/README.md index 8bbbdc70a5..5be286cee1 100644 --- a/ui/kotlinx-coroutines-android/README.md +++ b/ui/kotlinx-coroutines-android/README.md @@ -1,10 +1,27 @@ # Module kotlinx-coroutines-android -Provides `UI` context for Android applications. +Provides `Dispatchers.Main` context for Android applications. Read [Guide to UI programming with coroutines](https://github.com/Kotlin/kotlinx.coroutines/blob/master/ui/coroutines-guide-ui.md) for tutorial on this module. -# Package kotlinx.coroutines.experimental.android +# Optimization -Provides `UI` context for Android applications. +R8 and ProGuard rules are bundled into this module. +R8 is a replacement for ProGuard in Android ecosystem, it is enabled by default since Android gradle plugin 3.4.0 +(3.3.0-beta also had it enabled). +For best results it is recommended to use a recent version of R8, which produces a smaller binary. + +When optimizations are enabled with R8 version 1.6.0 or later +the following debugging features are permanently turned off to reduce the size of the resulting binary: + +* [Debugging mode](../../docs/debugging.md#debug-mode) +* [Stacktrace recovery](../../docs/debugging.md#stacktrace-recovery) +* The internal assertions in the library are also permanently removed. + +You can examine the corresponding rules in this +[`coroutines.pro`](resources/META-INF/com.android.tools/r8-from-1.6.0/coroutines.pro) file. + +# Package kotlinx.coroutines.android + +Provides `Dispatchers.Main` context for Android applications. diff --git a/ui/kotlinx-coroutines-android/android-unit-tests/build.gradle.kts b/ui/kotlinx-coroutines-android/android-unit-tests/build.gradle.kts new file mode 100644 index 0000000000..2f01e03d35 --- /dev/null +++ b/ui/kotlinx-coroutines-android/android-unit-tests/build.gradle.kts @@ -0,0 +1,14 @@ +project.configureAar() + +dependencies { + configureAarUnpacking() + + testImplementation("com.google.android:android:${version("android")}") + testImplementation("org.robolectric:robolectric:${version("robolectric")}") + // Required by robolectric + testImplementation("androidx.test:core:1.2.0") + testImplementation("androidx.test:monitor:1.2.0") + + testImplementation(project(":kotlinx-coroutines-test")) + testImplementation(project(":kotlinx-coroutines-android")) +} diff --git a/ui/kotlinx-coroutines-android/android-unit-tests/resources/META-INF/services/kotlinx.coroutines.CoroutineScope b/ui/kotlinx-coroutines-android/android-unit-tests/resources/META-INF/services/kotlinx.coroutines.CoroutineScope new file mode 100644 index 0000000000..2b6308a78d --- /dev/null +++ b/ui/kotlinx-coroutines-android/android-unit-tests/resources/META-INF/services/kotlinx.coroutines.CoroutineScope @@ -0,0 +1,13 @@ +kotlinx.coroutines.android.EmptyCoroutineScopeImpl1 +kotlinx.coroutines.android.EmptyCoroutineScopeImpl2 +# testing configuration file parsing # kotlinx.coroutines.service.loader.LocalEmptyCoroutineScope2 + +kotlinx.coroutines.android.EmptyCoroutineScopeImpl2 + +kotlinx.coroutines.android.EmptyCoroutineScopeImpl1 + + +kotlinx.coroutines.android.EmptyCoroutineScopeImpl1 + + +kotlinx.coroutines.android.EmptyCoroutineScopeImpl3#comment \ No newline at end of file diff --git a/ui/kotlinx-coroutines-android/android-unit-tests/src/EmptyCoroutineScopeImpl.kt b/ui/kotlinx-coroutines-android/android-unit-tests/src/EmptyCoroutineScopeImpl.kt new file mode 100644 index 0000000000..d1f920013c --- /dev/null +++ b/ui/kotlinx-coroutines-android/android-unit-tests/src/EmptyCoroutineScopeImpl.kt @@ -0,0 +1,20 @@ +package kotlinx.coroutines.android + +import kotlinx.coroutines.* +import kotlin.coroutines.* + +// Classes for testing service loader +internal class EmptyCoroutineScopeImpl1 : CoroutineScope { + override val coroutineContext: CoroutineContext + get() = EmptyCoroutineContext +} + +internal class EmptyCoroutineScopeImpl2 : CoroutineScope { + override val coroutineContext: CoroutineContext + get() = EmptyCoroutineContext +} + +internal class EmptyCoroutineScopeImpl3 : CoroutineScope { + override val coroutineContext: CoroutineContext + get() = EmptyCoroutineContext +} diff --git a/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/CustomizedRobolectricTest.kt b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/CustomizedRobolectricTest.kt new file mode 100644 index 0000000000..69dd0edf86 --- /dev/null +++ b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/CustomizedRobolectricTest.kt @@ -0,0 +1,53 @@ +package ordered.tests + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.test.* +import org.junit.Test +import org.junit.runner.* +import org.robolectric.* +import org.robolectric.annotation.* +import org.robolectric.shadows.* +import kotlin.test.* + + +class InitMainDispatcherBeforeRobolectricTestRunner(testClass: Class<*>) : RobolectricTestRunner(testClass) { + + init { + kotlin.runCatching { + // touch Main, watch it burn + GlobalScope.launch(Dispatchers.Main + CoroutineExceptionHandler { _, _ -> }) { } + } + } +} + +@Config(manifest = Config.NONE, sdk = [28]) +@RunWith(InitMainDispatcherBeforeRobolectricTestRunner::class) +@LooperMode(LooperMode.Mode.LEGACY) +class CustomizedRobolectricTest : TestBase() { + @Test + fun testComponent() { + // Note that main is not set at all + val component = TestComponent() + checkComponent(component) + } + + @Test + fun testComponentAfterReset() { + // Note that main is not set at all + val component = TestComponent() + Dispatchers.setMain(Dispatchers.Unconfined) + Dispatchers.resetMain() + checkComponent(component) + } + + + private fun checkComponent(component: TestComponent) { + val mainLooper = ShadowLooper.getShadowMainLooper() + mainLooper.pause() + component.launchSomething() + assertFalse(component.launchCompleted) + mainLooper.unPause() + assertTrue(component.launchCompleted) + } +} diff --git a/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/FirstMockedMainTest.kt b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/FirstMockedMainTest.kt new file mode 100644 index 0000000000..f56e4c4bdd --- /dev/null +++ b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/FirstMockedMainTest.kt @@ -0,0 +1,41 @@ +package ordered.tests + +import kotlinx.coroutines.testing.* +import kotlinx.coroutines.* +import kotlinx.coroutines.test.* +import org.junit.* +import org.junit.Test +import java.lang.IllegalStateException +import kotlin.test.* + +open class FirstMockedMainTest : TestBase() { + + @Before + fun setUp() { + Dispatchers.setMain(Dispatchers.Unconfined) + } + + @After + fun tearDown() { + Dispatchers.resetMain() + } + + @Test + fun testComponent() { + val component = TestComponent() + component.launchSomething() + assertTrue(component.launchCompleted) + } + + @Test + fun testFailureWhenReset() { + Dispatchers.resetMain() + val component = TestComponent() + try { + component.launchSomething() + throw component.caughtException + } catch (e: IllegalStateException) { + assertTrue(e.message!!.contains("Dispatchers.setMain")) + } + } +} diff --git a/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/FirstRobolectricTest.kt b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/FirstRobolectricTest.kt new file mode 100644 index 0000000000..8777ed939d --- /dev/null +++ b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/FirstRobolectricTest.kt @@ -0,0 +1,52 @@ +package ordered.tests + +import kotlinx.coroutines.* +import kotlinx.coroutines.test.* +import org.junit.Test +import org.junit.runner.* +import org.robolectric.* +import org.robolectric.annotation.* +import org.robolectric.shadows.* +import kotlin.test.* + +@RunWith(RobolectricTestRunner::class) +@Config(manifest = Config.NONE, sdk = [28]) +@LooperMode(LooperMode.Mode.LEGACY) +open class FirstRobolectricTest { + @Test + fun testComponent() { + // Note that main is not set at all + val component = TestComponent() + checkComponent(component) + } + + @Test + fun testComponentAfterReset() { + // Note that main is not set at all + val component = TestComponent() + Dispatchers.setMain(Dispatchers.Unconfined) + Dispatchers.resetMain() + checkComponent(component) + } + + @Test + fun testDelay() { + val component = TestComponent() + val mainLooper = ShadowLooper.getShadowMainLooper() + mainLooper.pause() + component.launchDelayed() + mainLooper.runToNextTask() + assertFalse(component.delayedLaunchCompleted) + mainLooper.runToNextTask() + assertTrue(component.delayedLaunchCompleted) + } + + private fun checkComponent(component: TestComponent) { + val mainLooper = ShadowLooper.getShadowMainLooper() + mainLooper.pause() + component.launchSomething() + assertFalse(component.launchCompleted) + mainLooper.unPause() + assertTrue(component.launchCompleted) + } +} diff --git a/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/MockedMainTest.kt b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/MockedMainTest.kt new file mode 100644 index 0000000000..221186e688 --- /dev/null +++ b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/MockedMainTest.kt @@ -0,0 +1,3 @@ +package ordered.tests + +class MockedMainTest : FirstMockedMainTest() diff --git a/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/RobolectricTest.kt b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/RobolectricTest.kt new file mode 100644 index 0000000000..eff0c33870 --- /dev/null +++ b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/RobolectricTest.kt @@ -0,0 +1,3 @@ +package ordered.tests + +open class RobolectricTest : FirstRobolectricTest() diff --git a/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/TestComponent.kt b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/TestComponent.kt new file mode 100644 index 0000000000..fbd6d8135a --- /dev/null +++ b/ui/kotlinx-coroutines-android/android-unit-tests/test/ordered/tests/TestComponent.kt @@ -0,0 +1,24 @@ +package ordered.tests + +import kotlinx.coroutines.* + +public class TestComponent { + internal lateinit var caughtException: Throwable + private val scope = + CoroutineScope(SupervisorJob() + Dispatchers.Main + CoroutineExceptionHandler { _, e -> caughtException = e}) + public var launchCompleted = false + public var delayedLaunchCompleted = false + + fun launchSomething() { + scope.launch { + launchCompleted = true + } + } + + fun launchDelayed() { + scope.launch { + delay(Long.MAX_VALUE / 2) + delayedLaunchCompleted = true + } + } +} diff --git a/ui/kotlinx-coroutines-android/api/kotlinx-coroutines-android.api b/ui/kotlinx-coroutines-android/api/kotlinx-coroutines-android.api new file mode 100644 index 0000000000..090c14e09c --- /dev/null +++ b/ui/kotlinx-coroutines-android/api/kotlinx-coroutines-android.api @@ -0,0 +1,13 @@ +public abstract class kotlinx/coroutines/android/HandlerDispatcher : kotlinx/coroutines/MainCoroutineDispatcher, kotlinx/coroutines/Delay { + public fun delay (JLkotlin/coroutines/Continuation;)Ljava/lang/Object; + public abstract fun getImmediate ()Lkotlinx/coroutines/android/HandlerDispatcher; + public fun invokeOnTimeout (JLjava/lang/Runnable;Lkotlin/coroutines/CoroutineContext;)Lkotlinx/coroutines/DisposableHandle; +} + +public final class kotlinx/coroutines/android/HandlerDispatcherKt { + public static final fun awaitFrame (Lkotlin/coroutines/Continuation;)Ljava/lang/Object; + public static final fun from (Landroid/os/Handler;)Lkotlinx/coroutines/android/HandlerDispatcher; + public static final fun from (Landroid/os/Handler;Ljava/lang/String;)Lkotlinx/coroutines/android/HandlerDispatcher; + public static synthetic fun from$default (Landroid/os/Handler;Ljava/lang/String;ILjava/lang/Object;)Lkotlinx/coroutines/android/HandlerDispatcher; +} + diff --git a/ui/kotlinx-coroutines-android/build.gradle.kts b/ui/kotlinx-coroutines-android/build.gradle.kts new file mode 100644 index 0000000000..adbafe4571 --- /dev/null +++ b/ui/kotlinx-coroutines-android/build.gradle.kts @@ -0,0 +1,108 @@ +configurations { + create("r8") +} + +repositories { + mavenCentral() +} + +project.configureAar() + +dependencies { + configureAarUnpacking() + + compileOnly("com.google.android:android:${version("android")}") + compileOnly("androidx.annotation:annotation:${version("androidx_annotation")}") + + testImplementation("com.google.android:android:${version("android")}") + testImplementation("org.robolectric:robolectric:${version("robolectric")}") + // Required by robolectric + testImplementation("androidx.test:core:1.2.0") + testImplementation("androidx.test:monitor:1.2.0") + + + testImplementation("org.smali:baksmali:${version("baksmali")}") + "r8"("com.android.tools.build:builder:8.1.0") +} + +val optimizedDexDir = layout.buildDirectory.dir("dex-optim/") +val unOptimizedDexDir = layout.buildDirectory.dir("dex-unoptim/") + +val optimizedDexFile = optimizedDexDir.map { it.dir("classes.dex") } .get().asFile +val unOptimizedDexFile = unOptimizedDexDir.map { it.dir("classes.dex") }.get().asFile + +val runR8 by tasks.registering(RunR8::class) { + outputDex = optimizedDexDir.get().asFile + inputConfig = file("testdata/r8-test-rules.pro") + + dependsOn("jar") +} + +val runR8NoOptim by tasks.registering(RunR8::class) { + outputDex = unOptimizedDexDir.get().asFile + inputConfig = file("testdata/r8-test-rules-no-optim.pro") + + dependsOn("jar") +} + +tasks.test { + // Ensure the R8-processed dex is built and supply its path as a property to the test. + dependsOn(runR8) + dependsOn(runR8NoOptim) + + inputs.files(optimizedDexFile, unOptimizedDexFile) + + systemProperty("dexPath", optimizedDexFile.absolutePath) + systemProperty("noOptimDexPath", unOptimizedDexFile.absolutePath) + + // Output custom metric with the size of the optimized dex + doLast { + println("##teamcity[buildStatisticValue key='optimizedDexSize' value='${optimizedDexFile.length()}']") + } +} + +externalDocumentationLink( + url = "https://developer.android.com/reference/" +) +/* + * Task used by our ui/android tests to test minification results and keep track of size of the binary. + */ +open class RunR8 : JavaExec() { + + @OutputDirectory + lateinit var outputDex: File + + @InputFile + lateinit var inputConfig: File + + @InputFile + val inputConfigCommon: File = File("testdata/r8-test-common.pro") + + @InputFiles + val jarFile: File = project.tasks.named("jar").get().archiveFile.get().asFile + + init { + classpath = project.configurations["r8"] + mainClass = "com.android.tools.r8.R8" + } + + override fun exec() { + // Resolve classpath only during execution + val arguments = mutableListOf( + "--release", + "--no-desugaring", + "--min-api", "26", + "--output", outputDex.absolutePath, + "--pg-conf", inputConfig.absolutePath + ) + arguments.addAll(project.configurations["runtimeClasspath"].files.map { it.absolutePath }) + arguments.add(jarFile.absolutePath) + + args = arguments + + project.delete(outputDex) + outputDex.mkdirs() + + super.exec() + } +} diff --git a/ui/kotlinx-coroutines-android/example-app/.gitignore b/ui/kotlinx-coroutines-android/example-app/.gitignore deleted file mode 100644 index 03d649e19a..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/.gitignore +++ /dev/null @@ -1,7 +0,0 @@ -local.properties -.gradle -.idea -build -example-app.iml -app/build -app/app.iml diff --git a/ui/kotlinx-coroutines-android/example-app/app/build.gradle b/ui/kotlinx-coroutines-android/example-app/app/build.gradle deleted file mode 100644 index fb1261a25f..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/build.gradle +++ /dev/null @@ -1,46 +0,0 @@ -apply plugin: 'com.android.application' -apply plugin: 'kotlin-android' -apply plugin: 'kotlin-android-extensions' - -android { - compileSdkVersion 25 - buildToolsVersion "25.0.2" - defaultConfig { - applicationId "com.example.app" - minSdkVersion 9 - targetSdkVersion 25 - versionCode 1 - versionName "1.0" - testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" - } - buildTypes { - release { - minifyEnabled false - proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro' - } - } -} - -repositories { - mavenLocal() - mavenCentral() -} - -dependencies { - compile fileTree(dir: 'libs', include: ['*.jar']) - androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', { - exclude group: 'com.android.support', module: 'support-annotations' - }) - compile 'com.android.support:appcompat-v7:25.2.0' - compile 'com.android.support.constraint:constraint-layout:1.0.1' - compile 'com.android.support:design:25.2.0' - testCompile 'junit:junit:4.12' - compile "org.jetbrains.kotlin:kotlin-stdlib-jre7:$kotlin_version" - compile "org.jetbrains.kotlinx:kotlinx-coroutines-android:0.15" -} - -kotlin { - experimental { - coroutines "enable" - } -} diff --git a/ui/kotlinx-coroutines-android/example-app/app/proguard-rules.pro b/ui/kotlinx-coroutines-android/example-app/app/proguard-rules.pro deleted file mode 100644 index a74eede944..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/proguard-rules.pro +++ /dev/null @@ -1,25 +0,0 @@ -# Add project specific ProGuard rules here. -# By default, the flags in this file are appended to flags specified -# in C:\Users\roman\AppData\Local\Android\Sdk/tools/proguard/proguard-android.txt -# You can edit the include path and order by changing the proguardFiles -# directive in build.gradle. -# -# For more details, see -# http://developer.android.com/guide/developing/tools/proguard.html - -# Add any project specific keep options here: - -# If your project uses WebView with JS, uncomment the following -# and specify the fully qualified class name to the JavaScript interface -# class: -#-keepclassmembers class fqcn.of.javascript.interface.for.webview { -# public *; -#} - -# Uncomment this to preserve the line number information for -# debugging stack traces. -#-keepattributes SourceFile,LineNumberTable - -# If you keep the line number information, uncomment this to -# hide the original source file name. -#-renamesourcefileattribute SourceFile diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/AndroidManifest.xml b/ui/kotlinx-coroutines-android/example-app/app/src/main/AndroidManifest.xml deleted file mode 100644 index a04a27e0a3..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/src/main/AndroidManifest.xml +++ /dev/null @@ -1,24 +0,0 @@ - - - - - - - - - - - - - - \ No newline at end of file diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/java/com/example/app/MainActivity.kt b/ui/kotlinx-coroutines-android/example-app/app/src/main/java/com/example/app/MainActivity.kt deleted file mode 100644 index 9f04427c15..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/src/main/java/com/example/app/MainActivity.kt +++ /dev/null @@ -1,38 +0,0 @@ -package com.example.app - -import android.os.Bundle -import android.support.design.widget.FloatingActionButton -import android.support.v7.app.AppCompatActivity -import android.view.Menu -import android.view.MenuItem -import android.widget.TextView -import kotlinx.android.synthetic.main.activity_main.* -import kotlinx.android.synthetic.main.content_main.* - -class MainActivity : AppCompatActivity() { - override fun onCreate(savedInstanceState: Bundle?) { - super.onCreate(savedInstanceState) - setContentView(R.layout.activity_main) - setSupportActionBar(toolbar) - setup(hello, fab) - } - - override fun onCreateOptionsMenu(menu: Menu): Boolean { - // Inflate the menu; this adds items to the action bar if it is present. - menuInflater.inflate(R.menu.menu_main, menu) - return true - } - - override fun onOptionsItemSelected(item: MenuItem): Boolean { - // Handle action bar item clicks here. The action bar will - // automatically handle clicks on the Home/Up button, so long - // as you specify a parent activity in AndroidManifest.xml. - val id = item.itemId - if (id == R.id.action_settings) return true - return super.onOptionsItemSelected(item) - } -} - -fun setup(hello: TextView, fab: FloatingActionButton) { - // placeholder -} diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/layout/activity_main.xml b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/layout/activity_main.xml deleted file mode 100644 index 13d32252c8..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/layout/activity_main.xml +++ /dev/null @@ -1,33 +0,0 @@ - - - - - - - - - - - - - - diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/layout/content_main.xml b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/layout/content_main.xml deleted file mode 100644 index 110dc678ff..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/layout/content_main.xml +++ /dev/null @@ -1,21 +0,0 @@ - - - - - - diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/menu/menu_main.xml b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/menu/menu_main.xml deleted file mode 100644 index c4ad098061..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/menu/menu_main.xml +++ /dev/null @@ -1,10 +0,0 @@ - - - diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-hdpi/ic_launcher.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-hdpi/ic_launcher.png deleted file mode 100644 index cde69bccce..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-hdpi/ic_launcher.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-hdpi/ic_launcher_round.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-hdpi/ic_launcher_round.png deleted file mode 100644 index 9a078e3e1a..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-hdpi/ic_launcher_round.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-mdpi/ic_launcher.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-mdpi/ic_launcher.png deleted file mode 100644 index c133a0cbd3..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-mdpi/ic_launcher.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-mdpi/ic_launcher_round.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-mdpi/ic_launcher_round.png deleted file mode 100644 index efc028a636..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-mdpi/ic_launcher_round.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xhdpi/ic_launcher.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xhdpi/ic_launcher.png deleted file mode 100644 index bfa42f0e7b..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xhdpi/ic_launcher.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xhdpi/ic_launcher_round.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xhdpi/ic_launcher_round.png deleted file mode 100644 index 3af2608a44..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xhdpi/ic_launcher_round.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxhdpi/ic_launcher.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxhdpi/ic_launcher.png deleted file mode 100644 index 324e72cdd7..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxhdpi/ic_launcher.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.png deleted file mode 100644 index 9bec2e6231..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png deleted file mode 100644 index aee44e1384..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png deleted file mode 100644 index 34947cd6bb..0000000000 Binary files a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png and /dev/null differ diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/colors.xml b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/colors.xml deleted file mode 100644 index 3ab3e9cbce..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/colors.xml +++ /dev/null @@ -1,6 +0,0 @@ - - - #3F51B5 - #303F9F - #FF4081 - diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/dimens.xml b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/dimens.xml deleted file mode 100644 index 59a0b0c4f5..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/dimens.xml +++ /dev/null @@ -1,3 +0,0 @@ - - 16dp - diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/strings.xml b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/strings.xml deleted file mode 100644 index a94b2dfb1b..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/strings.xml +++ /dev/null @@ -1,4 +0,0 @@ - - ExampleApp - Settings - diff --git a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/styles.xml b/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/styles.xml deleted file mode 100644 index d4ea9ae701..0000000000 --- a/ui/kotlinx-coroutines-android/example-app/app/src/main/res/values/styles.xml +++ /dev/null @@ -1,20 +0,0 @@ - - - - - - - -